Real-time concatenative synthesis for networked musical interactions Free

The recent proliferation of Networked Music Performances has led to the investigation of low-latency, low-bitrate musical encoding schemes, including audio codecs and control protocols that specifically address the requirements of live musical interactions across the Internet. This work presents an alternative perspective inspired by the 'synthesis by analysis' approach, strictly constrained in terms of processing latencies and rendering quality. The entire process is fully automated and involves an offline processing phase (that takes place prior to performance) and a real-time analysis-synthesis phase. The offline phase involves processing a solo recording of each musician's part so as to acquire audio segments corresponding to each note in the performance, and a trained Hidden Markov Model to be later used for online analysis. During live performance, online analysis encodes the position of the performance on a music score and resynthesizes the waveform by concatenating the audio segments of the offline phase. Although the synthesized waveform originates from an offline recording, it is synchronized to the live performance at note level, so as to allow for rendering a wide range of musical tempi as well as their expressive variations. The paper presents the complete methodology and reports on implementation details and preliminary evaluation results.