Multiple-timbre note tracking using linear dynamical systems

This work addresses the problem of multi-pitch detection and note tracking in multiple-instrument polyphonic music recordings. A system is developed that extends probabilistic latent component analysis (PLCA) and supports the use of a five-dimensional dictionary of spectral templates per instrument, pitch, deviation from ideal tuning, and sound state (e.g., attack, sustain, decay). A method based on linear dynamical systems (LDS) is introduced for note tracking, which assumes that the output of the PLCA model is the (noisy) observation in an LDS, with the latent states corresponding to the ideal multi-pitch activation output. The LDS-based process supports the tracking of multiple concurrent pitches and can also be integrated within the PLCA-based model, thus guiding the convergence of the multi-pitch detection process. Experiments performed on several datasets of multiple-instrument polyphonic music demonstrate that the LDS-based method leads to significant improvements in multi-pitch detection as compared to using the frame-based PLCA model alone.