Spectrotemporal Gabor filters for feature detection

Features (landmarks) in sound are located in time and spectrum. Two dimensional (time x spectrum) Gabor filters can be used to detect useful classes of these. We use a set of logarithmically spaced bandpass filters whose outputs are coded as spikes to perform spectral analysis. These are convolved with Gabor filters to create spectrotemporal feature maps. Using auditory-nerve like spikes to code zero-crossings retains precise timings and amplitude information, and makes convolution computation relatively straightforward. Gabor patches with “horizontal” bars (parallel to time axis) can be used to detect harmonicity, and patches with “vertical” bars (parallel to spectrum axis) can detect envelope modulations. Because the time resolution is maintained in the preprocessing, vertical bars may be close together (e.g., 5 to 10 ms apart), enabling detection of amplitude modulation due to unresolved harmonics. This is useful for both for speech voicing detection, and for animal utterances. Such filters may be localized in spectrum, allowing tracking of voicing energy. Filters with bars at other angles can detect frequency modulation. Using constellations of these features (and others, such as onsets), we can characterize and interpret sound sources.