Sensitivity to intermodal asynchrony between acoustic and structural vibrations

The purpose of this study was to discover the attributes of musical stimuli which facilitate sensory integration in bi‐modal music reproduction systems incorporating sound and whole‐body vibration. It was hypothesized that subjective judgments regarding bimodal synchrony would vary depending on the spectral, temporal, and spatial properties of the stimuli. To test this hypothesis, musical instruments with significant low frequency energy and a variety of spectra‐temporal envelopes were recorded. These stimuli were then reproduced with varying intermodal delay and overlap in frequency content between displayed vibratory and acoustic components. The air‐born component of the bimodal stimuli was presented via a multichannel loudspeaker array, with a direct sound component, as well as a reproduced indirect sound arriving from all around the observer. Psychometric functions were constructed for time order judgment (TOJ) over a range of intermodal delay values. Changes in the slope and intercept of the transformed psychometric functions gave a clear picture of the influence of spectra‐temporal and spatial parameters of the multimodal stimuli, the most striking results being the decreased tolerance for intermodal asynchrony associated with instruments recorded in reverberant environments. [Work supported by a Grant from VRQ of the Government of Quebec.]