An audio‐visual investigation of linguistic stress perception

The true acoustic correlate for linguistic stress has not yet been found. While it was originally thought that listeners base stress judgments on syllable intensity, it has been shown that intensity, fundamental frequency, duration, and spectral structure can all act as effective information for stress perception [D. Isenberg and T. Gay, J. Acoust. Soc. Am. Suppl. 1 63, S21 (1963)]. Because no single simple acoustic dimension seems to be dominant, it has been proposed that listeners base stress judgments on an articulatory property such as vocal effort. Since such properties can be specified optically as well as acoustically, a study to test this hypothesis was conducted which used conflicting audio‐visual presentations of a speaker producing tokens from two noun‐verb pairs (CONvict—conVICT and PERmit—perMIT). The prediction was made that if stress judgments are based on perception of articulatory dynamics rather than on simple acoustic parameters, then judgments should be affected by visual as well as auditory information. It is shown that stress judgments are affected by visual information even when subjects (1) are instructed to base their judgments on only what they hear and (2) cannot detect a discrepancy between the audio and visual components. Similar results are also shown for noun‐verb tokens distinguished by an auditory dimension that cannot be specified visually (fundamental frequency) indicating that a more general articulatory property, such as vocal effort, might be the basis of stress judgments.