Testing theories of vowel inherent spectral change^a)

Traditionally, the English vowel system is often said to comprise true diphthongs, /aɪ, aʊ, ɔɪ/, phonetic diphthongs, /e, o/ (frequently transcribed as /eɪ, oʊ/), and monophthongs, e.g., /i, ɪ, ε, æ/. However, for many North American dialects, it has been reported that several nominal monophthongs show significant vowel inherent spectral change (VISC), and VISC appears to be important for perception (Andruski and Nearey, 1992; Assmann and Katz, 2005; Assmann et al., 1982; Hillenbrand et al., 2001). Figure 1 provides examples of mean F1-F2 formant trajectories for natural productions of Western Canadian English /e/, /ɪ/, and /ε/ in the same context as used in the present study (data from Morrison, 2006).

There are three main accounts of the perceptually relevant aspects of VISC (Gottfried et al., 1993; Nearey and Assmann, 1986). All three hypotheses agree that the initial formant frequencies are perceptually relevant for vowel identification, but disagree on what additional cues are relevant. The $onset+offset$ hypothesis states that the relevant perceptual cues are the formant values at the end of the vowel. The $onset+slope$ hypothesis states that the relevant cue is the rate of change of formants over time. The $onset+direction$ hypothesis states that the only relevant factor is the direction of formant movement in an F1-F2 (or similar) space. To elucidate the difference among the three hypotheses: In the direction hypothesis the rate of change in time (hereafter speed) of formant movement and the formant values achieved at the end of the trajectory are irrelevant. In the slope hypothesis the direction and speed of formant movement are relevant, but the formant values achieved at the end of the trajectory are irrelevant. If the glide portion of one vowel is longer than the glide portion of another vowel, then they could both have the same formant slopes but different final formant values. In the offset hypothesis the formant values achieved at the end of the trajectory are relevant (i.e., direction and magnitude of formant movement are relevant), but the speed of movement toward those values is irrelevant. If the glide portion of one vowel is longer than the glide portion of another vowel, then they could both have the same final formant values but different formant slopes.

Nearey and Assmann (1986) and Gottfried et al. (1993) used pattern recognition models to test alternate parametrizations consistent with the three VISC hypotheses. In both studies, although there was a slight advantage for the offset model, all three parametrizations performed well in terms of correct identification (and, in the earlier study, correlation with listeners’ response patterns).¹ In contrast to the former studies, the stimuli in the present experiment are explicitly designed to differentiate among the three hypotheses. Specifically, the hypotheses are directly compared in a perceptual experiment using a synthetic /e/-/ɪ/-/ε/ continuum which has a fixed onset, but which varies in offset, duration, and slope.

Stimuli were synthesized in a male-speaker range using an implementation of the Klatt cascade formant synthesizer (Klatt and Klatt, 1990). The stimuli were /bVpə/, consisting of synthetic /bVp̚/ plus a natural recording of the final /pə/ syllable from the burst onward.² Care was taken (adjusting glottal slope and breathiness parameters) to match the voice quality of the synthetic speech to the natural portion of the stimuli. To limit the stimulus space, pilot studies were conducted to find a single initial set of F1-F2 values from which it was possible to obtain /e/, /ɪ/, and /ε/ percepts by changing only the final formant values and duration. The initial F1 and F2 values were 425 and $1900Hz$⁠, respectively (6.052 and $7.550logHz$⁠), see Fig. 1.

Three types of vowels were synthesized: Formant trajectories were either straight (in log hertz) from the beginning to the end of the vowel [Figs. 2(a), 2(c), and 2(e)]; or they were elbowed with an initial steady state for the first 25% of the vowel duration, followed by a glide [Figs. 2(b), 2(d), and 2(f)]. Elbowed diphthongs thus had the same initial and final formant values as straight diphthongs, but the slope of the glide portion was 33% steeper. The third vowel type was flat, having no formant movement during the vowel.

Each stimulus had one of three directions of movement. The flat stimuli had zero movement. The other two directions were opposite in the log F1-F2 space (see Fig. 1), stimuli either had converging VISC [F1 rose and F2 fell, see Figs. 2(c)–2(f)], or diverging VISC [F1 fell and F2 rose, see Figs. 2(a) and 2(b)].³ Nonflat stimuli had one of four magnitudes of VISC (multiples of $±0.0756logHz$ for F1 and $∓0.0367logHz$ for F2). Because natural productions of the three vowels examined here differ in duration as well as spectral properties, stimuli also ranged over three duration values: 60, 80, and $100ms$ (excluding consonant transitions). This resulted in a total of 51 stimuli: Eight straight, eight elbowed, and one flat, all multiplied by three durations.

Listeners were 23 undergraduate-student volunteers. They were monolingual English speakers who had grown up in western Canada, and reported no hearing deficits.

Listeners were tested one at a time in a sound booth. The stimuli were played at a comfortable volume (approximately $65dB$ SPL) via a Roland Edirol UA-30 sound card and a calibrated Mackie HR824 studio monitor. In each trial, the listeners heard a stimulus, then saw three buttons on a computer screen labeled baypa, bippa, and beppa representing /e/, /ɪ/, and /ε/, respectively (prior to the experiment, listeners were trained on the orthography to vowel-category relationship). They indicated their response via a mouse click. A new stimulus was automatically presented $500ms$ after a response to the previous stimulus was provided. The whole set of stimuli were presented eight times in randomized blocks, resulting in a total of 408 trials per listener.

Direct statistical tests of the three rival hypotheses are not possible because such tests require a strict nesting of terms, and the hypotheses are not related to each other in this manner. An indirect strategy analogous to partial correlation analysis was therefore adopted. The procedure is as follows: Models of two rival hypotheses (e.g., A and B) are fitted to the data, then a third larger model (e.g., C) including all the terms of the first two models is fitted. Thus, both smaller models are properly nested within the third. If the larger model fits significantly better than A but not significantly better than B, then it is reasonable to infer support for hypothesis B. Roughly speaking, B includes essentially all the relevant information in the larger hypothesis, while adding the extra terms from A adds little explanatory information.

In order to compare the adequacy of the offset versus the slope hypotheses, Vector smoothed generalized additive logistic regression models (Yee and Wild, 1996) were fitted to the perceptual response data (the raw counts of responses for each category given to each stimulus), and compared for goodness-of-fit. Three models were fitted to each of the listeners’ data sets. In all models, duration (three values) was entered in milliseconds and fitted via a (saturated) quadratic polynomial. Duration parameters were included to control for the effect of duration which is not of interest in the present letter. Other parameters entered into the models related directly to formant movement. These were fitted via smoothing splines with two effective degrees of freedom.⁴ Since all three models include initial formant specifications, the differences among them can be summarized by their characterization of spectral change:

(A) Offset model: $ΔF1$⁠, the change of F1 in log Hz from the beginning to the end of the vowel.

(B) Slope model: $ΔF1∕Δt$⁠, the slope of F1 in log Hz/s over the glide portion of the vowel (from the beginning or elbow of the vowel, for straight and elbowed stimuli, respectively, to the end of the vowel).

$(C=A+B)$ Combined model: A model containing both $ΔF1$ and $ΔF1∕Δt$⁠. This allows for both the offset and the slope to have effects on perception.

Parametrizations A and B are equivalent to those of Nearey and Assmann (1986). Since F2 in the synthetic stimuli in the present study was perfectly correlated with F1, it was only necessary to enter one set of formant values into the models.

The model-fitting procedure minimizes the deviance statistic $(G2)$⁠. Table I gives results of $ΔG2$ likelihood ratio tests comparing the differences in the goodness-of-fit between the larger model (model C) and each of the two smaller models (models A and B). Models fitted to data from 10 of the 23 listeners had a significant (nominal $p<0.05$⁠) improvement in goodness-of-fit when the offset parameters were added to a model already containing slope parameters. In contrast, when slope parameters were added to a model already containing offset parameters, there was a significant improvement in goodness-of-fit for data from only three listeners, and these three data sets had also had a significant improvement of fit when the offset parameters were added to the slope model. The results therefore provide greater support for the $onset+offset$ hypothesis than for the $onset+slope$ hypothesis.

If the direction hypothesis is correct, then there should be no difference in listeners’ responses to stimuli which have the same VISC direction in the F1-F2 plane but different final formant values. The direction versus offset hypotheses were tested in the same manner as the slope versus offset hypotheses. The formant-movement parameters in the models were:

(A) Offset model: $ΔF1$ (the same as noted earlier).

(D) Direction model: The three directions of formant movement in the stimuli, diverging, flat, and converging, were entered as three discrete interval levels, $−1$⁠, 0, and $+1$⁠.⁵

$(E=A+D)$ Combined model: A model containing both $ΔF1$ and direction parameters.

Results of the $ΔG2$ likelihood ratio tests are given in Table II. Models fitted to data from 16 of the 23 listeners had a significant (nominal $p<0.05$⁠) improvement in goodness-of-fit when the offset parameters were added to a model already containing direction parameters. In contrast, when the direction parameters were added to a model already containing offset parameters, there was a significant improvement in goodness-of-fit for data from only one listener. The results therefore provide much greater support for the offset hypothesis than for the direction hypothesis.

There was considerable interlistener variation in response patterns. Also, a number of listeners gave a preponderance of /e/ responses, even for stimuli with converging VISC. Note that interlistener variation and bias do not invalidate the results of the analyses. All that is required for the reasonable application of the modeling procedure is that the response probabilities change noticeably under at least some of the stimulus manipulations at issue. For each listener there was a significant improvement in goodness-of-fit when a model including only duration information was compared with the offset model (model A), this indicates that the requirement was met with respect to formant movement.

All hypotheses agree on the importance of initial formant values, and the onset point in the experiment may have given a strong cue for /e/ perception. Examination of Fig. 1 indicates that the ratio of initial F1 and F2 values were closest to the mean of natural /e/ productions [the Morrison (2006) data were collected after the present study was conducted]. It is also possible that no single onset point will allow for clear percepts of all three vowels by all listeners, and additional experiments using multiple onset points may be warranted.

Furthermore, it is conceivable that a perceptual mechanism registering rate of spectral change may have operational limits (e.g., temporal smearing) that produce differences in “effective slope” that are less than the nominal 33% differences between corresponding straight and elbowed stimuli. Thus, the distinction between slope and offset hypotheses should probably also be assessed in contexts with longer mean vowel durations (e.g., isolated CVCs), where absolute physical differences between straight and elbowed stimuli would be larger.

Finally, there is perhaps a weak indication that more complex characterizations of formant trajectories should be investigated (see footnote 1 for some possible parametrizations). The combined models C and E contain information not available in their constituents (⁠$A+B$ and $A+D$⁠, respectively). Three of 23 listeners for model C and one for model E showed significant improvement over both the relevant simpler models. Since at the nominal 0.05 $α$ level, one might expect to find only about two false positives (Type I errors), it would seem judicious to examine more complex trajectory models in a larger and presumably more powerful experiment.

The weight of evidence presented here indicates that the $onset+offset$ hypothesis is superior to the $onset+slope$ and $onset+direction$ hypotheses.

This research was supported by the Social Sciences and Humanities Research Council of Canada. Thanks to Peter F. Assmann, Michael Kiefte, the Associate Editor, and two anonymous reviewers for comments on earlier drafts of this letter.