The role of coarticulatory acoustic detail in the perception of verbal inflection

From a discriminative learning perspective (Ramscar et al., 2010; Ramscar et al., 2013), listeners use acoustic cues to predict three types of outcomes during comprehension. The first outcome type is semantic, such that an auditory input signal [tɑk] is understood as a talk and [tɑkt] is understood as past of talk. The second type is phonetic: an acoustic signal is perceived as [z] or [s]. The third type is predicted outcomes. For example, upon hearing [ɛkspɛkteɪ], listeners likely expect [ʃn] (Nixon and Tomaschek, 2020).

As the speech signal unfolds, an outcome decision of the prediction process can theoretically be made at every single time step. However, this is not the case. With the speech signal being inherently variable, language users have to learn which acoustic cues are informative about outcomes—and which cues are not informative. Studies on phonetic perception have shown that prediction, especially of phonetic contrasts, is based on acoustic cues from a time window that can extend over an entire syllable or more, therein included cues associated with anticipatory coarticulation (e.g., Denes, 1955; Heid and Hawkins, 2000; West, 1999; Whalen, 1984). However, little is known about how cues from coarticulatory detail are informative regarding outcomes that entail changes in meaning (including the morphological level).

Our aim is to investigate the degree to which acoustic detail associated with anticipatory coarticulation is informative about changes in meaning, more specifically meaning at the level of inflectional functions of English verbs. This is accomplished with two splicing experiments in which listeners classify (a) bare verbal stems and (b) original inflected verbs and inflected verbs where the inflectional suffix is spliced to an uninflected stem. We find that listener response latencies are slower when coarticulatory acoustic detail mismatched with the inflectional suffix and conclude that coarticulatory acoustic detail is informative and listeners actively use it to make predictions about the verbs' inflectional functions.

Next, we discuss systematic variation in acoustic cues and what systematic variation has been identified as informative about phonetic contrasts. From this we derive our hypothesis that coarticulatory phonetic detail provides informative cues about inflectional outcomes in inflected verbs.

Phonetic research has uncovered many contexts in which the characteristics of hypothetically identical phones vary. For example, vowels are longer preceding voiced consonants than preceding voiceless consonants (Denes, 1955; Kleber, 2020). In addition, spectral variation has been found to correlate with the identity of neighboring phones. This “coarticulation” not only occurs between a phone and its direct neighbors (e.g., Recasens, 1984; Öhman, 1966); it also has far reaching effects across multiple segments and even syllables (Heid and Hawkins, 2000; Magen, 1997).

In addition, phonetic detail is also associated with lexical characteristics such as frequency or conditional probability (e.g., Aylett and Turk, 2004; Tremblay and Tucker, 2011; Wright, 2004). Of importance for the present study is the increasing number of studies that document systematic variation in relation to morpholexical information (Bell and Plag, 2012; Cho, 2001; Kemps et al., 2005; Lee-Kim et al., 2012; Lõo et al., 2018; Plag et al., 2017; Tomaschek et al., 2021a). Recently, Tomaschek et al. (2021b) have demonstrated systematic differences in fine phonetic detail of the stem vowel in relation to the inflectional function of English verbs. The present study is a direct follow-up on this study as it investigates whether the differences documented in Tomaschek et al. are functional in perception.

As discussed above, there is evidence that phonetic variation is not random. Instead, variation is associated with phonetic sources and lexical sources. This kind of variation provides informative cues during speech perception, e.g., in classification tasks. Informative cues are not constrained to phones of interest but are also present in the preceding phonetic signal, especially in vowels. For example, in vowel-nasal sequences, non-nasal vowels are typically slightly nasalized. This anticipatory nasalization is informative enough such that when the postvocalic nasal is artificially spliced away, listeners are able to predict that a nasal should have followed (e.g., Ali et al., 1971). In vowel-obstruent sequences, the duration of the vowel is informative about the voicing of upcoming consonants, even when primary voicing cues in the consonant are missing (e.g., Denes, 1955). When listeners are presented with matching and non-matching coarticulatory phonetic detail in vocalic transitions during auditory lexical decision tasks, listeners' response latencies are slower in the case of non-matching coarticulatory phonetic detail (e.g., Whalen, 1984).

These and other studies illustrate that phonetic variation in the speech signal provides informative cues for speech comprehension [see also Nguyen (2012) for a review]. Listeners closely track coarticulatory phonetic cues (Beddor et al., 2013) and use them to predict the identity of upcoming phonetic contrasts. Discriminative learning (Ramscar et al., 2010; Ramscar et al., 2013) argues that listeners learn to use all informative cues about a specific upcoming event—be it a phonetic contrast or an upcoming phone [see Nixon and Tomaschek (2020) for application of discriminative learning to phonetics]. When the coarticulatory phonetic detail predicts one kind of phonetic contrast but listeners are presented with another one, they experience prediction error, which inhibits response latencies.

Since phonetic detail has been found to systematically vary with morphological structures, it follows that these cues should also be perceptually salient. Indeed, Davis et al. (2002) find that hypothetically homophonous syllables are informative about whether they were part of a larger inflected word or a free word in a phonetically matched phrase. Kemps et al. (2005) presented listeners with Dutch stems that were either originally singular nouns (without suffixation) or stems where the plural suffix was spliced away. Despite the missing suffix, listeners were able to identify which stem was originally singular and which was plural.

Considering these production and perception findings, we hypothesize that inflected stems contain systematically different phonetic detail due to coarticulation; these differences are informative and predictive about the suffix and, with it, the verb's inflectional function. We tested this hypothesis in two forced-choice experiments.

Following the findings by Kemps et al. (2005), Experiment 1 investigates whether listeners make use of anticipatory phonetic detail in stems to predict the verb's inflectional function when the suffix is missing.

Since the present study is a direct follow-up to Tomaschek et al. (2021b), the same 403 verbs (114 stems) were used here: 108 base verbs (stem), 98 second person present verbs (stem+s), 90 past verbs (stem+d), and 107 progressive verbs (stem+ing). The unequal distributions among the inflectional categories are a result of taking care that past and second person verbs were always regular and monosyllabic and that no irregular form was used [a complete list is available in Tomaschek and Tucker (2021)]. Since Tomaschek et al. investigated articulatory patterns in the stem vowel, stimuli were controlled for the vowel in the verbal stem ([iː], [ɪ], [ɒ], [ɑː]). A more detailed description of selection criteria can be found in Tomaschek et al. (2021b). A male native speaker of American English (aged 28) recorded all of the verbs, as part of a carrier sentence (“I said…. yesterday”) presented in a random list. To obtain bare stems of inflected verbs, we trimmed the suffixes in all the inflected verbs at the nearest zero crossing, producing bases only.

The bare stems were presented to 31 native speakers of English (one was left-handed and one reported some hearing loss). Testing was performed in a sound attenuated booth in the Department of Linguistics at the University of Alberta. Listeners received partial course credit for participation. All experiments were performed according to University of Alberta ethical standards.

Participants were seated in a sound attenuated booth in the Alberta Phonetics Laboratory. Audio stimuli were presented via over-the-ear MB-Quart (Obrigheim, Germany) headphones at an amplitude of 81 dB. The experiment was performed using a custom python script for the experiment. Listeners were instructed that they would hear a bare word stem and should guess which ending they expected the verb to end with: “A” = -ed verbs, “D” = -ing verbs, “J” = -s verbs, “L” = base verbs. As a reminder, response options were available on the screen throughout the experiment. Items were counterbalanced for presentation so that each listener only heard one version of each lexical item. We also reduced the items to 80 or 81 per experiment to keep the total duration of the experiment down. The variability between 80 and 81 was to accommodate all of the lexical items in the recordings.

We analyzed the listeners' error rates and their response latencies (2499 total responses) measured from stimulus offset. We excluded responses with response latencies longer than 10 s and shorter than zero (89 samples, 3.6% of the data).

Table 1 illustrates the confusion matrix in percentages for the responses in experiment 1. The relatively high percentages indicate that listeners preferred base and -ed verbs independently of the real inflectional function of the stimulus. By contrast, they were less likely to respond -ing and -s. The result indicates that listeners struggled to make use of coarticulatory phonetic detail in the bare stems to guess the inflectional function of the stem. This is supported by d′-values lower than 0.3 for all stimuli. We also inspected how this bias was distributed across listeners but were not able to find any systematic preference in the listeners' responses.

Does this result indicate that coarticulatory phonetic detail is not informative about a specific inflectional function? Not necessarily. One possible explanation for why participants responded base more often than the other inflectional functions is simply because the coarticulatory information was not enough to overcome the missing morpheme. It is possible that, rather than in their categorical response behavior, we may see the effects of coarticulatory phonetic detail in more gradient behavior, as measured by the listeners' response latencies. Since the current experiment very likely biased listeners to respond base, we focus in the following paragraph only on base responses.

Two possible listener responses can be observed here. Either the listeners' responses were correct because they were truly presented with a bare uninflected stem, or their responses were incorrect because they were presented with a stem missing the inflectional suffix. When listeners correctly responded base, the coarticulatory phonetic detail in the stem was consistent with their response. When they incorrectly responded base, the coarticulatory phonetic detail in the stem was inconsistent with their responses. We analyzed whether the listeners' response latencies were affected by this inconsistency. The coarticulatory phonetic detail in stems from which the suffix was trimmed predicted an upcoming suffix, thus a specific inflectional function of the verb. When this suffix does not occur, it rules out the possibility that the verb is inflected. The error between the predicted inflectional suffix and the missing inflectional suffix should have slowed down listeners' response latencies.

Figure 1 (left) illustrates the distribution of the raw response latencies (in seconds) for the items categorized as base depending on whether the answer was correct. As can be seen, response latencies are on average slower for false responses than for correct responses. This conclusion is supported by a linear mixed-effects regression model (Bates et al., 2014) fitting response latencies with the factorial predictor response (correct, incorrect). We added a control predictor duration to account for systematic differences in duration between suffixed stems and unsuffixed stems [base duration = 0.473 s; -ed inflected: β_ed = –0.037 s, standard deviation of the error (SDE)_ed = 0.007, t_ed = –5.63; -ing inflected: β_ing = –0.172 s, SDE_ing = 0.006, t_ing = –27.89; -s inflected: β_s = –0.018 s, SDE_s = 0.006, t_s = –2.93]. The model contained random intercepts for participants and lemma. We also tested the effects of word frequency and phonological neighborhood density, both of which failed to be significantly predictive of response latencies. We find that listeners' response latencies were marginally faster when they were presented with longer stimuli [β = –0.18, standard deviation (SD) = 0.08, t = –2.26]. Furthermore, we find that the listeners' base responses were slower when they incorrectly responded base (β = 0.09, SD = 0.02, t = 3.9), i.e., when listeners were presented with coarticulatory phonetic detail inconsistent with their responses.

In our data, there is a relationship between correct and incorrect responses and the duration of the stimuli, such that stems missing the inflectional suffix are significantly shorter than the uninflected bases (see above). As a post hoc analysis, we tested to what degree the duration of the bases affected the outcome of our model. We found that direction of the effects does not change when we remove one or the other of the predictors. The model containing response (correct, incorrect) provided a better model fit than the one containing duration (⁠$χ5,02=10.3693$⁠, p < 0.001), indicating that the categorical response predictor is better than the gradient duration predictor. Using both predictors in one model marginally improved the model fit (⁠$χ6,12=4.3104$⁠, p = 0.037 88). These results indicate that coarticulatory phonetic detail is informative about inflectional functions and is used by listeners during word recognition. In the next experiment, we investigate this finding in more detail.

In experiment 1, listeners were presented with bare stems and had to decide among four choices. Experiment 2 tested how coarticulatory phonetic detail affected listeners' response latencies when they were presented with inflected stems in which anticipatory phonetic detail inconsistently predicted the verb's inflectional function.

In this experiment, we manipulated tense for 100 verb stems from experiment 1. We created two kinds of stimuli: original and spliced across past and present second person singular. To achieve a maximal difference in acoustic phonetic cues between original and spliced inflected stimuli, spliced stimuli were created by cutting [s/z] and [t/d] from the originally inflected verbs (e.g., “talks”) and attaching them to the original, uninflected base verbs (e.g., “talk”). We took care that no artifacts were present at the transition between base and word final consonant by linearly fading out the last 0.9 ms of the base and fading in the initial 0.9 ms of the attached consonant. In this way, uninflected bases and suffixes were spliced at zero crossings. We suspect that both spectral and temporal anticipatory phonetic detail are potentially informative about upcoming inflectional suffixes. This is why we did not perform any time warping to match the timing between the original and spliced stems. Simultaneously, by using original uninflected bases, we avoided spectro-temporal artifacts due to the procedure.

The same listeners as in experiment 1 performed experiment 2.

The procedure in experiment 2 was identical with experiment 1. Listeners were instructed to press “D” if they heard past tense and “K” if they heard present tense. Original and spliced present and past stimuli were counter-balanced across listeners (2 × 2 across tense and splicing), resulting in four different randomized lists. Each participant completed 100 trials.

It turned out during the analysis that 13 verbs still contained minimal acoustic artifacts that had remained unnoticed during their creation. These verbs were excluded from the analysis.

Our response variable was the listeners' response latencies measured from the offset of the stimulus. From the 3100 total responses, we excluded all responses that were executed before stimulus offset and those that were longer than 5 s (N = 66, i.e., 2.2% of the data). We further excluded incorrect answers from the data set (N = 157, i.e., 5.2% of the data). Raw latencies were non-normally distributed with a long right tail. To obtain normal distribution, latencies were power-transformed (1/2.6). We modeled response latency using a linear mixed-effects model.

Our predictor of interest was stimulus type (original, spliced). To account for differences in response latency due to the inflectional function (and potentially the effect of the final consonant), tense (past, present) was included as a predictor, with “present” as the reference level. Participants and verbs were included as random intercepts.

We investigated whether phonetic context influenced response latency, testing effects of stimulus duration, the type of the stem vowel ([i, ɪ, ʋ, ɑ]), voicing of the word final consonants ([z, s, t, d]), and place of articulation of the consonant following the vowel. Apart from stimulus duration, none of the phonetic effects had a significant effect on response latency. We tested an interaction between tense and stimulus type, which was not significant (β = –0.02412, SDE = 0.01613, t = –1.496). We also tested lexical effects, fitting response latency with log-transformed frequency of occurrence and log-transformed phonological neighborhood density. Phonological neighborhood density did not significantly correlate with response latency.

Figure 1 (right) illustrates the effects in the data. Our control predictors show that listeners respond significantly faster to past verbs than to present verbs (β = –0.04, SD = 0.01, t = –3.64) and significantly faster to frequent verbs than to rare verbs (β = –0.02, SD = 0.01, t = –2.96). Our predictor of interest indicates that listeners respond more slowly to spliced stimuli (β = 0.03, SD = 0.01, t = 3.54); when the values are transformed back, this gives 1.02 s in the original condition vs 1.1 s in the spliced condition, i.e., 80 ms slower. This finding supports our hypothesis. One possibility for why listeners responded more slowly to spliced stimuli is that they are on average 36 ms longer than the original stimuli (β = 0.028, SD = 0.008, t = 3.37). However, the effect of duration in our experiment shows that listeners do not have longer response latencies for the longer stimuli (β = –0.22, SD = 0.04, t = –5.08). Thus, the effect of longer response latencies for spliced stimuli is unlikely due to stimulus length but emerges independently of this potential confound.

Though the speech signal is inherently variable, variation is not unsystematic. Quite the contrary. Anticipatory coarticulation and systematic changes in the temporal structure of the speech signal in relation to word length [Magen (1997) and Öhman (1966), among many others] produce informative cues that listeners make use of in language comprehension [Ali et al. (1971), Denes (1955), and Whalen (1984), among many others].

Many phonetic studies have reported that acoustic detail also depends on lexical sources such as morphological information (e.g., Cho, 2001; Lee-Kim et al., 2012; Plag et al., 2017). Here, we investigate the degree to which coarticulatory phonetic detail in verbal stems associated with inflectional suffixes predicts inflectional functions of American English verbs. To do so, listeners performed two forced-choice classification tasks.

In experiment 1, listeners were presented with bare stems from which inflectional suffixes were removed. We found that the coarticulatory phonetic detail in the verbal stem was not sufficient to classify the inflectional function. However, we found that response latencies to bare stem responses were faster when the stimulus was a real bare stem than when the stimulus was a stem from which the suffix was removed. In experiment 2, listeners classified spliced stimuli according to their inflectional function. We found that when listeners were presented with inconsistent coarticulatory phonetic detail, their response latencies were inhibited.

Our results confirm the findings by Ernestus and Baayen (2006) and Kemps et al. (2005), who have shown that systematic variation in acoustic detail provides informative cues that listeners use to discriminate a word's morphological and inflectional function. Taken together, this indicates an important role of lexical information during speech perception and comprehension. A perspective that is often left out of studies investigating the interaction between morphology and phonetics is the role of the prosodic domain and how it influences production (Cho, 2004). While the present study describes a set of perception experiments, the prosodic domain—in our case, the morphological boundary—may influence the coarticulatory information in our stimuli. Since the coarticulatory effects we are investigating cross a morphological boundary, we might expect that the coarticulatory effects are reduced in our stimuli. However, as we find inhibition in the response latencies when responding to items where the coarticulatory information is mismatched, our findings indicate that the coarticulatory effects are still available to the listeners across this domain.

Typically, shorter response latencies for mismatching coarticulatory detail have been explained as resulting from longer phone selection times due to the need to resolve conflicting acoustic information (e.g., Whalen, 1984). Applying this explanation to the mismatching stimuli in the present study, this means that the coarticulatory detail in the stem vowel indicates that no suffix needs to be selected. The occurrence of the suffix contradicts this choice, which takes time to resolve. While this resolution likely continues to play a role, it is also possible to explain the changes in response latencies from a discrimination perspective (e.g., Ramscar et al., 2010) and the informativity of cues about an outcome. The degree of informativity of acoustic cues about semantic outcomes depends on two factors: first, how often a cue and a semantic outcome co-occur; second, how often the cue co-occurs with other outcomes. Stronger association is mirrored by faster processing, as has been shown in multiple experiments using a computational implementation of discriminative learning to predict response latencies during speech comprehension [e.g., Baayen et al. (2011), but see also Walsh et al. (2010) for a computational implementation from the perspective of exemplar theory].

Since language users learn which cues are and which are not informative about a semantic outcome, the characteristics of cues and their location are not constrained. For example, the voicing of fricatives can be identified on the basis of the duration of preconsonantal vowels in the absence of true voicing (Denes, 1955). Listeners “simply” learn to use vowel duration as a cue for fricative voicing. This means that functions of cue characteristics such as vowel length or pitch emerge during learning.

How is this approach applied to the present experiment? First, listeners learn that the coarticulatory phonetic detail in the verb's stem predicts the upcoming suffix as well as the inflectional function of the verb. Second, they learn that the suffix also predicts and discriminates the verb's inflectional function. When listeners in the present experiment were presented with a stimulus in which coarticulatory phonetic detail matched the suffix, the coarticulatory phonetic detail and the suffix discriminated the same inflectional function. As a result, listeners exhibited faster responses. When listeners were presented with a non-matching acoustic signal, the coarticulatory phonetic signal predicted the non-occurrence of an inflectional suffix and the associated inflectional function. The experienced outcome, i.e., the perceiving of the spliced suffix, mismatched the prediction and caused an error. As a result, their response latencies slowed down.

In conclusion, acoustic detail not only discriminates phonetic outcomes but also semantic outcomes. This acoustic detail may be distributed throughout the signal, and language users learn which acoustic information is informative and which is uninformative about specific outcomes.

This research was funded by a collaborative grant from the Deutsche Forschungsgemeinschaft (BA 3080/3-1, BA 3080/3-2). We thank two anonymous reviewers for their helpful comments on preceding versions of this paper, and also Ryan Callihan for his help with the experiments.