Articulatory characteristics of preboundary lengthening in interaction with prominence on tri-syllabic words in American English

Preboundary lengthening is one of the important phonetic hallmarks for prosodic structure, and its effect is often taken to have a delimitative function of prosodic grouping by virtue of modulating the temporal realization of a word before a boundary (e.g., Turk and Shattuck-Hufnagel, 2007; Krivokapić, 2007; Katsika, 2016; see Fletcher, 2010, and Cho, 2016 for a review and Mitterer et al., 2016 for its perceptual role).

In an effort to further understand the nature of preboundary lengthening (henceforth PBL) in American English (AE), Turk and Shattuck-Hufnagel (2007) examined how PBL may be conditioned by the location of stress in polysyllabic words. Their acoustic study revealed that while PBL is robust in the final syllable regardless of stress, it may also be attracted to a non-final stressed syllable in a tri-syllabic word. They therefore hypothesized that there are “multiple targets” of PBL, i.e., the rime of the final syllable and the rime of the non-final stressed syllable. This study implies that PBL should be integrated into the phonetics-prosody interface of the language in such a way that one element of prosodic structuring, the boundary marking system, operates by making reference to another element of prosodic structuring, the prominence marking system of the language. While Turk and Shattuck-Hufnagel (2007) has thus provided new insights into the phonetics-prosody interface in connection with PBL, it has also left some important questions unanswered as laid out below.

First, in Turk and Shattuck-Hufnagel (2007), the lexical stress effect on preboundary lengthening (PBL) was examined mainly on words that received a (pre-)nuclear pitch accent (except for the data produced by one speaker), leaving a question open as to how stress constrains PBL across speakers in the presence vs absence of higher-order (phrase-level) accent (cf. Fletcher, 2010; Cho, 2016). Second, their study explored PBL only in the acoustic temporal dimension, so that a question remains as to what the articulatory-kinematic underpinnings are that may underlie the purported multiple targets for PBL. Note that some previous studies (e.g., Byrd et al., 2006; Krivokapić, 2007; Byrd and Riggs, 2008; Edwards et al., 1991) examined PBL in AE in the articulatory dimension, but their results were discussed without considering the interaction between stress and accent. The purpose of the present study is therefore to explore these unanswered questions by investigating articulatory-kinematic characteristics of PBL in a single set of tri-syllabic pseudo-words in AE with a view to understanding (1) how the temporal distribution of PBL over a tri-syllabic word is conditioned by the prominence (i.e., lexical stress and phrase-level accent) system, and (2) what the articulatory-kinematic underpinnings are that may underlie PBL in interaction with prominence.

The present study will also have implications for theories of speech production. In addition to the aforementioned multiple target hypothesis (Turk and Shattuck-Hufnagel, 2007), another well-developed model of speech production for boundary-related articulation is the pi-gesture model (Byrd and Saltzman, 2003). The tenet of the model is that kinematic realization of articulatory gestures in the vicinity of prosodic boundary is regulated by a modulation gesture of a dynamical system, the so-called pi-gesture, which governs the temporal realization of gestures under its governance, i.e., the more proximal the articulatory gesture to the boundary, the longer the gesture. An important theoretical question then is to what extent kinematic characteristics of preboundary lengthening that is assumed to interact with the prominence system may be accounted for in terms of the multiple target hypothesis as well as the pi-gesture theory. Results of the present study will be discussed in connection with this question.

Ten native speakers (5 female, 5 male) of AE in their 20s and 30s were paid for participation. They were from different parts of the United States, and were visiting Seoul, Korea at the time of recording.

Three tri-syllabic non-words were used with different stress patterns—bábaba, babába, and bababá (Table 1, left). The consonant was controlled to be [b], and the vowel was [ɑ] when stressed, and [ə] (or reduced to our best auditory judgement) when unstressed. The target word was placed in a carrier sentence in a mini discourse context (Table 1, right). It was located before a tag question for an Intonational Phrase (IP) boundary rendition and in the middle of the phrase for an IP-medial Word (Wd) boundary. When accented, the target word was focused (e.g., bábaba in contrast with mámama), receiving a nuclear pitch accent; when it was unaccented, the focus fell on the preceding title word (MR. Bababa vs MRS. Bababa).

Participants had a 30–60 min training session on a day before the experiment. The target words were introduced as last names (e.g., Mr. Bababa). Different stress patterns were visualized on a card along with a list of example words (Table 1, left) in order to aid speakers to practice intended stress patterns.

Articulatory data were collected using an Electromagnetic Midsagittal Articulography (Carstens AG200). Sensors were attached on different articulators including the upper and the lower lips and the data were processed with standard processing procedures (see Cho et al., 2016). (Note that two additional sensors were attached on the tongue blade and the tongue dorsum, but the data from these sensors have not been analyzed yet.) During the experiment, participants were provided with experimental sentences on a computer screen. They heard a pre-recorded prompt sentence of speaker A through a loudspeaker, and read the answer to the prompt question as speaker B (see Table 1). Test sentences were presented in a randomized order in four repetition blocks. In total, 480 sentences were collected [i.e., 3 target words × 2 boundaries (IP, Wd) × 2 accent types (accented, unaccented) × 4 repetitions × 10 speakers]. The prosodic renditions of the obtained sentences were further examined by the authors (trained prosodic transcribers), and 13 tokens were discarded due to unintended prosodic boundary or accent patterns.

The lip opening and closing movement profiles were obtained from the Euclidean distance of the sensors on the upper and lower lips, i.e., Lip Aperture (Byrd, 2000; Cho et al., 2016). Kinematic measures are summarized in Fig. 1. Two main temporal measures for preboundary lengthening (PBL) that were taken for each syllable were lip closing duration (①) which included both the lip closing movement duration and the closure duration up to the point of C release; and C release/lip opening duration (⑤) which included both the release movement duration and the lip opening plateau up to the onset of the following (lip closing) movement (which may potentially include any acoustic pause, if it exists, in the IP-final position). Time-to-peak velocity (acceleration duration) was also taken for both the lip closing and opening movement (②, ⑥) as an index of temporal variation related to a change in stiffness under the influence of the pi-gesture. (Note that while a change in stiffness may in principle influence both the acceleration and the deceleration durations, it is the former measure that is not influenced by timing of the following gesture; see Byrd and Saltzman, 2003). Peak velocity (③, ⑦) and displacement (④, ⑧) for both the lip closing and opening movements were measured in order to understand the overall kinematic characteristics that may underlie PBL. The lip closing/opening onset and target were defined as a point in time at which the velocity was 20% of its peak after or before the zero-crossing point, respectively. (As a reviewer pointed out, the 20% threshold point in time may vary depending on the actual peak velocity of a given movement. But our experience with kinematic data indicates that such variation is negligible, which is presumably why a great deal of previous studies have also employed a consistent threshold across different conditions as we do in the present study.)

The effect of Boundary on the kinematic measures was evaluated in Repeated Measures Analysis of Variance (ANOVAs) with Stress and Accent, using IBM SPSS version 21.0. Given the research questions of the present study, we will report just on main effects of Boundary and any interactions that involved Boundary. Each speaker's data were averaged across repetitions. Paired t-tests were carried out to examine PBL in each condition and observed interactions. The relationship between displacement and peak velocity was further evaluated by Analysis of Covariance (ANCOVA) with peak velocity (PK VEL) as a covariate and speaker as a random factor.

Results of three-way ANOVAs revealed a main effect of PBL on the rightmost measures: C3-rel/opening duration and C3 time-to-pkvel were significantly longer IP-finally than Wd-finally (see Fig. 2). The PBL effects, however, interacted with Stress and Accent, revealing prominence-dependent PBL. Most notably, Boundary interacted with Stress on both C3-rel/opening duration and C3 time-to-pkvel (F[2,18] = 25.9, p < 0.005; F[2,18] = 4.44, p < 0.05, respectively) with a weaker PBL effect when the final syllable was stressed [S3, bababá, t(9) = 2.92, p < 0.05, Δ = 36 ms, η_p²= 0.49; t(9) = 1.89, p > 0.1, respectively] than unstressed [S1, bábaba, t(9) = 4.36, p < 0.01, Δ = 62 ms, η_p²= 0.68; t(9) = 3.44, p < 0.01, Δ = 10 ms, η_p² = 0.61, respectively; S2, babába, t(9) = 5.29, p < 0.01, Δ = 69 ms, η_p²= 0.76; t(9) = 3.75, p < 0.01, Δ = 14 ms, η_p²= 0.57, respectively]. Boundary also interacted with Accent on both measures (F[1,9] = 6.37, p < 0.05; F[1,9] = 23.02, p < 0.005, respectively) with the PBL effect being weaker when the word was accented [t(9) = 3.96, p < 0.01, Δ = 50 ms, η_p²= 0.64] than unaccented [t(9) = 4.53, p < 0.01, Δ = 61 ms, η_p²= 0.70]. Paired t-tests for each prominence condition (Table 2) further revealed an extreme case of prominence-dependent PBL, showing a null effect when the final syllable was both stressed and accented, i.e., in the most prominent condition.

As for the second to the last measures, both C3-closing duration and C3-closing time-to-pkvel showed no main effect of boundary, but a significant Boundary × Stress interaction (F[2,18] = 7.41, p < 0.01; F[2,18] = 6.28, p < 0.05, respectively). As can be seen in Table 2, the interaction was primarily due to significant PBL effects on the C3-closing measures when immediately preceded by a stressed syllable (S2, babába), showing an attraction effect of PBL to a non-final stressed syllable. The interaction may also have stemmed in part from the fact that C3-closing duration showed a boundary-related shortening effect in the most prominent (stressed and accented) condition as shown in a cell in gray Table 2. Paired-t tests also indicated that there was a similar shortening effect on C2-closing duration when the second syllable was stressed and accented, but there was neither a main effect of stress nor a relevant interaction that involved stress.

The attraction of PBL to a non-final stressed syllable was further evident on C2-rel/opening duration. There was a significant three-way interaction, showing a PBL effect in S2 (babába) but again in the less prominent, unaccented, condition, while the interaction was also in part due to an opposite direction of PBL (i.e., shortening) in S1 (bábaba), as seen in Table 2.

As for the leftmost measures, there was a main effect of boundary on C1-closing duration, but as shown in Fig. 2, a small but significant distal preboundary shortening effect was observed. There was a significant Boundary × Accent interaction (F[1,9] = 8.09, p < 0.05), showing that the shortening effect was confined to initial syllables in the accented condition regardless of stress [accented, t(9) = –2.85, p < 0.05, Δ = –6 ms, η_p²= 0.47; unaccented, t(9) = –1, p > 0.1].

For C3-rel/opening, a significantly faster (PKVEL) and larger (displacement) movement (F[1,9] = 46.91; F[1,9] = 31.99, respectively, both p < 0.005) was observed before an IP than a Wd boundary [Fig. 3(a)]. PKVEL and displacement both showed a three-way interaction (F[2,18] = 6.86, p < 0.01; F[2,18] = 14.33, p < 0.005, respectively) which was again attributable to a prominence-dependent boundary effect, i.e., just like the preboundary lengthening (PBL) effect, C3-rel/opening was faster and larger in all but the stressed/accented (the most prominent) condition. The boundary-related strengthening effect (a longer, faster and larger movement) was further evident in the kinematic relationship between peak velocity and displacement as shown in Fig. 3(b): IP-final tokens were clustered orthogonally towards the upper-right plane in the plots, and Wd-final tokens towards the lower-left plane, indicating a spatial strengthening effect accompanied by a faster movement. The nature of the spatial strengthening effect was confirmed by results of ANCOVAs, i.e., the significant boundary effects on displacement of C3-rel/opening as shown in Fig. 3(a) remained significant in ANCOVAs in which PKVEL as a covariate was factored in (S1, acc: F[1,8.9] < 1, p > 0.1; S1, unacc: F[1,10.6] = 9.6, p < 0.05; S2, acc: F[1,23.8] = 37.67, p < 0.005; S2, unacc: F[1,14.3] = 33.3, p < 0.005; S3, acc: F[1,13.1] = 20.95, p < 0.005; S3, unacc: F[1,14.4] = 11.68, p < 0.005).

Results showed that preboundary lengthening (PBL) is largely localized to the final syllable (especially to C3-rel/opening duration and its time-to-pkvel) of a tri-syllabic word. While this finding is in line with the literature (e.g., Edwards et al., 1991; Krivokapić, 2007; Byrd and Riggs, 2008; Katsika, 2016), the study newly revealed how PBL in AE may be modulated by the degree of prominence. In particular, the PBL effect on C3-rel/opening and time-to-pkvel was most robust in the least prominent (unstressed/unaccented) condition, weaker in the moderately prominent (stressed/unaccented) condition, and non-observable in the most prominent (stressed/accented) condition. This indicates that the PBL of even the rightmost gesture interacts with hyperarticulation due to prominence, leaving no room for a further temporal extension due to boundary, which is interpretable as a ceiling effect.

Results also illuminated the extent to which preboundary lengthening (PBL) may be attracted to a non-final stressed syllable in connection with the multiple target hypothesis (Turk and Shattuck-Hufnagel, 2007). There was indeed some evidence that PBL was attracted to a stressed second syllable (as reflected in C2-rel/opening) consistent with the hypothesis, but as was the case with the stressed final syllable, the effect was modulated by prominence, i.e., significant only in the unaccented condition. On the other hand, C3-closing duration (in the final syllable) showed PBL only when immediately preceded by a stressed syllable. This may be better interpreted as a kind of gradual attraction to stress consistent with Katsika's (2016) view on the scope of preboundary lengthening in Greek, i.e., non-final lexical stress initiates PBL earlier presumably due to a kind of coupling between a mu-gesture (that modulates stress-related articulation) and a pi-gesture (that modulates boundary-related articulation). When the stress was in the initial syllable in the present study, however, there was no evidence of such a gradual attraction effect nor was there the multiple target effect (i.e., final lengthening of the stressed initial syllable), calling for more elaborative models of final lengthening.

There was instead a preboundary shortening of C1-closing gesture which was farthest away from the boundary, but only in the accented condition regardless of stress. This shortening effect may be attributable to the global speech planning process (cf. Krivokapić, 2007, 2014), i.e., the initiation of the articulatory gesture at the left edge of the word may be speeded up in anticipation (or compensation) for the extensive final lengthening at the right edge, regulating the global speech timing of the preboundary word (see Byrd et al., 2006, for related discussion especially on compensatory shortening on postboundary gestures). But it is also equally plausible that preboundary shortening in non-final syllables may arise as a consequence of an intricate mutual interaction between stress and boundary as proposed by Katsika (2016). The exact mechanism that may also differ across languages (e.g., English vs Greek) remains to be further elucidated.

Turning back to the discrepancy between the findings of the present study and of Turk and Shattuck-Hufnagel (2007), we do not have any corroborating explanation to offer, but as noted by Katsika (2016), previous studies have indeed shown inconsistent effects of distal stress on preboundary lengthening (PBL) in English (e.g., Byrd and Riggs, 2008). Furthermore, Cho et al. (2013) reported a case in which even an unstressed initial syllable of a tri-syllabic word may be lengthened due to the distal boundary, adding complexity to understanding the scope of PBL in English. The mixed results may be due to different speech rates employed by different studies or the difference between articulatory and acoustic data. The mixed results may also be due to speaker variation underlying the distal stress effect (Byrd and Riggs, 2008; Katsika, 2016), although 9 out of 10 speakers in the present study showed the pattern of consistent shortening. More data within and across languages will certainly be needed in order to understand the exact nature of the role of stress in determining the scope of PBL.

Another aspect of preboundary lengthening (PBL) newly revealed in the present study comes from its kinematic underpinnings. PBL was found to go hand in hand with a faster and larger articulatory movement of the rightmost C3-rel/opening gesture, showing a kind of articulatory strengthening attributable primarily to a spatial expansion (in displacement) accompanied by an increase in movement velocity being correlated with the spatial change. This suggests that while the prominence- vs the boundary-induced articulatory phenomena may differ in their detailed kinematic patterns and dynamic underpinnings, the spatio-temporal expansion that arise both phrase-finally and under prominence may be characterized in broad terms under the rubric of prosodic strengthening (e.g., Cho, 2016). This has further implications for developing theories of PBL from a dynamical point of view. The temporal variation underlying PBL is consistent with the assumption of the pi-gesture theory (e.g., Byrd and Saltzman, 2003), to the extent that PBL was evident on the final C3-rel/opening movement in all but the most prominent condition, especially in the domain of time-to-peak velocity. It is also important to note that some aspects of spatial expansion that arises at the prosodic juncture may be accounted for by a lesser intergestural overlap (i.e., due to a delayed following gesture) at a larger prosodic juncture consistent with the model (Byrd and Saltzman, 2003). It remains to be seen how the model devises a way of capturing a prominence-induced ceiling effect of PBL, and a boundary-related articulatory strengthening pattern that is seemingly similar to prominence-related strengthening in terms of spatio-temporal expansion, but different in their detailed kinematic relationships and dynamical underpinnings. Given that both temporal and spatial modulation gestures are available in a dynamical framework (Saltzman et al., 2008), the concept of the pi-gesture needs to be refined to encompass the types of spatio-temporal modulation observed in the present study by making reference to the prominence system of a given language (cf. Cho, 2016; Cho et al., 2017; Katsika, 2016).

In conclusion, the present study, though with a limited set of pseudo-words, provided some new insights into the temporal organization of preboundary words in AE, showing a combinatorial effect of a prominence-dependent preboundary lengthening (PBL) proximal to a boundary and a distal preboundary shortening possibly in compensation for the upcoming lengthening, both of which may ensue as a consequence of a speech planning process of prosodic structuring. The observed kinematic underpinnings of preboundary lengthening have further implications for how PBL may be captured from a dynamical point of view. Although it remains to be seen how variation observed within a language can be adequately understood, the proposed modulations could be parameterized in developing theories of PBL within and across different languages. It is hoped that the findings of the present study, which is limited to a specific data set, will further inspire future work on developing theories of speech production that model boundary-related speech phenomena within and across languages.

We thank the American English speakers who participated in our EMA experiment. This study has greatly benefited from constructive comments from Dani Byrd and two anonymous reviewers. This work was supported by the research fund of Hanyang University (HY-2017).