While recent research suggests that visual biofeedback can facilitate speech production training in clinical populations and second language (L2) learners, individual learners' responsiveness to biofeedback is highly variable. This study investigated the hypothesis that the type of biofeedback provided, visual-acoustic versus ultrasound, could interact with individuals' acuity in auditory and somatosensory domains. Specifically, it was hypothesized that learners with lower acuity in a sensory domain would show greater learning in response to biofeedback targeting that domain. Production variability and phonological awareness were also investigated as predictors. Sixty female native speakers of English received 30 min of training, randomly assigned to feature visual-acoustic or ultrasound biofeedback, for each of two Mandarin vowels. On average, participants showed a moderate magnitude of improvement (decrease in Euclidean distance from a native-speaker target) across both vowels and biofeedback conditions. The hypothesis of an interaction between sensory acuity and biofeedback type was not supported, but phonological awareness and production variability were predictive of learning gains, consistent with previous research. Specifically, high phonological awareness and low production variability post-training were associated with better outcomes, although these effects were mediated by vowel target. This line of research could have implications for personalized learning in both L2 pedagogy and clinical practice.
References
1.
Adler-Bock
, M.
, Bernhardt
, B. M.
, Gick
, B.
, and Bacsfalvi
, P.
(2007
). “The use of ultrasound in remediation of North American English /r/ in 2 adolescents
,” Am. J. Speech-Lang. Pathol.
16
(2
), 128
–139
.2.
Akahane-Yamada
, R.
, McDermott
, E.
, Adachi
, T.
, Kawahara
, H.
, and Pruitt
, J. S.
(1998
). “Computer-based second language production training by using spectrographic representation and HMM-based speech recognition scores
,” paper presented at the Fifth International Conference on Spoken Language Processing
.3.
Bacsfalvi
, P.
, and Bernhardt
, B. M.
(2011
). “Long-term outcomes of speech therapy for seven adolescents with visual feedback technologies: Ultrasound and electropalatography
,” Clin. Linguist. Phonet.
25
(11-12
), 1034
–1043
.4.
Bernhardt
, B.
, Gick
, B.
, Bacsfalvi
, P.
, and Adler-Bock
, M.
(2005
). “Ultrasound in speech therapy with adolescents and adults
,” Clin. Linguist. Phonet.
19
(6-7
), 605
–617
.5.
Best
, C. T.
(1995
). “A direct realist view of cross-language speech perception
,” in Speech Perception and Linguistic Experience: Issues in Cross-Language Research
, edited by W.
Strange
(York Press
, Baltimore, MD
), pp. 171
–204
.6.
Best
, C. T.
, Halle
, P.
, Bohn
, O.-S.
, and Faber
, A.
(2003
). “Cross-language perception of nonnative vowels: Phonological and phonetic effects of listeners' native languages
,” paper presented at the Proceedings of the 15th International Congress of Phonetic Sciences
.7.
Bliss
, H.
, Abel
, J.
, and Gick
, B.
(2018
). “Computer-assisted visual articulation feedback in L2 pronunciation instruction
,” J. Second Lang. Pronunciation
4
(1
), 129
–153
.8.
Boersma
, P.
, and Weenink
, D.
(2016
). “Praat: Doing phonetics by computer (version 6.0.18) [computer program]
,” http://www.praat.org (Last viewed 5 April 2019).9.
Borrie
, S. A.
, and Schäfer
, M. C.
(2015
). “The role of somatosensory information in speech perception: Imitation improves recognition of disordered speech
,” J. Speech Lang. Hear. Res.
58
(6
), 1708
–1716
.10.
Bradlow
, A. R.
, Pisoni
, D. B.
, Akahane-Yamada
, R.
, and Tohkura
, Y. I.
(1997
). “Training Japanese listeners to identify English /r/ and /l/: IV. Some effects of perceptual learning on speech production
,” J. Acoust. Soc. Am.
101
(4
), 2299
–2310
.11.
Carey
, M.
(2004
). “CALL visual feedback for pronunciation of vowels: Kay Sona-Match
,” CALICO J.
21
, 571
–601
.12.
Catford
, J. C.
, and Pisoni
, D.
(1970
). “Auditory versus articulatory training in exotic sounds
,” Mod. Lang. J.
54
, 477
–481
.13.
Chang
, C. B.
, Yao
, Y.
, Haynes
, E. F.
, and Rhodes
, R.
(2011
). “Production of phonetic and phonological contrast by heritage speakers of Mandarin
,” J. Acoust. Soc. Am.
129
(6
), 3964
–3980
.14.
Chen
, Y.
, Robb
, M.
, Gilbert
, H.
, and Lerman
, J.
(2001
). “Vowel production by Mandarin speakers of English
,” Clin. Linguist. Phonet.
15
(6
), 427
–440
.15.
Christiner
, M.
, and Reiterer
, S. M.
(2015
). “A Mozart is not a Pavarotti: Singers outperform instrumentalists on foreign accent imitation
,” Front. Hum. Neurosci.
9
, 482
.16.
Dörnyei
, Z.
(1998
). “Motivation in second and foreign language learning
,” Lang. Teach.
31
(3
), 117
–135
.17.
Dowd
, A.
, Smith
, J.
, and Wolfe
, J.
(1998
). “Learning to pronounce vowel sounds in a foreign language using acoustic measurements of the vocal tract as feedback in real time
,” Lang. Speech
41
(1
), 1
–20
.18.
Engwall
, O.
(2006
). “Feedback strategies of human and virtual tutors in pronunciation training
,” Speech Music and Hearing—Quarterly Progress and Status Report
Vol. 48
(1
), 11
–34
.19.
Fitch
, W. T.
(1997
). “Vocal tract length and formant frequency dispersion correlate with body size in rhesus macaques
,” J. Acoust. Soc. Am.
102
(2
), 1213
–1222
.20.
Fitch
, W. T.
, and Giedd
, J.
(1999
). “Morphology and development of the human vocal tract: A study using magnetic resonance imaging
,” J. Acoust. Soc. Am.
106
(3
), 1511
–1522
.21.
Flege
, J. E.
(1987
). “The production of ‘new’ and ‘similar’ phones in a foreign language: Evidence for the effect of equivalence classification
,” J. Phonetics
15
(1
), 47
–65
.22.
Flege
, J. E.
(1995
). “Second language speech learning: Theory, findings, and problems
,” in Speech Perception and Linguistic Experience
, edited by W.
Strange
(York Press
, Timonium, MD
), pp. 233
–277
.23.
Franken
, M. K.
, Acheson
, D. J.
, McQueen
, J. M.
, Eisner
, F.
, and Hagoort
, P.
(2017
). “Individual variability as a window on production-perception interactions in speech motor control
,” J. Acoust. Soc. Am.
142
(4
), 2007
–2018
.24.
Fucci
, D.
(1972
). “Oral vibrotactile sensation: An evaluation of normal and defective speakers
,” J. Speech Lang. Hear. Res.
15
(1
), 179
–184
.25.
Fucci
, D. J.
, and Robertson
, J. H.
(1971
). “ ‘Functional’ defective articulation: An oral sensory disturbance
,” Percept. Mot. Skills
33
(3
), 711
–714
.26.
Ghosh
, S. S.
, Matthies
, M. L.
, Maas
, E.
, Hanson
, A.
, Tiede
, M.
, Menard
, L.
, Guenther
, F. H.
, Lane
, H.
, and Perkell
, J. S.
(2010
). “An investigation of the relation between sibilant production and somatosensory and auditory acuity
,” J. Acoust. Soc. Am.
128
(5
), 3079
–3087
.27.
Gick
, B.
, Bernhardt
, B.
, Bacsfalvi
, P.
, Wilson
, I.
, and Zampini
, M.
(2008
). “Ultrasound imaging applications in second language acquisition
,” Phonol. Second Lang. Acquis.
36
, 315
–328
.28.
Gilbert
, J. B.
(2010
). “Pronunciation as orphan: What can be done?
,” Speak Out!
43
, 3
–7
.29.
Guenther
, F. H.
(1994
). “A neural network model of speech acquisition and motor equivalent speech production
,” Biol. Cybernet.
72
(1
), 43
–53
.30.
Guenther
, F. H.
(1995
). “Speech sound acquisition, coarticulation, and rate effects in a neural network model of speech production
,” Psychol. Rev.
102
(3
), 594
.31.
Guenther
, F. H.
(2016
). Neural Control of Speech
(MIT Press
, Cambridge, MA
).32.
Guenther
, F. H.
, Ghosh
, S. S.
, and Tourville
, J. A.
(2006
). “Neural modeling and imaging of the cortical interactions underlying syllable production
,” Brain Lang.
96
(3
), 280
–301
.33.
Guenther
, F. H.
, Hampson
, M.
, and Johnson
, D.
(1998
). “A theoretical investigation of reference frames for the planning of speech movements
,” Psychol. Rev.
105
(4
), 611
.34.
Halwani
, G. F.
, Loui
, P.
, Rueber
, T.
, and Schlaug
, G.
(2011
). “Effects of practice and experience on the arcuate fasciculus: Comparing singers, instrumentalists, and non-musicians
,” Front. Psychol.
2
, 156
.35.
Hanulíková
, A.
, Dediu
, D.
, Fang
, Z.
, Bašnaková
, J.
, and Huettig
, F.
(2012
). “Individual differences in the acquisition of a complex L2 phonology: A training study
,” Lang. Learn.
62
, 79
–109
.36.
Hardcastle
, W. J.
, Gibbon
, F. E.
, and Jones
, W.
(1991
). “Visual display of tongue-palate contact: Electropalatography in the assessment and remediation of speech disorders
,” Br. J. Disord. Commun.
26
(1
), 41
–74
.37.
Hattori
, K.
, and Iverson
, P.
(2010
). “Examination of the relationship between L2 perception and production: An investigation of English /r/-/l/ perception and production by adult Japanese speakers
,” paper presented at the Second Language Studies: Acquisition, Learning, Education and Technology
.38.
Hickok
, G.
(2012
). “Computational neuroanatomy of speech production
,” Nat. Rev. Neurosci.
13
(2
), 135
.39.
Hitchcock
, E. R.
, McAllister Byun
, T.
, Swartz
, M.
, and Lazarus
, R.
(2017
). “Efficacy of electropalatography for treating misarticulation of /r/
,” Am. J. Speech-Lang. Pathol.
26
(4
), 1141
–1158
.40.
Hu
, C. F.
(2003
). “Phonological memory, phonological awareness, and foreign language word learning
,” Lang. Learn.
53
(3
), 429
–462
.41.
Hu
, C. F.
(2010
). “Phonological bases for L2 morphological learning
,” J. Psycholinguist. Res.
39
(4
), 305
–322
.42.
Hu
, X.
, Ackermann
, H.
, Martin
, J. A.
, Erb
, M.
, Winkler
, S.
, and Reiterer
, S. M.
(2013
). “Language aptitude for pronunciation in advanced second language (L2) learners: Behavioural predictors and neural substrates
,” Brain Lang.
127
(3
), 366
–376
.43.
Hummel
, K. M.
(2009
). “Aptitude, phonological memory, and second language proficiency in nonnovice adult learner
,” Appl. Psycholinguist.
30
(2
), 225
–249
.44.
Jacobs
, R.
, Serhal
, C. B.
, and van Steenberghe
, D.
(1998
). “Oral stereognosis: A review of the literature
,” Clin. Oral Investig.
2
(1
), 3
–10
.45.
Kartushina
, N.
, and Frauenfelder
, U. H.
(2014
). “On the effects of L2 perception and of individual differences in L1 production on L2 pronunciation
,” Front. Psychol.
5
, 1246
.46.
Kartushina
, N.
, Hervais-Adelman
, A.
, Frauenfelder
, U. H.
, and Golestani
, N.
(2015
). “The effect of phonetic production training with visual feedback on the perception and production of foreign speech sounds
,” J. Acoust. Soc. Am.
138
(2
), 817
–832
.47.
Kartushina
, N.
, Hervais-Adelman
, A.
, Frauenfelder
, U. H.
, and Golestani
, N.
(2016
). “Mutual influences between native and non-native vowels in production: Evidence from short-term visual articulatory feedback training
,” J. Phonetics
57
, 21
–39
.48.
Kawahara
, H.
, Morise
, M.
, Banno
, H.
, and Skuk
, V. G.
(2013
). “Temporally variable multi-aspect N-way morphing based on interference-free speech representations
,” paper presented at the Signal and Information Processing Association Annual Summit and Conference (APSIPA)
, Asia-Pacific.49.
Kendall
, T.
, Thomas
, E. R.
, and Kendall
, M. T.
(2018
). “vowels: Vowel manipulation, normalization, and plotting [R package]
,” available at https://cran.r-project.org/web/packages/vowels/ (Last viewed 5 April 2019).50.
Kleber
, B. A.
, and Zarate
, J. M.
(2014
). “The neuroscience of singing
,” in The Oxford Handbook of Singing
(Oxford University Press
, Oxford, UK
).51.
Kocjančičo Antolík
, T. K.
, Pillot-Loiseau
, C.
, and Kamiyama
, T.
(2019
). “The effectiveness of real-time ultrasound visual feedback on tongue movements in L2 pronunciation training
,” J. Second Lang. Pronunciation
5
(1
), 72
–97
.52.
Lakens
, D.
(2017
). “Equivalence tests: A practical primer for t tests, correlations, and meta-analyses
,” Soc. Psychol. Personal. Sci.
8
(4
), 355
–362
.53.
Lametti
, D. R.
, Nasir
, S. M.
, and Ostry
, D. J.
(2012
). “Sensory preference in speech production revealed by simultaneous alteration of auditory and somatosensory feedback
,” J. Neurosci.
32
(27
), 9351
–9358
.54.
Lennes
, M.
(2003
). “Collect_formant_data_from_files, Praat [Praat script]
,” available at https://github.com/FieldDB/Praat-Scripts/blob/master/collect_formant_data_from_files.praat (Last viewed 5 April 2019).55.
Levy
, E. S.
, and Strange
, W.
(2008
). “Perception of French vowels by American English adults with and without French language experience
,” J. Phonetics
36
(1
), 141
–157
.56.
McAllister Byun
, T.
(2017
). “Efficacy of visual–acoustic biofeedback intervention for residual rhotic errors: A single-subject randomization study
,” J. Speech Lang. Hear. Res.
60
(5
), 1175
–1193
.57.
McAllister Byun
, T.
, and Campbell
, H.
(2016
). “Differential effects of visual-acoustic biofeedback intervention for residual speech errors
,” Front. Hum. Neurosci.
10
, 567
.58.
McAllister Byun
, T.
, and Hitchcock
, E. R.
(2012
). “Investigating the use of traditional and spectral biofeedback approaches to intervention for /r/ misarticulation
,” Am. J. Speech-Lang. Pathol.
21
(3
), 207
–221
.59.
McAllister Byun
, T.
, Hitchcock
, E. R.
, and Swartz
, M. T.
(2014
). “Retroflex versus bunched in treatment for rhotic misarticulation: Evidence from ultrasound biofeedback intervention
,” J. Speech Lang. Hear. Res.
57
(6
), 2116
–2130
.60.
McBride-Chang
, C.
(1996
). “Models of speech perception and phonological processing in reading
,” Child Dev.
67
(4
), 1836
–1856
.61.
McNutt
, J. C.
(1977
). “Oral sensory and motor behaviors of children with /s/ or /r/ misarticulations
,” J. Speech Lang. Hear. Res.
20
(4
), 694
–703
.62.
Milovanov
, R.
, Pietilä
, P.
, Tervaniemi
, M.
, and Esquef
, P. A.
(2010
). “Foreign language pronunciation skills and musical aptitude: A study of Finnish adults with higher education
,” Learn. Individ. Differ.
20
(1
), 56
–60
.63.
Mora
, J. C.
, Rochdi
, Y.
, and Kivistö-de Souza
, H.
(2014
). “Mimicking accented speech as L2 phonological awareness
,” Lang. Aware.
23
(1-2
), 57
–75
.64.
Nagle
, C. L.
(2018
). “Examining the temporal structure of the perception–production link in second language acquisition: A longitudinal study
,” Lang. Learn.
68
(1
), 234
–270
.65.
Nasir
, S. M.
, and Ostry
, D. J.
(2008
). “Speech motor learning in profoundly deaf adults
,” Nat. Neurosci.
11
(10
), 1217
–1222
.66.
Newman
, R. S.
(2003
). “Using links between speech perception and speech production to evaluate different acoustic metrics: A preliminary report
,” J. Acoust. Soc. Am.
113
(5
), 2850
–2860
.67.
Niziolek
, C. A.
, Nagarajan
, S. S.
, and Houde
, J. F.
(2013
). “What does motor efference copy represent? Evidence from speech production
,” J. Neurosci.
33
(41
), 16110
–16116
.68.
Nushi Kochaksaraie
, M.
, and Makiabadi
, H.
(2018
). “Second language learners' phonological awareness and perception of foreign accentedness and comprehensibility by native and non-native English speaking EFL teachers
,” J. Teach. Lang. Skills
36
(4
), 103
–140
.69.
Okuno
, T.
, and Hardison
, D. M.
(2016
). “Perception-production link in L2 Japanese vowel duration: Training with technology
,” Lang. Learn. Technol.
20
(2
), 61
–80
.70.
Olson
, D. J.
(2014
). “Benefits of visual feedback on segmental production in the L2 classroom
,” Lang. Learn. Technol.
18
(3
), 173
–192
.71.
Peperkamp
, S.
, and Bouchon
, C.
(2011
). “The relation between perception and production in L2 phonological processing
,” paper presented at the Twelfth Annual Conference of the International Speech Communication Association
.72.
Perkell
, J. S.
, Guenther
, F. H.
, Lane
, H.
, Matthies
, M. L.
, Stockmann
, E.
, Tiede
, M.
, and Zandipour
, M.
(2004a
). “The distinctness of speakers' productions of vowel contrasts is related to their discrimination of the contrasts
,” J. Acoust. Soc. Am.
116
(4
), 2338
–2344
.73.
Perkell
, J. S.
, Matthies
, M. L.
, Tiede
, M.
, Lane
, H.
, Zandipour
, M.
, Marrone
, N.
, Stockmann
, E.
, and Guenther
, F. H.
(2004b
). “The distinctness of speakers' /s/-/∫/ contrast is related to their auditory discrimination and use of an articulatory saturation effect
,” J. Speech Lang. Hear. Res.
47
(6
), 1259
–1269
.74.
Perrachione
, T. K.
, Lee
, J.
, Ha
, L. Y.
, and Wong
, P. C.
(2011
). “Learning a novel phonological contrast depends on interactions between individual differences and training paradigm design
,” J. Acoust. Soc. Am.
130
(1
), 461
–472
.75.
Posedel
, J.
, Emery
, L.
, Souza
, B.
, and Fountain
, C.
(2012
). “Pitch perception, working memory, and second-language phonological production
,” Psychol. Music
40
(4
), 508
–517
.76.
Preston
, J. L.
, and Leaman
, M.
(2014
). “Ultrasound visual feedback for acquired apraxia of speech: A case report
,” Aphasiology
28
(3
), 278
–295
.77.
Preston
, J. L.
, Leece
, M. C.
, and Maas
, E.
(2016
). “Intensive treatment with ultrasound visual feedback for speech sound errors in childhood apraxia
,” Front. Hum. Neurosci.
10
, 440
.78.
Preston
, J. L.
, McAllister
, T.
, Phillips
, E.
, Boyce
, S.
, Tiede
, M.
, Kim
, J. S.
, and Whalen
, D. H.
(2018
). “Treatment for residual rhotic errors with high-and low-frequency ultrasound visual feedback: A single-case experimental design
,” J. Speech Lang. Hear. Res.
61
(8
), 1875
–1892
.79.
Preston
, J. L.
, McAllister
, T.
, Phillips
, E.
, Boyce
, S. E.
, Tiede
, M.
, Kim
, J. S.
, and Whalen
, D. H.
(2019
). “Remediating residual rhotic errors with traditional and ultrasound-enhanced treatment: A single-case experimental study
,” Am. J. Speech-Lang. Pathol.
28
, 1167
–1183
.80.
Preston
, J. L.
, McCabe
, P.
, Rivera-Campos
, A.
, Whittle
, J. L.
, Landry
, E.
, and Maas
, E.
(2014
). “Ultrasound visual feedback treatment and practice variability for residual speech sound errors
,” J. Speech Lang. Hear. Res.
57
(6
), 2102
–2115
.81.
R Core Team
. (2015
). “R: A language and environment for statistical computing [software]
,” (R foundation for Statistical Computing, Vienna, Austria), available at http://www.R-project.org/ (Last viewed 5 April 2019).82.
Reiterer
, S. M.
, Hu
, X.
, Erb
, M.
, Rota
, G.
, Nardo
, D.
, Grodd
, W.
, and Ackermann
, H.
(2011
). “Individual differences in audio-vocal speech imitation aptitude in late bilinguals: Functional neuro-imaging and brain morphology
,” Front. Psychol.
2
, 271
.83.
Rota
, G.
, and Reiterer
, S.
(2009
). “Cognitive aspects of pronunciation talent
,” Lang. Talent Brain Act.
1
, 67
–96
.84.
Schmidt
, R.
(1990
). “The role of consciousness in second language learning
,” Appl. Linguist.
11
(2
), 129
–158
.85.
Schmidt
, R.
(1993
). “Awareness and second language acquisition
,” Annu. Rev. Appl. Linguist.
13
, 206
–226
.86.
Schwartz
, J.-L.
, Basirat
, A.
, Ménard
, L.
, and Sato
, M.
(2012
). “The Perception-for-Action-Control Theory (PACT): A perceptuo-motor theory of speech perception
,” J. Neurolinguist.
25
(5
), 336
–354
.87.
Simmonds
, A. J.
, Wise
, R. J.
, Dhanjal
, N. S.
, and Leech
, R.
(2011
). “A comparison of sensory-motor activity during speech in first and second languages
,” J. Neurophysiol.
106
(1
), 470
–478
.88.
Slevc
, L. R.
, and Miyake
, A.
(2006
). “Individual differences in second-language proficiency: Does musical ability matter?
,” Psychol. Sci.
17
(8
), 675
–681
.89.
Steele
, C. M.
, Hill
, L.
, Stokely
, S.
, and Peladeau-Pigeon
, M.
(2014
). “Age and strength influences on lingual tactile acuity
,” J. Texture Stud.
45
(4
), 317
–323
.90.
Villacorta
, V. M.
, Perkell
, J. S.
, and Guenther
, F. H.
(2007
). “Sensorimotor adaptation to feedback perturbations of vowel acoustics and its relation to perception
,” J. Acoust. Soc. Am.
122
(4
), 2306
–2319
.91.
Wagner
, R. K.
, Torgesen
, J. K.
, Rashotte
, C. A.
, and Pearson
, N. R.
(2013
). Comprehensive Test of Phonological Processing
, Second ed. (Pro-Ed
, Austin, TX
).92.
Watson
, B. U.
, and Miller
, T.
(1993
). “Auditory perception, phonological processing, and reading ability/disability
,” J. Speech Lang. Hear. Res.
36
(4
), 850
–863
.93.
Wickham
, H.
(2009
). “ggplot2: Elegant graphics for data analysis [R package]
,” available at (Last viewed 5 April 2019).94.
Wickham
, H.
(2016
). “tidyr: Easily tidy data with ‘spread()’ and ‘gather()’ functions [R package]
,” available at (Last viewed 5 April 2019).95.
Wickham
, H.
, and Francois
, R.
(2015
). “dplyr: A grammar of data manipulation [R package]
,” available at (Last viewed 5 April 2019).96.
Wong
, J. W. S.
(2013
). “The effects of perceptual and/or productive training on the perception and production of English vowels /ɪ/ and /iː/ by Cantonese ESL learners
,” in INTERSPEECH
, pp. 2113
–2117
.97.
Wong
, P. C.
, and Perrachione
, T. K.
(2007
). “Learning pitch patterns in lexical identification by native English-speaking adults
,” Appl. Psycholinguist.
28
(04
), 565
–585
.98.
Wong
, P. C.
, Vuong
, L. C.
, and Liu
, K.
(2017
). “Personalized learning: From neurogenetics of behaviors to designing optimal language training
,” Neuropsychologia
98
, 192
–200
.99.
Zandipour
, M.
, Perkell
, J.
, Guenther
, F.
, Tiede
, M.
, Honda
, K.
, and Murano
, E.
(2006
). “Speaking with a bite-block: Data and modeling
,” paper presented at the Proceedings of the 7th International Seminar on Speech Production
.© 2019 Acoustical Society of America.
2019
Acoustical Society of America
You do not currently have access to this content.