In order to perceive meaningful speech, the auditory system must recognize different phonemes amidst a noisy and variable acoustic signal. To better understand the processing mechanisms underlying this ability, evoked cortical responses to different spoken consonants were measured with electroencephalography (EEG). Using multivariate pattern analysis (MVPA), binary classifiers attempted to discriminate between the EEG activity evoked by two given consonants at each peri-stimulus time sample, providing a dynamic measure of their cortical dissimilarity. To examine the relationship between representations at the auditory periphery and cortex, MVPA was also applied to modelled auditory-nerve (AN) responses of consonants, and time-evolving AN-based and EEG-based dissimilarities were compared with one another. Cortical dissimilarities between consonants were commensurate with their articulatory distinctions, particularly their manner of articulation, and to a lesser extent, their voicing. Furthermore, cortical distinctions between consonants in two periods of activity, centered at 130 and 400 ms after onset, aligned with their peripheral dissimilarities in distinct onset and post-onset periods, respectively. In relating speech representations across articulatory, peripheral, and cortical domains, the understanding of crucial transformations in the auditory pathway underlying the ability to perceive speech is advanced.
References
1.
Allen
, J. B.
(1994
). “How do humans process and recognize speech?
,” IEEE Trans. Speech Audio Process.
2
(4
), 567
–577
.2.
Allen
, J. B.
(2005
). “Consonant recognition and the articulation index
,” J. Acoust, Soc, Am.
117
(4
), 2212
–2223
.3.
Bell
, A. J.
, and Sejnowski
, T. J.
(1995
). “An information-maximization approach to blind separation and blind deconvolution
,” Neural Comput.
7
(6
), 1129
–1159
.4.
Benjamini
, Y.
, and Yekutieli
, D.
(2001
). “The control of the false discovery rate in multiple testing under dependency
,” Ann. Stat.
29
, 1165
–1188
.5.
Bilger
, R. C.
, and Wang
, M. D.
(1976
). “Consonant confusions in patients with sensorineural hearing loss
,” J. Speech, Lang. Hear. Res.
19
(4
), 718
–748
.6.
Chang
, E. F.
, Rieger
, J. W.
, Johnson
, K.
, Berger
, M. S.
, Barbaro
, N. M.
, and Knight
, R. T.
(2010
). “Categorical speech representation in human superior temporal gyrus
,” Nat. Neurosci.
13
(11
), 1428
–1432
.7.
de Cheveigné
, A.
, and Simon
, J. Z.
(2008
). “Denoising based on spatial filtering
,” J. Neurosci. Methods
171
(2
), 331
–339
.8.
Delgutte
, B.
, and Kiang
, N. Y.
(1984a
). “Speech coding in the auditory nerve: III. Voiceless fricative consonants
,” J. Acoust. Soc. Am.
75
(3
), 887
–896
.9.
Delgutte
, B.
, and Kiang
, N. Y.
(1984b
). “Speech coding in the auditory nerve: IV. Sounds with consonant-like dynamic characteristics
,” J. Acoust. Soc. Am.
75
(3
), 897
–907
.10.
Dichter
, B. K.
, Bouchard
, K. E.
, and Chang
, E. F.
(2016
). “Dynamic structure of neural variability in the cortical representation of speech sounds
,” J. Neurosci.
36
(28
), 7453
–7463
.11.
Di Liberto
, G. M.
, O'Sullivan
, J. A.
, and Lalor
, E. C.
(2015
). “Low-frequency cortical entrainment to speech reflects phoneme-level processing
,” Curr. Biol.
25
(19
), 2457
–2465
.12.
Ding
, N.
, Melloni
, L.
, Zhang
, H.
, Tian
, X.
, and Poeppel
, D.
(2016
). “Cortical tracking of hierarchical linguistic structures in connected speech
,” Nat. Neurosci.
19
(1
), 158
–164
.13.
Ding
, N.
, and Simon
, J. Z.
(2011
). “Neural coding of continuous speech in auditory cortex during monaural and dichotic listening
,” J. Neurophysiol.
107
(1
), 78
–89
.14.
Ding
, N.
, and Simon
, J. Z.
(2013
). “Adaptive temporal encoding leads to a background-insensitive cortical representation of speech
,” J. Neurosci.
33
(13
), 5728
–5735
.15.
Doyle
, K. J.
, Danhauer
, J. L.
, and Edgerton
, B. J.
(1981
). “Features from normal and sensorineural listeners' nonsense syllable test errors
,” Ear Hear.
2
(3
), 117
–121
.16.
Duda
, R. O.
, Hart
, P. E.
, and Stork
, D. G.
(2012
). Pattern Classification
(Wiley
, New York
).17.
Grootswagers
, T.
, Wardle
, S. G.
, and Carlson
, T. A.
(2017
). “Decoding dynamic brain patterns from evoked responses: A tutorial on multivariate pattern analysis applied to time series neuroimaging data
,” J. Cognit. Neurosci.
29
(4
), 677
–697
.18.
Haxby
, J. V.
, Connolly
, A. C.
, and Guntupalli
, J. S.
(2014
). “Decoding neural representational spaces using multivariate pattern analysis
,” Ann. Rev. Neurosci.
37
, 435
–456
.19.
Heinz
, M. G.
(2010
). “Computational modeling of sensorineural hearing loss
,” in Computational Models of the Auditory System
, edited by R.
Meddis
, E. A.
Lopez-Poveda
, A. N.
Popper
, and R. R.
Fay
(Springer
, New York)
, pp. 177
–202
.20.
Huth
, A. G.
, de Heer
, W. A.
, Griffiths
, T. L.
, Theunissen
, F. E.
, and Gallant
, J. L.
(2016
). “Natural speech reveals the semantic maps that tile human cerebral cortex
,” Nature
532
(7600
), 453
–458
.21.
Khalighinejad
, B.
, da Silva
, G. C.
, and Mesgarani
, N.
(2017
). “Dynamic encoding of acoustic features in neural responses to continuous speech
,” J. Neurosci.
37
(8
), 2176
–2185
.22.
Kriegeskorte
, N.
, Mur
, M.
, and Bandettini
, P. A.
(2008
). “Representational similarity analysis—connecting the branches of systems neuroscience
,” Front. Syst. Neurosci.
2
, 4
.23.
Leonard
, M. K.
, and Chang
, E. F.
(2014
). “Dynamic speech representations in the human temporal lobe
,” Trends Cognit. Sci.
18
(9
), 472
–479
.24.
Lopez-Poveda
, EA.
(2005
). “Spectral processing by the peripheral auditory system: Facts and models
,” Int. Rev. Neurobiol.
70
, 7
–48
.25.
Martin
, B. A.
, Tremblay
, K. L.
, and Korczak
, P.
(2008
). “Speech evoked potentials: From the laboratory to the clinic
,” Ear Hear.
29
(3
), 285
–313
.26.
Martin
, J. G.
, and Bunnell
, H. T.
(1981
). “Perception of anticipatory coarticulation effects
,” J. Acoust. Soc. Am.
69
(2
), 559
–567
.27.
Martin
, J. G.
, and Bunnell
, H. T.
(1982
). “Perception of anticipatory coarticulation effects in vowel–stop consonant–vowel sequences
,” J. Exp. Psychol.: Hum. Percept. Perform.
8
(3
), 473
–488
.28.
Mesgarani
, N.
, Cheung
, C.
, Johnson
, K.
, and Chang
, E. F.
(2014
). “Phonetic feature encoding in human superior temporal gyrus
,” Science
343
(6174
), 1006
–1010
.29.
Miller
, G. A.
, and Nicely
, P. E.
(1955
). “An analysis of perceptual confusions among some English consonants
,” J. Acoust. Soc. Am.
27
(2
), 338
–352
.30.
Nili
, H.
, Wingfield
, C.
, Walther
, A.
, Su
, L.
, Marslen-Wilson
, W.
, and Kriegeskorte
, N.
(2014
). “A toolbox for representational similarity analysis
,” PLoS Comput. Biol.
10
(4
), e1003553
.31.
Peelle
, J. E.
, Gross
, J.
, and Davis
, M. H.
(2012
). “Phase-locked responses to speech in human auditory cortex are enhanced during comprehension
,” Cereb. Cortex
23
(6
), 1378
–1387
.32.
Phatak
, S. A.
, Lovitt
, A.
, and Allen
, J. B.
(2008
). “Consonant confusions in white noise
,” J. Acoust. Soc. Am.
124
(2
), 1220
–1233
.33.
Phatak
, S. A.
, Yoon
, Y. S.
, Gooler
, D. M.
, and Allen
, J. B.
(2009
). “Consonant recognition loss in hearing impaired listeners
,” J. Acoust. Soc. Am.
126
(5
), 2683
–2694
.34.
Phillips
, S. L.
, Richter
, S. J.
, and McPherson
, D.
(2009
). “Voiced initial consonant perception deficits in older listeners with hearing loss and good and poor word recognition
,” J. Speech, Lang. Hear. Res.
52
(1
), 118
–129
.35.
Rabinowitz
, N. C.
, Willmore
, B. D.
, King
, A. J.
, and Schnupp
, J. W.
(2013
). “Constructing noise-invariant representations of sound in the auditory pathway
,” PLoS Biol.
11
(11
), e1001710
.36.
Särelä
, J.
, and Valpola
, H.
(2005
). “Denoising source separation
,” J. Mach. Learn. Res.
6
(Mar), 233
–272
.37.
Walden
, B. E.
, and Montgomery
, A. A.
(1975
). “Dimensions of consonant perception in normal and hearing-impaired listeners
,” J. Speech, Lang., Hear. Res.
18
(3
), 444
–455
.38.
Wang
, M. D.
, Reed
, C. M.
, and Bilger
, R. C.
(1978
). “A comparison of the effects of filtering and sensorineural hearing loss on patterns of consonant confusions
,” J. Speech, Lang., Hear. Res.
21
(1
), 5
–36
.39.
Wesker
, T.
, Meyer
, B.
, Wagener
, K.
, Anemüller
, J.
, Mertins
, A.
, and Kollmeier
, B.
(2005
). “Oldenburg logatome speech corpus (OLLO) for speech recognition experiments with humans and machines
,” in Ninth European Conference on Speech Communication and Technology
.40.
Zilany
, M. S.
, and Bruce
, I. C.
(2006
). “Modeling auditory-nerve responses for high sound pressure levels in the normal and impaired auditory periphery
,” J. Acoust. Soc. Am.
120
(3
), 1446
–1466
.41.
Zilany
, M. S.
, Bruce
, I. C.
, and Carney
, L. H.
(2014
). “Updated parameters and expanded simulation options for a model of the auditory periphery
,” J. Acoust. Soc. Am.
135
(1
), 283
–286
.42.
Zilany
, M. S.
, Bruce
, I. C.
, Nelson
, P. C.
, and Carney
, L. H.
(2009
). “A phenomenological model of the synapse between the inner hair cell and auditory nerve: Long-term adaptation with power-law dynamics
,” J. Acoust. Soc. Am.
126
(5
), 2390
–2412
.© 2018 Acoustical Society of America.
2018
Acoustical Society of America
You do not currently have access to this content.