Successful speech communication often requires selective attention to a target stream amidst competing sounds, as well as the ability to switch attention among multiple interlocutors. However, auditory attention switching negatively affects both target detection accuracy and reaction time, suggesting that attention switches carry a cognitive cost. Pupillometry is one method of assessing mental effort or cognitive load. Two experiments were conducted to determine whether the effort associated with attention switches is detectable in the pupillary response. In both experiments, pupil dilation, target detection sensitivity, and reaction time were measured; the task required listeners to either maintain or switch attention between two concurrent speech streams. Secondary manipulations explored whether switch-related effort would increase when auditory streaming was harder. In experiment 1, spatially distinct stimuli were degraded by simulating reverberation (compromising across-time streaming cues), and target-masker talker gender match was also varied. In experiment 2, diotic streams separable by talker voice quality and pitch were degraded by noise vocoding, and the time alloted for mid-trial attention switching was varied. All trial manipulations had some effect on target detection sensitivity and/or reaction time; however, only the attention-switching manipulation affected the pupillary response: greater dilation was observed in trials requiring switching attention between talkers.
References
1.
S. A.
Shamma
, M.
Elhilali
, and C.
Micheyl
, “Temporal coherence and attention in auditory scene analysis
,” Trends Neurosci.
34
(3
), 114
–123
(2011
).2.
B. G.
Shinn-Cunningham
and V.
Best
, “Selective attention in normal and impaired hearing
,” Trends Amplif.
12
(4
), 283
–299
(2008
).3.
E. D.
Larson
and A. K. C.
Lee
, “Influence of preparation time and pitch separation in switching of auditory attention between streams
,” J. Acoust. Soc. Am.
134
(2
), EL165
–EL171 (2013
).4.
J.-P.
de Ruiter
, H.
Mitterer
, and N. J.
Enfield
, “Projecting the end of a speaker's turn: A cognitive cornerstone of conversation
,” Language
82
(3
), 515
–535
(2006
).5.
D.
Kahneman
and J.
Beatty
, “Pupil diameter and load on memory
,” Science
154
(3756
), 1583
–1585
(1966
).6.
J.
Beatty
, “Task-evoked pupillary responses, processing load, and the structure of processing resources
,” Psychol. Bull.
91
(2
), 276
–292
(1982
).7.
B.
Hoeks
and W. J. M.
Levelt
, “Pupillary dilation as a measure of attention: A quantitative system analysis
,” Behav. Res. Meth. Ins. C
25
(1
), 16
–26
(1993
).8.
S. M.
Wierda
, H.
van Rijn
, N. A.
Taatgen
, and S.
Martens
, “Pupil dilation deconvolution reveals the dynamics of attention at high temporal resolution
,” Proc. Natl. Acad. Sci. U.S.A.
109
(22
), 8456
–8460
(2012
).9.
D.
McCloy
, E.
Larson
, B.
Lau
, and A. K. C.
Lee
, “Temporal alignment of pupillary response with stimulus events via deconvolution
,” J. Acoust. Soc. Am.
139
(3
), EL57
–EL62
(2016
).10.
J. S.
Taylor
, “Pupillary response to auditory versus visual mental loading: A pilot study using super 8-mm photography
,” Percept. Mot. Skills
52
(2
), 425
–426
(1981
).11.
S.
Ahern
and J.
Beatty
, “Physiological evidence that demand for processing capacity varies with intelligence
,” in Intelligence and Learning
, edited by M. P.
Friedman
, J. P.
Das
, and N.
O'Connor
(Springer
, Boston
, 1981
), NATO Conference Series No. 14, pp. 121
–128
.12.
M. H.
Papesh
and S. D.
Goldinger
, “Pupil-BLAH-metry: Cognitive effort in speech planning reflected by pupil dilation
,” Atten. Percept. Psychophys.
74
(4
), 754
–765
(2012
).13.
E. H.
Hess
and J. M.
Polt
, “Pupil size in relation to mental activity during simple problem-solving
,” Science
143
(3611
), 1190
–1192
(1964
).14.
A. A.
Zekveld
, S. E.
Kramer
, and J. M.
Festen
, “Pupil response as an indication of effortful listening: The influence of sentence intelligibility
,” Ear Hear.
31
(4
), 480
–490
(2010
).15.
T.
Koelewijn
, A. A.
Zekveld
, J. M.
Festen
, J.
Rönnberg
, and S. E.
Kramer
, “Processing load induced by informational masking is related to linguistic abilities
,” Int. J. Otolaryngol.
2012
, 865731
.16.
M. B.
Winn
, J. R.
Edwards
, and R. Y.
Litovsky
, “The impact of auditory spectral resolution on listening effort revealed by pupil dilation
,” Ear Hear.
36
(4
), e153
–e165
(2015
).17.
R.
McGarrigle
, K. J.
Munro
, P.
Dawes
, A. J.
Stewart
, D. R.
Moore
, J. G.
Barry
, and S.
Amitay
, “Listening effort and fatigue: What exactly are we measuring? A British Society of Audiology Cognition in Hearing Special Interest Group ‘white paper
,’ ” Int. J. Audiol.
53
(7
), 433
–445
(2014
).18.
M. K.
Pichora-Fuller
, S. E.
Kramer
, M. A.
Eckert
, B.
Edwards
, B. W.
Hornsby
, L. E.
Humes
, U.
Lemke
, T.
Lunner
, M.
Matthen
, C. L.
Mackersie
, G.
Naylor
, N. A.
Phillips
, M.
Richter
, M.
Rudner
, M. S.
Sommers
, K. L.
Tremblay
, and A.
Wingfield
, “Hearing impairment and cognitive energy: The framework for understanding effortful listening (FUEL)
,” Ear Hear.
37
, 5S
–27S
(2016
).19.
A. A.
Zekveld
, S. E.
Kramer
, and J. M.
Festen
, “Cognitive load during speech perception in noise: The influence of age, hearing loss, and cognition on the pupil response
,” Ear Hear.
32
(4
), 498
–510
(2011
).20.
S. E.
Kuchinsky
, J. B.
Ahlstrom
, K. I.
Vaden
, S. L.
Cute
, L. E.
Humes
, J. R.
Dubno
, and M. A.
Eckert
, “Pupil size varies with word listening and response selection difficulty in older adults with hearing loss
,” Psychophysiology
50
(1
), 23
–34
(2013
).21.
V.
Best
, F. J.
Gallun
, C. R.
Mason
, G. D.
Kidd
, Jr., and B. G.
Shinn-Cunningham
, “The impact of noise and hearing loss on the processing of simultaneous sentences
,” Ear Hear.
31
(2
), 213
–220
(2010
).22.
T.
Koelewijn
, B. G.
Shinn-Cunningham
, A. A.
Zekveld
, and S. E.
Kramer
, “The pupil response is sensitive to divided attention during speech processing
,” Hear. Res.
312
, 114
–120
(2014
).23.
T.
Koelewijn
, H.
de Kluiver
, B. G.
Shinn-Cunningham
, A. A.
Zekveld
, and S. E.
Kramer
, “The pupil response reveals increased listening effort when it is difficult to focus attention
,” Hear. Res.
323
, 81
–90
(2015
).24.
D.
Kahneman
and J.
Beatty
, “Pupillary responses in a pitch-discrimination task
,” Percept. Psychophys.
2
(3
), 101
–105
(1967
).25.
A. K.
Nábělek
and P. K.
Robinson
, “Monaural and binaural speech perception in reverberation for listeners of various ages
,” J. Acoust. Soc. Am.
71
(5
), 1242
–1248
(1982
).26.
D. S.
Brungart
, “Informational and energetic masking effects in the perception of two simultaneous talkers
,” J. Acoust. Soc. Am.
109
(3
), 1101
–1109
(2001
).27.
R. A.
Cole
, Y.
Muthusamy
, and M.
Fanty
, “The ISOLET spoken letter database
,” Technical Report 90-004, Oregon Graduate Institute, Hillsboro, OR (1990), paper 205.28.
B. G.
Shinn-Cunningham
, N.
Kopco
, and T. J.
Martin
, “Localizing nearby sound sources in a classroom: Binaural room impulse responses
,” J. Acoust. Soc. Am.
117
(5
), 3100
–3115
(2005
).29.
L. T.
DeCarlo
, “Signal detection theory and generalized linear models
,” Psychol. Methods
3
(2
), 186
–205
(1998
).30.
C.-F.
Sheu
, Y.-S.
Lee
, and P.-Y.
Shih
, “Analyzing recognition performance with sparse data
,” Behav. Res. Meth.
40
(3
), 722
–727
(2008
).31.
D. R.
McCloy
and A. K. C.
Lee
, “Auditory attention strategy depends on target linguistic properties and spatial configuration
,” J. Acoust. Soc. Am.
138
(1
), 97
–114
(2015
).32.
33.
E.
Maris
and R.
Oostenveld
, “Nonparametric statistical testing of EEG- and MEG-data
,” J. Neurosci. Meth.
164
(1
), 177
–190
(2007
).34.
A.
Gramfort
, M.
Luessi
, E. D.
Larson
, D. A.
Engemann
, D.
Strohmeier
, C.
Brodbeck
, R.
Goj
, M.
Jas
, T.
Brooks
, L.
Parkkonen
, and M. S.
Hämäläinen
, “MEG and EEG data analysis with MNE-Python
,” Front. Neurosci.
7
, 267
(2013
).35.
J.-M.
Hupé
, C.
Lamirel
, and J.
Lorenceau
, “Pupil dynamics during bistable motion perception
,” J. Vision
9
(7
), 10
(2009
).36.
E. D.
Larson
and A. K. C.
Lee
, “The cortical dynamics underlying effective switching of auditory spatial attention
,” NeuroImage
64
, 365
–370
(2013
).37.
R. V.
Shannon
, F.-G.
Zeng
, V.
Kamath
, J.
Wygonski
, and M.
Ekelid
, “Speech recognition with primarily temporal cues
,” Science
270
(5234
), 303
–304
(1995
).38.
B. C. J.
Moore
and B. R.
Glasberg
, “Formulae describing frequency selectivity as a function of frequency and level, and their use in calculating excitation patterns
,” Hear. Res.
28
(2-3
), 209
–225
(1987
).39.
See supplementary material at http://dx.doi.org/10.1121/1.4979340 for details of statistical model specifications and results, post-hoc analyses, and location of data/code repository.
© 2017 Acoustical Society of America.
2017
Acoustical Society of America
You do not currently have access to this content.
