Successful speech communication often requires selective attention to a target stream amidst competing sounds, as well as the ability to switch attention among multiple interlocutors. However, auditory attention switching negatively affects both target detection accuracy and reaction time, suggesting that attention switches carry a cognitive cost. Pupillometry is one method of assessing mental effort or cognitive load. Two experiments were conducted to determine whether the effort associated with attention switches is detectable in the pupillary response. In both experiments, pupil dilation, target detection sensitivity, and reaction time were measured; the task required listeners to either maintain or switch attention between two concurrent speech streams. Secondary manipulations explored whether switch-related effort would increase when auditory streaming was harder. In experiment 1, spatially distinct stimuli were degraded by simulating reverberation (compromising across-time streaming cues), and target-masker talker gender match was also varied. In experiment 2, diotic streams separable by talker voice quality and pitch were degraded by noise vocoding, and the time alloted for mid-trial attention switching was varied. All trial manipulations had some effect on target detection sensitivity and/or reaction time; however, only the attention-switching manipulation affected the pupillary response: greater dilation was observed in trials requiring switching attention between talkers.

1.
S. A.
Shamma
,
M.
Elhilali
, and
C.
Micheyl
, “
Temporal coherence and attention in auditory scene analysis
,”
Trends Neurosci.
34
(
3
),
114
123
(
2011
).
2.
B. G.
Shinn-Cunningham
and
V.
Best
, “
Selective attention in normal and impaired hearing
,”
Trends Amplif.
12
(
4
),
283
299
(
2008
).
3.
E. D.
Larson
and
A. K. C.
Lee
, “
Influence of preparation time and pitch separation in switching of auditory attention between streams
,”
J. Acoust. Soc. Am.
134
(
2
),
EL165
–EL171 (
2013
).
4.
J.-P.
de Ruiter
,
H.
Mitterer
, and
N. J.
Enfield
, “
Projecting the end of a speaker's turn: A cognitive cornerstone of conversation
,”
Language
82
(
3
),
515
535
(
2006
).
5.
D.
Kahneman
and
J.
Beatty
, “
Pupil diameter and load on memory
,”
Science
154
(
3756
),
1583
1585
(
1966
).
6.
J.
Beatty
, “
Task-evoked pupillary responses, processing load, and the structure of processing resources
,”
Psychol. Bull.
91
(
2
),
276
292
(
1982
).
7.
B.
Hoeks
and
W. J. M.
Levelt
, “
Pupillary dilation as a measure of attention: A quantitative system analysis
,”
Behav. Res. Meth. Ins. C
25
(
1
),
16
26
(
1993
).
8.
S. M.
Wierda
,
H.
van Rijn
,
N. A.
Taatgen
, and
S.
Martens
, “
Pupil dilation deconvolution reveals the dynamics of attention at high temporal resolution
,”
Proc. Natl. Acad. Sci. U.S.A.
109
(
22
),
8456
8460
(
2012
).
9.
D.
McCloy
,
E.
Larson
,
B.
Lau
, and
A. K. C.
Lee
, “
Temporal alignment of pupillary response with stimulus events via deconvolution
,”
J. Acoust. Soc. Am.
139
(
3
),
EL57
EL62
(
2016
).
10.
J. S.
Taylor
, “
Pupillary response to auditory versus visual mental loading: A pilot study using super 8-mm photography
,”
Percept. Mot. Skills
52
(
2
),
425
426
(
1981
).
11.
S.
Ahern
and
J.
Beatty
, “
Physiological evidence that demand for processing capacity varies with intelligence
,” in
Intelligence and Learning
, edited by
M. P.
Friedman
,
J. P.
Das
, and
N.
O'Connor
(
Springer
,
Boston
,
1981
), NATO Conference Series No. 14, pp.
121
128
.
12.
M. H.
Papesh
and
S. D.
Goldinger
, “
Pupil-BLAH-metry: Cognitive effort in speech planning reflected by pupil dilation
,”
Atten. Percept. Psychophys.
74
(
4
),
754
765
(
2012
).
13.
E. H.
Hess
and
J. M.
Polt
, “
Pupil size in relation to mental activity during simple problem-solving
,”
Science
143
(
3611
),
1190
1192
(
1964
).
14.
A. A.
Zekveld
,
S. E.
Kramer
, and
J. M.
Festen
, “
Pupil response as an indication of effortful listening: The influence of sentence intelligibility
,”
Ear Hear.
31
(
4
),
480
490
(
2010
).
15.
T.
Koelewijn
,
A. A.
Zekveld
,
J. M.
Festen
,
J.
Rönnberg
, and
S. E.
Kramer
, “
Processing load induced by informational masking is related to linguistic abilities
,”
Int. J. Otolaryngol.
2012
,
865731
.
16.
M. B.
Winn
,
J. R.
Edwards
, and
R. Y.
Litovsky
, “
The impact of auditory spectral resolution on listening effort revealed by pupil dilation
,”
Ear Hear.
36
(
4
),
e153
e165
(
2015
).
17.
R.
McGarrigle
,
K. J.
Munro
,
P.
Dawes
,
A. J.
Stewart
,
D. R.
Moore
,
J. G.
Barry
, and
S.
Amitay
, “
Listening effort and fatigue: What exactly are we measuring? A British Society of Audiology Cognition in Hearing Special Interest Group ‘white paper
,’ ”
Int. J. Audiol.
53
(
7
),
433
445
(
2014
).
18.
M. K.
Pichora-Fuller
,
S. E.
Kramer
,
M. A.
Eckert
,
B.
Edwards
,
B. W.
Hornsby
,
L. E.
Humes
,
U.
Lemke
,
T.
Lunner
,
M.
Matthen
,
C. L.
Mackersie
,
G.
Naylor
,
N. A.
Phillips
,
M.
Richter
,
M.
Rudner
,
M. S.
Sommers
,
K. L.
Tremblay
, and
A.
Wingfield
, “
Hearing impairment and cognitive energy: The framework for understanding effortful listening (FUEL)
,”
Ear Hear.
37
,
5S
27S
(
2016
).
19.
A. A.
Zekveld
,
S. E.
Kramer
, and
J. M.
Festen
, “
Cognitive load during speech perception in noise: The influence of age, hearing loss, and cognition on the pupil response
,”
Ear Hear.
32
(
4
),
498
510
(
2011
).
20.
S. E.
Kuchinsky
,
J. B.
Ahlstrom
,
K. I.
Vaden
,
S. L.
Cute
,
L. E.
Humes
,
J. R.
Dubno
, and
M. A.
Eckert
, “
Pupil size varies with word listening and response selection difficulty in older adults with hearing loss
,”
Psychophysiology
50
(
1
),
23
34
(
2013
).
21.
V.
Best
,
F. J.
Gallun
,
C. R.
Mason
,
G. D.
Kidd
, Jr.
, and
B. G.
Shinn-Cunningham
, “
The impact of noise and hearing loss on the processing of simultaneous sentences
,”
Ear Hear.
31
(
2
),
213
220
(
2010
).
22.
T.
Koelewijn
,
B. G.
Shinn-Cunningham
,
A. A.
Zekveld
, and
S. E.
Kramer
, “
The pupil response is sensitive to divided attention during speech processing
,”
Hear. Res.
312
,
114
120
(
2014
).
23.
T.
Koelewijn
,
H.
de Kluiver
,
B. G.
Shinn-Cunningham
,
A. A.
Zekveld
, and
S. E.
Kramer
, “
The pupil response reveals increased listening effort when it is difficult to focus attention
,”
Hear. Res.
323
,
81
90
(
2015
).
24.
D.
Kahneman
and
J.
Beatty
, “
Pupillary responses in a pitch-discrimination task
,”
Percept. Psychophys.
2
(
3
),
101
105
(
1967
).
25.
A. K.
Nábělek
and
P. K.
Robinson
, “
Monaural and binaural speech perception in reverberation for listeners of various ages
,”
J. Acoust. Soc. Am.
71
(
5
),
1242
1248
(
1982
).
26.
D. S.
Brungart
, “
Informational and energetic masking effects in the perception of two simultaneous talkers
,”
J. Acoust. Soc. Am.
109
(
3
),
1101
1109
(
2001
).
27.
R. A.
Cole
,
Y.
Muthusamy
, and
M.
Fanty
, “
The ISOLET spoken letter database
,” Technical Report 90-004, Oregon Graduate Institute, Hillsboro, OR (1990), paper 205.
28.
B. G.
Shinn-Cunningham
,
N.
Kopco
, and
T. J.
Martin
, “
Localizing nearby sound sources in a classroom: Binaural room impulse responses
,”
J. Acoust. Soc. Am.
117
(
5
),
3100
3115
(
2005
).
29.
L. T.
DeCarlo
, “
Signal detection theory and generalized linear models
,”
Psychol. Methods
3
(
2
),
186
205
(
1998
).
30.
C.-F.
Sheu
,
Y.-S.
Lee
, and
P.-Y.
Shih
, “
Analyzing recognition performance with sparse data
,”
Behav. Res. Meth.
40
(
3
),
722
727
(
2008
).
31.
D. R.
McCloy
and
A. K. C.
Lee
, “
Auditory attention strategy depends on target linguistic properties and spatial configuration
,”
J. Acoust. Soc. Am.
138
(
1
),
97
114
(
2015
).
32.
E. D.
Larson
and
D. A.
Engemann
, “pyeparse (version 0.1.0),” .
33.
E.
Maris
and
R.
Oostenveld
, “
Nonparametric statistical testing of EEG- and MEG-data
,”
J. Neurosci. Meth.
164
(
1
),
177
190
(
2007
).
34.
A.
Gramfort
,
M.
Luessi
,
E. D.
Larson
,
D. A.
Engemann
,
D.
Strohmeier
,
C.
Brodbeck
,
R.
Goj
,
M.
Jas
,
T.
Brooks
,
L.
Parkkonen
, and
M. S.
Hämäläinen
, “
MEG and EEG data analysis with MNE-Python
,”
Front. Neurosci.
7
,
267
(
2013
).
35.
J.-M.
Hupé
,
C.
Lamirel
, and
J.
Lorenceau
, “
Pupil dynamics during bistable motion perception
,”
J. Vision
9
(
7
),
10
(
2009
).
36.
E. D.
Larson
and
A. K. C.
Lee
, “
The cortical dynamics underlying effective switching of auditory spatial attention
,”
NeuroImage
64
,
365
370
(
2013
).
37.
R. V.
Shannon
,
F.-G.
Zeng
,
V.
Kamath
,
J.
Wygonski
, and
M.
Ekelid
, “
Speech recognition with primarily temporal cues
,”
Science
270
(
5234
),
303
304
(
1995
).
38.
B. C. J.
Moore
and
B. R.
Glasberg
, “
Formulae describing frequency selectivity as a function of frequency and level, and their use in calculating excitation patterns
,”
Hear. Res.
28
(
2-3
),
209
225
(
1987
).
39.
See supplementary material at http://dx.doi.org/10.1121/1.4979340 for details of statistical model specifications and results, post-hoc analyses, and location of data/code repository.

Supplementary Material

You do not currently have access to this content.