Previous research has identified a “synchrony window” of several hundred milliseconds over which auditory-visual (AV) asynchronies are not reliably perceived. Individual variability in the size of this AV synchrony window has been linked with variability in AV speech perception measures, but it was not clear whether AV speech perception measures are related to synchrony detection for speech only or for both speech and nonspeech signals. An experiment was conducted to investigate the relationship between measures of AV speech perception and AV synchrony detection for speech and nonspeech signals. Variability in AV synchrony detection for both speech and nonspeech signals was found to be related to variability in measures of auditory-only (A-only) and AV speech perception, suggesting that temporal processing for both speech and nonspeech signals must be taken into account in explaining variability in A-only and multisensory speech perception.

1.
Boothroyd
,
A.
,
Hanin
,
L.
, and
Hnath
,
T.
(
1985
). “
A sentence test of speech perception: Reliability, set equivalence, and short term learning
,” Speech and Hearing Science Report No. RC110,
City University of New York
.
2.
Bushara
,
K. O.
,
Grafman
,
J.
, and
Hallett
,
M.
(
2001
). “
Neural correlates of auditory-visual stimulus onset asynchrony detection
,”
J. Neurosci.
21
(
1
),
300
304
.
3.
Cohen
,
J. D.
,
MacWhinney
,
B.
,
Flatt
,
M.
, and
Provost
,
J.
(
1993
). “
PsyScope: A new graphic interactive environment for designing psychology experiments
,”
Behav. Res. Methods Instrum. Comput.
25
(
2
),
257
271
.
4.
Conrey
,
B.
, and
Pisoni
,
D. B.
(
2003
). “
Audiovisual asynchrony detection for speech and nonspeech signals
,” paper presented at the
ISCA 2003 International Conference on Auditory-Visual Speech Processing
.
5.
Dixon
,
N.
, and
Spitz
,
L.
(
1980
). “
The detection of audiovisual desynchrony
,”
Perception
9
,
719
721
.
6.
Fujisaki
,
W.
,
Shinsuke
,
S.
,
Kashino
,
M.
, and
Nishida
,
S.
(
2004
). “
Recalibration of audiovisual simultaneity
,”
Nat. Neurosci.
7
,
773
778
.
7.
Grant
,
K. W.
, and
Greenberg
,
S.
(
2001
). “
Speech intelligibility derived from asynchronous processing of auditory-visual information
,” paper presented at the ISCA International Conference on Auditory-Visual Speech Processing.
8.
Grant
,
K. W.
, and
Seitz
,
P. F.
(
1998
). “
Measures of auditory-visual integration in nonsense syllables and sentences
,”
J. Acoust. Soc. Am.
104
,
2438
2450
.
9.
Grant
,
K. W.
,
van Wassenhove
,
V.
, and
Poeppel
,
D.
(
2003
). “
Discrimination of auditory-visual synchrony
,” paper presented at the
ISCA 2003 International Conference on Auditory-Visual Speech Processing
.
10.
King
,
A. J.
, and
Palmer
,
A. R.
(
1985
). “
Integration of visual and auditory information in bimodal neurones in the guinea-pig superior colliculus
,”
Exp. Brain Res.
60
,
492
500
.
11.
Lachs
,
L.
(
1999
). “
Use of partial stimulus information in spoken word recognition without auditory stimulation
,” in Research on Spoken Language Processing Progress Report No. 23, Speech Research Laboratory, Indiana University, Bloomington, IN, pp.
82
114
.
12.
Lachs
,
L.
, and
Hernandez
,
L. R.
(
1998
). “
Update: The Hoosier Audiovisual Multitalker Database
,” in Research on Spoken Language Processing Progress Report No. 22, Speech Research Laboratory, Indiana University, Bloomington, IN, pp.
377
388
.
13.
Lachs
,
L.
, and
Pisoni
,
D. B.
(
2004a
). “
Crossmodal source identification in speech perception
,”
Ecological Psychol.
16
,
159
187
.
14.
Lachs
,
L.
, and
Pisoni
,
D. B.
(
2004b
). “
Crossmodal source information and spoken word recognition
,”
J. Exp. Psychol. Hum. Percept. Perform.
30
(
2
),
378
396
.
15.
Lewkowicz
,
D. J.
(
1996
). “
Perception of auditory-visual temporal synchrony in human infants
,”
J. Exp. Psychol. Hum. Percept. Perform.
22
,
1094
1106
.
16.
Massaro
,
D.
, and
Cohen
,
M.
(
1993
). “
Perceiving asynchronous bimodal speech in consonant-vowel and vowel syllables
,”
Speech Commun.
13
,
127
134
.
17.
Massaro
,
D.
,
Cohen
,
M. M.
, and
Smeele
,
P. M. T.
(
1996
). “
Perception of asynchronous and conflicting visual and auditory speech
,”
J. Acoust. Soc. Am.
100
(
3
),
1777
1786
.
18.
McGrath
,
M.
, and
Summerfield
,
Q.
(
1985
). “
Intermodal timing relations and audio-visual speech recognition by normal-hearing adults
,”
J. Acoust. Soc. Am.
77
(
2
),
678
684
.
19.
Meredith
,
M. A.
(
2002
). “
On the neuronal basis for multisensory convergence: A brief overview
,”
Cognit. Brain Res.
14
,
31
40
.
20.
Meredith
,
M. A.
,
Nemitz
,
J. W.
, and
Stein
,
B. E.
(
1987
). “
Determinants of multisensory integration in superior colliculus neurons. I. Temporal factors
,”
J. Neurosci.
7
(
10
),
3215
3229
.
21.
Munhall
,
K. G.
,
Gribble
,
P.
,
Sacco
,
L.
, and
Ward
,
M.
(
1996
). “
Temporal constraints on the McGurk effect
,”
Percept. Psychophys.
58
(
3
),
351
362
.
22.
Pandey
,
C. P.
,
Kunov
,
H.
, and
Abel
,
M. S.
(
1986
). “
Disruptive effects of auditory signal delay on speech perception with lip-reading
,”
J. Aud. Res.
26
,
27
41
.
23.
Saberi
,
K.
, and
Perrott
,
D. R.
(
1999
). “
Cognitive restoration of reversed speech
,”
Nature (London)
398
,
760
.
24.
Sheffert
,
S. M.
,
Lachs
,
L.
, and
Hernandez
,
L. R.
(
1996
). “
The Hoosier Audiovisual Multitalker Database
,” in Research on Spoken Language Processing Progress Report No. 21, Speech Research Laboratory, Indiana University, Bloomington, IN, pp.
578
583
.
25.
Stein
,
B.
, and
Meredith
,
M. A.
(
1993
).
The Merging of the Senses
(
MIT
,
Cambridge, MA
).
26.
Stone
,
J. V.
,
Hunkin
,
N. M.
,
Porrill
,
J.
,
Wood
,
R.
,
Keeler
,
V.
,
Beanland
,
M.
, et al (
2001
). “
When is now? Perception of simultaneity
,”
Proc. R. Soc. London, Ser. B
268
,
31
38
.
27.
van Wassenhove
,
V.
,
Grant
,
K. W.
, and
Poeppel
,
D.
(
2002
). “
Temporal integration in the McGurk effect
,” poster presented at the
annual meeting of the Society for Cognitive Neuroscience
, San Francisco.
You do not currently have access to this content.