This study describes data on auditory-visual integration and visually-guided adaptation of auditory distance perception using the ventriloquism effect (VE) and ventriloquism aftereffect (VAE). In an experiment, participants judged egocentric distance of interleaved auditory or auditory-visual stimuli with the auditory component located from 0.7 to 2.04 m in front of listeners in a real reverberant environment. The visual component of auditory-visual stimuli was displaced 30% closer (V-closer), 30% farther (V-farther), or aligned (V-aligned) with respect to the auditory component. The VE and VAE were measured in auditory and auditory-visual trials, respectively. Both effects were approximately independent of target distance when expressed in logarithmic units. The VE strength, defined as a difference of V-misaligned and V-aligned response bias, was approximately 72% of the auditory-visual disparity regardless of the visual-displacement direction, while the VAE was stronger in the V-farther (44%) than the V-closer (31%) condition. The VAE persisted to post-adaptation auditory-only blocks of trials, although it was diminished. The rates of build-up/break-down of the VAE were asymmetrical, with slower adaptation in the V-closer condition. These results suggest that auditory-visual distance integration is independent of the direction of induced shift, while the re-calibration is stronger and faster when evoked by more distant visual stimuli.
References
1.
Alais
, D.
, and Burr
, D.
(2004
). “The ventriloquist effect results from near-optimal bimodal integration
,” Curr. Biol.
14
, 257
–262
.2.
Anderson
, P. W.
, and Zahorik
, P.
(2014
). “Auditory/visual distance estimation: Accuracy and variability
,” Front. Psychol.
5
, 1
–11
.3.
Bedford
, F. L.
(1993
). “Perceptual and cognitive spatial learning
,” J. Exp. Psychol. Hum. Percept. Perform.
19
, 517
–530
.4.
Bertelson
, P.
, Frissen
, I.
, Vroomen
, J.
, and de Gelder
, B.
(2006
). “The aftereffects of ventriloquism: Patterns of spatial generalization
,” Percept. Psychophys.
68
, 428
–436
.5.
Bertelson
, P.
, Vroomen
, J.
, de Gelder
, B.
, and Driver
, J.
(2000
). “The ventriloquist effect does not depend on the direction of deliberate visual attention
,” Percept. Psychophys.
62
, 321
–332
.6.
Berzborn
, M.
, Bomhardt
, R.
, Klein
, J.
, Richter
, J.-G.
, and Vorländer
, M.
(2017
). “The ITA-Toolbox: An Open Source matlab Toolbox for Acoustic Measurements and Signal Processing
,” in Proceedings of the DAGA 2017 - 43th Annual German Congress on Acoustics, 6–9 March 2017
, Kiel, Germany.7.
Bosen
, A. K.
, Fleming
, J. T.
, Allen
, P. D.
, O‘Neill
, W. E.
, and Paige
, G. D.
(2017
). “Accumulation and decay of visual capture and the ventriloquism aftereffect caused by brief audio-visual disparities
,” Exp. Brain Res.
235
, 585
–595
.8.
Bosen
, A. K.
, Fleming
, J. T.
, Allen
, P. D.
, O'Neill
, W. E.
, and Paige
, G. D.
(2018
). “Multiple time scales of the ventriloquism aftereffect
,” PLoS One
13
, e0200930
. 9.
Brungart
, D. S.
, and Rabinowitz
, W. M.
(1999
). “Auditory localization of nearby sources I: Head-related transfer functions
,” J. Acoust. Soc. Am.
106
, 1465
–1479
.10.
Bruns
, P.
, Dinse
, H. R.
, and Röder
, B.
(2020
). “Differential effects of the temporal and spatial distribution of audiovisual stimuli on cross‐modal spatial recalibration
,” Eur. J. Neurosci.
52
, 3763
–3775
.11.
Bruns
, P.
, Liebnau
, R.
, and Röder
, B.
(2020
). “Differential effects of the temporal and spatial distribution of audiovisual stimuli on cross‐modal spatial recalibration
,” Eur. J. Neurosci.
52
, 3763
–3775
.12.
Calcagno
, E. R.
, Abregú
, E. L.
, Eguía
, M. C.
, and Vergara
, R.
(2012
). “The role of vision in auditory distance perception
,” Perception
41
, 175
–192
.13.
Cubick
, J.
, Santurette
, S.
, Laugesen
, S.
, and Dau
, T.
(2015
). “The influence of visual cues on auditory distance perception
,” DAGA 2015 - 41. Annu. Ger. Congr. Acoust. 16–19 March 2015, Nürnberg, Germany, 1220–1223.14.
Frissen
, I.
, Vroomen
, J.
, and de Gelder
, B.
(2012
). “The aftereffects of ventriloquism: The time course of the visual recalibration of auditory localization
,” Seeing Perceiv.
25
, 1
–14
.15.
Gardner
, M. B.
(1968
). “Proximity image effect in sound localization
,” J. Acoust. Soc. Am.
43
, 163
–163
.16.
Hartmann
, W. M.
(1983
). “Localization of sound in rooms
,” J. Acoust. Soc. Am.
74
, 1380
–1391
.17.
Hendrikse
, M. M. E.
, Llorach
, G.
, Grimm
, G.
, and Hohmann
, V.
(2018
). “Influence of visual cues on head and eye movements during listening tasks in multi-talker audiovisual environments with animated characters
,” Speech Commun.
101
, 70
–84
.18.
Herron
, T.
(2005
). “C Language exploratory analysis of variance with enhancements
,” http://www.ebire.org/hcnlab/software/cleave.html Last Viewed: 1 November 2021.19.
Jack
, C. E.
, and Thurlow
, W. R.
(1973
). “Effects of degree of visual association and angle of displacement on the ‘ventriloquism’ effect
,” Percept. Mot. Skills
37
, 967
–979
.20.
Kopčo
, N.
, Huang
, S.
, Belliveau
, J. W.
, Raij
, T.
, Tengshe
, C.
, and Ahveninen
, J.
(2012
). “Neuronal representations of distance in human auditory cortex
,” Proc. Natl. Acad. Sci.
109
, 11019
–11024
.21.
Kopčo
, N.
, Lin
, I.-F.
, Shinn-Cunningham
, B. G.
, and Groh
, J. M.
(2009
). “Reference frame of the ventriloquism aftereffect
,” J. Neurosci.
29
, 13809
–13814
.22.
Kopčo
, N.
, Lokša
, P.
, Lin
, I.
, Groh
, J.
, and Shinn-Cunningham
, B.
(2019
). “Hemisphere-specific properties of the ventriloquism aftereffect
,” J. Acoust. Soc. Am.
146
, EL177
–EL183
.23.
Kopčo
, N.
, and Shinn-Cunningham
, B. G.
(2011
). “Effect of stimulus spectrum on distance perception for nearby sources
,” J. Acoust. Soc. Am.
130
, 1530
–1541
.24.
Lewald
, J.
(2002
). “Rapid Adaptation to Auditory-Visual Spatial Disparity
,” Learn. Mem.
9
, 268
–278
.25.
Mendonça
, C.
, Mandelli
, P.
, and Pulkki
, V.
(2016
). “Modeling the perception of audiovisual distance: Bayesian causal inference and other models
,” PLoS One
11
, e0165391
.26.
Mershon
, D. H.
, Desaulniers
, D. H.
, and Amerson
, J.
(1980
). “Visual capture in auditory distance perception: Proximity image effect reconsidered
,” J. Aud. Res.
20
, 129
–136
.27.
Mershon
, D. H.
, Desaulniers
, D. H.
, Kiefer
, S. A.
, Amerson
, J.
, and Mills
, J. T.
(1981
). “Perceived loudness and visually-determined auditory distance
,” Perception
10
, 531
–543
.28.
Min
, Y. K.
, and Mershon
, D. H.
(2005
). “An adjacency effect in auditory distance perception
,” Acta Acust. united Ac.
91
, 480
–489
.29.
Postma
, B. N. J.
, and Katz
, B. F. G.
(2017
). “The influence of visual distance on the room-acoustic experience of auralizations
,” J. Acoust. Soc. Am.
142
, 3035
–3046
.30.
Razavi
, B.
, O'Neill
, W. E.
, and Paige
, G. D.
(2007
). “Auditory spatial perception dynamically realigns with changing eye position
,” J. Neurosci.
27
, 10249
–10258
.31.
Rébillat
, M.
, Boutillon
, X.
, Corteel
, É.
, and Katz
, B. F. G.
(2012
). “Audio, visual, and audio-visual egocentric distance perception by moving subjects in virtual environments
,” ACM Trans. Appl. Percept.
9
, 1
–17
.32.
Recanzone
, G. H.
(1998
). “Rapidly induced auditory plasticity: The ventriloquism aftereffect
,” Proc. Natl. Acad. Sci. U.S.A.
95
, 869
–875
.33.
Recanzone
, G. H.
(2009
). “Interactions of auditory and visual stimuli in space and time
,” Hear. Res.
258
, 89
–99
.34.
Rife
, D. D.
, and Vanderkooy
, J.
(1989
). “Transfer-function measurement with maximum-length sequences
,” J. Audio Eng. Soc.
37
, 419
–444
.35.
Seeber
, B. U.
, Kerber
, S.
, and Hafter
, E. R.
(2010
). “A system to simulate and reproduce audio–visual environments for spatial hearing research
,” Hear. Res.
260
, 1
–10
.36.
Shinn-Cunningham
, B. G.
, Durlach
, N. I.
, and Held
, R. M.
(1998a
). “Adapting to supernormal auditory localization cues I: Bias and resolution
,” J. Acoust. Soc. Am.
103
, 3656
–3666
.37.
Shinn-Cunningham
, B. G.
, Durlach
, N. I.
, and Held
, R. M.
(1998b
). “Adapting to supernormal auditory localization cues. II. Constraints on adaptation of mean response
,” J. Acoust. Soc. Am.
103
, 3667
–3676
.38.
Shinn-Cunningham
, B. G.
, Kopčo
, N.
, and Martin
, T. J.
(2005
). “Localizing nearby sound sources in a classroom: Binaural room impulse responses
,” J. Acoust. Soc. Am.
117
, 3100
–3115
.39.
Stekelenburg
, J. J.
, Vroomen
, J.
, and de Gelder
, B.
(2004
). “Illusory sound shifts induced by the ventriloquist illusion evoke the mismatch negativity
,” Neurosci. Lett.
357
, 163
–166
.40.
Tresilian
, J. R.
, Mon-Williams
, M.
, and Kelly
, B. M.
(1999
). “Increasing confidence in vergence as a cue to distance
,” Proc. R. Soc. London. Ser. B Biol. Sci.
266
, 39
–44
.41.
Vanderkooy
, J.
(1994
). “Aspects of MLS measuring systems
,” J. Audio Eng. Soc.
42
, 219
–231
.42.
Voss
, P.
(2016
). “Auditory spatial perception without vision
,” Front. Psychol.
7
, 1
–7
.43.
Vroomen
, J.
, Bertelson
, P.
, and de Gelder
, B.
(2001
). “The ventriloquist effect does not depend on the direction of automatic visual attention
,” Percept. Psychophys.
63
, 651
–659
.44.
Watson
, D. M.
, Akeroyd
, M. A.
, Roach
, N. W.
, and Webb
, B. S.
(2019
). “Distinct mechanisms govern recalibration to audio-visual discrepancies in remote and recent history
,” Sci. Rep.
9
, 8513
.45.
Wozny
, D. R.
, and Shams
, L.
(2011
). “Recalibration of auditory space following milliseconds of cross-modal discrepancy
,” J. Neurosci.
31
, 4607
–4612
.46.
Zahorik
, P.
(2001
). “Estimating sound source distance with and without vision
,” Optom. Vis. Sci.
78
, 270
–275
.47.
Zahorik
, P.
(2003
). “Auditory and visual distance perception: The proximity image effect revisited
,” J. Acoust. Soc. Am.
113
, 2270
–2270
.48.
Zahorik
, P.
, Brungart
, D. S.
, and Bronkhorst
, A. W.
(2005
). “Auditory distance perception in humans: A summary of past and present research
,” Acta Acust. united Ac.
91
, 409
–420
.49.
Zahorik
, P.
, and Wightman
, F. L.
(2001
). “Loudness constancy with varying sound source distance
,” Nat. Neurosci.
4
, 78
–83
.50.
Zwiers
, M. P.
, Van Opstal
, A. J.
, and Paige
, G. D.
(2003
). “Plasticity in human sound localization induced by compressed spatial vision
,” Nat Neurosci.
6
, 175
–181
.© 2021 Acoustical Society of America.
2021
Acoustical Society of America
You do not currently have access to this content.
