When presented with a spatially discordant auditory-visual stimulus, subjects sometimes perceive the sound and the visual stimuli as coming from the same location. Such a phenomenon is often referred to as perceptual fusion or ventriloquism, as it evokes the illusion created by a ventriloquist when his voice seems to emanate from his puppet rather than from his mouth. While this effect has been extensively examined in the horizontal plane and to a lesser extent in distance, few psychoacoustic studies have focused on elevation. In the present experiment, sequences of a man talking were presented to subjects. His voice could be reproduced on different loudspeakers, which created disparities in both azimuth and elevation between the sound and the visual stimuli. For each presentation, subjects had to indicate whether the voice seemed to emanate from the mouth of the actor or not. Results showed that ventriloquism could be observed with larger audiovisual disparities in elevation than in azimuth.
References
1.
Agganis
, B. T.
, Muday
, J. A.
, and Schirillo
, J. A.
(2010
). “Visual biasing of auditory localization in azimuth and depth
,” Percept. Mot. Skills
111
, 872
–892
.2.
Alais
, D.
, and Burr
, D.
(2004
). “The ventriloquist effect results from near-optimal bimodal integration
,” Curr. Biol.
14
, 257
–262
.3.
André
, C.
, Corteel
, E.
, Embrechts
, J.-J.
, Verly
, J.
, and Katz
, B. F. G.
(2014
). “Subjective evaluation of the audiovisual spatial congruence in the case of stereoscopic-3D video and wave field synthesis
,” Int. J. Hum.-Comput. St.
72
, 23
–32
.4.
Ashmead
, D. H.
, Leroy
, D.
, and Odom
, R. D.
(1990
). “Perception of the relative distances of nearby sound sources
,” Percept. Psychophys.
47
, 326
–331
.5.
Battaglia
, P.
, Jacobs
, R.
, and Aslin
, R.
(2003
). “Bayesian integration of visual and auditory signals for spatial localization
,” J. Opt. Soc. Am.
20
, 1391
–1396
.6.
Bermant
, R. I.
, and Welch
, R. B.
(1976
). “Cross-modal bias and perceptual fusion with auditory-visual spatial discordance
,” Percept. Mot. Skills
43
, 487
–493
.7.
Bertelson
, P.
, and Aschersleben
, G.
(1998
). “Bayesian integration of visual and auditory signals for spatial localization
,” Psychon. Bull. Rev.
5
, 482
–489
.8.
Bertelson
, P.
, and Radeau
, M.
(1981
). “Cross-modal bias and perceptual fusion with auditory-visual spatial discordance
,” Percept. Psychophys.
29
, 578
–584
.9.
Bolia
, R. S.
, Nelson
, W. T.
, Ericson
, M. A.
, and Simpson
, B. D.
(2000
). “A speech corpus for multitalker communications research
,” J. Acoust. Soc. Am.
107
, 1065
–1066
.10.
Bowen
, A. L.
, Ramachandran
, R.
, Muday
, J. A.
, and Schirillo
, J. A.
(2011
). “Visual signals bias auditory targets in azimuth and depth
,” Exp. Brain. Res.
214
, 403
–414
.11.
Bruijn
, W. P. d.
, and Boone
, M. M.
(2002
). “Subjective experiments on the effects of combining spatialized audio and 2D video projection in audio-visual systems
,” in Proceedings of the 112th Convention of the Audio Engineering Society
, paper no. 5582.12.
Cavonius
, C.
, and Robbins
, D.
(1973
). “Relationships between luminance and visual acuity in the rhesus monkey
,” J. Physiol.
232
, 239
–246
.13.
Choe
, C. S.
, Welch
, R. B.
, Gilford
, R. M.
, and Juola
, J. F.
(1975
). “The ‘ventriloquist effect’: Visual dominance or response bias?
,” Percept. Psychophys.
18
, 55
–60
.14.
Ernst
, M. O.
, and Banks
, M. S.
(2002
). “Humans integrate visual and haptic information in a statistically optimal fashion
,” Nature
415
, 429
–433
.15.
Gardner
, M. B.
(1968
). “Proximity image effect in sound localization
,” J. Acoust. Soc. Am.
43
, 163
.16.
Giard
, M. H.
, and Peronet
, F.
(1999
). “Auditory-visual integration during multimodal object recognition in humans: A behavioral and electrophysiological study
,” J. Cogn. Neurosci.
11
, 473
–490
.17.
Hairston
, W.
, Wallace
, M.
, Vaughan
, J.
, Stein
, B.
, Norris
, J.
, and Schirillo
, J.
(2003
). “Visual localization ability influences cross-modal bias
,” J. Cogn. Neurosci.
15
, 20
–29
.18.
Hládek
, L.
, Le Dantec
, C. C.
, Kopco
, N.
, and Seitz
, A.
(2013
). “Ventriloquism effect and aftereffect in the distance dimension
,” Proc. Meet. Acoust.
19
, 050042
.19.
Ijsselsteijn
, W.
, Ridder
, H. d.
, Freeman
, J.
, Avons
, S. E.
, and Bouwhuis
, D.
(2001
). “Determination of optimal data placement for psychometric function estimation: A computer simulation
,” Presence-Teleop. Virt.
10
, 298
–311
.20.
ITU-R BS. 1284-1
(2003
). “General methods for the subjective assessment of sound quality
,” International Telecommunications Union.21.
Jack
, C. E.
, and Thurlow
, W. R.
(1973
). “Effects of degree of visual association and angle of displacement on the ‘ventriloquism’ effect
,” Percept. Mot. Skills
37
, 967
–979
.22.
Jackson
, C. V.
(1953
). “Visual factors in auditory localization
,” Q. J. Exp. Psychol.
5
, 52
–65
.23.
Knudsen
, E. I.
, and Konishi
, M.
(1979
). “Mechanisms of sound localization in the barn owl (Tyto alba)
,” J. Comp. Physiol.
133
, 13
–21
.24.
Komiyama
, S.
(1989
). “Subjective evaluation of angular displacement between picture and sound directions for HDTV sound systems
,” J. Audio Eng. Soc.
37
, 210
–214
.25.
Lam
, C. F.
, Dubno
, J. R.
, and Mills
, J. H.
(1999
). “Determination of optimal data placement for psychometric function estimation: A computer simulation
,” J. Acoust. Soc. Am.
106
, 1969
–1976
.26.
Levitt
, H.
(1971
). “Transformed up-down methods in psychoacoustics
,” J. Acoust. Soc. Am.
49
, 467
–477
.27.
Lewald
, J.
, Ehrenstein
, W.
, and Guski
, R.
(2001
). “Spatio-temporal constraints for auditory-visual integration
,” Behav. Brain Res.
121
, 69
–79
.28.
Makous
, J. C.
, and Middlebrooks
, J. C.
(1990
). “Two-dimensional sound localization by human listeners
,” J. Acoust. Soc. Am.
87
, 2188
–2200
.29.
Mannerheim
, P.
(2011
). “Spatial sound and stereoscopic vision
,” in Proceedings of the 130th Convention of the Audio Engineering Society
, paper no. 8424.30.
Martin
, R. L.
, McAnally
, K. I.
, Bolia
, R. S.
, Eberle
, G.
, and Brungart
, D. S.
(2012
). “Spatial release from speech-on-speech masking in the median sagittal plane
,” J. Acoust. Soc. Am.
131
, 378
–385
.31.
Melchior
, F.
, Brix
, S.
, Sporer
, T.
, Roder
, T.
, and Klehs
, B.
(2003
). “Wave field synthesis in combination with 2D video projection
,” in Proceedings of the 24th International Conference of the Audio Engineering Society
, paper no. 47.32.
Melchior
, F.
, Fischer
, J.
, and Vries
, D. d.
(2006
). “Audiovisual perception using Wave Field Synthesis in combination with augmented reality systems: Horizontal positioning
,” in Proceedings of the 28th International Conference of the Audio Engineering Society
, paper no. 3–2.33.
Mershon
, D. H.
, Desaulniers
, D. H.
, Amerson
, T. L.
, and Kiefer
, S. A.
(1980
). “Visual capture in auditory distance perception: Proximity image effect reconsidered
,” J. Aud. Res.
20
, 129
–136
.34.
Middlebrooks
, J. C.
, Makous
, J.
, and Green
, D. M.
(1989
). “Directional sensitivity of sound-pressure levels in the human ear canal
,” J. Acoust. Soc. Am.
86
, 89
–108
.35.
Pick
, H. L.
, Warren
, D. H.
, and Hay
, J. C.
(1969
). “Sensory conflict in judgments of spatial direction
,” Percept. Psychophys.
6
, 203
–205
.36.
Radeau
, M.
(1974
). “Adaptation au déplacement prismatique sur la base d’une discordance entre la vision et l’audition” (“Adaptation prismatic displacement based on a discrepancy between vision and hearing”)
, L’Année Psychologique
74
, 23
–24
.37.
Radeau
, M.
, and Bertelson
, P.
(1976
). “The effect of a textured visual field on modality dominance in a ventriloquism situation
,” Percept. Psychophys.
20
, 227
–235
.38.
Radeau
, M.
, and Bertelson
, P.
(1977
). “Adaptation to auditory-visual discordance and ventriloquism in semirealistic situations
,” Percept. Psychophys.
22
, 137
–146
.39.
Recanzone
, G. H.
, Makhamra
, S. D.
, and Guard
, D. C.
(1998
). “Comparison of relative and absolute sound localization ability in humans
,” J. Acoust. Soc. Am.
103
, 1085
–1097
.40.
Thurlow
, W. R.
, and Jack
, C. E.
(1973
). “Certain determinants of the ‘ventriloquism effect,’
” Percept. Mot. Skills
36
, 1171
–1184
.41.
Vatakis
, A.
, and Spence
, C.
(2007
). “Crossmodal binding: Evaluating the unity assumption using audiovisual speech stimuli
,” Percep. Psychophys.
69
, 744
–756
.42.
Wallace
, M.
, Roberson
, G.
, Hairston
, W.
, Stein
, B.
, Vaughan
, J.
, and Schirillo
, J.
(2004
). “Unifying multisensory signals across time and space
,” Exp. Brain. Res.
158
, 252
–258
.43.
Warren
, D. H.
(1979
). “Spatial localization under conflict conditions: Is there a single explanation?
,” Percept.
8
, 323
–337
.44.
Warren
, D. H.
, Welch
, R. B.
, and McCarthy
, T. J.
(1981
). “The role of visual-auditory compellingness in the ventriloquism effect: Implications for transitivity among the spatial senses
,” Percept. Psychophys.
30
, 557
–564
.45.
Weerts
, T. C.
, and Thurlow
, W. R.
(1971
). “The effect of eye position and expectation on sound localization
,” Percept. Psychophys.
9
, 35
–39
.46.
Welch
, R. B.
(1999
). “Meaning, attention, and the ‘unity assumption’ in the intersensory bias of spatial and temporal perceptions
,” Adv. Psychol.
129
, 371
–387
.47.
Welch
, R. B.
, and Warren
, D. H.
(1980
). “Immediate perceptual response to intersensory discrepancy
,” Psychol. Bull.
88
, 638
.48.
Werner
, S.
, Liebetrau
, J.
, and Sporer
, T.
(2013
). “Vertical sound source localization influenced by visual stimuli
,” Signal Process. Res.
2
, 29
–38
.49.
Zahorik
, P.
(2003
). “Auditory and visual distance perception: The proximity-image effect revisited
,” J. Acoust. Soc. Am.
113
, 2270
–2270
.50.
Zchaluk
, K.
, and Foster
, D. H.
(2009
). “Model-free estimation of the psychometric function
,” Percept. Psychophys.
71
, 1414
–1425
.© 2015 Acoustical Society of America.
2015
Acoustical Society of America
You do not currently have access to this content.