Auditory "capture" of visual motion

Hyungjun Kim, Brian Pasieka, Mark McCourt

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

It has recently been shown that aspects of visual perception can be significantly altered by auditory stimuli. We asked whether the perceived direction of visual motion could be affected by auditory signals. The visual stimulus was a spatiotemporally vignetted sinewave grating (0.5 cpd, 5% contrast). Multiple frames of this stimulus were combined into short movies. Each frame lasted 125 ms, and each movie consisted of 3 frames. Between successive frames the spatial phase of the grating was altered, with nine phases sampled, ranging from 90 deg to 270 deg in increments of 18 deg. To investigate the influence of auditory signals on visual motion we paired the presentation of the movies with auditory stimuli. The auditory stimulus was a binaural amplitude-gated burst of pink noise. The total duration of the noisebursts coincided with the duration of the motion movie. The relative onset, temporal phase and amplitude of the noisebursts were controlled to create a virtual soundsource whose location and motion in 3D auditory space could be manipulated. In the rightward sound condition soundsource onset occurred at a point 2 virtual meters to the left of the head and moved at a constant velocity to a location 2 virtual meters to the right of the head. The opposite occurred in the leftward sound condition. In the no motion control condition the two ears received the same toneburst, and the sound appeared stationary and centered within the head. There was also a no sound control condition. On each trial observers were instructed to ignore the auditory stimulus and to judge the direction of visual motion. The perceived direction of visual motion was strongly influenced by the direction of concurrent auditory motion. Auditory motion not only captured ambiguous visual motion, but could even reverse the perceived direction of unambiguous visual motion. These results suggest that visual motion, a dynamic and spatiotemporal aspect of visual perception, can be altered by sound.

Original languageEnglish (US)
Pages (from-to)777a
JournalJournal of vision
Volume3
Issue number9
DOIs
StatePublished - Dec 1 2003

Fingerprint

Motion Pictures
Visual Perception
Head
Ear
Direction compound

ASJC Scopus subject areas

  • Ophthalmology
  • Sensory Systems

Cite this

Auditory "capture" of visual motion. / Kim, Hyungjun; Pasieka, Brian; McCourt, Mark.

In: Journal of vision, Vol. 3, No. 9, 01.12.2003, p. 777a.

Research output: Contribution to journalArticle

Kim, Hyungjun ; Pasieka, Brian ; McCourt, Mark. / Auditory "capture" of visual motion. In: Journal of vision. 2003 ; Vol. 3, No. 9. pp. 777a.
@article{b57b629727e04f01b30e249a085dec6e,
title = "Auditory {"}capture{"} of visual motion",
abstract = "It has recently been shown that aspects of visual perception can be significantly altered by auditory stimuli. We asked whether the perceived direction of visual motion could be affected by auditory signals. The visual stimulus was a spatiotemporally vignetted sinewave grating (0.5 cpd, 5{\%} contrast). Multiple frames of this stimulus were combined into short movies. Each frame lasted 125 ms, and each movie consisted of 3 frames. Between successive frames the spatial phase of the grating was altered, with nine phases sampled, ranging from 90 deg to 270 deg in increments of 18 deg. To investigate the influence of auditory signals on visual motion we paired the presentation of the movies with auditory stimuli. The auditory stimulus was a binaural amplitude-gated burst of pink noise. The total duration of the noisebursts coincided with the duration of the motion movie. The relative onset, temporal phase and amplitude of the noisebursts were controlled to create a virtual soundsource whose location and motion in 3D auditory space could be manipulated. In the rightward sound condition soundsource onset occurred at a point 2 virtual meters to the left of the head and moved at a constant velocity to a location 2 virtual meters to the right of the head. The opposite occurred in the leftward sound condition. In the no motion control condition the two ears received the same toneburst, and the sound appeared stationary and centered within the head. There was also a no sound control condition. On each trial observers were instructed to ignore the auditory stimulus and to judge the direction of visual motion. The perceived direction of visual motion was strongly influenced by the direction of concurrent auditory motion. Auditory motion not only captured ambiguous visual motion, but could even reverse the perceived direction of unambiguous visual motion. These results suggest that visual motion, a dynamic and spatiotemporal aspect of visual perception, can be altered by sound.",
author = "Hyungjun Kim and Brian Pasieka and Mark McCourt",
year = "2003",
month = "12",
day = "1",
doi = "10.1167/3.9.777",
language = "English (US)",
volume = "3",
pages = "777a",
journal = "Journal of Vision",
issn = "1534-7362",
publisher = "Association for Research in Vision and Ophthalmology Inc.",
number = "9",

}

TY - JOUR

T1 - Auditory "capture" of visual motion

AU - Kim, Hyungjun

AU - Pasieka, Brian

AU - McCourt, Mark

PY - 2003/12/1

Y1 - 2003/12/1

N2 - It has recently been shown that aspects of visual perception can be significantly altered by auditory stimuli. We asked whether the perceived direction of visual motion could be affected by auditory signals. The visual stimulus was a spatiotemporally vignetted sinewave grating (0.5 cpd, 5% contrast). Multiple frames of this stimulus were combined into short movies. Each frame lasted 125 ms, and each movie consisted of 3 frames. Between successive frames the spatial phase of the grating was altered, with nine phases sampled, ranging from 90 deg to 270 deg in increments of 18 deg. To investigate the influence of auditory signals on visual motion we paired the presentation of the movies with auditory stimuli. The auditory stimulus was a binaural amplitude-gated burst of pink noise. The total duration of the noisebursts coincided with the duration of the motion movie. The relative onset, temporal phase and amplitude of the noisebursts were controlled to create a virtual soundsource whose location and motion in 3D auditory space could be manipulated. In the rightward sound condition soundsource onset occurred at a point 2 virtual meters to the left of the head and moved at a constant velocity to a location 2 virtual meters to the right of the head. The opposite occurred in the leftward sound condition. In the no motion control condition the two ears received the same toneburst, and the sound appeared stationary and centered within the head. There was also a no sound control condition. On each trial observers were instructed to ignore the auditory stimulus and to judge the direction of visual motion. The perceived direction of visual motion was strongly influenced by the direction of concurrent auditory motion. Auditory motion not only captured ambiguous visual motion, but could even reverse the perceived direction of unambiguous visual motion. These results suggest that visual motion, a dynamic and spatiotemporal aspect of visual perception, can be altered by sound.

AB - It has recently been shown that aspects of visual perception can be significantly altered by auditory stimuli. We asked whether the perceived direction of visual motion could be affected by auditory signals. The visual stimulus was a spatiotemporally vignetted sinewave grating (0.5 cpd, 5% contrast). Multiple frames of this stimulus were combined into short movies. Each frame lasted 125 ms, and each movie consisted of 3 frames. Between successive frames the spatial phase of the grating was altered, with nine phases sampled, ranging from 90 deg to 270 deg in increments of 18 deg. To investigate the influence of auditory signals on visual motion we paired the presentation of the movies with auditory stimuli. The auditory stimulus was a binaural amplitude-gated burst of pink noise. The total duration of the noisebursts coincided with the duration of the motion movie. The relative onset, temporal phase and amplitude of the noisebursts were controlled to create a virtual soundsource whose location and motion in 3D auditory space could be manipulated. In the rightward sound condition soundsource onset occurred at a point 2 virtual meters to the left of the head and moved at a constant velocity to a location 2 virtual meters to the right of the head. The opposite occurred in the leftward sound condition. In the no motion control condition the two ears received the same toneburst, and the sound appeared stationary and centered within the head. There was also a no sound control condition. On each trial observers were instructed to ignore the auditory stimulus and to judge the direction of visual motion. The perceived direction of visual motion was strongly influenced by the direction of concurrent auditory motion. Auditory motion not only captured ambiguous visual motion, but could even reverse the perceived direction of unambiguous visual motion. These results suggest that visual motion, a dynamic and spatiotemporal aspect of visual perception, can be altered by sound.

UR - http://www.scopus.com/inward/record.url?scp=4243055026&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=4243055026&partnerID=8YFLogxK

U2 - 10.1167/3.9.777

DO - 10.1167/3.9.777

M3 - Article

AN - SCOPUS:4243055026

VL - 3

SP - 777a

JO - Journal of Vision

JF - Journal of Vision

SN - 1534-7362

IS - 9

ER -