Dissociation of perception and action in audiovisual multisensory integration

Lynnette M. Leone, Mark E. Mccourt

Research output: Contribution to journalArticle

10 Scopus citations

Abstract

The 'temporal rule' of multisensory integration (MI) proposes that unisensory stimuli, and the neuronal responses they evoke, must fall within a window of integration. Ecological validity demands that MI should occur only for physically simultaneous events (which may give rise to non-simultaneous neural activations), and spurious neural response simultaneities unrelated to environmental multisensory occurrences must somehow be rejected. Two experiments investigated the requirements of simultaneity for facilitative MI. Experiment 1 employed an reaction time (RT)/race model paradigm to measure audiovisual (AV) MI as a function of AV stimulus-onset asynchrony (SOA) under fully dark adapted conditions for visual stimuli that were either rod- or cone-isolating. Auditory stimulus intensity was constant. Despite a 155-ms delay in mean RT to the scotopic vs. photopic stimulus, facilitative AV MI in both conditions occurred exclusively at an AV SOA of 0 ms. Thus, facilitative MI demands both physical and physiological simultaneity. Experiment 2 investigated the accuracy of simultaneity and temporal order judgements under the same stimulus conditions. Judgements of AV stimulus simultaneity or temporal order were significantly influenced by stimulus intensity, indicating different simultaneity requirements for these tasks. The possibility was considered that there are mechanisms by which the nervous system may take account of variations in response latency arising from changes in stimulus intensity in order to selectively integrate only those physiological simultaneities that arise from physical simultaneities. It was proposed that separate subsystems for AV MI exist that pertain to action and perception. Two experiments investigated the requirements of simultaneity for audiovisual multisensory integration (AVMI). Experiment 1 employed an RT/race model paradigm to measure AVMI as a function of AV stimulus onset asynchrony under fully dark adapted conditions for visual stimuli that were either rod- or cone-isolating. AVMI in both conditions occurred exclusively at an audiovisual stimulus onset asynchrony of 0 ms, showing that multisensory integration demands both physical and physiological simultaneity. Experiment 2 investigated the accuracy of simultaneity and temporal order judgments under the same stimulus conditions. Judgments of AV stimulus simultaneity or temporal order were significantly influenced by stimulus intensity indicating different simultaneity requirements for these tasks. We propose that separate subsystems for audiovisual multisensory integration exist which pertain to action and perception.

Original languageEnglish (US)
Pages (from-to)2915-2922
Number of pages8
JournalEuropean Journal of Neuroscience
Volume42
Issue number11
DOIs
Publication statusPublished - Dec 1 2015

    Fingerprint

Keywords

  • Audiovisual
  • Multisensory integration
  • Race model
  • Reaction time
  • Simultaneity judgment
  • Temporal order judgment

ASJC Scopus subject areas

  • Neuroscience(all)

Cite this