Multisensory object perception in infancy

4-month-olds perceive a mistuned harmonic as a separate auditory and visual object

Nicholas A Smith, Nicole A. Folland, Diana M. Martinez, Laurel J. Trainor

Research output: Contribution to journalArticle

2 Citations (Scopus)

Abstract

Infants learn to use auditory and visual information to organize the sensory world into identifiable objects with particular locations. Here we use a behavioural method to examine infants’ use of harmonicity cues to auditory object perception in a multisensory context. Sounds emitted by different objects sum in the air and the auditory system must figure out which parts of the complex waveform belong to different sources (auditory objects). One important cue to this source separation is that complex tones with pitch typically contain a fundamental frequency and harmonics at integer multiples of the fundamental. Consequently, adults hear a mistuned harmonic in a complex sound as a distinct auditory object (Alain, Theunissen, Chevalier, Batty, & Taylor, 2003). Previous work by our group demonstrated that 4-month-old infants are also sensitive to this cue. They behaviourally discriminate a complex tone with a mistuned harmonic from the same complex with in-tune harmonics, and show an object-related event-related potential (ERP) electrophysiological (EEG) response to the stimulus with mistuned harmonics. In the present study we use an audiovisual procedure to investigate whether infants perceive a complex tone with an 8% mistuned harmonic as emanating from two objects, rather than merely detecting the mistuned cue. We paired in-tune and mistuned complex tones with visual displays that contained either one or two bouncing balls. Four-month-old infants showed surprise at the incongruous pairings, looking longer at the display of two balls when paired with the in-tune complex and at the display of one ball when paired with the mistuned harmonic complex. We conclude that infants use harmonicity as a cue for source separation when integrating auditory and visual information in object perception.

Original languageEnglish (US)
Pages (from-to)1-7
Number of pages7
JournalCognition
Volume164
DOIs
StatePublished - Jul 1 2017

Fingerprint

Cues
infant
Auditory Perception
Evoked Potentials
Object Perception
Infancy
Hearing
Harmonics
Air
stimulus
air
event
Group
Sound

Keywords

  • Auditory development
  • Auditory scene analysis
  • Multisensory object perception
  • Simultaneous integration

ASJC Scopus subject areas

  • Experimental and Cognitive Psychology
  • Language and Linguistics
  • Developmental and Educational Psychology
  • Linguistics and Language
  • Cognitive Neuroscience

Cite this

Multisensory object perception in infancy : 4-month-olds perceive a mistuned harmonic as a separate auditory and visual object. / Smith, Nicholas A; Folland, Nicole A.; Martinez, Diana M.; Trainor, Laurel J.

In: Cognition, Vol. 164, 01.07.2017, p. 1-7.

Research output: Contribution to journalArticle

Smith, Nicholas A ; Folland, Nicole A. ; Martinez, Diana M. ; Trainor, Laurel J. / Multisensory object perception in infancy : 4-month-olds perceive a mistuned harmonic as a separate auditory and visual object. In: Cognition. 2017 ; Vol. 164. pp. 1-7.
@article{b8fe4a0bb00848a39fd36a37efe44170,
title = "Multisensory object perception in infancy: 4-month-olds perceive a mistuned harmonic as a separate auditory and visual object",
abstract = "Infants learn to use auditory and visual information to organize the sensory world into identifiable objects with particular locations. Here we use a behavioural method to examine infants’ use of harmonicity cues to auditory object perception in a multisensory context. Sounds emitted by different objects sum in the air and the auditory system must figure out which parts of the complex waveform belong to different sources (auditory objects). One important cue to this source separation is that complex tones with pitch typically contain a fundamental frequency and harmonics at integer multiples of the fundamental. Consequently, adults hear a mistuned harmonic in a complex sound as a distinct auditory object (Alain, Theunissen, Chevalier, Batty, & Taylor, 2003). Previous work by our group demonstrated that 4-month-old infants are also sensitive to this cue. They behaviourally discriminate a complex tone with a mistuned harmonic from the same complex with in-tune harmonics, and show an object-related event-related potential (ERP) electrophysiological (EEG) response to the stimulus with mistuned harmonics. In the present study we use an audiovisual procedure to investigate whether infants perceive a complex tone with an 8{\%} mistuned harmonic as emanating from two objects, rather than merely detecting the mistuned cue. We paired in-tune and mistuned complex tones with visual displays that contained either one or two bouncing balls. Four-month-old infants showed surprise at the incongruous pairings, looking longer at the display of two balls when paired with the in-tune complex and at the display of one ball when paired with the mistuned harmonic complex. We conclude that infants use harmonicity as a cue for source separation when integrating auditory and visual information in object perception.",
keywords = "Auditory development, Auditory scene analysis, Multisensory object perception, Simultaneous integration",
author = "Smith, {Nicholas A} and Folland, {Nicole A.} and Martinez, {Diana M.} and Trainor, {Laurel J.}",
year = "2017",
month = "7",
day = "1",
doi = "10.1016/j.cognition.2017.01.016",
language = "English (US)",
volume = "164",
pages = "1--7",
journal = "Cognition",
issn = "0010-0277",
publisher = "Elsevier",

}

TY - JOUR

T1 - Multisensory object perception in infancy

T2 - 4-month-olds perceive a mistuned harmonic as a separate auditory and visual object

AU - Smith, Nicholas A

AU - Folland, Nicole A.

AU - Martinez, Diana M.

AU - Trainor, Laurel J.

PY - 2017/7/1

Y1 - 2017/7/1

N2 - Infants learn to use auditory and visual information to organize the sensory world into identifiable objects with particular locations. Here we use a behavioural method to examine infants’ use of harmonicity cues to auditory object perception in a multisensory context. Sounds emitted by different objects sum in the air and the auditory system must figure out which parts of the complex waveform belong to different sources (auditory objects). One important cue to this source separation is that complex tones with pitch typically contain a fundamental frequency and harmonics at integer multiples of the fundamental. Consequently, adults hear a mistuned harmonic in a complex sound as a distinct auditory object (Alain, Theunissen, Chevalier, Batty, & Taylor, 2003). Previous work by our group demonstrated that 4-month-old infants are also sensitive to this cue. They behaviourally discriminate a complex tone with a mistuned harmonic from the same complex with in-tune harmonics, and show an object-related event-related potential (ERP) electrophysiological (EEG) response to the stimulus with mistuned harmonics. In the present study we use an audiovisual procedure to investigate whether infants perceive a complex tone with an 8% mistuned harmonic as emanating from two objects, rather than merely detecting the mistuned cue. We paired in-tune and mistuned complex tones with visual displays that contained either one or two bouncing balls. Four-month-old infants showed surprise at the incongruous pairings, looking longer at the display of two balls when paired with the in-tune complex and at the display of one ball when paired with the mistuned harmonic complex. We conclude that infants use harmonicity as a cue for source separation when integrating auditory and visual information in object perception.

AB - Infants learn to use auditory and visual information to organize the sensory world into identifiable objects with particular locations. Here we use a behavioural method to examine infants’ use of harmonicity cues to auditory object perception in a multisensory context. Sounds emitted by different objects sum in the air and the auditory system must figure out which parts of the complex waveform belong to different sources (auditory objects). One important cue to this source separation is that complex tones with pitch typically contain a fundamental frequency and harmonics at integer multiples of the fundamental. Consequently, adults hear a mistuned harmonic in a complex sound as a distinct auditory object (Alain, Theunissen, Chevalier, Batty, & Taylor, 2003). Previous work by our group demonstrated that 4-month-old infants are also sensitive to this cue. They behaviourally discriminate a complex tone with a mistuned harmonic from the same complex with in-tune harmonics, and show an object-related event-related potential (ERP) electrophysiological (EEG) response to the stimulus with mistuned harmonics. In the present study we use an audiovisual procedure to investigate whether infants perceive a complex tone with an 8% mistuned harmonic as emanating from two objects, rather than merely detecting the mistuned cue. We paired in-tune and mistuned complex tones with visual displays that contained either one or two bouncing balls. Four-month-old infants showed surprise at the incongruous pairings, looking longer at the display of two balls when paired with the in-tune complex and at the display of one ball when paired with the mistuned harmonic complex. We conclude that infants use harmonicity as a cue for source separation when integrating auditory and visual information in object perception.

KW - Auditory development

KW - Auditory scene analysis

KW - Multisensory object perception

KW - Simultaneous integration

UR - http://www.scopus.com/inward/record.url?scp=85016020616&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85016020616&partnerID=8YFLogxK

U2 - 10.1016/j.cognition.2017.01.016

DO - 10.1016/j.cognition.2017.01.016

M3 - Article

VL - 164

SP - 1

EP - 7

JO - Cognition

JF - Cognition

SN - 0010-0277

ER -