3.5 Dominance supports slow multisensory development

[The third in a series of 3 posts re sensory integration, from my book Ready to Learn.]

Objectives:

A. Understand that during development, sensory dominance can persist in guiding

task-related behavioural responses but that sensory systems are continually

calibrating one another.

B. Understand that with typical experience, children gradually learn to integrate

suitable/available sensory inputs to optimize task responses with adult level

proficiency.

C. Appreciate that sensory dominance and calibration involve many situation- and

child-specific factors.

Early auditory dominance in children should not surprise, according to David J. Lewkowicz,

a well-known developmental psychobiologist who, as a young researcher, enjoyed frequent

collaborations with Gerald Turkewicz. In the following passage, Lewkowicz summarized

the basis for auditory dominance in children in the context of sensory development:

Lewkowicz added that auditory experience before birth prepares an infant for crucial

activities during the first few months of life, such as discriminating its mother’s voice and

launching the discrimination and mapping of speech sound contrasts. Framed this way,

auditory dominance in infancy—and the idea of early sensory dominance in general—

makes a lot of sense.

Nava and Pavani (2013) examined the developmental trajectory of auditory dominance

in groups of school-aged children. At the time of their study, the available evidence suggested

that children under the age of eight often rely on a dominant sensory modality to guide behaviour until they can consistently apply optimal (adult) levels of sensory integration. Using Colavita-type audiovisual tasks with three age groups (6–7, 9–10, and 11–12 years), Nava and Pavani found support for those earlier findings. They observed auditory dominance in the 6- to 7-year-old group and visual dominance in the 9- to 10- and 11- to 12-year-old groups. The authors claimed their findings supported the view that sensory systems sometimes dominate in early childhood because they appear and develop in turn. They also cited interpretations of the day to address why auditory dominance extends so far into childhood. According to one explanation, auditory dominance persists to advantage language development (Benasich et al., 2002). A second interpretation, repopularized by Gori (2008) and Ernst (2008), held that a child’s growth requires sensory systems to undergo continuous sensory calibration. Relying on a calibrating (dominant) sensory system to inform situational decisions is the best course until the need for calibration subsides. We will soon revisit and offer a more detailed accounting of this idea.

A well-known illusion further illuminates the developmental shift from auditory to visual dominance. The McGurk effect describes how adults watching dubbed videos can misinterpret a spoken syllable when lip movements fail to match (McGurk & MacDonald, 1976). For example, adults hearing a voice speak /ba/ while watching a face pronounce /va/ will tend to mishear the sound as /va/. They will only perceive what they see. In other variations of the illusion, an altogether new syllable is heard; for example, hearing a voice speak /ba/ while watching a face pronounce /ga/ often results in mishearing /da/. Studies have documented many variations of the McGurk effect, all of which are thought to result from an individual’s attempt to integrate audiovisual speech and the dominating influence of the visual input when uncertainty arises (see footnote 1).

It is also well documented that children are much less susceptible to the illusion. A weakened McGurk effect has been reported in children aged 4–5 years (Dupont et al., 2005), 5–9 years (Tremblay et al., 2007), and 5–14 years (Schorr et al., 2005). The pattern of results in children is consistent; dominant auditory inputs typically override the mismatched visual inputs. However, as children mature through early and middle childhood and approach puberty, an experience-based competence for attending to the inputs of multiple sensory systems gradually improves. As the ability to integrate audiovisual speech information approaches adult levels, susceptibility to the McGurk illusion grows. Tremblay and colleagues (2007) found that 5- to 9-year-old participants correctly identified spoken syllables with a mismatched visual input on approximately 60 percent of trials, whereas 10- to18-year-olds could only do so on 20 to 30 percent of trials.

Hirst and colleagues (2018b) examined the performance of school-aged children and adults on McGurk trials when the reliability of sensory inputs is varied. They tested 3- to 6-year-olds, 7- to 9-year-olds, 10- to 12-year-olds, and adults while adding a varying level of blur to visual inputs (the face videos), and a varying level of white noise to auditory inputs (the dubbed syllable recordings). The authors reported that for all age groups, the McGurk effect was less likely to occur with either higher levels of visual blur or lower levels of auditory white noise. In all situations, however, adults were more susceptible to the illusion than the 3- to 6- and 7- to 9-year-old participants. In other words, adults were more likely than the youngest children to mishear the spoken syllable and report having heard the mismatched visual stimulus. Notably, the 10- to 12-year-old children performed like the adults, supporting the view that auditory dominance in children begins a shift toward visual dominance around 9 to 10 years of age.

Slow maturing multisensory integration has also been reported for other combinations of human sensory systems. For example, Gori and colleagues (2008) compared the ability sensory calibration of adults and 5- to 10-year-olds to integrate visual and touch cues when determining the size and orientation of objects. They found that only child participants beyond the age of 8 could integrate visual and touch cues as proficiently as adults. Interestingly, child participants under the age of 8 relied on one modality or the other depending on the task. Visual cues were dominant with orientation estimations (hypothetically because the retina provides more direct and therefore more reliable information), and touch cues were dominant with object size estimations (hypothetically because fingertip positions provide more direct and therefore more reliable information).

At the time, researchers in the field did not feel they could adequately explain the slow development observed in these findings. Even though early sensory dominance made a lot of sense, multisensory integration offers unrivalled benefits, such as helping to detect sensory events that are otherwise undetectable, clarifying perceptual ambiguities, and optimizing responses with enhanced accuracy and speed. The question loomed: Why does multisensory integration develop so slowly, with some varieties remaining immature into the second decade of life?

As some early researchers had speculated, it is now widely accepted that slow multisensory development in humans is due to sensory systems needing to calibrate one another during the first decade of life. Calibration occurs when sensory systems interact in goal-oriented situations and the estimations of one sensory system are determined to be more accurate than the estimations of another. The first sensory system is deemed more reliable than the second, and it will dominate a child’s response planning for that task. The job at hand will usually get done, however childishly. But because a discrepancy was flagged, the less reliable sensory system is calibrated in the direction of the more reliable sensory system to reduce the likelihood and magnitude of future discrepancies.

But that is only the beginning of an explanation. Researchers have identified a growing number of factors that can influence situational dominance, including the child’s age,

sensory systems involved (Gori et al., 2008), and maturity (Lewkowicz, 1988); task

demands and context (Ernst, 2008; Gori et al., 2008; Nava et al., 2013; Sinnett et al.,

2008); the child’s history of sensory experience (Lewkowicz, 2010; Hillock-Dunn et al.,

2011); and even semantic factors (Doehrmann & Naumer, 2008; Hirst et al., 2018b;

Spence, 2011). There is also a well-observed regularity that vision will dominate tasks

with spatial demands, and hearing will dominate tasks with timing demands (Barnhart

et al., 2018; Gori, 2010; Nava et al., 2013). Finally, it is important to re-emphasize that

calibration will only occur in the context of goal-oriented behaviour (Gori et al., 2011). In

summary, we may confidently say sensory dominance and calibration involve the interplay

of many situation- and child-specific factors (see footnote 2).

This is one of those times it is useful to state the obvious: A child needs time to acquire

experience. Our long childhood provides sufficient time to prepare a range of human

capabilities. Mutual calibration is how sensory systems prepare for more equitable and

precise collaborations. As the latency years draw to a close, collaborating systems are winding up their large-scale calibrations. Together, they will soon lead the child’s interactions with the world by providing precise situational assessments and flexible motor responses with

near-adult proficiency. Scheller et al. (2019) recently reviewed the literature to summarize

ages of onset for adultlike multisensory integration. They found that most combinations of sensory systems are integrating at near-optimal levels between the ages of 8 and 10 years. The most obvious exception is the integration of haptic (active touch) and auditory estimates of object size that does not occur until approximately 12–14 years of age.

Until sensory systems approach acceptable levels of precision and agreement, their inputs cannot profitably integrate. In the words of Marc Ernst, an acclaimed researcher in the field of human multisensory perception, “the benefits of integration are traded for extra plasticity” (Ernst, 2008, p. R520).

However, even mature sensory systems that have learned to integrate will on occasion

revert to a previous time in their relationship when one system situationally dominated the other. For example, when an individual encounters a novel, ambiguous, or challenging situation during adolescence and beyond, the typical response is to fall back on perceptual

processes from earlier life experience. For instance, when a voice is ambiguous, we direct

our eyes to the speaker’s lips just as we did during infancy. The older building-block process

from an earlier time has become a go-to process for finding meaning in an unsure situation.

Similarly, when we are reading along and cannot visually recognize a long or challenging

word, we back up and focus on sounding it out, just as we did in early childhood. In such

situations, dominance can temporarily reappear to offer its familiar benefits once again.

Notes:

1. Other varieties of the McGurk Effect can cause a strange combination of the

mismatched inputs (hearing /ga/ and seeing /ba/ results in perceiving /gba/).

Readers can find numerous online instructional videos to provide firsthand

experience of this illusion. Prepare for a bit of a shock.

2. Even in adulthood, out-of-the-ordinary task or context demands can call on

a dominating sensory system to step aside and allow another to take the lead

for the greater sensorimotor good. For example, Loria et al. (2016) reports how

the auditory system takes the lead from vision during a rapid throwing motion

because the timing talent of hearing can more precisely determine the optimal

instant to release a projectile.