– by Farhana Bokth

While it may be perceived by many that speech perception is typically auditory, it has been found contrary to popular belief, that visuals are just as important as audio. This multi-modal phenomenon is specifically known as the McGurk Effect. Psychologist McGurk and his assistant MacDonald accidentally discovered this illusion when the sound of one phoneme was dubbed onto a video voicing a different phoneme. For example, if /ba-ba/ is spoken over the lip movements of /ga-ga/, the perception is of /da-da/.

So why exactly does this happen? Well, when the brain receives two contradictory flows of information, one through the ears and one through the eyes, it has to decide which to interpret more. When this happens, the brain can either, combine the two perceptions and essentially create a mix of the two, or on the other hand, ignore the auditory information altogether and process just the visual information alone. So, when viewing a video of someone saying <far> while hearing audio of them saying <bar> may lead to perceiving the sound as <bar>. Our processing of information is immediate, automatic, and, to a large degree, unconscious. It all happens so quickly and mechanically through the brain’s effort to provide its best guess about the incoming information. Points for effort brain!

The McGurk effect is not though, equal for everyone:

❖ Individuals who are right handed are more likely to experience the McGurk response from the face having privileged access to the right hemisphere and words to the left hemisphere.

❖  In people with lesions to the left hemisphere of the brain, show a greater McGurk effect than normal controls. 

❖ There is a McGurk effect exhibited in people with damage to the right hemisphere of the brain but the effect is not as strong as a normal group. 

❖ Research has shown that the McGurk effect is much more prominent in English listeners compared to Japanese listeners. Although, this may be due to Japanese culture tending to have politeness and avoiding direct eye or face contact when interacting

There are also many variables which can vary regarding the degree to which the McGurk effect is experienced. This can be from your gender, your first language, to even whether you recognise the person you’re looking at. But, unlike normal optical illusions, the McGurk effect is robust, it works even after you realise what is going on!

For more information on the McGurk Effect, you can take a look at McGurk’s original 1976 paper titled ‘Hearing Lips and Seeing Voices’.