Did I hear you right? McGurk and other illusions

Written by: Captain Skellett // November 4th, 2011 // Science at Home

The other day I was chatting about muddled senses. Do we really see what’s around us, or do we just assume it’s the same as yesterday and fill in the blanks. How do we understand half-uttered mumblings we don’t properly hear, and when we think we’ve understood, have we actually listened to the other person or just heard what we want to hear?

It lead us to talk about some illusions that show how intertwined and untrustworthy our senses can be.

Case one: The McGurk Effect

What you see changes how you hear. Take the sound “ba”. When an audio recording of “ba” is dubbed over a silent video of someone saying “fa” – then “fa” is what you hear.

If the video silently mouths “ga” or “da”, while playing a “ba” audio – it turns into the harder sound “da.”

Close your eyes and the effect stops. Open them and it starts right back up again. No matter how much you try to hear “ba”, the visual information overrides the audio. Check it out.

Try it with your eyes open, then watch it again with them shut. Whaaaaa???

This BBC video has more of an explanation and the ba/fa illusion.

Nice, but what are the applications? Firstly, I should move my mouth more clearly when I talk to people instead of my usual pirate mumble-slur.

Second, if speech recognition software uses video as well, it could possibly become more accurate.

Captain Skellett

I be Captain Skellett. Me blog started in April 2009 when I was working full time and didn’t get a chance to talk science. Now I have changed jobs and talk science all the time, but that doesn’t stop me blogging. More About Captain Skellett   Google


2 Responses to “Did I hear you right? McGurk and other illusions”

  1. Michael says:

    A huge field where the McGurk effect is important is the dubbing of movies – I’ve been interested in this because I noticed that people can learn to switch off the effect when they live in places where movies are routinely dubbed, so visual input is usually wrong. Also see: http://proseandpassion.blogspot.com/2009/12/huffing-and-puffing.html

    Interesting, thanks for the comment. I watch movies in other languages sometimes to learn words, and always find the dubbing disconcerting because it doesn’t match up. I hadn’t thought about that impact on the McGurk effect though. Cool.

Buy me a Beer!
    If you don't want me to mention your donation just check the box above.
  • $ 0.00
Follow @CaptainSkellett (534 followers)
Find Me Writin’s