Select Page

The other day I was chatting about muddled senses. Do we really see what’s around us, or do we just assume it’s the same as yesterday and fill in the blanks. How do we understand half-uttered mumblings we don’t properly hear, and when we think we’ve understood, have we actually listened to the other person or just heard what we want to hear?

It lead us to talk about some illusions that show how intertwined and untrustworthy our senses can be.

Case one: The McGurk Effect

What you see changes how you hear. Take the sound “ba”. When an audio recording of “ba” is dubbed over a silent video of someone saying “fa” – then “fa” is what you hear.

If the video silently mouths “ga” or “da”, while playing a “ba” audio – it turns into the harder sound “da.”

Close your eyes and the effect stops. Open them and it starts right back up again. No matter how much you try to hear “ba”, the visual information overrides the audio. Check it out.

Try it with your eyes open, then watch it again with them shut. Whaaaaa???

This BBC video has more of an explanation and the ba/fa illusion.

Nice, but what are the applications? Firstly, I should move my mouth more clearly when I talk to people instead of my usual pirate mumble-slur.

Second, if speech recognition software uses video as well, it could possibly become more accurate.