What do you think of this fact?
Close your eyes and you hear a voice saying “ba ba”. Open them, watch the lips, and suddenly you hear “da da”. Same audio file, different result. Welcome to the McGurk effect.
A lucky accident in the lab
Psychologists Harry McGurk and John MacDonald described the effect in 1976 in the journal Nature, in a paper with the lovely title “Hearing lips and seeing voices”. According to Wikipedia, they stumbled on it by accident while studying how babies perceive language. A technician was asked to dub a video with a different syllable. When they played it back, both of them heard a third sound that was neither on the audio track nor on the lips.
The Illusions Index and Scientific American describe the classic version like this: in the video, a mouth forms “ga”, the audio says “ba”, and typically you hear “da”. Your brain gets two clues that don’t match and quietly settles on a compromise.
Knowing doesn’t help
Here comes the creepy part. Scientific American points out that knowing the trick changes nothing. You can understand exactly what is going on and still hear “da”. Close your eyes and the real “ba” is back. Open them and it is gone again.
That reveals a lot about how hearing works. Speech isn’t processed by your ears alone. Your brain combines the sound with what it sees on the speaker’s face, and when the two disagree, the eyes can win.
The effect isn’t equally strong for everyone, though. According to Wikipedia’s overview of the research, people differ in how strongly they merge sound and sight. Studies have found, for example, a weaker effect in Japanese and Chinese listeners than in English-speaking ones.
Bonus fact: The effect only really disappears once your gaze is at least 60 degrees away from the speaker’s mouth.
Your ears do the listening. Your eyes still get a vote on what you hear.