Hearing lips and seeing voices

[Permalink]

I heard “Bah” the first round, then “Vah”. The second time they ran the clips I heard “Vah” for both cycles.

We have noticed how much easier it is to understand people when we can see them at my workplace; whenever we work with groups that have people with heavy accents, we try to use the video conference. We find it is easier to understand them. Similarly, I am learning a portion of Hebrew right now and I videotape my cantor reading the Hebrew; it makes it a lot easier to get the pronunciation right.

1 Like

I discovered the truth of this in an amusing way. I was doing some public speaking at a time when I wore a beard. Spring came and I shaved it off… and the next time I spoke the old folks could “hear” me 100% better.

4 Likes

Boing boing really loves them some McGurk affect:

McGurk Affect on boingboing

1 Like

Buck me! That was cool.

2 Likes

It doesn’t work for me at all.
I paid very close attention the the video and saw the difference but it sounded 100% the same. I guess I don’t connect mouth movement to speech too closely, which could explain my slight difficulty in understanding actors speak in movies.
I’d like to see a study about how many people hear a difference!

1 Like

I’d like to see a study on “normal” people vs. Asperger’s / autistic / ADHD / OCD people. I heard exactly the same thing both times, and I’m OCD, with strong autistic tendencies.

1 Like

Didn’t work for me at all either.
This one on the other hand, works perfectly.

2 Likes

During this video, the word Faberge kept popping into my head.

When I took Intro to Psychology, way back in the late 80s, our professor talked about this (I didn’t remember that it’s called the McGurk Effect). He showed us an old, grainy video of someone intoning (different) syllables but the lips didn’t match (one at a time, every few seconds… “Ma…” “Ba…” “Ga…”).  What I remember is that the guy in the video vaguely looked like Jimmy Swaggart and had this kindly look on his face. There wasn’t any accompanying narration (as with this video), just straight-up McGurk Effect. The professor suggested that it would be great to leave that video running during a party.

1 Like

As anyone with hearing loss can tell you we all read lips. Try this, turn down the volume on the television news until it is hard to make out what they are saying. Then notice how it is easier to hear when the anchors faces are visible as apposed to someone talking off camera.

McGurk is a witch!!!

it is impossible for me not to associate this with the rapper Ol’ Dirty Bastard, who went by the alias Dirt McGurk. Or maybe it was McGirt? Understanding wtf he was saying is damn near impossible, which of course serves to re-enforce my mental association between him and the McGurk effect.

not sure if seeing his mouth would actually help, though.

If you are looking for the text to the latin song “Ó Four Tuna” in youtube video above, here it is:

When I see a Japanese person answer the phone in a movie I can never tell if they’re saying ‘wushi wushi’ or ‘mushi mushi,’ even if I do watch their lips.

I prefer this version…

O Fortuna, Apotheosis

1 Like

This effect is a great example of the brain’s effort to make sense out of conflicting sensory input. In this case, visual over-rides auditory. Other examples of this general (but fundamental) function include seeing faces in random images (or Jesus in burnt toast, depending on your predispositions…) or hearing voices or music in random noise.

This phenomenon also calls into serious question the general validity of personal experiences or testimony. In other words, the fact that you indisputably, undoubtedly heard “FAH” does not mean that “FAH” was spoken.

These studies have been done. It has been shown that the left superior temporal sulcus region of the brain integrates auditory and visual input. This area is less active in individuals who do not perceive the McGurk effect, which includes people with Autism or damage to that region (e.g., stroke).

1 Like

This topic was automatically closed after 5 days. New replies are no longer allowed.