Monkey See, Monkey Speak:
Facial Expressions as a Guide to Speech

By HENRY FOUNTAIN

Dr. Asif A. Ghazanfar

The monkey on the left is cooing. The one on the right is making a threatening sound. But you knew that - or you would if you were a rhesus monkey listening to a tape of the sounds that go with the expressions, researchers found.

Scientists have long known that there is more to speech perception than meets the ear. Humans, even infants who cannot yet speak, pick up visual cues from the movement of the lips and other parts of the face to help understand what it is they are hearing.

Now there is evidence that this ability may go back a long way. Researchers in Germany say they have found that rhesus monkeys can also combine visual and auditory information to perceive vocal signals, suggesting that the ability had some kind of evolutionary precursor before humans and other primates diverged millions of years ago.

The researchers, Dr. Asif A. Ghazanfar and Dr. Nikos K. Logothetis of the Max Planck Institute for Biological Cybernetics in Tübingen, tested monkeys by showing them side-by-side video images of another monkey making two sounds, while playing a soundtrack of just one of the sounds. Time after time, the test monkeys were able to match the sound with the correct video image. The research was reported in the current issue of the journal Nature.

"We know that for human speech perception, when you communicate one on one with someone, you glean a lot of information not only from sound, but from facial expressions," Dr. Ghazanfar said. But with other animals, he said, "we didn't know if perceivers use that multimodal means of perception as well."

The experiment is very similar to the kind that has been done with infants as young as 18 to 20 weeks to show that even at a prelinguistic age, they use facial expressions as an aid to perceiving speech.

But Dr. Ghazanfar said such studies had not been done on other animals, particularly closely related primates.

"In many ways it's kind of an obvious experiment to do," he said, and rhesus monkeys are an ideal subject because they have evolved a set of very elaborate vocal and facial expressions. But one reason it had not been done before, he said, is that it is difficult to get the right kind of videotaped images, free of body language or other visual activity that might skew the results.

By videotaping the animals in a standard primate restraint chair, Dr. Ghazanfar got good images of their two most common calls - a friendly, long and tonal coo, and a short, pulsed and noisy threat call.

The two calls have the advantage of being from different social contexts, and are different from a visual standpoint, too. The coo is formed with the mouth open slightly and the lips protruded; with the threat call, by contrast, the lips appear normal and the mouth is open wider. The researchers edited the video images to two-second loops each, and synchronized the sound with the beginning of them.

Dr. Ghazanfar, who describes himself as a neuro-ecologist, said he wanted to study the monkeys' behavior further to gain more understanding of the precise cues they use.

"You might imagine morphing the sounds together and then feature by feature asking at what point do monkeys no longer make a match," he said. Conversely, he might manipulate the video to make hybrid images, then alter the degree of mouth opening or lip-pursing to determine how variations in the visual cues make a difference.

Beyond that, Dr. Ghazanfar said, he would like to look at what is going on in the brain at the neuron level when speech or other vocalizations are perceived. One possibility is that the visual cues and auditory cues are being processed separately, in parallel. "The alternative," he said, "is that the brain is not as segregated as one might think."

HOME
Primates
Primate FAQ
Primate Species
Retired Chimpanzees
Monkey Opium Addicts
Monkey sex in the brain
Primates E-Mail Service
Monkey Sense of Justice
Chimpanzees and Animal Medicine
Is Miss Waldron's red colobus monkey extinct?