Facial Profiling

When does facial expression reading bleed over into facial profiling? Dr. Paul Ekman and other experts in the field of facial movement and expression are consistent in their message - you should see a person's neutral face, their baseline expressions, BEFORE you attempt to draw much conclusion from static photos, in particular.

And yet some research seems to point to people being able to predict some personality features by looking at pictures of faces. What's going on here? Is "facial profiling" ever reasonable, ever fair?

Slate Magazine has a great article on the subject:

But today, physiognomy is making a comeback. In the last decade, breakthroughs in 3-D modeling and animation software have opened up the field. At the same time, ideas from genetics and evolutionary psychology are reanimating old debates about biological determinism, race and gender differences, and why humans possess the faces and bodies that we do.

The new research suggests we are more skilled at "reading faces" than we knew. People are surprisingly adept at assessing sexual orientation from headshots. Five-year-olds can predict election outcomes based on photos of the candidates. We can even guess whether a face belongs to a Democrat or a Republican at a rate better than chance, according to a forthcoming study out of Princeton.

And here it is again...

How many of these shots have we seen now from press conferences involving some sort of accusation or resignation. What exactly is this expression? How many shots do the photographers need to take and sort through before they nail it -- is it an ongoing expression, or fleeting? Is it spontaneous, or an expression assumed to attempt to mask another?

This shot of Bank of America CEO Ken Lewis announcing his resignation seems to be the classic illustration for these stories. I can only assume photo editors know what they want to convey when they pick it. Let's take it apart:

The first thing you may notice is the pulled down mouth corners and downcast eyes -- but does the rest of his face look sad? The inner corners of his eyes are drawn in and down, they aren't lifting in and up like in a typical sad expression. His lips are drawn completely inside his mouth and pressed tightly together, and like the eyebrows, that's a strong indication of anger.

Could the intense clamping of the lips and the downward/no-eye contact gaze be an indication of an intense anger that must at all costs be restrained? A stony mask to wear during one of the more painful and infuriating experiences of your life? An attempt to appear serious and appropriately concerned when in fact extremely aggravated? What do you see?

The Eyes Have It

Have you ever noticed that human eyes are rather significantly different than the eyes of other mammals, including (or maybe even especially) those of other primates? A new study from the Max Planck Institute for Evolutionary Antropology in Germany has hypothesized that our white sclera and high-contrast irises have helped others understand where our gaze is directed, which in turn improved our social communication and cooperation abilities.

Why Eyes are So Alluring

Can you think of times when you've reacted to just a minor movement of someone's gaze?

The classic is the party scene where, even without the person moving their head, you notice their gaze leave your face and move over your shoulder to someone beyond. This is a subtle, but strong social cue that the conversation is drawing to a close and, unless you want to seem uncouth, you need to move aside in such a way to allow your conversational partner to leave.

Share other gaze-related body language cues in the comments!

Rating the Experts

Below is a five-minute local news video with the authors of a book on body language for job interviews. While most of what they say won't be news to readers of this blog, it's interesting to watch the body language on display during the interview. What sticks out for you? I share my impressions after the video:


Scroll down for impressions:





The first thing I noticed is the (reasonable) nervousness of the male author - he's appropriately using his hands but maybe he's thinking about his body language too much and so he trips over his speech a bit. The tall barstools are terrific for exposing the whole body here - compare the ramrod straight posture, big social smiles and submissive hands in laps of the guests with the upright but more relaxed and dominant bodies and faces of the hosts. Not unlike a job interview!

Note that the male guest mirrors the male host, and the women do this as well. My favorite part was the put-on look of surprise at one point from the female host - it lasts way too long and is a dead give-away that she's not actually surprised but that she wants to convey that emotion to the audience. What did you see?

Slip of the Lips?

Obama Leaks Emotion Over Gates Affair

What does a "pursed lip" mean? In this ABC News article, the author offers the evidence of several body language experts who parse this and other facial expressions for readers.

"I think Obama was disappointed in himself that he got sucked into this," said [TJ] Walker, CEO of Media Training Worldwide. "It's so rare that he does stumble into something. Compared to the last four presidents, he gets off message and sticks his foot in his mouth less often."

Later in the article, FBI profiler Brad Garrett was consulted:

"Lip tightening is usually read as "trying to control their speech or are getting angry," he said. "It makes sense that someone as cool as Obama was probably pissed that he used that one word [stupidly] and that the press made such a big deal out of it."

Crime suspects [and others] bite, gnaw, cover their mouths, put folders over their faces -- all as an "obstruction," according to Garrett."

What do you think? I know I purse and chew my lips all the time, but am not known for my verbal restraint either. Sometimes I notice it in myself and others during times of questioning or decision-making -- or could it be the urge to "keep one's mouth shut" until the answer has been reached?

Read the entire ABC News article.

Disgust

Both Kate Gosselin and Sarah Palin are showing disgust in these pictures. Notice the wrinkling along the sides of Kate's nose and the upper lip moving upwards and showing her teeth.

Sarah is showing a different version of this emotion. It doesn't look like there is any wrinkling along her nose, but the shadows and the bridge of her glasses makes this a bit difficult to determine. However, her upper lip is STRONGLY raised, so strongly, in fact, that she has pushed her lower lip up just as strongly, in order to keep her lips together.

Neither of them like what they're looking at in the moment these pictures were taken.

Kim Jong Il

Older, ailing and increasingly frail? Yes. Still in charge? Definitely:



Other side of the same shot:



Still the boss:



Photos and story at the New York Times.

Philby Follow-up

Interested in learning more about what you saw in Kim Philby's face, and how it all played out?

Read the fascinating article, The Naked Face, by Malcolm Gladwell - the precursor to the chapter in his best-selling book "Blink". Gladwell described the Philby micro-expressions and speaks with Dr. Paul Ekman and others about their experience with face reading.

Find out more about Philby and Cambridge Spies at the BBC and our old favorite, Wikipedia.

Kim Philby

This is a video of Kim Philby, who at the time of this video was accused of being a spy for the Soviet Union. The "Third Man" refers to a person believed to have tipped off two other British spies so that they escaped arrest by defecting to the USSR.

Watch for the micro-expressions appearing after Philby answers two questions.

Casey Anthony

Watch for the brief almost-smile that appears on Casey Anthony's face as she comes through the door and sees the media waiting for her. After that her face goes blank and she drops her gaze to the floor for a moment before looking ahead again.

What the Facial Action Coding System can tell us here is that she DOES smile for a brief moment. However, is this micro-expression of happiness a sign that she is guilty? Definitely not. Although a smile might seem to be a very odd reaction in this situation, it could be prompted by nervousness, or even fright. Sometimes people do smile when they are nervous (ever seen someone get the giggles at a funeral or other "serious" occasion?).

It's also important to remember that we do not know what she is looking at when she smiles. We know the media is there (you can hear them asking questions), but we do not know who or what else is in her line of sight. The smile could be prompted by something she is seeing, by relief at leaving, by excitement to see all the people waiting for her...the list is endless.

Why is This Man Smiling?

The following picture is from the New York Times - the man is smiling (a real smile, involving the eyes), but those around him are not. What do you think is happening?



A young movie star with his bodyguards stepping onto the red carpet? Or a suspect in international piracy arriving to the US court in manacles?

You may remember one of Amy's video from the workshop showing murder suspect Casey Anthony entering a room, handcuffed, flashing a smile at the reporters and photographers assembled there. Are these the smiles of narcissistic sociopaths delighting in their crimes? Or are we so conditioned to smile at experiencing such strong attention (name called, photos) that we can't help it? Or, is it something else entirely?

Paul Ekman grades "Lie to Me"--the show he inspired

Reading Between The Lies
By Marc D. Allan
Special to The Washington Post
Sunday, July 12, 2009

With the first season of "Lie to Me" completed, Paul Ekman is ready to assign grades to the Fox drama he inspired:

A-minus for entertainment value, B-plus for realism.

Ekman, a professor emeritus at the University of California at San Francisco and a social psychologist who works in the area of deception and demeanor, said 85 percent of what "Lie to Me" depicts is accurate.

"Baum does care to get it right," Ekman said.

He was speaking of Samuel Baum, who created the show after learning about Ekman's work using facial expressions and verbal tics to help determine whether someone is telling the truth.
"I thought a franchise set around someone who did that kind of work would give real scope to the kinds of cases you could explore -- one week a political thriller, one week a family drama," Baum said. "A little movie every week."

He centered the show on a character named Cal Lightman (played by Tim Roth), an in-your-face investigator who leads a firm that helps law enforcement and government agencies.
The Lightman Group -- partner Gillian Foster (Kelli Williams), newcomer Ria Torres (Monica Raymund) and researcher Eli Loker (Brendan Hines) -- is typically hired to investigate crimes. But occasionally its clients are people such as a multimillionaire who wants to know whether his fiancee really loves him.

Read the rest of the article at the Washington Post

Anxiety and Othello's Error

In her workshop, Amy discussed the importance of avoiding "Othello's Error" when interpreting facial expressions, which Paul Ekman described in a 2004 interview:

But the emotion doesn't tell you its source. Otherwise, you'll make "Othello's error." Othello killed Desdemona because he thought that her signs of fear were of a woman caught in a betrayal. She was afraid of being disbelieved. The fear of being disbelieved looked just like the fear of being caught. Fear is fear. You have to find out which it is. That's a little disappointing, because people would like to think, "Oh, if you look afraid, that means you did it." No, it doesn't mean that; it means you're afraid.
Paul Ekman Interview: Conversations with History; Institute of International Studies, UC Berkeley

A study published in the August 2006 Journal of Personality indicates that people with anxiety tended to be faster to perceive changes facial expressions than a control group, but that they also tended to jump to conclusions about the source of the emotions more quickly as well, introducing significant errors of perception.

Interestingly, when the anxious individuals were asked to take a longer period to link expressions to possible emotional states, they tended to perform more accurately than the control. Teaching people with anxiety to take this time may be important - in the press release from University of Illinois Urbana-Champaign, lead author, R. Chris Fraley, states:

“This ‘hair trigger’ style of perceptual sensitivity may be one reason why highly anxious people experience greater conflict in their relationships,” Fraley said. “The irony is that they have the ability to make their judgments more accurately than less-anxious people, but, because they are so quick to make judgments about others’ emotions, they tend to mistakenly infer other people’s emotional states and intentions."

Not-So-Neutral Neutral Faces

The April Edition of the Journal Emotion is full of wonderful "face-based" research. One that particularly caught my eye was how having a neutral face that has some of the features of a facial expression can affect the way you are perceived, rather strikingly:

Structural resemblance to emotional expressions predicts evaluation of emotionally neutral faces. Said, C., et. al.

People make trait inferences based on facial appearance despite little evidence that these inferences accurately reflect personality. The authors tested the hypothesis that these inferences are driven in part by structural resemblance to emotional expressions. The authors first had participants judge emotionally neutral faces on a set of trait dimensions. The authors then submitted the face images to a Bayesian network classifier trained to detect emotional expressions. By using a classifier, the authors can show that neutral faces perceived to possess various personality traits contain objective resemblance to emotional expression. In general, neutral faces that are perceived to have positive valence resemble happiness, faces that are perceived to have negative valence resemble disgust and fear, and faces that are perceived to be threatening resemble anger. These results support the idea that trait inferences are in part the result of an overgeneralization of emotion recognition systems. Under this hypothesis, emotion recognition systems, which typically extract accurate information about a person's emotional state, are engaged during the perception of neutral faces that bear subtle resemblance to emotional expressions. These emotions could then be misattributed as traits. (PsycINFO Database Record (c) 2009 APA, all rights reserved)

Those of you who took the workshop may remember a couple of the Ekman Microexpressions Training Tool faces that already had a strong "sad" or one-sided "contempt" neutral expression, and some participants stated that they found those faces confusing. Do you know of people who have identifiable "emotional expressions" in their neutral faces? Has it ever affected the way you interact with them?

What a Nod Says

Do we "thin-slice" body language to quickly gauge someone's socioeconomic status? Some preliminary research indicates what many of us know as common TV tropes -- the eager young striver, the bored, contemptuous socialite -- bears itself out with body language during stranger interactions. Not only do those with lower SES show more engagement behavior and those with higher SES less in this study, third parties were more likely (than chance) to recognize and assign SES correctly after watching these interactions.

A lay media piece can be found at Muzi.com: Body Language Reveals Wealth

They found that students whose parents were from higher socioeconomic status (SES) backgrounds engaged in more of what he called "impolite" behaviors, such as grooming, doodling and fidgeting. Lower SES students showed more "I'm interested" gestures, including laughter and raising of the eyebrows.

The full text of the original article is available at the journal Psychological Science, Jan 2009: Signs of Socioeconomic Status: A Thin-Slicing Approach

ABSTRACT—Socioeconomic status (SES) is a primary determinant of health vulnerabilities and social affiliations. To ascertain if SES is signaled in brief patterns of nonverbal behavior, we had participants of varying SES backgrounds engage in a brief interaction with a stranger. Videos of 60-s slices of these interactions were coded for nonverbal cues of disengagement and engagement, and estimates of participants' SES were provided by naive observers who viewed these videos. As predicted by analyses of resource dependence and power, upper-SES participants displayed more disengagement cues (e.g., doodling) and fewer engagement cues (e.g., head nods, laughs) than did lower-SES participants. Results were also consistent with the thin-slicing literature, in that observers' estimates of SES were reliable with each other and accurately predicted targets' family income, maternal education, and subjective SES. Finally, nonverbal displays of disengagement and engagement predicted observers' estimates of SES, which suggests that these cues are systematic signs of SES. These results have implications for understanding the effect of SES on social interactions and patterns of disengagement and engagement in other realms.

While it's worth noting that they couldn't mask from the researchers or the naive third parties other SES "tells" like accents, clothing, accessories, or word choice, the study is still intriguing. When you are people watching, look for non-mirrored nods and laughs, or doodles and self-grooming during conversation - do they tell you something about the status (socioeconomic or other) of the participants? When you meet with your supervisor, who nods more?

Analysis of Body Language - Joe Navarro

Once you start looking at this stuff, you can't stop! Where in the past a flight delay may have made you frown like the rest, now you can kick back and enjoy the smorgasboard of body language available to you in the airport, or anywhere else humans gather and you have a spare moment.

Below, Joe Navarro talks with Ann Curry about decoding body language in pictures and video:

Visit msnbc.com for Breaking News, World News, and News about the Economy

(Sorry for the heaping helping of US politicians - but they are a body language gold mine!)

Is Facial Motion the Key?

If you took Amy's workshop, you may have eventually found yourself looking for some tell-tale movements as a neutral face picture changed to one with a specific expression. The start of the wrinkles around the nose denoting disgust, or the lengthening of a face showing surprise, as the eyebrows arch and the jaw drops.

The Science Daily article "The Enigmatic Face: Motion Is The Key To Understanding Expressions" cites a study that seems to indicate that it's not so much seeing the finished facial expression that triggers our recognition, but rather our experience of the movement of the face in the direction of that expression - meaning that we may be detecting, and reacting to, these expressions even more quickly than we realize!

Body Language by Joe

Much of the material Amy shared in the Body Language portion of the workshop was adapted from the terrific new book "What Every Body Is Saying" by Joe Navarro, an ex-FBI agent and body language expert.

Here he talks about signs of discomfort, especially those people show facially:

Joe Navaro: How to Read Faces

My favorite part is at the end where he discusses how people in a discussion use body language to demonstrate whether or not you are welcome to join the conversation. Most of us have probably experienced this and understood the message being conveyed, even without realizing exactly what on someone's body was telling us "hey, get lost!"

I (Jennie) really enjoyed Joe Navarro's book. They aren't all so well written or as backed up by research, so definitely recommended.

Workshop Photos

Hello Everyone! I hope that you have been seeing lots of micro-expressions and body language since our workshop last Sunday! As promised, here are the photos we viewed during the workshop. The videos will follow shortly!

Disgust and Contempt

A very strong disgust expression from Samantha Ronson in the left picture. Notice the wrinkling on the sides of the nose and the action following the center line of her face. Her upper lip is raised, and her chin and lower lip are pushed up as well. In that same picture, Lindsay Lohan is showing a very social smile--no involvement of the eyes at all.

In the second (right) picture, the image is grainy, but you can see the left side of Lindsay's mouth being pulled in the classic expression of contempt.

Fear

An unilateral (one-sided) fear expression from Robert Pattinson. His eyes (especially his right eye) may be widened a bit, but it's hard to tell without a neutral picture of him to compare it with. The left side of his lips are pulled back, not downwards as we see in the sad expressions. He may be trying to suppress an expression of fear, or simply "putting on" a fearful expression for emphasis.

Surprise and Happiness

Surprise? Happiness? Or both?

Both show the high, rounded brows of surprise. However, remember that surprise is the briefest emotion of all. George may have been surprised a moment ago, but now he's communicating happiness. The high arch of his brows makes it a little tougher to tell if he's genuinely happy with what/whom he's seeing, or if he is sending a social smile (he's wearing a tuxedo, so it's probably a very social and public arena he's in). Obama also has the arched brows, but it's more likely that he's instinctively giving the baby an open, friendly, happy face to look at.

Social Smiles

Social Smiles. For a different perspective, try covering the lower half of the face and see what eyes and forehead communicate on their own. Then cover the eyes and just look at the lower half of the face.

Sad expressions

Heidi definitely is feeling sorry for herself in the jungle.

Most of Brett's face is neutral, but his lips are pulled down strongly in a classic sad expression.

Casey Anthony has her brows pushing upward, causing the horizontal wrinkles to appear in her forehead. The visible side of her mouth shows the end of the lips being pulled down just like Brett above. It's possible that the pull is being counteracted by her chin and lips pressing together--perhaps an attempt on her part to keep quiet despite her crying.

Anger pics

Here is A-Rod showing some restrained anger--brow wrinkled, eyes narrowed, chin tight.

We can see Amy Winehouse only from the side but her chin appears jutted out and pushed upwards, almost like a petulant child. Her "victim" is showing some anger and/or disgust (and who wouldn't?)

Body Language/Facial Expressions in the Media

Much of the workshop's first half featured the research of Dr. Paul Ekman, a leader in the study of facial expression. You can learn more about his advanced training at his website, but you can also learn quite a bit from the blog he's keeping as technical advisor to the U.S. TV show "Lie to Me", which is loosely based on the work of Dr. Ekman and his colleagues.

On the main show website, be sure to check out some of the videos -- here's a great one describing the genuine "Duchenne Smile".

Elsewhere, find the user-contributed "Faces in the News". Do you have a good example of a famous face "saying it all" in contradiction to their words?

--