emotion

Emotion

情绪和文化实验室|浙江大学

Outline

1. Characterizing Emotion
2. Theories of emotion:
   • Evolutionary
   • Appraisal
   • Constructionist
3. How to Study Emotions?
   • Inducing Emotions
   • Measuring Emotions
4. Facial Expressions of Emotion

Defining Emotion: Is It Possible?

Everybody knows what an emotion is, until asked to give a definition

Fehrand Russell, 1984, p.464

Defining Emotion

It’s a
Fan!

It’s
It’s
a Wall!
It’s a
Spear!
a
Rope!

It’s
a Snake!
It’s a
Tree!

Defining Emotion: Your list

Write down a list of emotions (1 minute)

Defining Emotion

Episodic, relatively short-term, biologically-based patterns of perception,
experience, physiology, action, and communication that occur in response to specific physical and social challenges and opportunities.

Keltner and Gross, 1999

针对特定的身体和社会挑战和机会而发生的、片段性的、相对短期的、 于
生物的感知、体验、生理、行为和沟通模式。

Defining Emotion: Your List

• Evaluate your list of emotions against Keltner and Gross’ definition

Episodic, relatively short-term, biologically-based patterns of perception,
experience, physiology, action, and communication that occur in response to specific physical and social challenges and opportunities.

Emotion Words

• Anger, Disgust, Fear, Surprise, Sadness, Joy
• Contempt

• 《礼记· 礼运》：喜、怒、哀、惧、爱、恶、欲

• 目前我们通过文献搜索汇集了与情绪研究相关的81篇英文文献和14篇中 文文献，在此 础上共整理出581个情绪词汇。
• 经过专家初步筛选，排除422个近似词汇，得到159个相似度较低的情绪 词汇。

焦虑 vs. 焦躁

1 2 3 4 5 6 7 8 9
非常不相似 非常相似

Emotion Words

确定情绪词汇								
待定情绪词汇								
自编情绪词汇								
微信表情情绪								

(无高亮颜色) 待选情绪词汇

安宁 安心 畅快 放松 放松(relaxation/relax 放心 幸福 康乐 怡和 宽心
大笑 欢 欢畅 欢快 欢乐 欢乐(joy) 欢喜 欢欣 欢娱 欢愉
苦乐参半(bittersweet) 哭笑不得
奸笑 坏笑 偷笑 苦笑 幸灾乐祸(schadenfreud 忍俊不禁
爱好 应迷 炽烈 炽热 感兴趣(interest) 兴趣
入神(entrancing) 美(beautiful) 神往 审美鉴赏(aesthetic ppreciation) 专注、专心(concentrat
H 投入
楼火(dreamy) 消魂 缠绵
爱(love 爱慕(adoration) 宠 浪漫(romance) 溺爱 偏爱 热爱 喜爱 喜爱(affection) 喜欢
友谊(friendship 利他的(altruism)
感动 感激 感谢 感激、感谢(gratitude) 起鸡皮疙瘩的(goose bumps)
自满 自傲 自卑 自大 自 负 炫耀 骄傲 骄做(pride/proud 骄横 骄矜
撒娇 娇
情欲(lust 色情的(erotic 性快感(sexua
pleasure) 性欲(sexual desire)
关怀 关切 关心 关心(caring) 关注 在乎 留神 珍爱 珍视 珍惜
崇敬 崇尚 崇奉 崇拜 钦佩 钦佩(admiration) 景慕 景仰 敬慕 敬佩
称心 称意 可心 可意 满意 满意(satisfaction) 满足(contentment) 如意 充实
欣慰 (尤指对子女成就的) 快慰 解恨 (不快过后的)密辱
解脱(relief)

Emotion Words

·Judgments of the similarity of
600 different emotion terms

challega
grit
···pride.
. ● confidence

● ingēr
irtatign
hate.

self-pity
·contempt
indienatior
righteous
·bitterness
teeling oppressed Sham
·disdain
.repugnance
·dhgg,
·disapprow ·humiliatior
● dieppointmgnt
● amusement
.iiugheer
●demoraliati helplessnes

●foy*
● giddiness
· elntion
?snarkiness
thrill ●feeling creepedeout
jealousy : feeling let down
…eelingstuck
urge to escape reluctance
●unease
·doibt
worr
·leeling betrayed
. ∵● pain
crushed ● anguish
. ● heartbreak
nvigoratioi
··feeling
rejuver
anquility ·
disinhibited .shyness
guilte pleasure
*
● feeling torn
yearning
·feeling
·nicismdismay
stress
desperation

lonaliness
feeling lost
·detachment melayicholy
?sadrtess
·low spirits 。
●despair
● satisfaction,
●gratification
·savoring
ety

● queasiness
·restlessness
. · ●dare · :● confusion
● bewilderment
● nostalgia
?eagerness
gratitude
·sollidarity
i contemplation
· interest
dediratiofbcus
·hionor
· entrancement , ● epiphany
boredom
·atigue

·reyerence *,
· awe
●adoration
ust

●ast8.SFBCiSe
shock

●dread
● shuddering

● orgasm ·omnfatgh?

What is a Theory of Emotion?

• Testable statement about origin and structure of emotion
• Differ on:
o Causes
o Biological givens
o Components
o Integration

Physiological Changes

Appraisal

Other Behavior

Emotion Components

Nonverbal Expressions

Emotion Episode

Subjective Experience

Major Theories of Emotion

Evolutionary

Appraisal

Constructionism

Evolutionary Theories of Emotion

·Evolved and Adaptive
Anger
Disgust
·Separate
· Distinct physiology
· Universal
· Innate
Fear
Joy

Surprise

Sadness

Evolutionary Theories of Emotion

What is a Theory of Emotion?

· Causes: Emotions caused by activation of Affect Program
· Biological givens: Basic emotions are biological givens
· Components: Facial expressions,behavior (action tendencies),
physiology,changes to cognition;Subjective experience less central component
· Integration: High integration between components

Appraisal Theories of Emotion

Novelty/Expectation

Intrinsic Pleasantness

Goal Conduciveness

Agency

Coping Potential

Appraisal Theories of Emotion

Novelty/Expectation

Intrinsic Pleasantness

Goal Conduciveness

Agency

Coping Potential

Did you expect this situation
to occur?

Did you find the event itself
pleasant or unpleasant?

Did the event help or hinder
you to achieve your aims?

Who caused it?

Can I deal with it?

Appraisal Theories of Emotion

Did you expect
Novelty/
Expectation
this situation to
NO
occur?

Did you find the
Intrinsic
Pleasantness
NO
event itseli
pleasant?

Did the event help
Goal
Conduciveness
you to achieve
NO
your aims?

Who caused it?
Other Persons
Agency

Coping
Can I deal with it?
NO
Potential

Appraisal Theories of Emotion

TABLE 1
Predictions on Emotion Differentiation based on Stimulus Evaluation Checks

Joy	Fear	Anger	Sadness	Disgust	Shame	Guilt

Novelty/Expectancy
Expectedness Open Low Open Open Open Open Open
Intrinsic Pleasantness
Unpleasantness Low High Open Open Very high Open Open
GoallNeed-conduciveness
Goal hindrance Very low High High High Open Open Low
Coping Potential
External causation Open External External Open External Internal Internal
Coping ability Medium Very low High Low Open Open Open
Compatibility
with standards
Immorality Open Open High Open Open Open Very high
Self-consistency Open Open Low Open Open Very low Very low
Note:This table is based on table 2 in Scherer(1986)and table 5 in Scherer(1988).Open =different evaluation results are compatible with the respective emotion and/or further detail (on subtype of emotion and/or subchecks)needed for predictions.

Appraisal Theories of Emotion

· Causes:Emotions caused by appraisals
· Biological givens: Novelty and valence (at least)are biological givens
· Components:Appraisals can affect other components,but not necessarily
all together
· Integration: Less strong integration between components

Constructionist Theories of Emotion

Arousal

Valence

Constructionist Theories of Emotion

tense

nervous

stressed

upset

Unpleasant-

sad

depressed

sluggish
bored

Activated、
alert
excited
Arousal enthusiastic
elated
happy

Valence ——Pleasant
contented

serene

relaxed
calm

Deactivated

Constructionist Theories of Emotion

· Causes: Emotions caused by valence and arousal (core affect)in combination with conceptual knowledge
· Biological givens: Core affect is a biological given
· Components: Components vary across experiences;Subjective experience central
· Integration: Little integration between components

Core
Stimulus
Emotion
Categorization
Affect

Constructionist Theories of Emotion

·You aren’t at the mercy of your emotions
·https://www.ted.com/talks/lisa feldman barrett you aren t at the merc y_of your emotions your brain creates_them?language=en

How to study emotions?

How to study emotions

• Inducing Emotions
• Measuring Emotions

How to induce emotions?

Inducing Emotions

Think about different ways to induce emotions (3 minutes)

Inducing emotions

1. Pictures
2. Movies
3. Music
4. Memories
5. Scripted social interactions

Inducing emotions -Pictures

· Validated stimulus sets
· Induces weak,short-lived emotions
· Good for multiple measures
· Works only for some emotions

Inducing emotions -Movies

· Validated stimulus sets
· Can elicit specific emotions
· Can elicit strong emotions
· Few repetitions
· Works only for some emotions

Inducing emotions -Music

· Validated stimulus sets
· Induces diffuse emotions or moods
· Works only for some emotions

Inducing emotions -Music

Music can elicit 13
emotions across USA and
China.

D-Beautiful

H-Erotic,
desiroug

J-Joyful,
A-Amusingsheerfu

E-Galm,

depressing
R-Sad,

M-Friumphant, heroie
G-Energizing
pump-up

1-Indtgnant,
deflant fearful

Inducing emotions -Memories

· Can elicit intense emotions
· Can elicit specific emotions
· Few repetitions
· Works for most emotions

Inducing emotions - Scripted social interactions

• Can elicit intense emotions
• Can elicit specific emotions
• Typically only one trial per participant
• Works only for some emotions

Inducing emotions -Naturally occurring emotions

How to measure emotions?

Measuring Emotions

Think about different ways to measure emotions (3 minutes)

Measuring emotions

1. Self-report (questionnaires)
   

2. Biological measures
   

3. Nonverbal expressions
   

Measuring emotions - Self-report (questionnaires)

How angry do you feel?

1……2……3……4……5……6……7……8……9

Not at all angry Extremely angry

Measuring emotions -Biological measures

1. Skin Conductance (EDA/GSR)
2. Brain Activity (EEG,fMRI)
3. Heart Rate (ECG)

Measuring emotions - Nonverbal expressions

1. Facial Expressions
   

a) Facial EMG
b) Automatic FACS-based Software: FaceReader, iMotion,
c) Manual FACS Coding
2) Pupil Dilation (eye tracking)
3) Gesture
4) Touch
5) Voice

Measuring emotions -Nonverbal expressions

1. Facial Expressions
   a) Facial EMG
   b) Automatic FACS-based Software:FaceReader,iMotion,
   c) Manual FACS Coding

SAD
DEPRESSOR
CORNER PULLER

FEAR
FRONTALIS
BROW RAISER

ANGRY
CORRUGATOR
BROW FROWNER

HAPPV
ZHGOMxTC
LP PAESE

depressor	frontalis	corrugator	zygomaticusQ

SAD 十 十 十
FEAR 十
ANGRY 十
RAPPW 十

Measuring emotions -Nonverbal expressions

2. Pupil Dilation (eye tracking)
   演示

How eye tracking works

An eye tracker consists of cameras,
projectors and algorithms.
The prorectors create a pattem
2
of near-infrared light on the eyes.
The cameras take high-resolution
3
images of the user’s eyes and the
pattern.
Gaze point
Machine leaning,image processing
and mathematical algorithms are
used to determine the eyes’position
and gaze point.

Measuring emotions -Nonverbal expressions

3. Gesture
   The Body Action and Posture coding system (BAP)

a
anger disgust fear happiness sadness surprise neutra

b
disgust fear happiness sadness Surprise neutral

Measuring emotions -Nonverbal expressions

4. Touch

The Science of
Happiness

Measuring emotions -Nonverbal expressions

Touch

Measuring emotions -Nonverbal expressions

4. Touch

(A)

(B)

Measuring emotions - Nonverbal expressions

4. Touch
   

Sympathy, Gratitude, Love, Anger, Fear, Disgust
Dacher Keltner

Measuring emotions -Nonverbal expressions

5. Voice
   a) Vocalization (叹词)?
   b) Speech Prosody (韵律)?

Chinese Emotional Speech Prosody and Vocal Bursts

Adoration 喜爱
Amusement 好笑
Anger 气愤
Awe 令人惊叹
Confusion 困惑
Contempt 蔑视
Contentment 满足
Desire 欲望
Desire (for food) 性欲
Desire (for sex) 食欲
Disgust 厌恶
Disgust (physical) 恶心
Distress 苦恼
Embarrassment 尴尬
Excitement 兴奋
Fear 恐惧

Interest 兴趣
moved 感动
Neutral 中性
praise 赞扬
pride 自豪/骄傲
Sadness 悲伤
Sajiao 撒娇
shame/guilt 愧疚/内疚
snicker/grin 坏笑
Surprise 惊讶
Surprise
(positive) 惊喜
Triumph 胜利

Participants: 62 Chinese participants (M = 21.48, SD = 2.42; 31 females)

vocal bursts (32)

Interest 兴趣
moved 感动
Pain (physical) 痛苦
praise 赞扬
pride 自豪/骄傲
Realization 恍然大悟
Relief 解脱
Sadness 悲伤
Sajiao 撒娇
shame/guilt 愧疚/内疚
snicker/grin 坏笑
snigger 偷笑
Surprise 惊讶
Surprise (positive) 惊喜
Sympathy 同情
Triumph 胜利

speech prosody (25)

Adoration 喜爱
Amusement 好笑
Anger 气愤
Awe 令人惊叹
Confusion 困惑
Contempt 蔑视
Desire 欲望
Disgust 厌恶
Distress 苦恼
Excitement 兴奋
Fear 恐惧
Gratitude 感谢
Happiness 开心

Facial Expressions of Emotion

Facial expressions

How many facial expressions of emotion are there?

A study on the face and including nearby
hand and body movements showed that
there are 28 facial-bodily expressions of
emotion in the United States (Cowen &
Keltner,2019).

Platysma (cut)
-i- (cut)

Facial muscles

Galea aponeurotica一

Frontalis—

Orbicu[Palpebral-
Orbital:

Nasalis—
Levator labii—
superioris
Zygomaticus minor.
Zygomaticus maior
Orbicularis oris-
Risorius 一
s
Platysma—
Depressor labii
inferioris

Superficial Facial
Muscles—
Anterior
View
—Temporalis
-Corrugator
supercilli
-Orbicularis oculi
Zygomaticus minor
and major (cut)
oirilut)
tor anguli oris
-Buccinator
-Masseter
Orbicularis oris
.Depressor labii

Sternocleidomastoid

Facial
Action
Coding
System

The Manual
on CD ROM
by
Paul Ekman,Ph.D.

CHARLES DARWIN
The Expresion ofthe Emotions
in Man and Animals
Wallace V.Friesen,Ph.D.
Joseph C.Hager,Ph.D.

1872
Charles

Darwin

1978
Paul

Ekman

2023

面部动作编码系统
(Facial Action Coding System)

44 个面部动作单元

Facial Action Coding System(FACS)

Upper Face Action Units
AU 1 AU 2 AU 4 AU 5 AU 6 AU 7

Inner Brow Raiser

Lid
Tightener
Outer Brow Raiser Brow
Lowerer Upper Lid Raiser Cheek
Raiser
*AU 41 *AU 42 *AU 43 AU 44 AU 45 AU 46

Lid
Droop

Slit

Eyes
Closed

Squint

Blink

Wink
Lower Face Action Units
AU 9 AU 10 AU 11 AU 12 AU 13 AU 14

Nose
Wrinkler

Upper Lip Raiser

Nasolabial Deepener

Lip Corner Puller

Cheek
Puffer

Dimpler
AU 15 AU 16 AU 17 AU 18 AU 20 AU 22

Lip Corner Depressor

Lower Lip Depressor

Chin
Raiser

Lip
Puckerer

Lip
Stretcher

Lip
Funneler
AU 23 AU 24 *AU 25 *AU 26 *AU 27 AU 28

Lip
Tightener

Lip
Pressor

Lips
Part

Jaw
Drop

Mouth Stretch

Lip
Suck

肌肉动作单元
(Action Unit)

AU4: 皱眉

● AU5: 上眼睑上抬 AU24: 上下唇按压 AU17: 下巴上抬

肌肉动作单元
(Action Unit)

● AU7: 内眼眶缩紧

● AU9: 皱鼻

● AU10: 上嘴唇上抬 AU25: 上下唇分开

Collecting Chinese Emotional Faces

Happiness

Anger

Sadness

Disgust

Surprise

Fear

Contempt

Grievance

Confusion

Positive
Surprise

Confusion matrix for static facial expression
Perceived Emotion
的
为
名
色

Anger-
Anxiousness -
Awe- CanNotHelpLaughing
Chuckle- Confusion -
Contempt-
Discouragement-
Disgust-
Distress-
Droop-
Excitement - Fear- Fidget- ForcedSmile -
Grief - Grievance -
Happiness -
Neutral-
PhysicalDisgust -
PositiveSurprise - Sadness - Shyness -
SinisterSmile-
Smile-
Snicker-
Surprise -
Triumph -
Other -

25.15
3.31
2.06
0.74
0.37 6.77
5.46
2.33
1.38
0,73 0.96
0.37
16.11
0,81
0,74 0.44
0.15
051
11.67
13.66 0.07
0.16
0.22
14.31
23.16 3.77
2.59
1.63
0.81
0.15 2.44
0.74
0.8
1.18
2.07 2.44
1.55
0.66
0,8
0,22 6.10 2.57 1.18 .03 0,29 5.33
. 8 104 0,44
0,22 3.01
0.8
0.59
0,22
0.22 0.5
0.15
4.1E
2.98
2.77 6.11
.84
787
0.96
0.37 7.86
3.16
0.59
0,81
0,22 0.81
0.44
066
2.88
4.65 4.43
3.33
)89
1.18
0.37 5.92
2.5
05
0.6
0.15 0.15 1.46 4.68
0.30 0.95 2.62
074 0.80 187
,40 0.95 2.43
2,44 0,80 0.9 1.03
0.22
9.54
.0
0,88 3.16
2.13
073
0.66
0.59 1.62
.47
0.66
6.76
11.2 0.8
0.30
0.5S
2.88
8,4
.11
1.3
0
.03
18
037
20
0.07
0.59
2
22
81 0.15
0.30
0.44
2.29
3.62
.48
2.14
0.74
0.59
0.96
0.8
.1
0.15
0.59
25
A
0.52 0.67
0.15
0.30
5.25
7,40
1.18
9.91
0.44
.4
0.52
0.44
2.00
0.44
0.37
6.80
0.07
0.22 1.69
0.51
14.49
0.74
0.59
9.62
1.91
0.7
1.25
1.18
D.96
0.15
2.79
0.51
0,44
D.15
0.66 2.11
0.15
1.82
3.78
3.86
1.53
1.31
0.07
0.15
0.66
0.36
9.24
0.15
0.66
3.71
0.51
0.36
11.40
4.63
2.43
8.09
6.03
1.84
0.37
1.10
5.37
1.10
0.59
3.24
0.51
8.16
2.72
0.44
2.50
0.15
0.15
0.37
0.22
5.15
0.07
1.76 9.36
3.35
3.13
4.73
0.33
2.18
0.15
0.80
5.82
2.91
1.82
6.62
0.44
8.44
0.87
0.15
4.00
0.22
0.15
1.53
0.15
3.71
0.36
2.11 5.15
2.06
0.81
0.66
0.37
0.66
1.03
1.10
0.22
1.18
0.15
0.15
3.0
6.62
D.22
5.00
D.22
0,07
1.03
2.35
1.25

0.37
1.10 0.73
2.79
0.5
0.59
0.7
0.37
26
0.15
0.00
2
15
7
24.0
1.17
0.15
051
0.07
2.57
3.38
5.57
7.49
0.51
0.73
1.17 0.66
3.32
0.74
174
66
0.66
2.29
0.37
0.00
5.68
0,22
96
18
9
0.22
0.5
0.44
3.6
4.28
11.65
11.36
0.44
0.59
0.52 35.70
6.1
3.7
36
76

0.15
0.59
4.8
33
03

15
4.2
0.89
0.0

0.37
0.30
0.15
0.67
4.07
0.00
1.92 7.03
4.8
74
584
96
85

0.30
11
2
59
03
.70
5.10
33
0.5
.
0.96
2.66
2.74
4.66
0.67
0.52
2.51 0.34
6.87
0.64
244
27
15 5.15
1.99
2.50
66/
5.2
91
U.
0.72
28
24
25

D.2
53
4.63
0.07
.9
D,2
0.8
D.88
0.74
1.10
0.22
2.72 20.34
4.29
5.99
3.62

7 9.12
B.22
9.8
2.94
90
90 .46
0.5
0.44
05
D.44
2
10.19
0.51
2G
00
15 2.21
2.87
0.88
567
53
06
)U
5.15
06
88 7.41 3.10 10.57 10.95 0.74
.6
0.5
02
D.4
0.37
63
0.15
0.22
5.0
)(
0,07 3.79
2.12
2.12
.95
2,4
3
U.U
0.44
.17
8
0.2
3.36
73 9.2
8.98
1.22
9.08

50
0.4
1.87
3.72
2.0
5.5

0.37
0.8 2.93
1.98
0.37
0.22
29
0.1 2.4
5.95
02
2.86
18
5.7
0.00
0.66
26
32
3.67
02 1.32
.26
.4
2.0
34
13
8.74
50
7.8

3,82

9.18 7.96
3.76
1.6
3.2

U.b
0.1E
.70
18
D. 2.22
3.7
3.7
3.7
3.9
)U
0.89
4.2
3.26
3.8
35 2.8
7

85
02
0.44
62

.4
24
0.15

517

63
U.UL
8.7
0.52
0.15
16
0.66
01e
0.15
0.07
0.44
0.07
3.25 0.00 0.37
99
92
41
69 0.0
0.37
75
96
54
31 b.7
0.7
0.00
2.0

0.2 0.66
0.66
87
22
15
88
2 0.07 2
2
0.36
9.83
D.00
0.44
2.04
8.8
3.48
8.30
1.75
1,0
8.3 03 0.2
6.87 6.07 39.96 D.2
2 36 35 06 3.9 D.66 32.75
73
0.74
15
10
0.15
).30 .44 0.22
0.96
0.00
.85 .
0.3
0.29
,43
0.29
0.29
0.37
0.29
0.5
0.07
2.2
2.87
0.74
06
0.44
0.29
0.44
0.22
24.43
0.07
2.72

15
.0
0.3
0.37
1.17
0.37
1.40
0.07
2.28 3.9
0.07
0.22

0.36
0.07
4.23
0.07
0.88
0.58
2.48 4.7 D.6 2g
0.22
0.52
0.37
.33
10.50
10.42
18.77
0.44
1.18
0.74 7.60
0.00
0.81
0.00
1.70
.48
40.68
2.14
0.44
0.59
1.33 3.18
0.37
0.44
0.22
1.26
1.39
2.06
17.01
0.52
0.89
1.18 6.03
0.29
2.50
0.66
0.07
0.00
1.03
0.37
48.82
0.37
0.81 4.80
0.15
3.06
0.15
1.09
2.91
13.25
5.60
2.91
2.26
0.66
0.66
0.44
,32
.70
2.65
7.26
3.98
0.52
0.59
1.84 1.55
.44
4.04
D.30
044
.44
0.22
0.8
0.15
1.55 0.44
0.22
.40
0.15
0.1
0.22
0.00
0.74
0.22
1.1 0.07
2.14
0.07
4
2.72
21.27
4.21
0.89
1.55
1.03 1.23
0.28
4.5
0.28
1.03
0.37
0.75
2.90
0.00
1.68 0.00
3.8
0.07
D.2
1.76
4.84
2.5 0.8
0.15
3.07
0.73
0.07
0.73
0.15
0.51
0.22
1.69 0.15
0.3
2.28
6.6
.54
8.01
2.79
1.98
0.29
2.87
20.76
0.81
0.88
c
9
9
Q 分
分
C
。
分
c>
Expressed Emotion

Percentage%
80
60
40
20
0

Confusion matrix for dynamic facial expression
Perceived Emotion
6
名
Percentage%
80
60
40
20
0

1.16
Anger-
Anxiousness -
Awe- 30.71
2.01
0.85
0.00
0.15 8.55
9.17
0.39
0.77
0.62 0.62
0.15
21.70
0.62
0.3 0.31
0.00
0.00
24.88
6.26 0.39
0.08
0.00
23.8
27.58 3.24 2.08 .93 4.01
.93 0.54 .16 0.69
0.62 0.08 0.00 0.00
0.00 0.15 0.08 0.31
0.31 0.23 0 5.17
2.85
0.23
0.31
0.25 1.47
0.62
0.08
0.23
0.08 0.92
0.08
5.08
4.31
2.47
1.00
1.69
0.15
0.46
08
J.15
7.01
0.39
0.39
4.4
0.08
0.08 5.95
2.63
3.40
0.08
0.08 11.33
2.77
0.0
0.2
0.2 0.85
0.15
0.00
3.00
4.76 2.23
2.62
D.3
0.7
0.38 4.55
85
0.00
0.46
0.6 0.2
0.08
0.54
2.47
1.3
0.39
2.08
02
0.46
0.4E
0.2
3.0
0.0
0.15 2.16
0.3
0.0
0.69
0.85
2.00
2.3
1.00
0.9
0.69
4.7
0.0
0.46
0.69
0.62
0.6
2.0
0.1 2.07
0.69
0.79
0.89
0.59
8.07
3.74
0.49
.1
3.54
0.30
0.00
1.97
2.17
1.57
1.87
2.46
0.00 0.3
0.15
9.48
.3
0.2
1.54
31
0.0C
0.39
08
0.15
3.7(
0.46
.23
2.3
0.08
0.08 2.32
.70
0.08
0.54
0.39
9.96
2.39
0.87
4.6
342
6.72
0.0C
.00
3.17 0.77
1.08
0.08
7.24
3.16

1.62
2.85
.00
5.02
.92
0.0
0.9
.54 0.39
0.00
0.15
4.93
6.09
0.39
3.64 0.54 0.9 0.39
0.15
6
15
0.39
3.4
0.00
0.46 0.23
0.00
0.15
2.47
3.86
0.31
.85
0.54
0.46
0.85
D
A
0.08
0.54
1.25
0.0C
0.2 0.85
0.23
0.08
7.56
9.34
0.15
3.66
0.31
1.54
0.3
0.28
0.77
0.00
0.15
8.18
0.08
0.15 0.62
0.46
11.19
0.31
0.46
11.88
1.77
0.15
0.85
0.46
0.54
0.15
1.47
0.31
0.15
0.08
0.31 2.62
0.31
2.00
3.24
3.08
0.77
2.62
0.62
0.62
0.23
0.23
6.78
0.15
0.77
5.47
0.15
0.08
anNotHelpLaughing
C
Chuckle -
Confusion-
Contempt-
Discouragement -
14.74 16.10 2.78 )15 48
.3
2.85
4.70
24
08
00
46
85
85
.85

0.00 574
02 1588
10.40
16 22.07
47
6.71
667

6.4
b
0,0 10.5E 12.40 3 6 0.70
1.6
3.35
4.16
0.0

0.08
.69
3.08
2.85 8.78
.00
EE
5.78
9.32
2.23
0.
0.62
54
0.54
3.01 1.00 10.65
4.7
0.77
0.0C
31
5.32 1.39
4,47
7.32
10.7
2.08
0.00
1.08
7.63 2.85 08 .78
,08
49
.08
93
00
.36
39
69 6.47
5.24
17
10.48
69
08
15
4.8 0.5F
2.70
.85
2

0,0

.54
0.3

D.4
0.1
0C

0.15 .23 08
0.69 77
0.46 77
15
16

.08 08
24 2.65

5.U

.0
39
Disgust- Distress -
Droop-

D5

58
47
Excitement - Fear -
Fidget -
46
8 56
47
08
ForcedSmile
Grief - 0.6
0.39
3.40
0.08
5.94
2.47
0.46
1.85
0.00
0.08
0.08
0.39 2.39
1.6
6.4
0.08
5.86
1.85
0.00
4.85
0.39
0.00
0.15
0.08
0.15
U.2
2.1
3.4
0.08
0 42
0.08
0.6
0.08
1.08
1.00 40 2.70 4. 9.42 24 .47
D.OC 3
0.5 1.08 46 15 0,39 0.9 3 0.6 0.0E
2.3 14.79
11.06 0.77 0.0
0.1 2.78
20.62 0
1.47
1.85
5.85
0.62
0.38
2.39
4.24
1.31
5.08
2.00
0.77
0.23
6.47
Grievance · Happiness ·
Neutral - ZZ 1.1 5 86 5.5 34.98
5.18
22
0.08
0.92
0.08
4.55
2.39
0.7
3.78 E
2.31
0.23
F
),46
5.62
39
6.55
6.47 0.00 0.00 0.08 23.65 0.00 0.23 2.54 0.54 0.00 36.06 10.1C 0.00
6.41
0.62
0.00
5.91
0.54
0.08
0.08
0.00
0.46
0.08
3.01 8.47
1.31
0.00
0.23
0.31 2.00
14.71
9.71
20.49
0.46
1.39
2.23 5.70
15.18
0.00
0.46
0.39
1.16
1.08
46.53
2.16
0.23
0.31
3.08 8.56
2.47
0.39
0.46
0.15
4.17
2.81
7.48
15.74
0.23
0.85
3.09 0.15 25.73
08
.77
.00
293
0.08
D.08
0.0
0.15 24
23
00

0.5
0
85
6.09 6.4
23
0.00 85
10
.00 93
46
0.00 8.18
0.39
0.00 2.70
0.15
7.33 4.71
2.86 3.24
.69 5.93
0.3
0.00 8.47 7.0 5.4
0.0E
0.00 0.69
28.54
0.00 2.54
0.31
29.76
0.0
0.46
1.08
2.6
1.70 2.31
0.23
2.85
0,46
0.15
0.00
0.23
0.31

0.15
1.77 3.93
0.31
7.47
0,08
2.39
2.70
11.09
3.39
3.24
7.01
2.93
PhysicalDisgust - PositiveSurprise - 3.70
0.00 .3
0.00 0.15
3.09
0.00
1.78
0.9
22.55
2.24
2.08
0.85
039 .00
Sadness- Shyness - SinisterSmile - 5.7
0.3
0.08
0.15
0.15 8
0.54
0,46
0.39
0.15
1.47
0.00 .86
.&
0,00
0.08
0.15
1.16
0.00 7.72
0.15
0.00 0.23
0.00
0.46
0.00 0,23
.00
2.23
6.39
2.31 24
0.15 0,00 0.00
0.08 2.70 0.16
0.08
0.15
0.15 2.38
.93
1.77
11.71
3.31
1.00
0.39 9.7
0.39
0.08
0.2
0.15
0.92
0.08 8.17
0.85
高
0.08
0.0
0.31
0.00 3.2
0.38
0.0E
3.55
0.15
0.62
0.15 2.17 0.1C
0.20
0.00
0.49
3.54
0.00
Smile - Snicker- Surprise - Triumph -
Other -
6.25
0.00
2.78 2.85
0.00
3.08 46.33
0.15
2.86 0.39
0.46
2.23 0.08
0.23
.69 3.54
0.08 1.31
0.77 0.08
0.00 10.40
2.08 28.1
0.00 1.0
0.08 25.98
1.08
3.16 3.54 4.94 4.40 3.16 5.48 2.70 3.78 4.47 6.7E 2.93 2.70 3.55 2.62 3.94 2.47
⁵
g
Q
5
g
分
分
分
0
Q 分
。
C
Expressed Emotion

Accuracy
Emotions that could be conveyed through dynamic and static facial expressions

Condition
Dynamic
Static

Expressed Emotion

Participants could identify 13 emotions from static facial expressions,but 23 emotions from dynamic facial expressions.

Fixed Effects
Anger
Anxiousness
Awe
Can Not Help Laughing Chuckle
Confusion
Contempt
Discouragement
Disgus
Distress
Droop
Excitement
Fear
Fidget
ForcedSmile
Grief
Grievance
Happiness
Neutra
PhysicalDisgust
PositiveSurprise
Sadness
Shyness
SinisterSmile
Smile
Snicker
Surprise
Triumph

Dynamic
Z
21.14
-1.642
7.888
11.25
13.93
36.24
32.13
9.952
16.27
3.343
8.452
-2.004
-0.144
3.216
12.52
5.707
21.45
23.79
35.09
12.22
18.47
3.487
1.142
3.767
28.71
6.812
39.64
-3.326 Pr(>\z
<0.001
0.101 <0.001 <0.001 <0.001 <0.001 <0.001
<0.001 <0.001 <0.001 <0.001
0.045 0.885
0.001
<0.001 <0.001 <0.001 <0.001 <0.001
<0.001
<0.001 <0.001 0.254
<0.001 <0.001 <0.001
≤0.001
<0.001

Static
Z
11.58
-3.633 2.913 -2.425 7.045 22.64
24.39
-1.013
3.633
1.442
5.848
-2.419
-4.053
-2.569
4.69
-1.992
11.09
25.93
31.9
-2.188
1.869
1.093
-3.88
-1.447
23.36
6.764
35.03
-2302 Pr(>|z|)
<0.001
<0.001
0.004
0.015
<0.001 <0.001 <0.001
0.311
<0.001
0.149 <0.001
0.016 <0.001 0.010
<0.001 0.046 <0.001
<0.001 <0.001
0.02g
0.062
0.274 <0.001 0.148
<0.001 <0.001 <0.001
<0.001

Significance level was adjusted using the Bonferroni correction (0.05/28)

Active AU
ActiveAU
动作单元27 -嘴唇张大延伸- 动作单元26-下巴落下-
动作单元25 -微张嘴唇-
动作单元24-紧压嘴唇- 动作单元23 -绷紧嘴唇-
动作单元20 -嘴唇拉伸-
动作单元18 -嘴唇皱起- 动作单元17-下巴提起-
动作单元15-嘴角下撇-
动作单元14-挤出酒窝-
动作单元12-拉动嘴角-
动作单元10 -上嘴唇提起-
动作单元09 -鼻子起皱- 动作单元07 -眼睑收紧-
动作单元06 -面颊提起-
动作单元05-上眼睑提起-
动作单元04 -眉毛降下- 动作单元02 -外侧眉毛提起-
动作单元01-内侧眉毛提起-
Percentage%
100
75
50
25
0

2.3 0.0 18,4 0.0 0.0 1.9 0.0 0.0 0.0 0.0 0.0 25.0 0.0 0.0 0.0 7.7 0.0 1.9 0.0 5.7 17,4 0.0 0.0 0.0 0.0 0.0 11.9 0.0
32.6 47.8 98.0 26.7 18.5 37.0 26.4 37.5 18.0 36.4 13.0 80.0 66.7 37.8 20.4 65.4 32.7 61.5 3.1 57.1 95.7 22.9 33.3 31.2 8.1 25.0 84.7 57.1
58.1 73.9 98.0 57.8 37.0 57.4 50.9 50.0 42.0 54.5 30.4 85.0 74.1 62.2 34.7 69.2 23.1 90.4 12.5 77.1 97.8 22.9 26.7 68.8 14.5 44.4 88.1 64.3
27.9 26.1 0.0 13.3 20.4 18.5 0.0 12.5 6.0 31.8 6.5 0.0 3.7 13.5 4.1 15.4 17.3 0.0 0.0 14.3 2.2 11.4 26.7 0.0 1.6 0.0 0.0 7.1
48.8 52.2 10.2 31.1 33.3 37.0 11.3 15.0 50.0 36.4 6.5 5.0 14.8 48.6 14.3 15.4 28.8 1.9 1.6 45.7 10.9 20.0 26.7 21.9 6.5 22.2 1.7 21.4
25.6 34.8 14.3 15.6 27.8 11.1 15.1 17.5 26.0 15.9 6.5 25.0 29.6 24.3 26.5 46.2 11.5 13.5 4.7 37.1 17.4 20.0 23.3 6.2 22.6 11.1 13.6 42.9
0.0 0.0 12.2 0.0 3.7 0.0 0.0 2.5 0.0 2.3 0.0 0.0 0.0 0.0 0.0 0.0 1.9 0.0 0.0 5.7 0.0 0.0 0.0 0.0 0.0 0.0 1.7 0.0
81.4 69.6 22.4 57.8 81.5 68.5 41.5 77.5 68.0 81.8 63.0 10.0 37.0 40.5 59.2 57.7 100.0 9.6 39.1 54.3 15.2 85.7 83.3 28.1 24.2 33.3 47.5 35.7
11.6 13.0 4.1 22.2 14.8 22.2 15.1 37.5 20.0 27.3 30.4 5.0 3.7 16.2 8.2 26.9 71.2 0.0 6.2 25.7 2.2 57.1 10.0 3.1 3.2 5.6 6.8 0.0
16.3

11.6 26.1 2.0 31.1 35.2 11.1 18.9 25.0 18.0 38.6 0.0 15.0 0.0 29.7 51.0 15.4 5.8 32.7 1.6 5.7 4.3 5.7 43.3 43.8 72.6 33.3 3.4 35.7
26.1 30.6 91.1 88.9 3.7 34.0 7.5 10.0 9.1 6.5 100.0 3.7 16.2 77.6 19.2 1.9 100.0 6.2 8.6 80.4 2.9 56.7 96.9 79.0 88.9 10.2 92.9
30.2 17.4 4.1 20.0 22.2 24.1 20.8 5.0 50.0 11.4 2.2 15.0 33.3 27.0 6.1 61.5 7.7 15.4 0.0 74.3 10.9 22.9 10.0 12.5 1.6 25.0 10.2 21.4
14.0 4.3 2.0 8.9 13.0 11.1 0.0 0.0 30.0 4.5 0.0 10.0 7.4 16.2 2.0 26.9 3.8 0.0 0.0 42.9 0.0 2.9 6.7 9.4 0.0 11.1 1.7 14.3
67.4 87.0 22.4 82.2 79.6 72.2 49.1 40.0 88.0 75.0 39.1 45.0 37.0 81.1 55.1 88.5 53.8 65.4 12.5 85.7 23.9 71.4 50.0 68.8 29.0 63.9 15.3 64.3
2.3 4.3 12.2 71.1 57.4 1.9 7.5 5.0 8.0 4.5 2.2 85.0 3.7 5.4 34.7 15.4 1.9 84.6 1.6 14.3 52.2 0.0 33.3 65.6 27.4 58.3 0.0 64.3
51.2 34.8 69.4 8.9 18.5 35.2 11.3 2.5 30.0 22.7 2.2 50.0 51.9 29.7 6.1 19.2 19.2 15.4 14.1 31.4 73.9 8.6 26.7 15.6 12.9 16.7 72.9 28.6
|36.0 87.0 32.7 24.4 31.5 90.7 37.7 50.0 86.0 86.4 43.5 20.0 66.7 91.9 30.6 96.2 78.8 15.4 31.2 82.9 13.0 80.0 20.0 43.8 17.7 25.0 49.2 21.4
9.3 0.0 53.1 2.2 1.9 7.4 9.4 5.0 0.0 0.0 0.0 45.0 22.2 2.7 2.0 0.0 1.9 13.5 4.7 5.7 63.0 0.0 3.3 6.2 1.6 2.8 57.6 0.0
16.3 26.1 61.2 11.1 5.6 40.7 24.5 22.5 8.0 15.9 6.5 50.0 40.7 16.2 12.2 19.2 21.2 15.4 14.1 22.9 69.6 37.1 10.0 6.2 4.8 0.0 79.7 0.0

Emotion

Decoding Facial Expressions

Manual vs. Automatic FACS Coding

Kpeg
Facial Action Coding System Final Test

Xia Fang
Your scores on the FACS Final Test show you have
PASSED
this test of skill in scoring facial actions.

From the authors of FACS
Paul Ekman,Wallace V.Friesen,Joseph C.Hager

Decoding Facial Expressions

FACS:
https://www.paulekman.com/

Facial
Action
Coding
System

The Manual
on CD ROM
by
Paul Ekman,Ph.D.
Wallace V.Friesen,Ph.D.
Joseph C.Hager.Ph.D.

Pause (k)

0:00/1:32

女

Inside Out

ki-szt.tumtbir nm

hap
Child. 13man

hid s,neutal
Ch
Chid 7,sad

Chid 4,neutek
Chid 12,nau

Child 1 neutral

Chid 11,neute

DISTRACTION
DROWSY
YAWN
EYE CLOSURE
JOY

100
80
0
100

Do faces always tell the truth about feelings ?

nature

画外音：我们
对面部表情的理解程度如何?
Play (k)

image qualit Expression Summary
日 | Settings
No Reut

Surprsed
Dhe

Neuta
Expression Intensity
Neutra
Happy
Sad
Angry
Surprised
Scared
Disgusted
e|n
da 0N Expressions·Action Unts·Events w Facial States
0242 0029.0 00331 0037.31 00414
Stimuli
Event Marken
Neutral
Happy

Noldus
Information Technology

eMOTleNT’

Affectiva

Pleasant

Decoding Facial Expressions

Faceater

Inactive
Expression Intensity
Neutral
Happy
Sad
Angry
Surprised
Scared
Disgusted

Contempt
[]

Automatic FACS Coding

问题：
》识别情绪种类和面部动作有
限； 比如FaceReader 目前只
能判定7种表情，20个面部
动作。

对自然情境下的自发表情识
别信度和效度较低。

The question of universality

·Are there universal expressions of emotion?

LESSONSWORTHSHARING

翻译人员：Chloe Ma
校 对 人 员 ：Jenny Yang

0:00/4:51

The question of universality

Culture A

Culture B

主观感受

行为意图

外在表达
情绪

生理反应
认知评价

⽂化

⾯部表情
声⾳
肢体动作

⾳乐
绘画
电影

A Brief History of Cross-Cultural Studies of Emotion

Tomkins,1962/1963 Frijda,1986 Elfenbein,2002
Izard &Ekman,1977/1992 Ellsworth &Scherer,2003
Russell &Barrett,2003/2006

Cultural Similarities Cultural Differences Ingroup Advantage
Evolutionary Theories of Appraisal and Constructionist Dialect Theories of Emotion
Emotion Theories of Emotion

Universal facial expressions?

·Remote population
· Emotion scenarios
· Six emotions
· Bi-directional

“Basic”facial expressions?

a) Anger
b) Disgust
c) Fear
d) Sadness
e) Surprise
Happiness

Accuracy
十
十

Ingroup advantage

90
80
70
60
50
40
30
20
10
0

十

Culture B
Culture A
Perceiver
group

Culture A
Culture B

Bonjourl
NPMBETI

Salut! Hallo!
ao!
Hello!

   您好

Hit

Solut!
Hola!

Emotional Dialects
Wit

People from different cultural backgrounds have slightly different expressions
of emotions (“dialects”)due to cultural differences and social environments.

Emotional Dialects

Individuals are better at recognizing facial expressions of emotion from their
own culture than from another culture.

Ingroup Advantage

Elfenbein et al.(2007),Emotion

Ingroup Advantage

Prototype

AU1+4+15b

Quebecois

Sadness

AU 4

Gabonese

AU 56

Ingroup Advantage

Table 2
Results of Discriminant Analyses Comparing Culturally Normative Facial Expressions of Emotion From Quebecois Versus Gabonese Posers:Study l

%Correctly classified

Emotion

Wilks’s λ

Chi-square

P

df

Canonica
correlation

Overall

Quebecois

Gabonese

Serenity
Sadness
Contempt
Shame
Happiness
Anger
Surprise .29
.52
.56
.61
.76
.63
.79 66.76
35.05
30.37
26.11
15.29
23.72
13.00 001 .001 .004 .004 .004 .022 .043 8 .84 93.3 100.0 86.7
9 69 81.7 93.3 70.0
13 .67 86.7 90.0 83.3
11 .60 86.7 66.7 76.7
5 .49 76.7 83.3 70.0
12 .61 71.7 66.7 76.7
6 .46 66.7 50.0 83.3

Disgust .75 14.90 .385 14 .50
Fear .83 10.00 .441 10 .42
Embarrassment .74 14.72 .615 17 .51

Ingroup Advantage

Table 5
Means and Standard Deviations for Percentage of Judgment Accuracy With Culturally Normative Expressions as a Function of Intended Emotion,Expressor Cultural Group,and Perceiver Cultural Group:Study 2

Culturally normative
Quebecois Gabonese
Quebecois Gabonese Quebecois Gabonese

Intended
emotion M
SD
M
SD
M
SD
M
SD In-group advantage
Anger 47 19 39 22 26 16 33 26 8
Contemp 37 17 23 14 29 17 42 22 14
Disgust 54 17 30 18 40 16 35 19 10
Embarrassment 31 17 28 18 31 16 26 16 -1
Fear 34 20 28 17 19 18 18 15 3
Happiness 80 18 70 27 78 17 68 33 0
Sadness 39 18 29 18 38 18 38 21 5
Serenity 55 17 43 20 50 21 53 27 8
Shame 28 11 28 15 21 13 25 18
Surprise
Total 56
46 13
17 45
36 23
19 54
39 15
17
39 23
22
45

在不同文化中都很相似。
are quite similar across cultures.

Concurrent Context

16
2 16
7

(Masuda et al.,JPSP,2008)

Concurrent Context

Compared to Westerners,Easterners are influenced more by the
concurrent emotional context when judging others’facial expressions.
(lto et al.,PSPB,2013)

7-38-55 RULE OF COMMUNICATION

WORDS

7%
55%
BODY
LANGUAGE
38%
VOICE
TONE
SPOKEN
WORDS

Mehrabian A.&Wiener M(1967).Decoding of inconsistent communications.JPSP,6(1).

情绪和文化实验室|浙江大学

方霞
x.fang@zju.edu.cn