Implicit Analysis（二）

The perceptual experience of multimedia has diverse factors, including QoE, emotion, aesthetic satisfaction, preference,fatigue, attention
多媒体引起的知觉信息包括——用户体验，情感，美学满足，偏好，疲劳和注意力

Physiological signals can be divided into two categories: brain activities and peripheral physiological activities
生理信号包括脑活动信号和外周神经活动信号

其中，脑信号为：Cerebral signals measured by electroencephalography (EEG), magnetoencephalography (MEG), functional magnetic resonance imaging (fMRI), functional near-infrared spectroscopy (fNIRS)
外周神经活动信号为：galvanic skin response (GSR), skin temperature, respiration, heart rate, etc., are
induced by the activities of the peripheral nervous system

EEG信号的分类：
1. ERP时间相关电位：The event induces changes of phases and amplitudes of EEG signals, which typically occurs in a few hundred milliseconds before and after the event. Commonly, changes that occur within 100 ms after an event are influenced by the sensory inputs, whereas later changes reflect cognitive processes
2. Power-Based Features：alpha (8-13 Hz), beta (14-30 Hz), gamma (31-50 Hz), delta (1-3 Hz), and theta (4-7 Hz),are separately extracted from EEG signals. Generally, high amplitude and low frequency signals are observed in calm states,and low amplitude and high frequency signals are observed in alert states
3. Connectivity-Based Features：Structural connectivity concerns how different brain regions are anatomically connected. Functional connectivity measures (undirected) statistical dependence between separate brain regions. Finally, effective connectivity describes (directed) causal relationships between brain regions
（其他不再赘述）

脑信号和外周神经信号的优劣对比：Cerebral features tend to be more informative than features extracted from the peripheral signals. However, cerebral features are more likely to contain information irrelevant to the target factor than peripheral features

情感计算的最大问题——many-to-many

the difficulty of physiological analysis of emotion stems from the fact that physiological signals and emotion do not have one-to-one relationship but many-to-many relationship

脑信号——

• 杏仁核与情绪有关：In [53], negative emotions such as fear and sadness were induced by electrical
  stimulations of the right amygdala, and both positive and negative emotions were induced by electrical stimulations of the left amygdala
• ERP：ERP changes due to affective stimuli are more prominently observed in their amplitudes rather than latencies.
• 信号的不对称反映情绪：The asymmetry index, which refers to the level of asymmetry of neural activities in the left and right hemispheres of the brain, is regarded as an effective representation of emotion
• 相位同步反映情绪：the phase synchronization was observed during emotional stimulation. In particular, happiness was associated with synchronization in a wide area of frontal and occipital sites, while sadness influenced only the frontal lobe

外周神经信号——
多媒体刺激更容易被外周神经信号捕获

• average accuracies using peripheral signal features for classification of high vs. low valence and liking vs. disliking were higher than those using EEG features
• The EMG electrodes were placed at the upper trapezius muscle to measure the stress level
• the features extractedfrom ECG and respiration were effective for classification of the valence level

QoE:It is closely related to but more user-centricthan the traditional quality of service (QoS) that is rather device-, infrastructure-, and signal-centric, such as signal-to-noise ratio(SNR), delay, packet loss rate, etc
理解为画质模糊，影像重叠，声音破音等用户体验问题对电生理的影响
Aesthetics：美学
Fatigue：疲劳

未来发展

1. 可穿戴设备
2. 深度学习
   1）无监督特征提取
   2）多模态，跨模态
3. 偏好分析
4. 个人化
   the performance of recognizing perceptual experience in a subject-independent scheme is mostly poorer than that in a subject-dependent scheme
   it is impossible to completely overcome such subjectivity, it will be desirable in the future to investigate ways of discovering physiological features common across different individuals
5. 环境相关
   perception of the stimulus varies depending on the persons environment, e.g., mobile vs. desktop conditions, home vs. workplace, and alone vs. in crowds

6. 公开数据集

   【1】Implicit Analysis of Perceptual Multimedia Experience Based on Physiological Response: A Review