5 multimodal perception 笔记

processing multiple signals

theories of information processing:

• for at least speech/audio and visual/spatial information
• not more demanding than unimodal processing
• more salient, can result in stronger neuronal activity
  1 super-additivity, large than sum of 2
  2 reduced variance in object recognition, more reliable

Using multimodal system output will improve …

• reaction times(perception, not necessarily interaction)
• salience/prominence(degree of attention grabbed)
• decreased workload
• robustness(certainty of being perceived grabbed)
• naturalness/authenticity(really synthesize)
• shorter task completion time

each modality is unique

• visual system: detailed/spatial information; complex information in HCI
• auditory system: general/temporal information;