假想观众
This is part of a series describing the development of Moviegoer, a multi-disciplinary data science project with the lofty goal of teaching machines how to “watch” movies and interpret emotion and antecedents (behavioral cause/effect).
这是描述 Moviegoer 开发的系列文章的一部分, Moviegoer 是一个跨学科的数据科学项目,其崇高目标是教学机器如何“观看”电影以及解释情感和前因(行为因果)。
Continued progress has helped clarify the overall goals of the project — we’ve identified four broad categories of knowledge that Moviegoer must identify and recognize. These categories aren’t tied to any specific aspect of the tech stack, and advances in one category may support another. Much like a human viewer, a machine must be able to parse four categories of comprehension to “watch a movie”: film structure; characters; plot and events; and emotional and style features.
持续的进展有助于阐明该项目的总体目标-我们确定了Moviegoer必须识别和认识的四大类知识。 这些类别与技术堆栈的任何特定方面都没有关系,一个类别的进步可能会支持另一类别。 就像人类的观众一样,机器必须能够解析四种理解以“观看电影”:电影结构; 字符 情节和事件; 以及情感和风格特征。
Structure
结构体
An individual scene is a granular, self-contained component of every film. It has a fixed location, a set number of characters, and conveys one or more story beats. A scene can be analyzed individually, or compared against other scenes. Earlier in the project, we created an algorithm to identify a specific type of scene: the two-character dialogue scene. But we’ll need to be able to divide the entire film into its individual scenes.
单个场景是每部电影的颗粒状,独立的组件。 它具有固定的位置,一定数量的角色,并传达一个或多个故事节拍。 可以单独分析一个场景,也可以将其与其他场景进行比较。 在项目的早期,我们创建了一种算法来识别特定类型的场景:两个字符的对话场景。 但是我们需要能够将整部电影分为各个场景。
We’ll also want to divide the film into its eight sequences. Many films follow the eight-sequence-approach, which can be thought of a more detailed breakdown of the three-act structure. These eight sequences, each lasting roughly 15 minutes in a two-hour film, denote (broadly) when major plot points are supposed to be unfold and when new characters might be introduced. Each of the eight sequences ends in a climax — this could be an important clue when identifying major plot points.
我们还想将电影分为八个序列 。 许多电影都遵循八序列方法,可以考虑对三作用结构进行更详细的分解。 这八个序列在一个两小时的电影中持续大约15分钟,表示(广泛地)表示主要的情节点应该被展示以及何时引入新角色。 八个序列中的每个序列都以高潮结尾-这可能是识别主要绘图点时的重要线索。
Characters
性格
We’ll need to persistently track characters throughout the entire film, to track their events and emotional changes. We can look for the vectorizations of their face and voice throughout the entire film, locating in which scenes they appear. We’ll also need to attribute dialogue to each character, using NLP on the subtitles to understand what they’re saying.
我们需要在整部电影中持续跟踪角色,以跟踪其事件和情感变化。 我们可以在整部电影中寻找他们的脸和声音的矢量化,并确定它们出现在哪个场景中。 我们还需要将对话赋予每个角色,使用字幕上的NLP理解他们在说什么。
Films elicit responses through their characters’ emotions, and we’ll also need to monitor their emotions throughout the film. We can track their ups and downs through analysis of their facial expression, voice tone, and word choice, and see what antecedents triggered those emotional changes.
电影通过角色的情感引起回应,我们还需要在整部电影中监控他们的情感。 我们可以通过分析他们的面部表情,语气和单词选择来追踪他们的兴衰,并查看是哪些前因触发了这些情感变化。
Plot and Events
情节和事件
A plot consists of many different events and happenings. We’ll need to use context to understand where a scene is taking place, and what’s happening. Maybe an outdoor scene on a boat can be identified by the sound of waves crashing. Dialogue with a previously-unknown character about appetizers or entrees may hint a character is ordering with the waiter at a restaurant.
一个情节包括许多不同的事件和事件。 我们需要使用上下文来了解场景在哪里发生以及正在发生什么。 也许可以通过海浪的撞击声来识别船上的户外场景。 与先前未知的有关开胃菜或主菜的人物进行对话可能暗示该人物正在餐厅与服务员点菜。
This particular category might be the most difficult to populate, and its conclusions might be filled with qualifiers and “best guesses”.
这个特定类别可能是最难填充的类别,其结论可能包含限定词和“最佳猜测”。
Emotional and Style Features
情感和风格特征
Emotional and style features are somewhat “intangible”, and subject to interpretation. These are directorial choices used to elicit specific emotions in the audience. Music score is the most prominent example — although we understand this music doesn’t actually exist within the scene, it’s been layered on top of it to make the audience feel sad, excited, tense, or a multitude of other emotions.
情感和风格特征在某种程度上是“无形的”,并且需要解释。 这些是导演的选择,用于引起观众的特定情感。 乐谱是最突出的例子-尽管我们知道这种音乐实际上并不存在于场景中,但它已经被叠加在音乐之上,以使听众感到悲伤,激动,紧张或其他多种情感。
Color and brightness can easily be quantified with computer vision. Dark scenes might be moody or uneasy. A scene tinted blue is “cool”, and the location or situation could be inhospitable or foreign.
颜色和亮度可以通过计算机视觉轻松量化。 黑暗的场景可能让人情绪低落或不安。 蓝色场景是“凉爽”的,其位置或情况可能不友好或陌生。
The cinematography, or shot choice, can also be scrutinized. A character’s face may fill the frame to emphasize a facial reaction, or we may see his entire body from afar to emphasize loneliness or emotional distance. A shot might be looking down at a character to make her seem powerless, or looking up at her for the opposite effect.
也可以检查摄影或镜头选择。 角色的脸可能会填满整个框架以强调面部React,或者我们可能从远处看到他的整个身体以强调孤独或情感距离。 投篮可能是低头看着一个角色,使她看起来无能为力,或者抬头看着她,以获得相反的效果。
This category of comprehension is simultaneously the most powerful as well as the most debatable. Recognizing these clues (and coding them into the project) relies heavily on domain knowledge in filmmaking. These empirical rules have evolved from over a century’s worth of advances in filmmaking, and require a strong understanding of the craft. At the same time, some directors will consciously flout these rules as an artistic choice, and Moviegoer must be ready to accept these scenarios. But, if a style rule helps us interpret emotion in 99% of films, across all genres, it’ll greatly help in interpreting films.
这种理解同时是最强大和最有争议的。 识别这些线索(并将它们编码到项目中)在很大程度上依赖于电影制作中的领域知识。 这些经验法则是在电影制作上一个多世纪的发展过程中发展而来的,需要对电影制作有深刻的理解。 同时,一些导演会自觉地视这些规则为艺术选择,Moviegoer必须准备接受这些场景。 但是,如果风格规则可以帮助我们解释所有类型的99%电影中的情感,那么它将对解释电影有很大帮助。
Wanna see more?
想看更多吗?
Repository: Moviegoer
资料库: Moviegoer
Project Intro: Can a Machine Watch a Movie?
项目简介: 机器可以看电影吗?
Previous Story: Dialogue Attribution
上一故事: 对话归因
Next Story: Vision Features
下一个故事: 视觉功能
翻译自: https://medium.com/@moviegoer/moviegoer-four-categories-of-comprehension-ea0d25ef0886
假想观众