gaugan使用教程_在真实环境中使用gaugan绘制故事的内部世界

gaugan使用教程

Both AI and interactive storytelling are complex and unpredictable systems. As we deepened Marrow’s design process, the challenge of combining those two systems into one coherent experience became apparent. On the one hand, as the authors, we developed AI systems and real-time interactions to lead the flow of the experience. On the other hand, we wished to tell a story that also provokes the participants’ imagination and emotions.

AI和交互式讲故事都是复杂且不可预测的系统。 随着我们对Marrow的设计过程的加深，将这两个系统组合为一个连贯的体验的挑战变得显而易见。 一方面，作为作者，我们开发了AI系统和实时交互来引导体验。 另一方面，我们希望讲一个故事，它也激发了参与者的想象力和情感。

Marrow is a story about the possibility of mental illness in machine learning models, focusing mainly on Generative Adversarial Networks (GAN). We question what kind of mental disorders could emerge in advanced AIs, and invite participants to play in an interactive theater scene operated by GAN. Together they play as one dysfunctional family of AIs. Since we are dealing with very abstract and complex concepts, we wanted to explore multiple ways to communicate the story, more than just through dialogue between the family members. Our tactic was to make the room more ‘alive,’ reflecting on the embodied models’ mental state. We wanted to dissolve the barriers between the participants and the environment; to slowly immerse them in an unfamiliar magical experience within that room. The room and the dinner scene were an invitation to let go and indulge in an emotional affair with three other strangers.

Marrow是有关机器学习模型中精神疾病的可能性的故事，主要关注于生成对抗网络(GAN)。 我们质疑先进的AI中会出现哪种精神障碍，并邀请参与者在GAN运营的互动剧场中表演。 他们在一起扮演着功能失调的AI家族。 由于我们处理的是非常抽象和复杂的概念，我们希望探索多种交流故事的方式，而不仅仅是通过家庭成员之间的对话。 我们的策略是让房间更加“活跃”，以体现具体模型的心理状态。 我们希望消除参与者与环境之间的障碍； 让他们慢慢沉浸在那个房间里不熟悉的魔法体验中。 房间和晚餐现场都是一个邀请，让他们放纵自己，与其他三个陌生人沉迷于情感上。

Photo by Andre Bendahan at the NFB labs 2020 © All rights reserved

安德烈·本达汉(Andre Bendahan)在NFB实验室2020中的照片©版权所有

In practice, this meant that we had to implement GAN networks that frequently interacted with the environment and with the participants. Since GAN’s training process does not happen in real-time, this became a challenge of manipulating the output of a pre-trained GAN network in response to real-time changes in the environment. To explain our solution, we first need to look at the difference between standard GANs and conditional GANs.

实际上，这意味着我们必须实施经常与环境和参与者交互的GAN网络。 由于GAN的培训过程不是实时发生的，因此，应对环境的实时变化来操纵经过预训练的GAN网络的输出成为一个挑战。 为了解释我们的解决方案，我们首先需要研究标准GAN与条件GAN之间的区别。

标准VS条件GAN (Standard VS Conditional GANs)

In its basic form, GAN is trained to produce new images that are visually similar to the training set. If we used a dataset of faces, it would generate new faces. If we trained it on cats, it would render new cats. It can maintain variability (not producing the same image every time) by taking an input ‘noise’ vector (essentially a series of random numbers) and using them as the basis for the output image. Thus, if we want to connect GAN’s output to changes in the environment, we need to manipulate the noise vector based on those changes. However, as we showed in our previous post, there is hardly any control over what kind of change would emerge as a result of changing the noise vector.

在其基本形式中，GAN受过训练以生成视觉上类似于训练集的新图像。 如果我们使用面部数据集，它将生成新的面部。 如果我们在猫身上训练它，它将渲染出新的猫。 通过获取输入的“噪声”矢量(本质上是一系列随机数)并将其用作输出图像的基础，它可以保持可变性(每次都不会生成相同的图像)。 因此，如果我们想将GAN的输出与环境的变化联系起来，则需要基于这些变化来操纵噪声矢量。 但是，正如我们在前一篇文章中所展示的那样，几乎没有任何控制权会由于更改了噪声矢量而产生什么样的变化。

We could link different variables in the physical room (such as the participants’ position, the position of objects, and mood analysis of the participants) to the generated output, but the lack of precise control over the output results in a tenuous connection to the environment.

我们可以将物理空间中的不同变量(例如，参与者的位置，对象的位置以及参与者的情绪分析)链接到生成的输出，但是由于缺乏对输出的精确控制，导致与输出的紧密联系。环境。