深度卷积对抗神经网络 进阶 第一部分 GANs 在数据增强和隐私领域的应用 GANs for Data Augmentation and Privacy

深度卷积对抗神经网络 进阶 第一部分 GANs 在数据增强和隐私领域的应用 GANs for Data Augmentation and Privacy

GANs可以创造性地生成数据，这样就可以用在数据增强领域，在某些缺乏数据的行业认为地添加数据。此外，生成的数据如果用于神经网络训练，那么以防某些人利用逆向工程的方式窃取原本的个人数据，那么如何保证个人隐私的安全便也是迫在眉睫的事情。

1. GANs的应用回顾 Application recall

• 图像的转换 image to image:

  

  • 转换风格：translate one style to another using gans (gau GAN)
  • 超清图片：GANs super-resolution GAN
  • 多模态图像转化： Multimodal image-to-image translation

• 文字转化图像Text to image: Image and landmark to Video: Image and face landmarks

• 图像加工和处理：Image filter et image editing et stylized images

• 数据增强： Date Augmentation: increase dataset size

  • Medicine: simulating tissues
  • Climate Change
  • Media: deep fakes

2. GANs 应用的优点和缺点 PROs & CONs

优点 Pros:

• 比手工的更加精细 Can be better than hand-crafted synthetic examples

• 可以生成更多标记的实例用于训练 （使用conditional GAN）Can generate mor labeled examples using conditional gans

• 可以改善下游模型的泛化能力，也就是更加的多样化地生成模型对应的实例。Can improve a downstream model’s generalization (mimic the expert’s method, image segmentation)

缺点 Cons:

• 在数据的多样性，尤其是生成空间的模态数可能很有限，取决于模型训练时所输入到数据的多样性。Can be limited by the available data in diversity
• 可能会有过拟合的现象。Can overfit to the real training data

3. GANs在数据增强上的应用 Data Augmentation

传统的数据增强方法如下图所示，通过旋转，裁切或者更换颜色，灰度等等来进行。但是如果我们有图像的生成器，我们就能够生成不通的狗狗的图片，其拥有不同的模态，而且是多模态的多样性。

4. GANs在个人隐私保护上的应用 GANs for Privacy

GANs生成或者简介改变图片的功能可以有以下的作用：

• 保护实际病人的数据 Protects real patient data
• 鼓励机构之间的数据分享 Can encourage data-sharing between institutions
• 其生成的数据更加廉价以及更加丰富 Less expensive and more abundant than real data

5. GANs在匿名功能上的应用 GANs for Anonymity

其可以将一些敏感的个人信息或者个人视频进行匿名处理，防止泄露个人的肖像照片，或者隐匿个人身份。比如说：

• 隐藏身份 Concealing identity

• 盗取身份Stealing identity

• 深度伪造的一些图片或者视频 DeepFakes： Deepfakes put words into people’s mouths

• 提供安全的表达环境 Provide safe environment for expression

  

6. 参考信息 Reference

(Optional Notebook) Generative Teaching Networks

Please note that this is an optional notebook, meant to introduce more advanced concepts if you’re up for a challenge, so don’t worry if you don’t completely follow! The first author of this work, Felipe Such, reviewed this notebook for you.

Click on this link to access the optional Colab notebook.

In this notebook, you’ll be implementing a Generative Teaching Network (GTN), first introduced in Generative Teaching Networks: Accelerating Neural Architecture Search by Learning to Generate Synthetic Training Data (Such et al. 2019). Essentially, a GTN is composed of a generator (i.e. teacher), which produces synthetic data, and a student, which is trained on this data for some task. The key difference between GTNs and GANs is that GTN models work cooperatively (as opposed to adversarially).

(Optional) Talking Heads

Fascinated by how you can use GANs to create talking heads and deepfakes? Take a look at the paper!

Few-Shot Adversarial Learning of Realistic Neural Talking Head Models (Zakharov, Shysheya, Burkov, and Lempitsky, 2019): https://arxiv.org/abs/1905.08233

(Optional) De-identification

Curious to learn more about how you can de-identify (anonymize) a face while preserving essential facial attributes in order to conceal an identity? Check out this paper!

De-identification without losing faces (Li and Lyu, 2019): https://arxiv.org/abs/1902.04202

(Optional) GAN Fingerprints

Concerned about distinguishing between real images and fake GAN generated images? See how GANs leave fingerprints!

Attributing Fake Images to GANs: Learning and Analyzing GAN Fingerprints (Yu, Davis, and Fritz, 2019): https://arxiv.org/abs/1811.08180