通过AI为黑白照片着色。-CSDN博客

本文介绍了如何利用深度学习中的生成对抗网络(GAN)为黑白照片着色。作者参照了DeOldify项目，用ImageNet数据集训练了一个GAN。在预训练了生成器和鉴别器后，模型能够为树木、草地和人的皮肤等进行较为逼真的着色，尽管某些颜色可能不完全现实。此外，还使用Streamlit创建了一个简单的Web应用程序来展示这一过程。

摘要由CSDN通过智能技术生成

介绍 (Introduction)

When I see a historical black and white photo, I always wonder what was the real color, what did the photographer see when taking the photo.

当我看到一张历史性的黑白照片时，我总是想知道真正的色彩是什么，摄影师在拍摄照片时看到了什么。

It is not easy to discover the exact color of the B&W photos, but it is possible to colorize the photo based on experience and imagination. It can be done by investigating the possible colors of the objects in the photos, for example, the color of cloths, building, trees, cars, etc., and colorized them manually using some software such as Photoshop.

发现黑白照片的确切颜色并不容易，但是可以根据经验和想象为照片着色。可以通过研究照片中对象的可能颜色(例如，衣服，建筑物，树木，汽车等的颜色)并使用某些软件(例如Photoshop)手动为它们上色来完成此操作。

The other approach is to train the computer to understand the colors of different objects by providing a large of amount color photos, then ask the computer to colorize by identifying plausible color for the detected objection. Thanks to the development of deep learning, it can be done by using Generative Adversarial Network (GAN).

另一种方法是通过提供大量彩色照片来训练计算机以了解不同对象的颜色，然后通过识别检测到的异物的合理颜色来要求计算机着色。由于深度学习的发展，可以使用生成对抗网络(GAN)来完成。

生成对抗网络(GAN) (Generative Adversarial Network (GAN))

The idea of Generative Adversarial Networks is quite straightforward, it contains two neural networks, generator, and discriminator. The generator’s job is to predict the color based on the black and white photo and then generate colorized photos. The discriminator’s job is then to judge if the generated photo is real enough compared to the real photo.

生成对抗网络的思想非常简单，它包含两个神经网络，即生成器和鉴别器。生成器的工作是根据黑白照片预测颜色，然后生成彩色照片。鉴别者的工作是判断所生成的照片与真实照片相比是否足够真实。

If the discriminator can easily tell the photo is generated, which means the generator is not good enough, the generator needs more training. When the generator is improving and the discriminator cannot tell the difference anymore, the discriminator will be trained more in order to tell the difference.

如果辨别器可以轻松辨别出照片已生成，这意味着生成器不够好，则生成器需要更多的培训。当生成器正在改进并且区分器无法再区分差异时，将对区分器进行更多的训练以区分差异。

In this work, I took Jason Antic’s DeOldify as reference (https://github.com/jantic/DeOldify) and trained a GAN using fastai.

在这项工作中，我以Jason Antic的DeOldify作为参考( https://github.com/jantic/DeOldify )，并使用fastai训练了GAN。

数据集 (Dataset)

I used ImageNet as my training dataset. However, due to the computation limit (12 hours runtime limit on Colab), it was not easy to use all the images, so I ended up using only 5000 images from it.

我使用ImageNet作为训练数据集。但是，由于计算限制(Colab的运行时间限制为12小时)，要使用所有图像并不容易，所以我最终只使用了其中的5000张图像。

Image for post — Part of the training dataset

制作黑白照片 (Make black and white photos)

First I need to make the color image black and white for training. I also added a little bit of noise to give the images some kind of old photo feeling.

首先，我需要对彩色图像进行黑白训练。我还添加了一些噪点，使图像具有某种旧照片的感觉。

预训练发生器 (Pre-train generator)

Now I have the dataset ready, I can start training. I created U-Net learner for the generator and pre-trained it before the GAN. It can be done easily by fastai library.

现在我已经准备好数据集，我可以开始训练了。我为生成器创建了U-Net学习器，并在GAN之前对其进行了预培训。可以通过fastai库轻松完成。

I started with a small size (64 pixel), so I can train it faster with a larger batch size.

我从一个小尺寸(64像素)开始，所以我可以以更大的批量来更快地训练它。

Then I increased the size to 128 and 256. I started to see some good results with more epochs.

然后将大小增加到128和256。随着时间的推移，我开始看到一些不错的结果。

训练前鉴别器 (Pre-train discriminator)

After the generator pre-training, I used the generator to generate color photos from the black&white dataset. Then I used these generated photos and the original photos to train the discriminator.

在对生成器进行预训练之后，我使用生成器从黑白数据集中生成彩色照片。然后，我使用这些生成的照片和原始照片来训练鉴别器。

After 10 epochs, the discriminator was able to tell if the image is generated with 85% accuracy.

经过10个时间段后，鉴别器便能够分辨出图像是否以85％的精度生成。

甘 (GAN)

Now it is time to put them together. Fastai provides a very useful tool for creating GAN, combining the pre-trained generator and discriminator. After 50 epoch training, I got my preliminary results.

现在是时候将它们放在一起了。 Fastai提供了一个非常有用的工具来创建GAN，将预训练的生成器和鉴别器结合在一起。经过50历时的训练，我得到了初步结果。

结果 (Results)

I downloaded some black and white photos which the model has not seen before for testing. Here are some of the results.

我下载了一些黑白照片，这些照片是该模型以前没有进行测试的。这是一些结果。

The model is able to colorize the tree and grass. It also did a good job of colorizing human’s skin.

该模型能够为树木和草着色。它还可以很好地使人的皮肤着色。

Some colors are not really realistic but still quite convincing.

有些颜色不是很真实，但仍然很有说服力。

Please be noted that this was trained by only 5000 images and not a lot of epoch, it might be improved further by doing more intensive training.

请注意，这仅通过5000张图像进行了训练，而且时间不多，可以通过进行更深入的训练来进一步改善。

Web应用程序 (Web Application)

A web app prototype can be quickly created using Streamlit. It does not require a lot of HTML and Java skills, so it is really helpful for web dummy like me.

可以使用Streamlit快速创建Web应用程序原型。它不需要很多HTML和Java技能，因此对于像我这样的网络虚拟人真的很有帮助。