论文速读：自监督弱光图像增强：Self-supervised Image Enhancement Network: Training with Low Light Images Only

Self-supervised Image Enhancement Network: Training with Low Light Images Only

Yu Zhang, Xiaoguang Di, Bin Zhang, and Chunhui Wang

 

 

本文用信息熵理论和Retinex 模型构建了一个自监督图像增强网络，实现了弱光图像中，亮度与反射系数的分解。其中，最为重要的约束是：反射系数的最大通道与弱光图像的最大通道是conform的，并且前者的熵在本文的模型中应该是最大的。

问题：什么是最大通道？为什么 conform？为什么熵最大？

1. 最大通道是指对每个位置，r、g、b最大的值取出来，构成的灰度图像。
2. 对于弱光图像，最大通道对其视觉效果的影响最大；所以，两幅图像的最大通道应该相符合；
3. 第三点，没大明白，希望懂的读者大神指教。

 

Abstract

This paper proposes a self-supervised low light image enhancement method based on deep learning.

Inspired by information entropy theory and Retinex model, we proposed a maximum entropy based Retinex model. With this model, a very simple network can separate the illumination and reflectance, and the network can be trained with low light images only. We introduce a constraint that the maximum channel of the reflectance conforms to the maximum channel of the low light image and its entropy should be largest in our model to achieve self-supervised learning. 

Our model is very simple and does not rely on any well-designed data set (even one low light image can complete the training). The network only needs minute-level training to achieve image enhancement. It can be proved through experiments that the proposed method has reached the state-of-the-art in terms of processing speed and effect.

 

Problem

As for low light image enhancement task, in the previous work, some solutions such as synthesizing low light images [7], using different exposure time images to obtain data [8], and so on, have achieved good visual effects. However, there are still two problems with those methods. One is how to ensure that the pre-trained network can be used for images collected from different devices, different scenes, and different lighting conditions rather than building new training data set. The other is how to determine whether the normal light image used for supervision is the best, there can be lots of normal light images for a low light image. Usually, the builder of the data set gets the normal light images by experience or artificial adjustment, which will cost lots of time and energy and we cannot make sure that the enhanced image can show the information contained in the low light image to the greatest extent with those normal light images.

传统基于数据集网络模型的两个问题：

1. 传统的数据集都采用一种摄像机采集不同曝光下的图像对；采用这种数据集训练的pre-trained work不能保证对不同摄像机拍摄的、或者不同曝光时间下的、或者其他没有在数据集中出现的场景的图像也都有较好的效果；
2. 给出的真实图像，未必是最好的图像。

 

Strategy

1. Retinex model : decompose low light image into reflectance and illumination;
2. Self-supervised;
3. Information entropy theory : histogram distribution of the maximum channel of the enhanced image should conform to the histogram distribution of the maximum channel of the low light image histogram equalization.

 

Method

• Maximum Entropy Based Retinex model (损失函数的构建）

采用了三个损失函数，分别如下：

Retinex model : S = R ◦ I

reconstruction loss : lrcon = || S − R ◦ I ||1

reflectance loss : lR =|| max c∈R,G,B Rc − F( max c∈R,G,B Sc ) ||1 + λ||△R||1 ,

where, F() means the histogram equalization operator to input image.

这个损失函数是本文的核心，下面解释了为什么提出这样一个损失函数：

This loss function means that maximum channel of the reflectance should conform to the maximum channel of the low light image and has the maximum entropy.

There are three main reasons why we choose the maximum channel to constrain.

Firstly, for a low light image, the maximum channel has the greatest impact on its visual effect.

Secondly, if other channels are selected, there is no doubt that saturation will occur according to the prior that the maximum channel must be greater than the other two channels.

Thirdly, if we choose one of the color channel, such as R, G or B channel, it is obviously not in line with the natural image.

本文的一个重要先验：反射图像的损失。

该损失函数意味着反射系数的最大通道应符合弱光图像的最大通道，且具有最大熵。这个先验的原因在于：

1. 对于弱光图像，最大通道对其视觉效果的影响最大；
2. 如果选择其他通道，那么最大通道将过饱和；
3. 如果我们选择其中一个颜色通道，如R、G或B通道，它显然是不符合自然图像的。
4.  

illumination loss : adopt structure-aware smoothness loss lI = ||△I ◦ exp (−λ3 △R)||1

最后的损失函数就是三者相加。这个公式的解很难，且费时。因此，这个混合损失函数，采用深度学习的方式求其解。

 

• Self-supervised Network Based Solution

only need a very simple CNN structure to achieve the decomposition of the illumination and reflectance. 

网络结构如下图所示。