Image Inpainting for Irregular Holes Using Partial Convolutions理解与电镜图像应用

最新推荐文章于 2022-10-01 09:36:04 发布

Dr.S

最新推荐文章于 2022-10-01 09:36:04 发布

阅读量1.6k

点赞数

文章标签： Image Inpainting 电镜图像处理

本文链接：https://blog.csdn.net/mamawoaini1314v/article/details/87653118

版权

Image Inpainting for Irregular Holes Using Partial Convolutions

Partial Convolutional Layer

在这里插入图片描述
只对non-hole的部分做卷积运算

可以理解为对mask做卷积运算，卷积核的权值都为1，相当于对slide window对应的mask小区域求和，也就是公式中的sum(M)，如果结果大于0，该小区域卷积之后的结果为1，otherwise为0。这就做到了mask的更新只要网络深度足够大，mask最终都为1。

Network Architecture

在这里插入图片描述 ## Loss Functions

在这里插入图片描述

These are the L 1 losses on the network output for the hole and the non-hole pixels respectively.

The perceptual loss computes the L 1 distances between both I out and I comp and the ground truth, but after projecting these images into higher level feature spaces using an ImageNet-pretrained VGG-16. Ψ p I ∗ is the activation map of the pth selected layer given original input I ∗ . We use layers pool1, pool2 and pool3 for our loss.
在这里插入图片描述
We further include the style-loss term, which is similar to the perceptual loss, but we first perform an autocorrelation (Gram matrix) on each feature map before applying the L 1 .

Our final loss term is the total variation (TV) loss L tv : which is the smoothing penalty on R, where R is the region of 1-pixel dilation of the hole region.