![image-20210629172000459](https://i-blog.csdnimg.cn/blog_migrate/f65380366e3b61acdbcaf00aad9070f3.png)
1. Motivation
- For many tasks, paired training data will not be available.
- We present an approach for learning to translate an image from a source domain X to a target domain Y in the absence of paired examples.
对于Cycle consistent的引入很重要,否则会出现模式崩溃的问题,所有输入图像都映射到相同的输出图像并且优化无法取得进展。
- Moreover, in practice,where all input images map to the same output image and the optimization fails to make progress.
![image-20210629200450492](https://i-blog.csdnimg.cn/blog_migrate/9f59bc3cff9138cbb9152f0c510bdd45.png)
2. Contribution
-
We apply our method to a wide range of applications, including collection style transfer, object transfiguration, season transfer and photo enhancement.
-
We also compare against previous approaches that rely either on hand-defined factorizations of style and content, or on shared embedding functions, and show that our method outperforms these baselines.
3. Method
![image-20210629204834678](https://i-blog.csdnimg.cn/blog_migrate/07adb79a57034920849fe1f58cadc053.png)
3.1 Adversarial Loss
![image-20210629204927778](https://i-blog.csdnimg.cn/blog_migrate/f306689701e063cd026d22f3f10ab038.png)
3.2 Cycle Consistency Loss
![image-20210629205554803](https://i-blog.csdnimg.cn/blog_migrate/b0a36337c64b33a82fd7630c63a6236d.png)
3.3 Full Objective
![image-20210629205635887](https://i-blog.csdnimg.cn/blog_migrate/fe245494251ba1386d929cbd7045407c.png)
![image-20210629205651098](https://i-blog.csdnimg.cn/blog_migrate/15e3211b81f89b3523e60e354f8cf9fd.png)
Cylce GAN中的loss部分,其中与pixel2pixelGAN不同的是,CycleGAN采用的是先update G在D的方法,使用detach()。
训练的时候,是A–>B; B–>A 同时进行的,一次迭代过程需要优化G_A, G_B, D_A, D_B。
def forward(self):
"""Run forward pass; called by both functions <optimize_parameters> and <test>."""
self.fake_B = self.netG_A(self.real_A) # G_A(A)
self.rec_A = self.netG_B(self.fake_B) # G_B(G_A(A))
self.fake_A = self.netG_B(self.real_B) # G_B(B)
self.rec_B = self.netG_A(self.fake_A) # G_A(G_B(B))
def backward_D_basic(self, netD, real, fake):
"""Calculate GAN loss for the discriminator
Parameters:
netD (network) -- the discriminator D
real (tensor array) -- real images
fake (tensor array) -- images generated by a generator
Return the discriminator loss.
We also call loss_D.backward() to calculate the gradients.
"""
# Real
pred_real = netD(real)
loss_D_real = self.criterionGAN(pred_real, True)
# Fake
pred_fake = netD(fake.detach())
loss_D_fake = self.criterionGAN(pred_fake, False)
# Combined loss and calculate gradients
loss_D = (loss_D_real + loss_D_fake) * 0.5
loss_D.backward()
return loss_D
def backward_D_A(self):
"""Calculate GAN loss for discriminator D_A"""
fake_B = self.fake_B_pool.query(self.fake_B)
self.loss_D_A = self.backward_D_basic(self.netD_A, self.real_B, fake_B)
def backward_D_B(self):
"""Calculate GAN loss for discriminator D_B"""
fake_A = self.fake_A_pool.query(self.fake_A)
self.loss_D_B = self.backward_D_basic(self.netD_B, self.real_A, fake_A)
def backward_G(self):
"""Calculate the loss for generators G_A and G_B"""
lambda_idt = self.opt.lambda_identity
lambda_A = self.opt.lambda_A
lambda_B = self.opt.lambda_B
# Identity loss
if lambda_idt > 0:
# G_A should be identity if real_B is fed: ||G_A(B) - B||
self.idt_A = self.netG_A(self.real_B)
self.loss_idt_A = self.criterionIdt(self.idt_A, self.real_B) * lambda_B * lambda_idt
# G_B should be identity if real_A is fed: ||G_B(A) - A||
self.idt_B = self.netG_B(self.real_A)
self.loss_idt_B = self.criterionIdt(self.idt_B, self.real_A) * lambda_A * lambda_idt
else:
self.loss_idt_A = 0
self.loss_idt_B = 0
# GAN loss D_A(G_A(A))
self.loss_G_A = self.criterionGAN(self.netD_A(self.fake_B), True)
# GAN loss D_B(G_B(B))
self.loss_G_B = self.criterionGAN(self.netD_B(self.fake_A), True)
# Forward cycle loss || G_B(G_A(A)) - A||
self.loss_cycle_A = self.criterionCycle(self.rec_A, self.real_A) * lambda_A
# Backward cycle loss || G_A(G_B(B)) - B||
self.loss_cycle_B = self.criterionCycle(self.rec_B, self.real_B) * lambda_B
# combined loss and calculate gradients
self.loss_G = self.loss_G_A + self.loss_G_B + self.loss_cycle_A + self.loss_cycle_B + self.loss_idt_A + self.loss_idt_B
self.loss_G.backward()
def optimize_parameters(self):
"""Calculate losses, gradients, and update network weights; called in every training iteration"""
# forward
self.forward() # compute fake images and reconstruction images.
# G_A and G_B
self.set_requires_grad([self.netD_A, self.netD_B], False) # Ds require no gradients when optimizing Gs
self.optimizer_G.zero_grad() # set G_A and G_B's gradients to zero
self.backward_G() # calculate gradients for G_A and G_B
self.optimizer_G.step() # update G_A and G_B's weights
# D_A and D_B
self.set_requires_grad([self.netD_A, self.netD_B], True)
self.optimizer_D.zero_grad() # set D_A and D_B's gradients to zero
self.backward_D_A() # calculate gradients for D_A
self.backward_D_B() # calculate graidents for D_B
self.optimizer_D.step() # update D_A and D_B's weights