MMD(p,q) > or = 0 ,当p=q的时候才会为0
当时使用pytorch计算的时候可能会出现 为 nan 的情况
这是因为 p和q矩阵中每个元素都是相同的
# 基于pytorch
import torch
from torch.autograd import Variable
def guassian_kernel(source, target, kernel_mul=2.0, kernel_num=5, fix_sigma=None):
n_samples = int(source.size()[0])+int(target.size()[0])
total = torch.cat([source, target], dim=0)
total0 = total.unsqueeze(0).expand(int(total.size(0)), int(total.size(0)), int(total.size(1)))
total1 = total.unsqueeze(1).expand(int(total.size(0)), int(total.size(0)), int(total.size(1)))
L2_distance = ((total0-total1)**2).sum(2)
if fix_sigma:
bandwidth = fix_sigma
else:
bandwidth = torch.sum(L2_distance.data) / (n_samples**2-n_samples)
bandwidth /= kernel_mul ** (kernel_num // 2)
bandwidth_list = [bandwidth * (kernel_mul**i) for i in range(kernel_num)]
kernel_val = [torch.exp(-L2_distance / bandwidth_temp) for bandwidth_temp in bandwidth_list]
return sum(kernel_val)#/len(kernel_val)
def mmd(source, target, kernel_mul=2.0, kernel_num=5, fix_sigma=None):
batch_size = int(source.size()[0])
kernels = guassian_kernel(source, target, kernel_mul=kernel_mul, kernel_num=kernel_num, fix_sigma=fix_sigma)
XX = kernels[:batch_size, :batch_size]
YY = kernels[batch_size:, batch_size:]
XY = kernels[:batch_size, batch_size:]
YX = kernels[batch_size:, :batch_size]
#print(kernels)
#print("XX:{}\n,YY:{}\n,XY{}\n,YX{}\n".format(XX, YY,XY,YX))
#print("-"*50)
loss = torch.mean(XX + YY - XY -YX)
return loss
情况1:p=q但元素不全部相等
情况2:p=q且 全部元素相等(一般是这个)
出现这个情况推荐将 学习率lr降低(正常数据都不会完全相同的,可能出现的情况是经过了只有bias的神经网络)
或者 行向量不是线性无关组