最大均值差异MMD计算出现nan情况

SCUT-Chung

于 2023-03-23 15:11:54 发布

阅读量380

点赞数

文章标签： python pytorch

本文链接：https://blog.csdn.net/m0_46861439/article/details/129731054

版权

文章讨论了在使用PyTorch计算最大均值差异(MMD)时可能会遇到的NaN问题，特别是在p和q矩阵元素相同时。解决方案包括检查数据的线性无关性和调整学习率。代码示例展示了如何计算高斯核以及MMD损失，并提出了在元素全相等时可能出现的问题。

摘要由CSDN通过智能技术生成

MMD(p,q) > or = 0 ,当p=q的时候才会为0
当时使用pytorch计算的时候可能会出现为 nan 的情况
这是因为 p和q矩阵中每个元素都是相同的

# 基于pytorch
import torch
from torch.autograd import Variable

def guassian_kernel(source, target, kernel_mul=2.0, kernel_num=5, fix_sigma=None):
    n_samples = int(source.size()[0])+int(target.size()[0])
    total = torch.cat([source, target], dim=0)
    total0 = total.unsqueeze(0).expand(int(total.size(0)), int(total.size(0)), int(total.size(1)))
    total1 = total.unsqueeze(1).expand(int(total.size(0)), int(total.size(0)), int(total.size(1)))
    L2_distance = ((total0-total1)**2).sum(2)
    if fix_sigma:
        bandwidth = fix_sigma
    else:
        bandwidth = torch.sum(L2_distance.data) / (n_samples**2-n_samples)
    bandwidth /= kernel_mul ** (kernel_num // 2)
    bandwidth_list = [bandwidth * (kernel_mul**i) for i in range(kernel_num)]
    kernel_val = [torch.exp(-L2_distance / bandwidth_temp) for bandwidth_temp in bandwidth_list]
    return sum(kernel_val)#/len(kernel_val)

def mmd(source, target, kernel_mul=2.0, kernel_num=5, fix_sigma=None):
    batch_size = int(source.size()[0])
    kernels = guassian_kernel(source, target, kernel_mul=kernel_mul, kernel_num=kernel_num, fix_sigma=fix_sigma)
    XX = kernels[:batch_size, :batch_size]
    YY = kernels[batch_size:, batch_size:]
    XY = kernels[:batch_size, batch_size:]
    YX = kernels[batch_size:, :batch_size]
    #print(kernels)
    #print("XX:{}\n,YY:{}\n,XY{}\n,YX{}\n".format(XX, YY,XY,YX))
    #print("-"*50)
    loss = torch.mean(XX + YY - XY -YX)
    return loss