pytorch训练模型时,因为broadcast机制踩的坑:loss无法收敛

在训练一个非常简单的拟合曲线的模型时,遇到下面的报错:

C:/Users/user/Desktop/test/test.py:58: UserWarning: Using a target size (torch.Size([30000])) that is different to the input size (torch.Size([30000, 1])). This will likely lead to incorrect results due to broadcasting. Please ensure they have the same size.
  loss = F.mse_loss(out, yt)

mse_loss的两个输入参数input和target维度不一样,一个是torch.Size([30000, 1],一个是torch.Size([30000],这两个维度在进行计算时都会将维度广播成torch.Size([30000, 30000],这个时候就出问题了:

mse_loss函数最终输出值是(target-input)每个元素数字平方和除以width x height,也就是在batch和特征维度上都做了平均。但这两个维度在进行广播之后tensor会变成下面这样:

# 原tensor:yt
yt tensor([ 96.6252, -83.4613,  -1.6751,  ...,   1.8656, -15.8007, -30.5789]) torch.Size([30000])

# 广播之后:yt
yt tensor([[ 96.6252, -83.4613,  -1.6751,  ...,   1.8656, -15.8007, -30.5789],
        [ 96.6252, -83.4613,  -1.6751,  ...,   1.8656, -15.8007, -30.5789],
        [ 96.6252, -83.4613,  -1.6751,  ...,   1.8656, -15.8007, -30.5789],
        ...,
        [ 96.6252, -83.4613,  -1.6751,  ...,   1.8656, -15.8007, -30.5789],
        [ 96.6252, -83.4613,  -1.6751,  ...,   1.8656, -15.8007, -30.5789],
        [ 96.6252, -83.4613,  -1.6751,  ...,   1.8656, -15.8007, -30.5789]])

###############################################################################

# 原tensor:out
out tensor([[62.2171],
        [34.9442],
        [92.2927],
        ...,
        [16.6877],
        [35.8723],
        [60.5973]], grad_fn=<MmBackward>) torch.Size([30000, 1])

# 广播之后:out
out tensor([[62.2171, 62.2171, 62.2171,  ..., 62.2171, 62.2171, 62.2171],
        [34.9442, 34.9442, 34.9442,  ..., 34.9442, 34.9442, 34.9442],
        [92.2927, 92.2927, 92.2927,  ..., 92.2927, 92.2927, 92.2927],
        ...,
        [16.6877, 16.6877, 16.6877,  ..., 16.6877, 16.6877, 16.6877],
        [35.8723, 35.8723, 35.8723,  ..., 35.8723, 35.8723, 35.8723],
        [60.5973, 60.5973, 60.5973,  ..., 60.5973, 60.5973, 60.5973]],
       grad_fn=<ExpandBackward>)

这样在进行逐元素计算时,结果就和预想的不一样了,计算出来的loss就发生了变化

评论 3
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

半个女码农

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付:¥1
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值