模型梯度为零/莫名很多nan值-CSDN博客

本文链接：https://blog.csdn.net/yihang___/article/details/140723750

复现：

a1 = torch.tensor(([[-1.8462517], [-1.6996475], [-1.6996043], [-1.6536701], [-1.5806862], [-1.576904 ], [-1.7009722], [-1.7114887], [-1.9303117], [-1.8057854], [-1.9342886], [-1.8744069], [-1.9387085], [-1.8926618], [-1.7802298], [-1.8745576], [-1.798627 ], [-1.8225772], [-1.7066717], [-1.7475389], [-1.7566582], [-1.7850085], [-1.8122517], [-1.6183958], [-1.622589 ], [-1.5926841], [-1.7001287], [-1.530619 ], [-1.8174226], [-1.922854 ], [-1.8523171], [-1.863019 ], [-1.828721 ], [-1.8192344], [-1.7720382], [-1.6729505], [-1.7575488], [-1.8308074], [-1.8519098], [-1.8641762], [-1.7812943], [-1.7846534], [-1.8493922], [-1.9107132], [-1.8032849], [-1.8501979], [-1.887317 ], [-1.9013412], [-1.7885242], [-1.8767337], [-1.810638 ], [-1.8970709], [-1.8163861], [-1.8840438], [-1.820437 ], [-1.8448343], [-1.848794 ], [-1.8697724], [-1.7473991], [-1.8704672], [-1.893554 ], [-1.7552618], [-1.7717416], [-1.7990098], [-1.7435725], [-1.8348565], [-1.7796655], [-1.7834994], [-1.8406482], [-1.7503937], [-1.8263179], [-1.7891648], [-1.8076115], [-1.6707844], [-1.7044897], [-1.9526197], [-1.7945414], [-1.7960532], [-1.8125875], [-1.7580117], [-1.7450215], [-1.8912759], [-1.5726353], [-1.6368845], [-1.7623049], [-1.7626779], [-1.8233726], [-1.7546546], [-1.8273476], [-1.7913294], [-1.8337822], [-1.806838 ], [-1.7181054], [-1.81465  ], [-1.6504749], [-1.6457516], [-1.642711 ], [-1.6748763], [-1.7524449], [-1.7269801], [-1.7843764], [-1.7518194], [-1.8675231], [-1.9139432], [-1.8547994], [-1.8093313], [-1.8078493], [-1.776361 ], [-1.6497005], [-1.6589589], [-1.668573 ], [-1.6395924], [-1.778931 ], [-1.8940191], [-1.7269216], [-1.7559772], [-1.6983002], [-1.7044072], [-1.647506 ], [-1.4078465], [-1.6137681], [-1.6815753], [-1.6097116], [-1.6609125], [-1.7553643]]))

a1.shape
Out[172]: torch.Size([125, 1])

a2 = 0.999

a3 = 0.9999503

p9_  = ((a1 + a2) ** a3).float()

p9_
Out[176]: 
tensor([[nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan],
        [nan]])

搞了我一天，干！

torch.tenso

torch.tensor([-0.9]) ** torch.tensor(2.1)
Out[178]: tensor([nan])

因为在模型中用了负数+指数。