模型搭建好后,训练,发现loss不下降,精度不上升…
我枯了。。
iter[110]: accuracy = 0.8114 loss = 899.3430
iter[120]: accuracy = 0.8013 loss = 917.3421
iter[130]: accuracy = 0.8103 loss = 901.3432
iter[140]: accuracy = 0.7969 loss = 925.3407
iter[150]: accuracy = 0.7974 loss = 924.3414
iter[160]: accuracy = 0.7974 loss = 924.3428
iter[170]: accuracy = 0.8131 loss = 896.3424
iter[180]: accuracy = 0.8075 loss = 906.3424
iter[190]: accuracy = 0.8052 loss = 910.3427
iter[200]: accuracy = 0.8002 loss = 919.3431
iter[210]: accuracy = 0.8013 loss = 917.3414
iter[220]: accuracy = 0.7913 loss = 935.3404
iter[230]: accuracy = 0.8131 loss = 896.3430
iter[240]: accuracy = 0.8103 loss = 901.3420
iter[250]: accuracy = 0.8108 loss = 900.3426
iter[260]: accuracy = 0.7963 loss = 926.3401
iter[270]: accuracy = 0.8108 loss = 900.3425
iter[280]: accuracy = 0.8069 loss = 907.3419
iter[290]: accuracy = 0.8025 loss = 915.3429
iter[300]: accuracy = 0.7946 loss = 929.3417
iter[310]: accuracy = 0.8008 loss = 918.3424
iter[320]: accuracy = 0.8041 loss = 912.3424
iter[330]: accuracy = 0.7958 loss = 927.3420
iter[340]: accuracy = 0.7919 loss = 934.3403
iter[350]: accuracy = 0.8013 loss = 917.3424
首先,网上一些loss不下降常见的解决方法,你要对应的检查,尝试一下,是不是有问题
参考博客:https://blog.csdn.net/zongza/article/details/89185852
其中:
- 模型结构是否有问题
- 权重的初始化
- 你的数据和label是否正确
- 优化器的使用是否合理
- 最重要的,损失函数是否正确
- 是否忘记写loss.backward(), 即梯度回传
- 学习率是否过大
按照顺序排除,我确实发现我的人脸当嘴部遮挡的时候,标签写错了,但改过来之后,并没有影响它精度不上升。。。
因为当时同事帮我看了一眼,说搭建的网络结构没问题,我就尝试了其他所有,发现还是不行,此时我觉得应该是网络结构有问题,在模型最后加了几个fc层,每一个fc判断一个部位二分类是否遮挡,在经过sigmoid层,得到概率与label做交叉熵操作,貌似也没问题,最后的改正方法是将fc层的输出直接与label求交叉熵损失,去掉了sigmoid层,模型就收敛了。
好像很容易就改好了,其实并不是,找这个bug,找了一天多,我裂开了。。。。
(这个还是我在另一个代码里发现fc层的输出可以直接和label求损失,不用必须经过sigmoid函数)
原因是因为损失函数用的是:
criterion = nn.CrossEntropyLoss()
而这个函数已经包含了nn.LogSoftmax()和nn.NLLLoss()两个函数,所以不用再写sigmoid或者softmax函数将fc转换为概率了。
优化器也很重要,用的是:
optimizer = torch.optim.Adam(net.parameters(), lr=opt.lr)
发现用SGD,模型就不收敛
另外学习率也很重要,一开始设置的0.1,收敛不明显,震荡,当改成0.01后,收敛就非常快了
lr = 0.0100 iter[145]: accuracy = 0.9916 loss = 37.7496
lr = 0.0100 iter[147]: accuracy = 0.9888 loss = 49.2694
lr = 0.0100 iter[149]: accuracy = 0.9944 loss = 33.9792
lr = 0.0100 iter[151]: accuracy = 0.9905 loss = 51.7469
lr = 0.0100 iter[153]: accuracy = 0.9927 loss = 38.7818
lr = 0.0100 iter[155]: accuracy = 0.9939 loss = 37.6417
lr = 0.0100 iter[157]: accuracy = 0.9905 loss = 46.7828
lr = 0.0100 iter[159]: accuracy = 0.9944 loss = 29.3018
lr = 0.0100 iter[161]: accuracy = 0.9922 loss = 33.4684
lr = 0.0100 iter[163]: accuracy = 0.9939 loss = 27.3714
lr = 0.0100 iter[165]: accuracy = 0.9961 loss = 26.4873
lr = 0.0100 iter[167]: accuracy = 0.9989 loss = 8.3629
lr = 0.0100 iter[169]: accuracy = 0.9955 loss = 24.1883
lr = 0.0100 iter[171]: accuracy = 0.9866 loss = 131.9818
lr = 0.0100 iter[173]: accuracy = 0.9955 loss = 29.4185
lr = 0.0100 iter[175]: accuracy = 0.9883 loss = 57.8468
lr = 0.0100 iter[177]: accuracy = 0.9961 loss = 31.3515
lr = 0.0100 iter[179]: accuracy = 0.9955 loss = 23.5201
说点别的,有些人可能天生性格就不是特别乐于助人的那种,别人帮你是情分,不帮忙是正常的,但你自己可以成长起来,强大起来,成为灯塔,用你认为对的方式去行为处事,去影响别人,照亮别人,所以,继续加油鸭~~~