resnet跑猫狗数据集

记录

1 learning rate 0.01 batch_size 64 全部采用迁移学习，优化函数用的SGD 看到loss一直在0.25左右，准确率也上不去，此时想到几个方案
1 实现手动调节学习率
2 换自适应学习率优化函数 Adam
acc 0.953125
iter177 loss is 0.23283
iter178 loss is 0.29332
iter179 loss is 0.27237
iter180 loss is 0.27875
iter181 loss is 0.29585
iter182 loss is 0.31252
iter183 loss is 0.27854
iter184 loss is 0.30047
acc 0.90625
iter185 loss is 0.25069
iter186 loss is 0.27696
iter187 loss is 0.24047
iter188 loss is 0.25004
iter189 loss is 0.25326
iter190 loss is 0.26660
iter191 loss is 0.25707
iter192 loss is 0.22706
acc 0.96875
iter193 loss is 0.30918
iter194 loss is 0.30031
iter195 loss is 0.26041
iter196 loss is 0.22081
iter197 loss is 0.28045
iter198 loss is 0.23174
iter199 loss is 0.29546
iter200 loss is 0.25297
acc 0.90625
iter201 loss is 0.28606
iter202 loss is 0.21978
iter203 loss is 0.20347
iter204 loss is 0.29359
iter205 loss is 0.23292
iter206 loss is 0.20534
iter207 loss is 0.26971
iter208 loss is 0.26564
acc 0.90625

第二次

使用Adam batch_size=32 初始学习率0.001，120步，最后准确率9.5左右

acc 0.96875
iter105 loss is 0.14243
iter106 loss is 0.13628
iter107 loss is 0.14267
iter108 loss is 0.13569
iter109 loss is 0.13622
iter110 loss is 0.08332
iter111 loss is 0.08634
iter112 loss is 0.11523
acc 0.953125
iter113 loss is 0.21412
iter114 loss is 0.14907
iter115 loss is 0.10260
iter116 loss is 0.12300
iter117 loss is 0.06373
iter118 loss is 0.09578
iter119 loss is 0.11955
iter120 loss is 0.15617
acc 0.9375
训练到第120步
test acc is 0.96875
test acc is 0.9609375
test acc is 0.96907216

第三次

手动调整，初始学习率0.005，看情况调整。几次实验都证明0.005的学习率很不好，放弃这种参数组合。又试了0.01loss跳的更是离谱，还是先0.001吧
acc 0.390625
输入是否暂停0，继续1 或者直到多少再讯问40
iter1 loss is 1.11672
iter2 loss is 1.08927
iter3 loss is 0.83577
iter4 loss is 1.34358
iter5 loss is 1.09488
iter6 loss is 1.57081
iter7 loss is 0.92379
iter8 loss is 1.09501
acc 0.53125
iter9 loss is 1.25106
iter10 loss is 1.25390
iter11 loss is 0.97132
iter12 loss is 1.26648
iter13 loss is 1.11743
iter14 loss is 1.00292
iter15 loss is 1.06702
iter16 loss is 1.34261
acc 0.453125
iter17 loss is 0.80170
iter18 loss is 1.11988
iter19 loss is 0.95434
iter20 loss is 1.44541
iter21 loss is 1.17612
iter22 loss is 1.19897

第四次

手动调整，优化函数 Adam 初始学习率0.001，batch_size64，看情况调整。在70步的时候loss0.15左右，不在下降，此时调整学习率到0.0001，发现loss有所下降，准确率也升至0.98以上，停止训练，查看在测试集上的效果，测试两次，每次128个图片，发现准确率也基本是0.98左右
acc 0.90625
iter49 loss is 0.17640
iter50 loss is 0.17131
iter51 loss is 0.15204
iter52 loss is 0.19296
iter53 loss is 0.24765
iter54 loss is 0.15905
iter55 loss is 0.16615
iter56 loss is 0.25073
acc 0.875
iter57 loss is 0.20932
iter58 loss is 0.14780
iter59 loss is 0.15539
iter60 loss is 0.16140
iter61 loss is 0.20160
iter62 loss is 0.11687
iter63 loss is 0.20982
iter64 loss is 0.15173
acc 0.9375
iter65 loss is 0.20064
iter66 loss is 0.13343
iter67 loss is 0.27181
iter68 loss is 0.17003
iter69 loss is 0.18234
iter70 loss is 0.15138
学习率调整为0.0001，并继续训练到80
iter72 loss is 0.12749
acc 0.96875
iter73 loss is 0.17898
iter74 loss is 0.15269
iter75 loss is 0.14632
iter76 loss is 0.17868
iter77 loss is 0.15235
iter78 loss is 0.13781
iter79 loss is 0.15078
iter80 loss is 0.13460
acc 0.96875
输入是否暂停0，或者直到多少再讯问,或者调整学习率直接输入1
继续训练到90
iter81 loss is 0.14586
iter82 loss is 0.21333
iter83 loss is 0.12632
iter84 loss is 0.12707
iter85 loss is 0.14055
iter86 loss is 0.15701
iter87 loss is 0.11231
iter88 loss is 0.09056
acc 1.0
iter89 loss is 0.07519
iter90 loss is 0.18150
输入是否暂停0，或者直到多少再讯问,或者调整学习率直接输入1
继续训练到100
iter91 loss is 0.11225
iter92 loss is 0.18190
iter93 loss is 0.08836
iter94 loss is 0.11323
iter95 loss is 0.16332
iter96 loss is 0.10887
acc 0.984375
iter97 loss is 0.13629
iter98 loss is 0.14404
iter99 loss is 0.09832
iter100 loss is 0.20285
输入是否暂停0，或者直到多少再讯问,或者调整学习率直接输入1
继续训练到110
iter101 loss is 0.10193
iter102 loss is 0.13885
iter103 loss is 0.12020
iter104 loss is 0.12967
acc 0.984375

test acc is 0.9765625
test acc is 0.984375

总结

第四次学习效果最好，学习率初始0.001，loss不再下降时，调整学习率到更低（0.0001），可以让loss进一步减小，Adam要比SGD效果好