实验比较[参数初始化方式]与[ImageNet预训练参数]对模型训练和最终性能的影响

最新推荐文章于 2021-09-12 17:13:45 发布

VIP文章 huangpg丶

最新推荐文章于 2021-09-12 17:13:45 发布

阅读量3k

点赞数 2

分类专栏： Deep Learning

本文链接：https://blog.csdn.net/u013978977/article/details/84670305

版权

这篇文章主要是通过实验比较模型的参数初始化方式，以及是否使用ImageNet预训练参数初始化网络对训练过程和模型性能的影响。

实验项目采用ReID领域比较通用的PCB模型，数据集为DukeMTMC-ReID，深度学习框架使用PyTorch0.3版本。模型效果分别用Top1，Top5，Top10和mAP进行比较。

分别设置以下三个实验：

不对模型进行初始化操作

对ResNet50和替换的全连接层结构使用 nn.init 进行初始化操作

加载ResNet50的ImageNet预训练参数，对替换的全连接层结构使用 nn.init 进行初始化操作

一、不对模型进行初始化操作

1.1 参数设置

# 1. 没有加载ImageNet预训练参数
# 2. Resnet50网络结构没有进行初始化操作
# 3. 替换的全连接层结构没有进行初始化操作

1.2 训练过程信息

Epoch 0/99
----------
train Loss: 0.0978 Acc: 0.0455
val Loss: 0.0998 Acc: 0.0185

Epoch 1/99
----------
train Loss: 0.0783 Acc: 0.1134
val Loss: 0.0757 Acc: 0.1154

Epoch 2/99
----------
train Loss: 0.0594 Acc: 0.2307
val Loss: 0.0590 Acc: 0.2393

Epoch 3/99
----------
train Loss: 0.0449 Acc: 0.3641
val Loss: 0.0491 Acc: 0.3248

Epoch 4/99
----------
train Loss: 0.0348 Acc: 0.4849
val Loss: 0.0283 Acc: 0.6097

Epoch 5/99
----------
train Loss: 0.0281 Acc: 0.5709
val Loss: 0.0271 Acc: 0.6011

Epoch 6/99
----------
train Loss: 0.0234 Acc: 0.6373
val Loss: 0.0238 Acc: 0.6738

Epoch 7/99
----------
train Loss: 0.0199 Acc: 0.6839
val Loss: 0.0173 Acc: 0.7422

Epoch 8/99
----------
train Loss: 0.0172 Acc: 0.7247
val Loss: 0.0178 Acc: 0.7293

Epoch 9/99
----------
train Loss: 0.0157 Acc: 0.7461
val Loss: 0.0116 Acc: 0.8234

Epoch 10/99
----------
train Loss: 0.0138 Acc: 0.7768
val Loss: 0.0096 Acc: 0.8632

Epoch 11/99
----------
train Loss: 0.0125 Acc: 0.7980
val Loss: 0.0191 Acc: 0.7293

Epoch 12/99
----------
train Loss: 0.0113 Acc: 0.8207
val Loss: 0.0084 Acc: 0.8818

Epoch 13/99
----------
train Loss: 0.0104 Acc: 0.8284
val Loss: 0.0075 Acc: 0.9060

Epoch 14/99
----------
train Loss: 0.0098 Acc: 0.8428
val Loss: 0.0061 Acc: 0.9088

Epoch 15/99
----------
train Loss: 0.0091 Acc: 0.8532
val Loss: 0.0068 Acc: 0.8946

Epoch 16/99
----------
train Loss: 0.0084 Acc: 0.8641
val Loss: 0.0079 Acc: 0.8761

Epoch 17/99
----------
train Loss: 0.0080 Acc: 0.8707
val Loss: 0.0083 Acc: 0.8903

Epoch 18/99
----------
train Loss: 0.0074 Acc: 0.8800
val Loss: 0.0082 Acc: 0.8818

Epoch 19/99
----------
train Loss: 0.0073 Acc: 0.8802
val Loss: 0.0053 Acc: 0.9302

Epoch 20/99
----------
train Loss: 0.0066 Acc: 0.8976
val Loss: 0.0038 Acc: 0.9516

Epoch 21/99
----------
train Loss: 0.0061 Acc: 0.9056
val Loss: 0.0044 Acc: 0.9430

Epoch 22/99
----------
train Loss: 0.0061 Acc: 0.9052
val Loss: 0.0042 Acc: 0.9558

Epoch 23/99
----------
train Loss: 0.0059 Acc: 0.9080
val Loss: 0.0037 Acc: 0.9615

Epoch 24/99
----------
train Loss: 0.0054 Acc: 0.9165
val Loss: 0.0073 Acc: 0.8903

Epoch 25/99
----------
train Loss: 0.0052 Acc: 0.9208
val Loss: 0.0037 Acc: 0.9587

Epoch 26/99
----------
train Loss: 0.0052 Acc: 0.9228
val Loss: 0.0032 Acc: 0.9615

Epoch 27/99
----------
train Loss: 0.0049 Acc: 0.9292
val Loss: 0.0024 Acc: 0.9744

Epoch 28/99
----------
train Loss: 0.0046 Acc: 0.9348
val Loss: 0.0026 Acc: 0.9715

Epoch 29/99
----------
train Loss: 0.0043 Acc: 0.9360
val Loss: 0.0024 Acc: 0.9658

Epoch 30/99
----------
train Loss: 0.0042 Acc: 0.9394
val Loss: 0.0024 Acc: 0.9701

Epoch 31/99
----------
train Loss: 0.0045 Acc: 0.9338
val Loss: 0.0021 Acc: 0.9858

Epoch 32/99
----------
train Loss: 0.0040 Acc: 0.9431
val Loss: 0.0033 Acc: 0.9558

Epoch 33/99
----------
train Loss: 0.0041 Acc: 0.9423
val Loss: 0.0035 Acc: 0.9573

Epoch 34/99
----------
train Loss: 0.0039 Acc: 0.9461
val Loss: 0.0026 Acc: 0.9644

Epoch 35/99
----------
train Loss: 0.0038 Acc: 0.9464
val Loss: 0.0044 Acc: 0.9487

Epoch 36/99
----------
train Loss: 0.0038 Acc: 0.9475
val Loss: 0.0022 Acc: 0.9729

Epoch 37/99
----------
train Loss: 0.0034 Acc: 0.9551
val Loss: 0.0023 Acc: 0.9758

Epoch 38/99
----------
train Loss: 0.0033 Acc: 0.9554
val Loss: 0.0034 Acc: 0.9530

Epoch 39/99
----------
train Loss: 0.0036 Acc: 0.9509
val Loss: 0.0022 Acc: 0.9687

Epoch 40/99
----------
train Loss: 0.0017 Acc: 0.9812
val Loss: 0.0004 Acc: 0.9986

Epoch 41/99
----------
train Loss: 0.0011 Acc: 0.9917
val Loss: 0.0004 Acc: 0.9986

Epoch 42/99
----------
train Loss: 0.0009 Acc: 0.9943
val Loss: 0.0003 Acc: 1.0000

Epoch 43/99
----------
train Loss: 0.0008 Acc: 0.9948
val Loss: 0.0003 Acc: 1.0000

Epoch 44/99
----------
train Loss: 0.0008 Acc: 0.9952
val Loss: 0.0003 Acc: 0.9986

Epoch 45/99
----------
train Loss: 0.0008 Acc: 0.9965
val Loss: 0.0003 Acc: 1.0000

Epoch 46/99
----------
train Loss: 0.0007 Acc: 0.9959
val Loss: 0.0003 Acc: 0.9986

Epoch 47/99
----------
train Loss: 0.0008 Acc: 0.9960
val Loss: 0.0003 Acc: 0.9986

Epoch 48/99
----------
train Loss: 0.0007 Acc: 0.9969
val Loss: 0.0003 Acc: 0.9986

Epoch 49/99
----------
train Loss: 0.0008 Acc: 0.9971
val Loss: 0.0003 Acc: 1.0000

Epoch 50/99
----------
train Loss: 0.0007 Acc: 0.9970
val Loss: 0.0003 Acc: 1.0000

Epoch 51/99
----------
train Loss: 0.0007 Acc: 0.9978
val Loss: 0.0003 Acc: 1.0000

Epoch 52/99
----------
train Loss: 0.0008 Acc: 0.9973
val Loss: 0.0003 Acc: 1.0000

Epoch 53/99
----------
train Loss: 0.0007 Acc: 0.9974
val Loss: 0.0003 Acc: 1.0000

Epoch 54/99
----------
train Loss: 0.0008 Acc: 0.9973
val Loss: 0.0003 Acc: 0.9986

Epoch 55/99
----------
train Loss: 0.0007 Acc: 0.9975
val Loss: 0.0003 Acc: 0.9986

Epoch 56/99
----------
train Loss: 0.0007 Acc: 0.9982
val Loss: 0.0003 Acc: 1.0000

Epoch 57/99
----------
train Loss: 0.0007 Acc: 0.9984
val Loss: 0.0003 Acc: 0.9986

Epoch 58/99
----------
train Loss: 0.0008 Acc: 0.9978
val Loss: 0.0003 Acc: 0.9986

Epoch 59/99
----------
train Loss: 0.0008 Acc: 0.9976
val Loss: 0.0004 Acc: 0.9986

Epoch 60/99
----------
train Loss: 0.0008 Acc: 0.9979
val Loss: 0.0004 Acc: 1.0000

Epoch 61/99
----------
train Loss: 0.0008 Acc: 0.9980
val Loss: 0.0004 Acc: 0.9986

Epoch 62/99
----------
train Loss: 0.0007 Acc: 0.9983
val Loss: 0.0004 Acc: 1.0000

Epoch 63/99
----------
train Loss: 0.0008 Acc: 0.9987
val Loss: 0.0004 Acc: 1.0000

Epoch 64/99
----------
train Loss: 0.0008 Acc: 0.9982
val Loss: 0.0004 Acc: 1.0000

Epoch 65/99
----------
train Loss: 0.0008 Acc: 0.9987
val Loss: 0.0004 Acc: 1.0000

Epoch 66/99
----------
train Loss: 0.0008 Acc: 0.9985
val Loss: 0.0004 Acc: 1.0000

Epoch 67/99
----------
train Loss: 0.0008 Acc: 0.9986
val Loss: 0.0004 Acc: 0.9986

Epoch 68/99
----------
train Loss: 0.0008 Acc: 0.9985
val Loss: 0.0004 Acc: 0.9986

Epoch 69/99
----------
train Loss: 0.0008 Acc: 0.9985
val Loss: 0.0004 Acc: 0.9986

Epoch 70/99
----------
train Loss: 0.0008 Acc: 0.9985
val Loss: 0.0004 Acc: 1.0000

Epoch 71/99
----------
train Loss: 0.0008 Acc: 0.9987
val Loss: 0.0004 Acc: 0.9986

Epoch 72/99
----------
train Loss: 0.0008 Acc: 0.9993
val Loss: 0.0004 Acc: 1.0000

Epoch 73/99
----------
train Loss: 0.0008 Acc: 0.9986
val Loss: 0.0004 Acc: 1.0000

Epoch 74/99
----------
train Loss: 0.0008 Acc: 0.9990
val Loss: 0.0004 Acc: 1.0000

Epoch 75/99
----------
train Loss: 0.0008 Acc: 0.9992
val Loss: 0.0004 Acc: 1.0000

Epoch 76/99
----------
train Loss: 0.0008 Acc: 0.9981
val Loss: 0.0004 Acc: 1.0000

Epoch 77/99
----------
train Loss: 0.0008 Acc: 0.9992
val Loss: 0.0004 Acc: 1.0000

Epoch 78/99
----------
train Loss: 0.0008 Acc: 0.9990
val Loss: 0.0004 Acc: 1.0000

Epoch 79/99
----------
train Loss: 0.0008 Acc: 0.9990
val Loss: 0.0004 Acc: 1.0000

Epoch 80/99
----------
train Loss: 0.0008 Acc: 0.9989
val Loss: 0.0004 Acc: 1.0000

Epoch 81/99
----------
train Loss: 0.0007 Acc: 0.9992
val Loss: 0.0004 Acc: 1.0000

Epoch 82/99
----------
train Loss: 0.0007 Acc: 0.9991
val Loss: 0.0004 Acc: 1.0000

Epoch 83/99
----------
train Loss: 0.0007 Acc: 0.9990
val Loss: 0.0004 Acc: 1.0000

Epoch 84/99
----------
train Loss: 0.0007 Acc: 0.9990
val Loss: 0.0004 Acc: 1.0000

Epoch 85/99
----------
train Loss: 0.0007 Acc: 0.9994
val Loss: 0.0004 Acc: 1.0000

Epoch 86/99
----------
train Loss: 0.0007 Acc: 0.9991
val Loss: 0.0004 Acc: 1.0000

Epoch 87/99
----------
train Loss: 0.0007 Acc: 0.9988
val Loss: 0.0004 Acc: 1.0000

Epoch 88/99
----------
train Loss: 0.0007 Acc: 0.9991
val Loss: 0.0004 Acc: 1.0000

Epoch 89/99
----------
train Loss: 0.0007 Acc: 0.9989
val Loss: 0.0004 Acc: 1.0000

Epoch 90/99
----------
train Loss: 0.0007 Acc: 0.9992
val Loss: 0.0004 Acc: 1.0000

Epoch 91/99
----------
train Loss: 0.0008 Acc: 0.9990
val Loss: 0.0004 Acc: 1.0000

Epoch 92/99
----------
train Loss: 0.0007 Acc: 0.9987
val Loss: 0.0004 Acc: 1.0000

Epoch 93/99
----------
train Loss: 0.0008 Acc: 0.9992
val Loss: 0.0004 Acc: 1.0000

Epoch 94/99
----------
train Loss: 0.0008 Acc: 0.9988
val Loss: 0.0004 Acc: 0.9986

Epoch 95/99
----------
train Loss: 0.0007 Acc: 0.9990
val Loss: 0.0004 Acc: 1.0000

Epoch 96/99
----------
train Loss: 0.0007 Acc: 0.9990
val Loss: 0.0004 Acc: 1.0000

Epoch 97/99
----------
train Loss: 0.0007 Acc: 0.9991
val Loss: 0.0004 Acc: 1.0000

Epoch 98/99
----------
train Loss: 0.0007 Acc: 0.9993
val Loss: 0.0004 Acc: 1.0000

Epoch 99/99
----------
train Loss: 0.0007 Acc: 0.9991
val Loss: 0.0004 Acc: 1.0000

Training complete in 122m 6s

1.3 模型性能测试

迭代次数	train_loss	accuracy
epoch-10	0.0157	Rank@1:0.353680 Rank@5:0.537702 Rank@10:0.625673 mAP:0.214494
epoch-20	0.0073	Rank@1:0.486984 Rank@5:0.663375 Rank@10:0.732047 mAP:0.306539
epoch-30	0.0043	Rank@1:0.540395 Rank@5:0.725763 Rank@10:0.789048 mAP:0.360594
epoch-40	0.0036	Rank@1:0.592908 Rank@5:0.764812 Rank@10:0.814183 mAP:0.395242
epoch-50	0.0008	Rank

最低0.47元/天解锁文章

huangpg丶

关注

2
点赞
踩
9

收藏

觉得还不错? 一键收藏
2
评论
实验比较[参数初始化方式]与[ImageNet预训练参数]对模型训练和最终性能的影响

这篇文章主要是通过实验比较模型的参数初始化方式，以及是否使用ImageNet预训练参数初始化网络对训练过程和模型性能的影响。实验项目采用ReID领域比较通用的PCB模型，数据集为DukeMTMC-ReID，深度学习框架使用PyTorch0.3版本。模型效果分别用Top1，Top5，Top10和mAP进行比较。分别设置以下三个实验：不对模型进行初始化操作对ResNet50和替换...
复制链接

扫一扫