使用ResNet的caffe实现进行图片分类

最新推荐文章于 2024-05-29 09:37:45 发布
最新推荐文章于 2024-05-29 09:37:45 发布
阅读量346 收藏 1
点赞数
分类专栏： caffe 文章标签： caffe 深度学习
原文链接：https://blog.csdn.net/CVAIDL/article/details/82152091
版权

使用ResNet的caffe实现进行图片分类

Gemfield

A CivilNet Maintainer

背景

图片分类是深度学习在图像领域的初级应用，更高阶的应用如目标检测、图像分割等。那就从最初级的应用-图像分类做起吧。

准备数据

准备数据包括训练集和测试集。

比方说gemfield要做个224分类的应用，那么先准备好224个子目录，每个目录里放的是同一分类的照片：

 

  1. gemfield@ai:/bigdata/video_annotation_web/annotation_output$ find . -type f

  2. ./0/wzry223.mp4_7670_1051_385_126_140.jpg

  3. ./0/wzry110.mp4_2740_562_293_160_148.jpg

  4. ......

  5. ./65/wzry223.mp4_7670_1021_385_126_140.jpg

  6. ./65/wzry110.mp4_2740_572_293_160_148.jpg

  7. ......

  8. ./153/wzry61.mp4_1240_561_298_182_150.jpg

  9. ./153/wzry73.mp4_3660_792_424_280_238.jpg

  10. ......

  11. ./223/wzry73.mp4_3510_830_426_252_232.jpg

  12. ./223/wzry73.mp4_1450_820_434_274_240.jpg

那么如果我已有的照片目录并不是从0开始按照数字顺序排列的，而是一些用中文命名的目录，那怎么办呢？使用下面的shell命令可以做到：

 

  1. #先保存个映射表

  2. num=0;for f in $(ls);do echo "$num,$f" >> /tmp/gemfield.dict;num=$((num + 1));done

  3.  

  4. #改文件夹的名字

  5. num=0;for f in $(ls);do mv $f $num ;num=$((num + 1));done

 

224分类的label是0到223，一定要从0开始，caffe框架决定的。那现在有了224个目录的照片，怎么生成对应的label文件呢？可以使用下面的python代码：

 

  1. gemfield@ai:/bigdata/video_annotation_web/annotation_output$ find . -type f > ../gemfield.txt

  2. gemfield@ai:/bigdata/video_annotation_web/annotation_output$ cd ..

  3. gemfield@ai:/bigdata/video_annotation_web$ python

  4. Python 2.7.13 (default, Nov 23 2017, 15:37:09)

  5. [GCC 6.3.0 20170406] on linux2

  6. Type "help", "copyright", "credits" or "license" for more information.

  7. >>> l = open('gemfield.txt').readlines()

  8. >>> ll = [x.strip() + ' ' + x.split('/')[1] + '\n' for x in l]

  9. >>> import random

  10. >>> random.shuffle(ll)

  11.  

  12. >>> with open('gemfield.lst', 'w') as f:

  13. ... f.writelines(ll)

  14. ...

  15. >>>

 

现在label文件也生成了，我们需要将其分为训练集和测试集，针对本次实验，gemfield按照6:1的比例划分了训练集和测试集；这个划分过程不用动那224个图片目录，而只需要将上面的那个gemfield.txt按照6:1的比例划分为2个label文件：train.lst和val.lst 文件：

 

  1. gemfield@ai:/bigdata/video_annotation_web$ tail -24319 gemfield.lst > val.lst

  2. gemfield@ai:/bigdata/video_annotation_web$ head -120000 gemfield.lst > train.lst

 

准备Caffe环境

下载官方的caffe仓库，修改makefile.config的参数，编译就ok了。参考本专栏的前述文章。

 

准备ResNet的网络

Gemfield使用的是ResNet50版本：

KaimingHe/deep-residual-networks

这个下载下来的prototxt文件，拷贝2份出来，一份命名为ResNet-50-deploy-by-gemfield.prototxt，这个不作修改；另一份命名为ResNet-50-train-val-by-gemfield.prototxt，然后这个网络结构文件要进行修改，修改部分如下所示：

第一部分：修改input部分：

ResNet-50-deploy-by-gemfield.prototxt中的

 

  1. input: "data"

  2. input_dim: 1

  3. input_dim: 3

  4. input_dim: 224

  5. input_dim: 224

改为ResNet-50-train-val-by-gemfield.prototxt中的

 

  1. layer {

  2. name: "data"

  3. type: "Data"

  4. top: "data"

  5. top: "label"

  6. include {

  7. phase: TRAIN

  8. }

  9. transform_param {

  10. mirror: false

  11. # crop_size: 224

  12. mean_value: 0

  13. mean_value: 0

  14. mean_value: 0

  15. }

  16. data_param {

  17. source: "wzry_224_train_lmdb"

  18. batch_size: 16

  19. backend: LMDB

  20. }

  21. }

  22.  

  23. layer {

  24. name: "data"

  25. type: "Data"

  26. top: "data"

  27. top: "label"

  28. include {

  29. phase: TEST

  30. }

  31. transform_param {

  32. mirror: false

  33. # crop_size: 224

  34. mean_value: 0

  35. mean_value: 0

  36. mean_value: 0

  37. }

  38. data_param {

  39. source: "wzry_224_val_lmdb"

  40. batch_size: 1

  41. backend: LMDB

  42. }

  43. }

第二部分：

1，将全连接层的fc1000改为fc224，因为我们的分类是224种；

2，将softmax改为softmax with loss，因为训练的时候必须有loss作为反馈；

将ResNet-50-deploy-by-gemfield.prototxt中的

 

  1. layer {

  2. bottom: "pool5"

  3. top: "fc1000"

  4. name: "fc1000"

  5. type: "InnerProduct"

  6. inner_product_param {

  7. num_output: 1000

  8. }

  9. }

  10.  

  11. layer {

  12. bottom: "fc1000"

  13. top: "prob"

  14. name: "prob"

  15. type: "Softmax"

  16. }

改为ResNet-50-train-val-by-gemfield.prototxt中的

 

  1. layer {

  2. bottom: "pool5"

  3. top: "fc224"

  4. name: "fc224"

  5. type: "InnerProduct"

  6. inner_product_param {

  7. num_output: 224

  8. }

  9. }

  10. layer {

  11. bottom: "fc224"

  12. bottom: "label"

  13. name: "loss"

  14. type: "SoftmaxWithLoss"

  15. top: "loss"

  16. }

  17.  

  18. layer {

  19. bottom: "fc224"

  20. bottom: "label"

  21. top: "acc/top-1"

  22. name: "acc/top-1"

  23. type: "Accuracy"

  24. include {

  25. phase: TEST

  26. }

  27. }

第三部分：

ResNet的输入参数是224x224，因此，在下面生成lmdb的时候，要将图像resize成224x224，除非你的图片本来就是224x224。

准备solver.prototxt

 

  1. net: "ResNet-50-train-val-by-gemfield.prototxt"

  2. #每次测试跑多少次迭代，照片数量/test的batch_size

  3. test_iter: 24000

  4. #每多少次迭代进行一次测试

  5. test_interval: 2000

  6. test_initialization: false

  7.  

  8. #初始的学习率

  9. base_lr: 0.001

  10. #学习率的下降策略

  11. lr_policy: "step"

  12. gamma: 0.1

  13. stepsize: 5000

  14.  

  15. #每迭代多少次打印一次loss信息

  16. display: 100

  17. #最多迭代多少次

  18. max_iter: 75000

  19. #梯度下降策略，90%依赖这次，10%依赖上次

  20. momentum: 0.9

  21. #正则项的比重

  22. weight_decay: 0.0001

  23. #迭代多少次生成一次中间的caffemodel

  24. snapshot: 5000

  25. #caffemodel的前缀

  26. snapshot_prefix: "models/resnet_gemfield_cls224"

  27. solver_mode: GPU

生成lmdb

使用Caffe自带的工具convert_imageset来生成lmdb文件（在caffe仓库目录里），注意，resnet的输入都是224x224,所以在制作数据集的时候需要进行resize；其次，convert_imageset会简单的将下列命令中的root目录"./"和train.lst中的文件路径进行直接的字符串拼接；再次，生成的lmdb目录不能已经存在。

先生成训练集：

 

  1. gemfield@ai:~$ convert_imageset -resize_height 224 -resize_width 224 ./ train.lst wzry_224_train_lmdb

  2. I0113 12:13:44.050431 27 convert_imageset.cpp:89] A total of 120000 images.

  3. E0113 12:13:44.051177 27 common.cpp:114] Cannot create Cublas handle. Cublas won't be available.

  4. E0113 12:13:44.051475 27 common.cpp:121] Cannot create Curand generator. Curand won't be available.

  5. I0113 12:13:44.051504 27 db_lmdb.cpp:35] Opened lmdb wzry_224_train_lmdb

  6. I0113 12:13:45.643923 27 convert_imageset.cpp:147] Processed 1000 files.

  7. I0113 12:13:47.270226 27 convert_imageset.cpp:147] Processed 2000 files.

  8. I0113 12:13:48.633996 27 convert_imageset.cpp:147] Processed 3000 files.

  9. I0113 12:13:50.994860 27 convert_imageset.cpp:147] Processed 4000 files.

  10. I0113 12:13:52.172212 27 convert_imageset.cpp:147] Processed 5000 files.

  11. I0113 12:13:53.344650 27 convert_imageset.cpp:147] Processed 6000 files.

  12. I0113 12:13:55.925756 27 convert_imageset.cpp:147] Processed 7000 files.

  13. I0113 12:13:57.467402 27 convert_imageset.cpp:147] Processed 8000 files.

  14. I0113 12:13:59.090173 27 convert_imageset.cpp:147] Processed 9000 files.

  15. I0113 12:14:00.676419 27 convert_imageset.cpp:147] Processed 10000 files.

  16. I0113 12:14:02.851028 27 convert_imageset.cpp:147] Processed 11000 files.

  17. I0113 12:14:04.404832 27 convert_imageset.cpp:147] Processed 12000 files.

  18. I0113 12:14:05.979625 27 convert_imageset.cpp:147] Processed 13000 files.

  19. I0113 12:14:07.152940 27 convert_imageset.cpp:147] Processed 14000 files.

  20. I0113 12:14:09.217108 27 convert_imageset.cpp:147] Processed 15000 files.

  21. I0113 12:14:10.806496 27 convert_imageset.cpp:147] Processed 16000 files.

  22. I0113 12:14:12.268177 27 convert_imageset.cpp:147] Processed 17000 files.

  23. I0113 12:14:13.508546 27 convert_imageset.cpp:147] Processed 18000 files.

  24. I0113 12:14:15.016147 27 convert_imageset.cpp:147] Processed 19000 files.

  25. I0113 12:14:17.189391 27 convert_imageset.cpp:147] Processed 20000 files.

  26. I0113 12:14:19.469425 27 convert_imageset.cpp:147] Processed 21000 files.

  27. I0113 12:14:20.869881 27 convert_imageset.cpp:147] Processed 22000 files.

  28. I0113 12:14:22.358198 27 convert_imageset.cpp:147] Processed 23000 files.

  29. I0113 12:14:23.811714 27 convert_imageset.cpp:147] Processed 24000 files.

  30. I0113 12:14:25.611598 27 convert_imageset.cpp:147] Processed 25000 files.

  31. I0113 12:14:27.188184 27 convert_imageset.cpp:147] Processed 26000 files.

  32. I0113 12:14:28.578914 27 convert_imageset.cpp:147] Processed 27000 files.

  33. I0113 12:14:30.235432 27 convert_imageset.cpp:147] Processed 28000 files.

  34. I0113 12:14:31.867501 27 convert_imageset.cpp:147] Processed 29000 files.

  35. I0113 12:14:33.497715 27 convert_imageset.cpp:147] Processed 30000 files.

  36. I0113 12:14:34.925644 27 convert_imageset.cpp:147] Processed 31000 files.

  37. I0113 12:14:36.906508 27 convert_imageset.cpp:147] Processed 32000 files.

  38. I0113 12:14:38.464900 27 convert_imageset.cpp:147] Processed 33000 files.

  39. I0113 12:14:40.249969 27 convert_imageset.cpp:147] Processed 34000 files.

  40. I0113 12:14:42.774039 27 convert_imageset.cpp:147] Processed 35000 files.

  41. I0113 12:14:44.158181 27 convert_imageset.cpp:147] Processed 36000 files.

  42. I0113 12:14:45.778302 27 convert_imageset.cpp:147] Processed 37000 files.

  43. I0113 12:14:47.252451 27 convert_imageset.cpp:147] Processed 38000 files.

  44. I0113 12:14:49.799836 27 convert_imageset.cpp:147] Processed 39000 files.

  45. I0113 12:14:51.288296 27 convert_imageset.cpp:147] Processed 40000 files.

  46. I0113 12:14:52.484359 27 convert_imageset.cpp:147] Processed 41000 files.

  47. I0113 12:14:54.095209 27 convert_imageset.cpp:147] Processed 42000 files.

  48. I0113 12:14:56.525655 27 convert_imageset.cpp:147] Processed 43000 files.

  49. I0113 12:14:58.033469 27 convert_imageset.cpp:147] Processed 44000 files.

  50. I0113 12:14:59.409360 27 convert_imageset.cpp:147] Processed 45000 files.

  51. I0113 12:15:00.880760 27 convert_imageset.cpp:147] Processed 46000 files.

  52. I0113 12:15:02.602565 27 convert_imageset.cpp:147] Processed 47000 files.

  53. I0113 12:15:04.345715 27 convert_imageset.cpp:147] Processed 48000 files.

  54. I0113 12:15:05.799033 27 convert_imageset.cpp:147] Processed 49000 files.

  55. I0113 12:15:06.995527 27 convert_imageset.cpp:147] Processed 50000 files.

  56. I0113 12:15:08.686816 27 convert_imageset.cpp:147] Processed 51000 files.

  57. I0113 12:15:11.079591 27 convert_imageset.cpp:147] Processed 52000 files.

  58. I0113 12:15:12.673241 27 convert_imageset.cpp:147] Processed 53000 files.

  59. I0113 12:15:14.083250 27 convert_imageset.cpp:147] Processed 54000 files.

  60. I0113 12:15:15.629169 27 convert_imageset.cpp:147] Processed 55000 files.

  61. I0113 12:15:17.027292 27 convert_imageset.cpp:147] Processed 56000 files.

  62. I0113 12:15:18.703470 27 convert_imageset.cpp:147] Processed 57000 files.

  63. I0113 12:15:20.178828 27 convert_imageset.cpp:147] Processed 58000 files.

  64. I0113 12:15:21.541697 27 convert_imageset.cpp:147] Processed 59000 files.

  65. I0113 12:15:23.018353 27 convert_imageset.cpp:147] Processed 60000 files.

  66. I0113 12:15:24.517725 27 convert_imageset.cpp:147] Processed 61000 files.

  67. I0113 12:15:26.077165 27 convert_imageset.cpp:147] Processed 62000 files.

  68. I0113 12:15:27.476835 27 convert_imageset.cpp:147] Processed 63000 files.

  69. I0113 12:15:28.695413 27 convert_imageset.cpp:147] Processed 64000 files.

  70. I0113 12:15:30.552235 27 convert_imageset.cpp:147] Processed 65000 files.

  71. I0113 12:15:32.182060 27 convert_imageset.cpp:147] Processed 66000 files.

  72. I0113 12:15:33.655457 27 convert_imageset.cpp:147] Processed 67000 files.

  73. I0113 12:15:34.957446 27 convert_imageset.cpp:147] Processed 68000 files.

  74. I0113 12:15:36.233927 27 convert_imageset.cpp:147] Processed 69000 files.

  75. I0113 12:15:37.818439 27 convert_imageset.cpp:147] Processed 70000 files.

  76. I0113 12:15:39.956657 27 convert_imageset.cpp:147] Processed 71000 files.

  77. I0113 12:15:41.336088 27 convert_imageset.cpp:147] Processed 72000 files.

  78. I0113 12:15:42.587082 27 convert_imageset.cpp:147] Processed 73000 files.

  79. I0113 12:15:44.044525 27 convert_imageset.cpp:147] Processed 74000 files.

  80. I0113 12:15:45.632407 27 convert_imageset.cpp:147] Processed 75000 files.

  81. I0113 12:15:47.075654 27 convert_imageset.cpp:147] Processed 76000 files.

  82. I0113 12:15:48.567188 27 convert_imageset.cpp:147] Processed 77000 files.

  83. I0113 12:15:49.854224 27 convert_imageset.cpp:147] Processed 78000 files.

  84. I0113 12:15:51.043192 27 convert_imageset.cpp:147] Processed 79000 files.

  85. I0113 12:15:52.586591 27 convert_imageset.cpp:147] Processed 80000 files.

  86. I0113 12:15:54.092438 27 convert_imageset.cpp:147] Processed 81000 files.

  87. I0113 12:15:55.546066 27 convert_imageset.cpp:147] Processed 82000 files.

  88. I0113 12:15:56.811760 27 convert_imageset.cpp:147] Processed 83000 files.

  89. I0113 12:15:57.987690 27 convert_imageset.cpp:147] Processed 84000 files.

  90. I0113 12:15:59.671317 27 convert_imageset.cpp:147] Processed 85000 files.

  91. I0113 12:16:01.283619 27 convert_imageset.cpp:147] Processed 86000 files.

  92. I0113 12:16:02.826110 27 convert_imageset.cpp:147] Processed 87000 files.

  93. I0113 12:16:04.015786 27 convert_imageset.cpp:147] Processed 88000 files.

  94. I0113 12:16:05.246563 27 convert_imageset.cpp:147] Processed 89000 files.

  95. I0113 12:16:07.631480 27 convert_imageset.cpp:147] Processed 90000 files.

  96. I0113 12:16:09.503032 27 convert_imageset.cpp:147] Processed 91000 files.

  97. I0113 12:16:10.790294 27 convert_imageset.cpp:147] Processed 92000 files.

  98. I0113 12:16:12.273556 27 convert_imageset.cpp:147] Processed 93000 files.

  99. I0113 12:16:14.271823 27 convert_imageset.cpp:147] Processed 94000 files.

  100. I0113 12:16:15.744318 27 convert_imageset.cpp:147] Processed 95000 files.

  101. I0113 12:16:17.090734 27 convert_imageset.cpp:147] Processed 96000 files.

  102. I0113 12:16:18.353124 27 convert_imageset.cpp:147] Processed 97000 files.

  103. I0113 12:16:19.863163 27 convert_imageset.cpp:147] Processed 98000 files.

  104. I0113 12:16:21.596539 27 convert_imageset.cpp:147] Processed 99000 files.

  105. I0113 12:16:23.036448 27 convert_imageset.cpp:147] Processed 100000 files.

  106. I0113 12:16:24.533990 27 convert_imageset.cpp:147] Processed 101000 files.

  107. I0113 12:16:25.711519 27 convert_imageset.cpp:147] Processed 102000 files.

  108. I0113 12:16:27.407729 27 convert_imageset.cpp:147] Processed 103000 files.

  109. I0113 12:16:29.793790 27 convert_imageset.cpp:147] Processed 104000 files.

  110. I0113 12:16:31.340585 27 convert_imageset.cpp:147] Processed 105000 files.

  111. I0113 12:16:32.955618 27 convert_imageset.cpp:147] Processed 106000 files.

  112. I0113 12:16:34.523054 27 convert_imageset.cpp:147] Processed 107000 files.

  113. I0113 12:16:36.220069 27 convert_imageset.cpp:147] Processed 108000 files.

  114. I0113 12:16:37.792441 27 convert_imageset.cpp:147] Processed 109000 files.

  115. I0113 12:16:39.185384 27 convert_imageset.cpp:147] Processed 110000 files.

  116. I0113 12:16:40.366708 27 convert_imageset.cpp:147] Processed 111000 files.

  117. I0113 12:16:41.930851 27 convert_imageset.cpp:147] Processed 112000 files.

  118. I0113 12:16:43.605252 27 convert_imageset.cpp:147] Processed 113000 files.

  119. I0113 12:16:45.682309 27 convert_imageset.cpp:147] Processed 114000 files.

  120. I0113 12:16:47.001965 27 convert_imageset.cpp:147] Processed 115000 files.

  121. I0113 12:16:48.201941 27 convert_imageset.cpp:147] Processed 116000 files.

  122. I0113 12:16:50.482267 27 convert_imageset.cpp:147] Processed 117000 files.

  123. I0113 12:16:53.129992 27 convert_imageset.cpp:147] Processed 118000 files.

  124. I0113 12:16:55.065909 27 convert_imageset.cpp:147] Processed 119000 files.

  125. I0113 12:16:56.272369 27 convert_imageset.cpp:147] Processed 120000 files.

再生成测试集：

 

  1. gemfield@ai:~$ convert_imageset -resize_height 224 -resize_width 224 ./ val.lst wzry_224_val_lmdb

  2. I0113 12:35:57.135522 28 convert_imageset.cpp:89] A total of 24319 images.

  3. E0113 12:35:57.136313 28 common.cpp:114] Cannot create Cublas handle. Cublas won't be available.

  4. E0113 12:35:57.136673 28 common.cpp:121] Cannot create Curand generator. Curand won't be available.

  5. I0113 12:35:57.136708 28 db_lmdb.cpp:35] Opened lmdb wzry_224_val_lmdb

  6. I0113 12:35:58.789322 28 convert_imageset.cpp:147] Processed 1000 files.

  7. I0113 12:36:00.379387 28 convert_imageset.cpp:147] Processed 2000 files.

  8. I0113 12:36:01.759858 28 convert_imageset.cpp:147] Processed 3000 files.

  9. I0113 12:36:02.987604 28 convert_imageset.cpp:147] Processed 4000 files.

  10. I0113 12:36:04.426345 28 convert_imageset.cpp:147] Processed 5000 files.

  11. I0113 12:36:05.960880 28 convert_imageset.cpp:147] Processed 6000 files.

  12. I0113 12:36:07.313741 28 convert_imageset.cpp:147] Processed 7000 files.

  13. I0113 12:36:08.496649 28 convert_imageset.cpp:147] Processed 8000 files.

  14. I0113 12:36:10.013240 28 convert_imageset.cpp:147] Processed 9000 files.

  15. I0113 12:36:11.585314 28 convert_imageset.cpp:147] Processed 10000 files.

  16. I0113 12:36:12.984436 28 convert_imageset.cpp:147] Processed 11000 files.

  17. I0113 12:36:14.225903 28 convert_imageset.cpp:147] Processed 12000 files.

  18. I0113 12:36:15.468725 28 convert_imageset.cpp:147] Processed 13000 files.

  19. I0113 12:36:17.084775 28 convert_imageset.cpp:147] Processed 14000 files.

  20. I0113 12:36:18.566448 28 convert_imageset.cpp:147] Processed 15000 files.

  21. I0113 12:36:19.855185 28 convert_imageset.cpp:147] Processed 16000 files.

  22. I0113 12:36:21.040355 28 convert_imageset.cpp:147] Processed 17000 files.

  23. I0113 12:36:22.528937 28 convert_imageset.cpp:147] Processed 18000 files.

  24. I0113 12:36:23.917870 28 convert_imageset.cpp:147] Processed 19000 files.

  25. I0113 12:36:25.164480 28 convert_imageset.cpp:147] Processed 20000 files.

  26. I0113 12:36:27.153021 28 convert_imageset.cpp:147] Processed 21000 files.

  27. I0113 12:37:18.769785 28 convert_imageset.cpp:147] Processed 22000 files.

  28. I0113 12:37:19.995821 28 convert_imageset.cpp:147] Processed 23000 files.

  29. I0113 12:37:21.388216 28 convert_imageset.cpp:147] Processed 24000 files.

  30. I0113 12:37:21.853869 28 convert_imageset.cpp:153] Processed 24319 files.

值得注意的是，1.5G的图片生成的lmdb大小为20.5GB！

 

获取pretrained的model

pre trained model主要是用来初始化当前网络结构的参数。

 

  1. gemfield@ai:~$ ls -l ResNet-50-model.caffemodel

  2. -rw-r--r-- 1 gemfield gemfield 102462397 1月 13 21:16 ResNet-50-model.caffemodel

开始训练

gemfield@ai:~$ build/tools/caffe train -solver solver.prototxt -weights ResNet-50-model.caffemodel

 

一些训练日志

 

  1. I0114 03:52:23.357553 21226 solver.cpp:218] Iteration 73900 (6.61567 iter/s, 15.1156s/100 iters), loss = 0.0242063

  2. I0114 03:52:23.357609 21226 solver.cpp:237] Train net output #0: loss = 0.0242095 (* 1 = 0.0242095 loss)

  3. I0114 03:52:23.357619 21226 sgd_solver.cpp:105] Iteration 73900, lr = 1e-17

  4. I0114 03:52:38.312636 21226 solver.cpp:330] Iteration 74000, Testing net (#0)

  5. I0114 03:54:58.760177 21231 data_layer.cpp:73] Restarting data prefetching from start.

  6. I0114 03:57:33.356952 21226 solver.cpp:397] Test net output #0: acc/top-1 = 0.969708

  7. I0114 03:57:33.357126 21226 solver.cpp:397] Test net output #1: loss = 0.115373 (* 1 = 0.115373 loss)

  8. I0114 03:57:33.503705 21226 solver.cpp:218] Iteration 74000 (0.322433 iter/s, 310.142s/100 iters), loss = 0.0952036

  9. I0114 03:57:33.503748 21226 solver.cpp:237] Train net output #0: loss = 0.0952069 (* 1 = 0.0952069 loss)

  10. I0114 03:57:33.503764 21226 sgd_solver.cpp:105] Iteration 74000, lr = 1e-17

  11. I0114 03:57:48.466397 21226 solver.cpp:218] Iteration 74100 (6.68344 iter/s, 14.9624s/100 iters), loss = 0.00265225

  12. I0114 03:57:48.466521 21226 solver.cpp:237] Train net output #0: loss = 0.00265556 (* 1 = 0.00265556 loss)

  13. I0114 03:57:48.466542 21226 sgd_solver.cpp:105] Iteration 74100, lr = 1e-17

  14. I0114 03:58:03.440867 21226 solver.cpp:218] Iteration 74200 (6.67818 iter/s, 14.9741s/100 iters), loss = 0.011983

  15. I0114 03:58:03.441133 21226 solver.cpp:237] Train net output #0: loss = 0.0119865 (* 1 = 0.0119865 loss)

  16. I0114 03:58:03.441164 21226 sgd_solver.cpp:105] Iteration 74200, lr = 1e-17

  17. I0114 03:58:18.470480 21226 solver.cpp:218] Iteration 74300 (6.65372 iter/s, 15.0292s/100 iters), loss = 0.234399

  18. I0114 03:58:18.470551 21226 solver.cpp:237] Train net output #0: loss = 0.234403 (* 1 = 0.234403 loss)

  19. I0114 03:58:18.470561 21226 sgd_solver.cpp:105] Iteration 74300, lr = 1e-17

  20. I0114 03:58:33.493978 21226 solver.cpp:218] Iteration 74400 (6.65636 iter/s, 15.0232s/100 iters), loss = 0.00225647

  21. I0114 03:58:33.494228 21226 solver.cpp:237] Train net output #0: loss = 0.0022598 (* 1 = 0.0022598 loss)

  22. I0114 03:58:33.494269 21226 sgd_solver.cpp:105] Iteration 74400, lr = 1e-17

  23. I0114 03:58:48.538324 21226 solver.cpp:218] Iteration 74500 (6.6472 iter/s, 15.0439s/100 iters), loss = 2.68966e-05

  24. I0114 03:58:48.538383 21226 solver.cpp:237] Train net output #0: loss = 3.02486e-05 (* 1 = 3.02486e-05 loss)

  25. I0114 03:58:48.538391 21226 sgd_solver.cpp:105] Iteration 74500, lr = 1e-17

  26. I0114 03:59:03.605232 21226 solver.cpp:218] Iteration 74600 (6.63717 iter/s, 15.0667s/100 iters), loss = 0.00452782

  27. I0114 03:59:03.605480 21226 solver.cpp:237] Train net output #0: loss = 0.00453112 (* 1 = 0.00453112 loss)

  28. I0114 03:59:03.605512 21226 sgd_solver.cpp:105] Iteration 74600, lr = 1e-17

  29. I0114 03:59:18.693054 21226 solver.cpp:218] Iteration 74700 (6.62804 iter/s, 15.0874s/100 iters), loss = 0.173986

  30. I0114 03:59:18.693130 21226 solver.cpp:237] Train net output #0: loss = 0.173989 (* 1 = 0.173989 loss)

  31. I0114 03:59:18.693142 21226 sgd_solver.cpp:105] Iteration 74700, lr = 1e-17

  32. I0114 03:59:33.762369 21226 solver.cpp:218] Iteration 74800 (6.63611 iter/s, 15.0691s/100 iters), loss = 0.237659

  33. I0114 03:59:33.762547 21226 solver.cpp:237] Train net output #0: loss = 0.237662 (* 1 = 0.237662 loss)

  34. I0114 03:59:33.762562 21226 sgd_solver.cpp:105] Iteration 74800, lr = 1e-17

  35. I0114 03:59:48.851802 21226 solver.cpp:218] Iteration 74900 (6.62731 iter/s, 15.0891s/100 iters), loss = 0.0473921

  36. I0114 03:59:48.851860 21226 solver.cpp:237] Train net output #0: loss = 0.0473955 (* 1 = 0.0473955 loss)

  37. I0114 03:59:48.851868 21226 sgd_solver.cpp:105] Iteration 74900, lr = 1e-17

  38. I0114 04:00:03.242220 21230 data_layer.cpp:73] Restarting data prefetching from start.

  39. I0114 04:00:03.808995 21226 solver.cpp:447] Snapshotting to binary proto file models/resnet_gemfield_cls224_iter_75000.caffemodel

  40. I0114 04:00:04.100811 21226 sgd_solver.cpp:273] Snapshotting solver state to binary proto file models/resnet_gemfield_cls224_iter_75000.solverstate

  41. I0114 04:00:04.314110 21226 solver.cpp:310] Iteration 75000, loss = 0.225416

  42. I0114 04:00:04.314160 21226 solver.cpp:315] Optimization Done.

  43. I0114 04:00:04.314165 21226 caffe.cpp:259] Optimization Done.

 

清洗了一次训练集，把一些模糊的照片去掉后，再次用同样的参数去训练，得到的训练日志如下：

 

  1. I0116 01:29:12.525300 61 sgd_solver.cpp:105] Iteration 73800, lr = 1e-17

  2. I0116 01:29:45.552590 61 solver.cpp:218] Iteration 73900 (3.02776 iter/s, 33.0277s/100 iters), loss = 0.121681

  3. I0116 01:29:45.554165 61 solver.cpp:237] Train net output #0: loss = 0.121683 (* 1 = 0.121683 loss)

  4. I0116 01:29:45.554219 61 sgd_solver.cpp:105] Iteration 73900, lr = 1e-17

  5. I0116 01:30:18.147012 61 solver.cpp:330] Iteration 74000, Testing net (#0)

  6. I0116 01:33:34.854918 67 data_layer.cpp:73] Restarting data prefetching from start.

  7. I0116 01:44:08.726112 61 solver.cpp:397] Test net output #0: acc/top-1 = 0.976

  8. I0116 01:44:08.726408 61 solver.cpp:397] Test net output #1: loss = 0.0893586 (* 1 = 0.0893586 loss)

  9. I0116 01:44:09.004158 61 solver.cpp:218] Iteration 74000 (0.115814 iter/s, 863.456s/100 iters), loss = 0.272815

  10. I0116 01:44:09.004226 61 solver.cpp:237] Train net output #0: loss = 0.272817 (* 1 = 0.272817 loss)

  11. I0116 01:44:09.004240 61 sgd_solver.cpp:105] Iteration 74000, lr = 1e-17

  12. I0116 01:44:41.944175 61 solver.cpp:218] Iteration 74100 (3.03601 iter/s, 32.938s/100 iters), loss = 0.00181933

  13. I0116 01:44:41.944440 61 solver.cpp:237] Train net output #0: loss = 0.00182139 (* 1 = 0.00182139 loss)

  14. I0116 01:44:41.944473 61 sgd_solver.cpp:105] Iteration 74100, lr = 1e-17

  15. I0116 01:45:15.054388 61 solver.cpp:218] Iteration 74200 (3.0204 iter/s, 33.1082s/100 iters), loss = 0.0241835

  16. I0116 01:45:15.054733 61 solver.cpp:237] Train net output #0: loss = 0.0241856 (* 1 = 0.0241856 loss)

  17. I0116 01:45:15.054776 61 sgd_solver.cpp:105] Iteration 74200, lr = 1e-17

  18. I0116 01:45:48.085701 61 solver.cpp:218] Iteration 74300 (3.02762 iter/s, 33.0293s/100 iters), loss = 0.0254915

  19. I0116 01:45:48.085886 61 solver.cpp:237] Train net output #0: loss = 0.0254935 (* 1 = 0.0254935 loss)

  20. I0116 01:45:48.085896 61 sgd_solver.cpp:105] Iteration 74300, lr = 1e-17

  21. I0116 01:46:21.485394 61 solver.cpp:218] Iteration 74400 (2.99422 iter/s, 33.3977s/100 iters), loss = 0.000650348

  22. I0116 01:46:21.485787 61 solver.cpp:237] Train net output #0: loss = 0.000652366 (* 1 = 0.000652366 loss)

  23. I0116 01:46:21.485879 61 sgd_solver.cpp:105] Iteration 74400, lr = 1e-17

  24. I0116 01:46:54.568955 61 solver.cpp:218] Iteration 74500 (3.02283 iter/s, 33.0815s/100 iters), loss = 0.209352

  25. I0116 01:46:54.569267 61 solver.cpp:237] Train net output #0: loss = 0.209354 (* 1 = 0.209354 loss)

  26. I0116 01:46:54.569316 61 sgd_solver.cpp:105] Iteration 74500, lr = 1e-17

  27. I0116 01:47:27.618690 61 solver.cpp:218] Iteration 74600 (3.02593 iter/s, 33.0477s/100 iters), loss = 0.0229688

  28. I0116 01:47:27.618928 61 solver.cpp:237] Train net output #0: loss = 0.0229709 (* 1 = 0.0229709 loss)

  29. I0116 01:47:27.618969 61 sgd_solver.cpp:105] Iteration 74600, lr = 1e-17

  30. I0116 01:48:00.857776 61 solver.cpp:218] Iteration 74700 (3.0085 iter/s, 33.2391s/100 iters), loss = 0.00383524

  31. I0116 01:48:00.857947 61 solver.cpp:237] Train net output #0: loss = 0.00383731 (* 1 = 0.00383731 loss)

  32. I0116 01:48:00.857959 61 sgd_solver.cpp:105] Iteration 74700, lr = 1e-17

  33. I0116 01:48:34.184334 61 solver.cpp:218] Iteration 74800 (3.00065 iter/s, 33.3261s/100 iters), loss = 0.126383

  34. I0116 01:48:34.184636 61 solver.cpp:237] Train net output #0: loss = 0.126385 (* 1 = 0.126385 loss)

  35. I0116 01:48:34.184814 61 sgd_solver.cpp:105] Iteration 74800, lr = 1e-17

  36. I0116 01:49:07.174204 61 solver.cpp:218] Iteration 74900 (3.03142 iter/s, 32.9879s/100 iters), loss = 0.327584

  37. I0116 01:49:07.174506 61 solver.cpp:237] Train net output #0: loss = 0.327586 (* 1 = 0.327586 loss)

  38. I0116 01:49:07.174561 61 sgd_solver.cpp:105] Iteration 74900, lr = 1e-17

  39. I0116 01:49:38.800302 66 data_layer.cpp:73] Restarting data prefetching from start.

  40. I0116 01:49:40.076367 61 solver.cpp:447] Snapshotting to binary proto file models/resnet_gemfield_cls224_iter_75000.caffemodel

  41. I0116 01:49:40.382776 61 sgd_solver.cpp:273] Snapshotting solver state to binary proto file models/resnet_gemfield_cls224_iter_75000.solverstate

  42. I0116 01:49:40.630769 61 solver.cpp:310] Iteration 75000, loss = 0.0174639

  43. I0116 01:49:40.630820 61 solver.cpp:315] Optimization Done.

  44. I0116 01:49:40.630826 61 caffe.cpp:259] Optimization Done.

嗯，效果还是好了些。

等待破茧
关注
  • 0
    点赞
  • 踩
  • 1
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
专栏目录
caffe Resnet-50
05-21
残差resnet网络 ImageNet pre-trained ResNet-50的caffemodel
resnet101-caffe(2).pth
10-22
ResNet-101在ImageNet数据集上的caffe预训练模型，可以用于分类和目标检测网络。
Caffe Resnet 简单的图像分类示例
m0_37357063的博客
12-02 7759
环境： OS：Ubuntu Caffe环境（CUDA、OpenCV、cuDNN、...） Nvidia显卡 TITIAN X 目录 1.数据准备（使用自己的数据） 1.1生成 所有图像数据的 每一幅图的路径 类别标签的 txt文件 2.利用1中的txt文件生成 lmdb文件 3.准备网络模型：网络定义文件prototxt 4.准备Caffe的Solver 文件：solver.pr...
caffe分类网络训练及测试步骤_Resnet50
cumtchw
09-22 2254
1.生成txt文件 分类网络可以不用制作lmdb数据，直接用txt文件作为输入源，一般习惯创建一个images文件夹，然后里面每一类单独一个文件夹，每个文件夹里面存放某一类的图片，然后用Python生成txt，脚本如下。 import os import random base_dir = "/data/chw/changjing_fenlei_20200622/yugeshuju/images" f_train = open("./train_origin.txt", 'w') #f_val
推荐使用PyTorch-ResNet：无缝转换Caffe预训练模型至PyTorch
最新发布
gitblog_00030的博客
05-29 382
推荐使用PyTorch-ResNet：无缝转换Caffe预训练模型至PyTorch 项目地址:https://gitcode.com/ruotianluo/pytorch-resnet PyTorch-ResNet是一个非常实用的开源项目，专为那些希望将Caffe框架中预训练的ResNet模型转换到PyTorch环境中的开发者设计。该项目由Kaiming He开发的Deep Residual Ne...
Resnet的caffe实现
Graceying的博客
01-15 222
https://www.lijl888.com/archives/477
caffe运行resnet网络
jiao_mrswang的博客
02-18 741
resnet利用caffe自带的制作lmdb的命令： /caffe/build/tools/convert_imageset -resize_height 224 -resize_width 224 /data/images/ /train.txt resnet_lmdb 以上参数的注释： convert_imageset:制作lmdb的命令 resize_height:将图像处理为高为2...
ResNet caffe编写（转）
gzd1996的博客
11-23 145
https://www.cnblogs.com/Charles-Wan/p/6535395.html
【caffe学习】项目实战：用caffe进行Resnet -50的图像分类任务示例
zl3090的博客
03-02 2505
这是我接触caffe跑的第一个项目，跑的时候对caffe几乎一无所知，从零开始跟着博客一步步做，对caffe有了初步的认识，所以像我一样的新手小白不要恐慌，先做一遍反而更快学到caffe内部的原理。此处非常感谢下面博主的清晰介绍，将我成功带进了caffe的大门，此篇文章也转载于此，整个流程非常清晰，亲测行得通，详情见链接： https://blog.csdn.net/m0_37357063/ar...
inception-resnet-v2 protxt caffemodel
01-11
接着，根据实际需求，可以进行模型的前向传播，实现图片分类。 6. **训练与验证**：在`Inception-resnet-v2.protxt`文件中，包含了训练和验证阶段的配置，比如损失函数、优化器的选择以及数据读取的设定等。通常，...
resnet_imagenet_python接口.zip
11-26
在本项目中，我们主要探讨如何使用Caffe的Python接口进行图像分类，特别是基于ResNet-18模型的实现。Caffe是一种广泛使用的深度学习框架，以其高效、灵活性和易于扩展而闻名。在这个"resnet_imagenet_python接口.zip...
史上最全的ResNet网络文件(for Caffe)
12-12
整合了在Caffe框架下使用的主流ResNet网络结构文件Prototxt. 主要包括Resnet-18,34,50,101,152 的训练及测试网络结构文件;还有个solver文件用于配置参数.
resnet50-caffe.pth
10-22
resnet-50在ImageNet数据集上的caffe预训练模型，可以用于分类，目标检测等。
resnet50_caffe.pth
12-16
resnet50_caffe.pth 预训练模型之一。 在Detectron.pytorch等项目中的config文件（*.yaml）中会用到
ResNet-101-model.caffemodel
03-27
ImageNet pre-trained ResNet-101的caffemodel 来源：https://github.com/KaimingHe/deep-residual-networks中提供的OneDrive链接
ResNet-50-model.caffemodel
03-27
ImageNet pre-trained ResNet-50的caffemodel 来源：https://github.com/KaimingHe/deep-residual-networks中提供的OneDrive链接
Faster-RCNN pytorch1.0，vgg16-caffe.pth和resnet101-caffe.pth
03-11
使用这些预训练模型时，需要注意的是，由于Caffe和PyTorch的卷积层实现可能存在差异，因此在迁移过程中可能需要进行一些适配工作。此外，预训练模型通常是为ImageNet分类任务训练的，对于目标检测任务，可能还需要...
验证码识别caffe实现.txt
09-17
本方案的核心在于利用Caffe框架中的ResNet模型进行验证码图像的识别工作。ResNet（Residual Network）是一种深度残差网络结构，通过引入跳过连接解决了深层神经网络训练时容易出现的梯度消失问题，使得模型能够更...
【OpenCV】基于ResNet和Caffe模型的图像分类
李白白的博客
04-14 3895
文章目录前言一、准备文件二、代码总结 前言 环境：群辉docker内安装Anaconda3，已装好OpenCV4.5。jupyter notebook编写代码。 一、准备文件 ResNet-50-deploy.prototxt 定义模型结构配置文件 ResNet-50-model.caffemodel 实际层权重训练模型文件 classes.txt 标签文件 二、代码 import cv2 import numpy as np from matplotlib import pyplot as plt
使用ResNet进行图片分类
06-10
好的，使用ResNet进行图片分类的步骤如下： 1.准备数据集：首先需要准备一个图片分类的数据集，包含训练集、验证集和测试集。 2.加载数据集：使用Python中的各种图像处理和数据加载库，如Pillow和PyTorch自带的...
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值