caffe下使用faster-rcnn中end2end训练结束绘制loss曲线

caffe下使用faster-rcnn中end2end训练结束绘制loss曲线

设计好网络结构后，在神经网络训练的过程中，迭代输出的log信息中，一般包括，迭代次数，训练损失代价，测试损失代价，测试精度等。在py-faster-rcnn-master\experiments\logs目录下找到最新的txt文件，调用函数画出loss曲线。

import os
import sys
import numpy as np
import matplotlib.pyplot as plt
import math
import re
import pylab
from pylab import figure, show, legend
from mpl_toolkits.axes_grid1 import host_subplot

# read the log file
fp = open('faster_rcnn_end2end_VGG_CNN_M_1024_.txt.2019-03-22_09-18-24', 'r')

train_iterations = []
train_loss = []
test_iterations = []
# test_accuracy = []


# 遍历每一行
for ln in fp:
  # get train_iterations and train_loss
  if '] Iteration ' in ln and 'loss = ' in ln:
    # 以上图第一行为例，以下的命令匹配的是'ion 22000,'
    arr = re.findall(r'ion \b\d+\b,', ln)
    train_iterations.append(int(arr[0].strip(',')[4:]))

    # 以下寻找的是0.173712
    #  train_loss.append(float(ln.strip().split(' = ')[-1])) 报错：ValueError: invalid literal for float(): 0.069speed: 0.056s / iter
    # 原因：模型训练次数大于10000时，日志文件中出现“I0312 19:08:53.461860  2864 solver.cpp:228] Iteration 18340, loss = 0.05526speed: 0.057s / iter”
    # 解决一：在日志文件中报错地方的“speed”前加回车键修改
    # 解决二：修改代码如下
    y = ln.strip().split(' = ')[-1]
    try:
      x = float(y)
    except ValueError:
      ind = y.index('speed')
      print(y)
      x = float(y[0:ind])
    train_loss.append(x)

fp.close()

host = host_subplot(111)
plt.subplots_adjust(right=0.8)  # ajust the right boundary of the plot window
# par1 = host.twinx()
# set labels
host.set_xlabel("iterations")
host.set_ylabel("loss")
# par1.set_ylabel("validation accuracy")
# plot curves
p1, = host.plot(train_iterations, train_loss, label="train loss")
# p2, = par1.plot(test_iterations, test_accuracy, label="validation accuracy")

# set location of the legend,
# 1->rightup corner, 2->leftup corner, 3->leftdown corner
# 4->rightdown corner, 5->rightmid ...
host.legend(loc=1)

# set label color
host.axis["left"].label.set_color(p1.get_color())
# par1.axis["right"].label.set_color(p2.get_color())
# set the range of x axis of host and y axis of par1
host.set_xlim([-100, 70000])
host.set_ylim([-0.2, 2])
plt.draw()
plt.show()

报错：

train_loss.append(float(ln.strip().split(' = ')[-1]))

ValueError: invalid literal for float(): 0.069speed: 0.056s / iter

原因：

模型训练次数大于10000时，日志文件中出现“I0312 19:08:53.461860  2864 solver.cpp:228] Iteration 18340, loss = 0.05526speed: 0.057s / iter”
    # 解决一：在日志文件中报错地方的“speed”前加回车键修改
    # 解决二：修改代码