caffe绘制训练过程中的accuracy、loss曲线

利用caffe中的脚本工具绘制出来的accuracy、loss曲线分别在两张图中，本人通过代码实现了在一张图中同时绘制两条曲线。

本文以mnist的训练为例说明：

1、保存训练模型时的日志文件

修改train_lenet.sh文件，加入“2>&1   | tee examples/mnist/mnist_train_log.log” 保存日志到mnist_train_log.log中

2、提取日志文件训练数据（accuracy、loss）

caffe/tools/extra/parse_log.sh examples/mnist/mnist_train_log.log

脚本运行后会生成两个文件mnist_train_log.log.test和mnist_train_log.log.train

test部分的数据截图如下：

train部分的数据截图如下：

3、解析上述两个文件中的数据并绘制曲线

实现的代码示例如下：

import math
import os
import re
import sys
import matplotlib.pyplot as plt
import numpy as np
import pylab
from mpl_toolkits.axes_grid1 import host_subplot
from pylab import figure, show, legend

train_iterations = []
train_loss = []
test_iterations = []
test_accuracy = []

train_log_path = 'mnist/mnist_train_log.log.train'
test_log_path = 'mnist/mnist_train_log.log.test'
ftrain = open(test_log_path, 'r')
for index, train_line in enumerate(ftrain.readlines()):
    if index == 0:
        continue
    iteration = int(train_line.strip().split(' ')[0])
    accuracy = float(train_line.strip()[18:24])
    test_iterations.append(iteration)
    test_accuracy.append(accuracy)
ftrain.close()

ftest = open(train_log_path, 'r')
for index, test_line in enumerate(ftest.readlines()):
    if index == 0:
        continue
    iteration = int(test_line.strip().split(' ')[0])
    loss = float(test_line.strip()[18:29])
    train_iterations.append(iteration)
    train_loss.append(loss)
ftest.close()

host = host_subplot(111)
plt.subplots_adjust(right=0.8) # ajust the right boundary of the plot window
par1 = host.twinx()
# set labels
host.set_xlabel("iterations")
host.set_ylabel("loss")
par1.set_ylabel("accuracy")
 
# plot curves
p1, = host.plot(train_iterations, train_loss, 'ob-',label="loss")
p2, = par1.plot(test_iterations, test_accuracy, 'xg-',label="accuracy")
#p2, = par1.plot(test_iterations, test_accuracy, 'xg-',label="accuracy")
 
# set location of the legend, 
# 1->rightup corner, 2->leftup corner, 3->leftdown corner
# 4->rightdown corner, 5->rightmid ...
host.legend(loc=5)
 
# set label color
host.axis["left"].label.set_color(p1.get_color())
par1.axis["right"].label.set_color(p2.get_color())
# set the range of x axis of host and y axis of par1
host.set_xlim([0, 10000])
par1.set_ylim([0., 2.6])
plt.draw()
plt.show()

绘制的路线图如下：

4、caffe中自带工具绘制曲线

      首先将文件夹caffe/tools/extra下的parse_log.sh 、extract_seconds.py、plot_training_log.py.example复制到上一步log日志文件的保存目录下，将plot_training_log.py.example改为plot_training_log.py，并执行以下命令就可以绘制曲线：
python plot_training_log.py 6 train_loss.png mnist_train_log.log

caffe支持多种曲线绘制，指定不同的类型参数即可，具体参数如下:
Notes: 
    1. Supporting multiple logs. 
    2. Log file name must end with the lower-cased ".log". 
Supported chart types: 
    0: Test accuracy  vs. Iters 
    1: Test accuracy  vs. Seconds 
    2: Test loss  vs. Iters 
    3: Test loss  vs. Seconds 
    4: Train learning rate  vs. Iters 
    5: Train learning rate  vs. Seconds 
    6: Train loss  vs. Iters 
    7: Train loss  vs. Seconds