yolo训练之训练结果评估环节

在yolo中，怎么知道自己已经训练的怎么样了呢？自然是希望把训练过程中的loss等数据可视化一下，这篇文章中，我们主要就介绍一下这些。

1.记录终端信息到文件

首先在训练开始的时候需要把终端信息记录到文件，我这里使用的命令是| tee train_log.txt ，可参考：Linux中记录终端（Terminal）输出到文本文件 。我们会得到这样一个文本文件：

2. python数据处理

下面我们就可以用python对其进行处理了，这里我写了两个jupyter notebook，第一个可视化了loss，第二个可视化了iou。

#### **A.可视化loss**

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

result = pd.read_csv('trainlog.txt', skiprows=[x for x in range(60000) if x%10!=9] ,error_bad_lines=False, names=['loss', 'avg', 'rate', 'seconds', 'images'])
result.head()

	loss	avg	rate	seconds	images
0	1: 121.201286	121.201286 avg	0.000500 rate	5.607964 seconds	64 images
1	2: 116.259315	120.707092 avg	0.000500 rate	5.027367 seconds	128 images
2	3: 92.172997	117.853683 avg	0.000500 rate	5.162279 seconds	192 images
3	4: 85.307167	114.599030 avg	0.000500 rate	4.845596 seconds	256 images
4	5: 81.885292	111.327652 avg	0.000500 rate	5.068791 seconds	320 images

result['loss']=result['loss'].str.split(' ').str.get(1)
result['avg']=result['avg'].str.split(' ').str.get(1)
result['rate']=result['rate'].str.split(' ').str.get(1)
result['seconds']=result['seconds'].str.split(' ').str.get(1)
result['images']=result['images'].str.split(' ').str.get(1)
result.head()
result.tail()

	loss	avg	rate	seconds	images
5006	0.903784	0.923844	0.010000	4.661594	320448
5007	0.856821	0.917142	0.010000	4.695950	320512
5008	0.917766	0.917204	0.010000	4.740102	320576
5009	0.866580	0.912142	0.010000	4.713165	320640
5010	1.071078	0.928035	0.010000	4.728881	320704

# print(result.head())
# print(result.tail())
# print(result.dtypes)

result['loss']=pd.to_numeric(result['loss'])
result['avg']=pd.to_numeric(result['avg'])
result['rate']=pd.to_numeric(result['rate'])
result['seconds']=pd.to_numeric(result['seconds'])
result['images']=pd.to_numeric(result['images'])
result.dtypes

loss float64 avg float64 rate float64 seconds float64 images int64 dtype: object

result['loss'].plot()

p=result['avg'].plot()

 图

result['avg'].values

array([ 121.201286, 120.707092, 117.853683, …, 0.917204, 0.912142, 0.928035])

fig = plt.figure();
ax = fig.add_subplot(1, 1, 1)
ax.plot(result['avg'].values,label='avg_loss')
ax.legend(loc='best')
ax.set_title('The loss curves')
ax.set_xlabel('batches')
fig.savefig('avg_loss')

 图 #### **B.可视化IOU**

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

result = pd.read_csv('trainlog.txt', skiprows=[x for x in range(500000) if x%10==9 or x%10==0] ,error_bad_lines=False, names=['Detection Avg IOU', 'Pos Cat', 'All Cat', 'Pos Obj', 'Any Obj', 'count'])
result.head()

	Detection Avg IOU	Pos Cat	All Cat	Pos Obj	Any Obj	count
0	Detection Avg IOU: 0.061472	Pos Cat: -0.054766	All Cat: 0.160247	Pos Obj: -0.081178	Any Obj: -0.016451	count: 16
1	Detection Avg IOU: 0.083749	Pos Cat: 0.272016	All Cat: 0.223963	Pos Obj: 0.130788	Any Obj: -0.018666	count: 14
2	Detection Avg IOU: 0.105311	Pos Cat: 0.178397	All Cat: 0.069925	Pos Obj: -0.062407	Any Obj: -0.016685	count: 17
3	Detection Avg IOU: 0.056007	Pos Cat: 0.142428	All Cat: 0.043840	Pos Obj: -0.197250	Any Obj: -0.051494	count: 15
4	Detection Avg IOU: 0.085293	Pos Cat: 0.108593	All Cat: 0.033600	Pos Obj: 0.020100	Any Obj: -0.012297	count: 15

result['Detection Avg IOU']=result['Detection Avg IOU'].str.split(': ').str.get(1)
result['Pos Cat']=result['Pos Cat'].str.split(': ').str.get(1)
result['All Cat']=result['All Cat'].str.split(': ').str.get(1)
result['Pos Obj']=result['Pos Obj'].str.split(': ').str.get(1)
result['Any Obj']=result['Any Obj'].str.split(': ').str.get(1)
result['count']=result['count'].str.split(': ').str.get(1)
result.head()

	Detection Avg IOU	Pos Cat	All Cat	Pos Obj	Any Obj	count
0	0.061472	-0.054766	0.160247	-0.081178	-0.016451	16
1	0.083749	0.272016	0.223963	0.130788	-0.018666	14
2	0.105311	0.178397	0.069925	-0.062407	-0.016685	17
3	0.056007	0.142428	0.043840	-0.197250	-0.051494	15
4	0.085293	0.108593	0.033600	0.020100	-0.012297	15

result['Detection Avg IOU']=pd.to_numeric(result['Detection Avg IOU'])
result['Pos Cat']=pd.to_numeric(result['Pos Cat'])
result['All Cat']=pd.to_numeric(result['All Cat'])
result['Pos Obj']=pd.to_numeric(result['Pos Obj'])
result['Any Obj']=pd.to_numeric(result['Any Obj'])
result['count']=pd.to_numeric(result['count'])
result.dtypes

Detection Avg IOU    float64
Pos Cat              float64
All Cat              float64
Pos Obj              float64
Any Obj              float64
count                  int64
dtype: object

result['Detection Avg IOU'].plot()

<matplotlib.axes._subplots.AxesSubplot at 0x7f4cdc6691d0>

result['Pos Cat'].plot()

<matplotlib.axes._subplots.AxesSubplot at 0x7f4cdc66c390>

3. 看看结果至少还是符合趋势的，哈哈