0.前言
这一篇主要是一些实验数据,实验结果以及分析。
1.训练网络过程记录及分析
1.1 影响网络结果的因素
影响网络预测的因素大致为两类,第一类是网络自身结构造成的影响,第二类是数据方面造成的影响。
第一类中主要起影响作用的是:
(1) batch_size; (2) firstLayer; (3) secondeLayer; (4) thirdLayer; (5) epochs; (6) data_train;
第二类中主要起影响作用的是:
(1) dataFormat; (2) samplesNum; (3) isRandom;
1.2 实验假定条件
目前,定第一类的情况不变,只改变第二类的情况,观察结果。
在数据准备方面,准备两种数据格式:
if dataFormat == 2:
print("dataFormat = 2")
trainData = np.empty((sampleNums, 2, rowColTotalNum))
elif dataFormat == 1:
print("dataFormat = 1")
trainData = np.empty((sampleNums, rowColTotalNum*2))
samplesNum代表样本数,定为:11000, 33000 , 55000, 77000, 222000, 444000, 666000, 888000, 1111000这九种情况。
isRandom代表数据样本选取的形式,若为真,则代表,以samplesNum为样本数,随机选取符合总数为samplesNum的样本;若为FALSE,则表示,按顺序选取样本数为samplesNum的样本;
这两种情况代表的,一种是有序,另一种是无序随机。这两种结合起来,进行对比足以说明问题。
2. 实验结果记录及分析
2.1 按照1.2的假定条件进行实验
定第一类不变,第二类中样本数和isRandom不变,只改变数据格式,实验结果(由于全部信息太多,只截取部分信息)如下:
实验结果:
dataFormat为2的情况:
Using TensorFlow backend.
load data begin...
load data..., no random!
load done
Data.shape= (11000, 2, 429)
label.shape= (11000, 5)
dataFormat = 2, need to reshape to 1
reshape data shape= (11000, 858)
reshape label shape= (11000, 5)
data_train.shape= (8000, 858)
data_test.shape= (3000, 858)
label_train.shape= (8000, 5)
label_test.shape= (3000, 5)
construct net begin...
Training...
Epoch 1/200
200/8000 [..............................] - ETA: 3:41 - loss: 0.2301 - acc: 0.1100
1400/8000 [====>.........................] - ETA: 27s - loss: 0.5132 - acc: 0.6029
2800/8000 [=========>....................] - ETA: 10s - loss: 0.2827 - acc: 0.6343
4200/8000 [==============>...............] - ETA: 5s - loss: 0.1952 - acc: 0.6555
5600/8000 [====================>.........] - ETA: 2s - loss: 0.1478 - acc: 0.7166
7000/8000 [=========================>....] - ETA: 0s - loss: 0.1191 - acc: 0.7551
8000/8000 [==============================] - 6s 749us/step - loss: 0.1048 - acc: 0.7755
Epoch 2/200
200/8000 [..............................] - ETA: 0s - loss: 0.0039 - acc: 0.9100
1600/8000 [=====>........................] - ETA: 0s - loss: 0.0040 - acc: 0.9262
3000/8000 [==========>...................] - ETA: 0s - loss: 0.0039 - acc: 0.9330
4400/8000 [===============>..............] - ETA: 0s - loss: 0.0037 - acc: 0.9327
5800/8000 [====================>.........] - ETA: 0s - loss: 0.0036 - acc: 0.9348
7200/8000 [==========================>...] - ETA: 0s - loss: 0.0036 - acc: 0.9342
8000/8000 [==============================] - 0s 40us/step - loss: 0.0035 - acc: 0.9326
Epoch 3/200
200/8000 [..............................] - ETA: 0s - loss: 0.0021 - acc: 0.9450
1600/8000 [=====>........................] - ETA: 0s - loss: 0.0020 - acc: 0.9181
3000/8000 [==========>...................] - ETA: 0s - loss: 0.0020 - acc: 0.9227
4400/8000 [===============>..............] - ETA: 0s - loss: 0.0020 - acc: 0.9225
5800/8000 [====================>.........] - ETA: 0s - loss: 0.0020 - acc: 0.9205
7200/8000 [==========================>...] - ETA: 0s - loss: 0.0020 - acc: 0.9224
8000/8000 [==============================] - 0s 39us/step - loss: 0.0020 - acc: 0.9215
Epoch 182/200
200/8000 [..............................] - ETA: 0s - loss: 0.0024 - acc: 0.9250
1600/8000 [=====>........................] - ETA: 0s - loss: 0.0021 - acc: 0.9231
3000/8000 [==========>...................] - ETA: 0s - loss: 0.0020 - acc: 0.9203
4400/8000 [===============>..............] - ETA: 0s - loss: 0.0020 - acc: 0.9180
5800/8000 [====================>.........] - ETA: 0s - loss: 0.0020 - acc: 0.9179
7200/8000 [==========================>...] - ETA: 0s - loss: 0.0020 - acc: 0.9206
8000/8000 [==============================] - 0s 39us/step - loss: 0.0020 - acc: 0.9215
Epoch 183/200
200/8000 [..............................] - ETA: 0s - loss: 0.0020 - acc: 0.9100
1600/8000 [=====>........................] - ETA: 0s - loss: 0.0019 - acc: 0.9150
3000/8000 [==========>...................] - ETA: 0s - loss: 0.0019 - acc: 0.9160
4400/8000 [===============>..............] - ETA: 0s - loss: 0.0020 - acc: 0.9186
5800/8000 [====================>.........] - ETA: 0s - loss: 0.0020 - acc: 0.9184
7200/8000 [==========================>...] - ETA: 0s - loss: 0.0020 - acc: 0.9218
8000/8000 [==============================] - 0s 40us/step - loss: 0.0020 - acc: 0.9215
Epoch 184/200
200/8000 [..............................] - ETA: 0s - loss: 0.0019 - acc: 0.9200
1600/8000 [=====>........................] - ETA: 0s - loss: 0.0020 - acc: 0.9194
3000/8000 [==========>...................] - ETA: 0s - loss: 0.0020 - acc: 0.9143
4400/8000 [===============>..............] - ETA: 0s - loss: 0.0020 - acc: 0.9195
5800/8000 [====================>.........] - ETA: 0s - loss: 0.0020 - acc: 0.9207
7200/8000 [==========================>...] - ETA: 0s - loss: 0.0020 - acc: 0.9219
8000/8000 [==============================] - 0s 40us/step - loss: 0.0020 - acc: 0.9215
Epoch 00184: early stopping
Testing ------------
200/3000 [=>............................] - ETA: 0s
2600/3000 [=========================>....] - ETA: 0s
3000/3000 [==============================] - 0s 26us/step
test cost: [0.001527793175773695, 0.89100000063578289]
./saveProcessResult/processb_200FIL_858FO_572SO_572EO_5_0521-11_00dataFormat2.png
贴一张训练过程图:
dataFormat为1时的实验过程(只截取部分)如下:
Using TensorFlow backend.
load data begin...
load data..., no random!
load done
Data.shape= (11000, 858)
label.shape= (11000, 5)
reshape data shape= (11000, 858)
reshape label shape= (11000, 5)
data_train.shape= (8000, 858)
data_test.shape= (3000, 858)
label_train.shape= (8000, 5)
label_test.shape= (3000, 5)
construct net begin...
Training...
Epoch 1/200
2018-05-21 11:05:41.724062: I C:\tf_jenkins\home\workspace\rel-win\M\windows-gpu\PY\35\tensorflow\core\platform\cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX AVX2
2018-05-21 11:05:42.025240: I C:\tf_jenkins\home\workspace\rel-win\M\windows-gpu\PY\35\tensorflow\core\common_runtime\gpu\gpu_device.cc:1030] Found device 0 with properties:
name: Quadro K1200 major: 5 minor: 0 memoryClockRate(GHz): 1.0325
pciBusID: 0000:01:00.0
totalMemory: 4.00GiB freeMemory: 3.35GiB
2018-05-21 11:05:42.025796: I C:\tf_jenkins\home\workspace\rel-win\M\windows-gpu\PY\35\tensorflow\core\common_runtime\gpu\gpu_device.cc:1120] Creating TensorFlow device (/device:GPU:0) -> (device: 0, name: Quadro K1200, pci bus id: 0000:01:00.0, compute capability: 5.0)
200/8000 [..............................] - ETA: 1:24 - loss: 0.2852 - acc: 0.3250
1600/8000 [=====>........................] - ETA: 8s - loss: 0.4600 - acc: 0.2925
3000/8000 [==========>...................] - ETA: 3s - loss: 0.2971 - acc: 0.4343
4400/8000 [===============>..............] - ETA: 1s - loss: 0.2098 - acc: 0.5180
5800/8000 [====================>.........] - ETA: 0s - loss: 0.1605 - acc: 0.6069
7200/8000 [==========================>...] - ETA: 0s - loss: 0.1301 - acc: 0.6676
8000/8000 [==============================] - 2s 309us/step - loss: 0.1175 - acc: 0.6936
Epoch 2/200
200/8000 [..............................] - ETA: 0s - loss: 0.0038 - acc: 0.9550
1600/8000 [=====>........................] - ETA: 0s - loss: 0.0036 - acc: 0.9356
3000/8000 [==========>...................] - ETA: 0s - loss: 0.0035 - acc: 0.9363
4400/8000 [===============>..............] - ETA: 0s - loss: 0.0034 - acc: 0.9386
5800/8000 [====================>.........] - ETA: 0s - loss: 0.0034 - acc: 0.9422
7200/8000 [==========================>...] - ETA: 0s - loss: 0.0033 - acc: 0.9421
8000/8000 [==============================] - 0s 39us/step - loss: 0.0033 - acc: 0.9420
Epoch 3/200
200/8000 [..............................] - ETA: 0s - loss: 0.0033 - acc: 0.9400
1600/8000 [=====>........................] - ETA: 0s - loss: 0.0032 - acc: 0.9406
3000/8000 [==========>...................] - ETA: 0s - loss: 0.0031 - acc: 0.9357
4400/8000 [===============>..............] - ETA: 0s - loss: 0.0030 - acc: 0.9357
5800/8000 [====================>.........] - ETA: 0s - loss: 0.0031 - acc: 0.9345
7200/8000 [==========================>...] - ETA: 0s - loss: 0.0031 - acc: 0.9335
8000/8000 [==============================] - 0s 39us/step - loss: 0.0030 - acc: 0.9346
Epoch 4/200
200/8000 [..............................] - ETA: 0s - loss: 0.0032 - acc: 0.9250
1600/8000 [=====>........................] - ETA: 0s - loss: 0.0030 - acc: 0.9162
3000/8000 [==========>...................] - ETA: 0s - loss: 0.0030 - acc: 0.9267
4400/8000 [===============>..............] - ETA: 0s - loss: 0.0030 - acc: 0.9302
5800/8000 [====================>.........] - ETA: 0s - loss: 0.0030 - acc: 0.9288
7200/8000 [==========================>...] - ETA: 0s - loss: 0.0029 - acc: 0.9310
8000/8000 [==============================] - 0s 40us/step - loss: 0.0029 - acc: 0.9311
。。。。
200/8000 [..............................] - ETA: 0s - loss: 0.0024 - acc: 0.9550
1600/8000 [=====>........................] - ETA: 0s - loss: 0.0020 - acc: 0.9275
3000/8000 [==========>...................] - ETA: 0s - loss: 0.0020 - acc: 0.9227
4400/8000 [===============>..............] - ETA: 0s - loss: 0.0020 - acc: 0.9218
5800/8000 [====================>.........] - ETA: 0s - loss: 0.0020 - acc: 0.9221
7200/8000 [==========================>...] - ETA: 0s - loss: 0.0020 - acc: 0.9219
8000/8000 [==============================] - 0s 39us/step - loss: 0.0020 - acc: 0.9210
Epoch 172/200
200/8000 [..............................] - ETA: 0s - loss: 0.0020 - acc: 0.9150
1600/8000 [=====>........................] - ETA: 0s - loss: 0.0019 - acc: 0.9188
3000/8000 [==========>...................] - ETA: 0s - loss: 0.0019 - acc: 0.9193
4400/8000 [===============>..............] - ETA: 0s - loss: 0.0020 - acc: 0.9180
5800/8000 [====================>.........] - ETA: 0s - loss: 0.0020 - acc: 0.9210
7200/8000 [==========================>...] - ETA: 0s - loss: 0.0020 - acc: 0.9215
8000/8000 [==============================] - 0s 40us/step - loss: 0.0020 - acc: 0.9210
Epoch 173/200
200/8000 [..............................] - ETA: 0s - loss: 0.0024 - acc: 0.9550
1600/8000 [=====>........................] - ETA: 0s - loss: 0.0021 - acc: 0.9300
3000/8000 [==========>...................] - ETA: 0s - loss: 0.0020 - acc: 0.9237
4200/8000 [==============>...............] - ETA: 0s - loss: 0.0020 - acc: 0.9236
5600/8000 [====================>.........] - ETA: 0s - loss: 0.0020 - acc: 0.9238
7000/8000 [=========================>....] - ETA: 0s - loss: 0.0020 - acc: 0.9224
8000/8000 [==============================] - 0s 40us/step - loss: 0.0020 - acc: 0.9210
Epoch 174/200
200/8000 [..............................] - ETA: 0s - loss: 0.0020 - acc: 0.9200
1600/8000 [=====>........................] - ETA: 0s - loss: 0.0020 - acc: 0.9250
3000/8000 [==========>...................] - ETA: 0s - loss: 0.0020 - acc: 0.9237
4200/8000 [==============>...............] - ETA: 0s - loss: 0.0019 - acc: 0.9238
5600/8000 [====================>.........] - ETA: 0s - loss: 0.0020 - acc: 0.9245
7000/8000 [=========================>....] - ETA: 0s - loss: 0.0020 - acc: 0.9217
8000/8000 [==============================] - 0s 40us/step - loss: 0.0020 - acc: 0.9210
Epoch 175/200
200/8000 [..............................] - ETA: 0s - loss: 0.0020 - acc: 0.8950
1600/8000 [=====>........................] - ETA: 0s - loss: 0.0020 - acc: 0.9244
3000/8000 [==========>...................] - ETA: 0s - loss: 0.0020 - acc: 0.9237
4400/8000 [===============>..............] - ETA: 0s - loss: 0.0020 - acc: 0.9230
5800/8000 [====================>.........] - ETA: 0s - loss: 0.0020 - acc: 0.9224
7200/8000 [==========================>...] - ETA: 0s - loss: 0.0020 - acc: 0.9214
8000/8000 [==============================] - 0s 39us/step - loss: 0.0020 - acc: 0.9210
Epoch 00175: early stopping
Testing ------------
200/3000 [=>............................] - ETA: 0s
2600/3000 [=========================>....] - ETA: 0s
3000/3000 [==============================] - 0s 29us/step
test cost: [0.0015478261494232962, 0.89099999666213991]
./saveProcessResult/processb_200FIL_858FO_572SO_572EO_5_0521-11_04dataFormat1.png
贴一张训练图:
从以上信息可以得到以下结论:
(1)dataFormat格式对于实验结果的准确率影响微弱;
(2)快速下降都是在第二个epoch;第二个epoch结束后,dataFormat为1时的loss: 0.0033 - acc: 0.9420;dataFormat为2时的loss: 0.0035 - acc: 0.9326;可见dataFormat对于loss的影响还是有点的,准确率高出1%, loss低了0.02%;
(3)dataFormat为1时, Epoch 00175: early stopping, loss: 0.0020 - acc: 0.9210;dataFormat为2时, Epoch 00184: early stopping, loss: 0.0020 - acc: 0.9215;蛮奇怪的,准确率dataFormat为2时还高一点点, 0.05%,好吧,可以忽略了。loss一致;
综上考虑,选择dataFormat为1,相关性增强了,提高了训练速度;