对于训练过程的数据分析

最近跑了一波bert模型，分析一下相应的训练过程以及自己的心得体会

Epoch 1/10
184/184 [==============================] - 91s 411ms/step - loss: 69.9343 - sparse_accuracy: 0.6796

100%|██████████| 1970/1970 [03:19<00:00,  9.87it/s]

准确率 = 
{1: 1.0, 2: 0.9789741956036955, 3: 0.9465433050629658, 4: 0.9165302782324058, 5: 0.9289281598155974, 6: 0.7776752767527675, 7: 1.0, 8: 0.9415481832543444, 9: 0.9766816143497757, 10: 0.7871687587168759, 11: 0.8740458015267175, 12: 0.9035469819539514, 13: 0.9772079772079773, 14: 0.6666666666666666, 15: 0, 16: 0, 17: 0, 18: 0, 19: 0, 20: 0}
召回率 = 
{1: 0.9961210240496509, 2: 0.9743183259353202, 3: 0.9354838709677419, 4: 0.9024979854955681, 5: 0.8764044943820225, 6: 0.7960339943342776, 7: 0.1092436974789916, 8: 0.9786535303776683, 9: 0.981081081081081, 10: 0.9538617542673652, 11: 0.1497710922171354, 12: 0.9465449804432855, 13: 0.8772378516624041, 14: 0.003838771593090211, 15: 0, 16: 0, 17: 0.0, 18: 0.0, 19: 0.0, 20: 0}
f1_score = 
{1: 0.9980567431014381, 2: 0.9766407119021133, 3: 0.9409810935104751, 4: 0.9094600081201786, 5: 0.9019022752704214, 6: 0.7867475501633223, 7: 0.19696969696969696, 8: 0.959742351046699, 9: 0.9788764044943821, 10: 0.8625353404141514, 11: 0.25572305974316023, 12: 0.924546322827125, 13: 0.9245283018867925, 14: 0.0076335877862595426, 15: 0, 16: 0, 17: 0, 18: 0, 19: 0, 20: 0}
总分数为
33.780953100665556
Epoch 2/10
184/184 [==============================] - 78s 423ms/step - loss: 9.4470 - sparse_accuracy: 0.9569

100%|██████████| 1970/1970 [03:23<00:00,  9.69it/s]

准确率 = 
{1: 0.9972868217054264, 2: 0.9800695982284088, 3: 0.9672258064516129, 4: 0.9332260659694288, 5: 0.957967137944211, 6: 0.8636363636363636, 7: 0.8991596638655462, 8: 0.9735548794757781, 9: 0.975609756097561, 10: 0.8174137190319347, 11: 0.8604954367666232, 12: 0.9743589743589743, 13: 0.9922480620155039, 14: 0.9655172413793104, 15: 0, 16: 0, 17: 0.7912621359223301, 18: 0, 19: 0.98, 20: 0}
召回率 = 
{1: 0.9980605120248255, 2: 0.9822447685478757, 3: 0.9519939039878079, 4: 0.934730056406124, 5: 0.9086625588981515, 6: 0.789423984891407, 7: 0.8991596638655462, 8: 0.9758386113065916, 9: 0.984984984984985, 10: 0.9646780463072503, 11: 0.4316546762589928, 12: 0.9413298565840938, 13: 0.9820971867007673, 14: 0.9673704414587332, 15: 0, 16: 0, 17: 0.8534031413612565, 18: 0.0, 19: 0.7424242424242424, 20: 0}
f1_score = 
{1: 0.9976735168670028, 2: 0.9811559778305621, 3: 0.9595494111623144, 4: 0.9339774557165862, 5: 0.9326636904761905, 6: 0.8248643315244203, 7: 0.8991596638655462, 8: 0.9746954076850984, 9: 0.9802749551703527, 10: 0.8849612403100776, 11: 0.5749128919860628, 12: 0.9575596816976126, 13: 0.9871465295629821, 14: 0.9664429530201343, 15: 0, 16: 0, 17: 0.8211586901763224, 18: 0, 19: 0.8448275862068965, 20: 0}
总分数为
43.75811228211583
Epoch 3/10
184/184 [==============================] - 75s 409ms/step - loss: 5.8964 - sparse_accuracy: 0.9705

100%|██████████| 1970/1970 [03:10<00:00, 10.32it/s]

准确率 = 
{1: 0.9968980224893369, 2: 0.9685063922669161, 3: 0.9413793103448276, 4: 0.9409209383145091, 5: 0.9214566226347733, 6: 0.8599605522682445, 7: 0.9426229508196722, 8: 0.9799043062200957, 9: 0.9816981698169817, 10: 0.8370972598614875, 11: 0.875, 12: 0.9657669519420672, 13: 0.9922879177377892, 14: 0.9718045112781954, 15: 0, 16: 0, 17: 0.8434343434343434, 18: 0, 19: 0.9821428571428571, 20: 0}
召回率 = 
{1: 0.9972847168347556, 2: 0.9847812301838935, 3: 0.9707899415798832, 4: 0.8726833199033038, 5: 0.9354838709677419, 6: 0.8234183191690274, 7: 0.9663865546218487, 8: 0.9608257095941825, 9: 0.9825825825825826, 10: 0.939665370965016, 11: 0.4898626553302812, 12: 0.9563233376792699, 13: 0.9872122762148338, 14: 0.9923224568138196, 15: 0, 16: 0, 17: 0.8743455497382199, 18: 0.0, 19: 0.8333333333333334, 20: 0}
f1_score = 
{1: 0.9970913321698662, 2: 0.9765760100613112, 3: 0.955858446917594, 4: 0.9055183946488294, 5: 0.9284172661870504, 6: 0.8412928123492522, 7: 0.9543568464730291, 8: 0.9702712306052351, 9: 0.9821401770974035, 10: 0.8854208137590572, 11: 0.6280922431865829, 12: 0.961021945627252, 13: 0.9897435897435897, 14: 0.9819563152896487, 15: 0, 16: 0, 17: 0.858611825192802, 18: 0, 19: 0.9016393442622951, 20: 0}
总分数为
44.2861909256549
Epoch 4/10
184/184 [==============================] - 75s 405ms/step - loss: 4.5023 - sparse_accuracy: 0.9732

100%|██████████| 1970/1970 [03:16<00:00, 10.03it/s]

准确率 = 
{1: 0.9969016266460109, 2: 0.9875080076873799, 3: 0.9597964376590331, 4: 0.9145569620253164, 5: 0.9124555160142349, 6: 0.8894907908992417, 7: 0.9590163934426229, 8: 0.975932043416706, 9: 0.9914373088685016, 10: 0.8467397928092626, 11: 0.8508771929824561, 12: 0.9521683673469388, 13: 0.9922879177377892, 14: 0.9663551401869159, 15: 0, 16: 0, 17: 0.8870967741935484, 18: 1.0, 19: 0.9824561403508771, 20: 0}
召回率 = 
{1: 0.9984484096198604, 2: 0.9774889029803424, 3: 0.9580899161798324, 4: 0.9315068493150684, 5: 0.9293222181949982, 6: 0.7752596789423984, 7: 0.9831932773109243, 8: 0.9702087731644382, 9: 0.9735735735735735, 10: 0.9393273618387696, 11: 0.5709614126880314, 12: 0.9732724902216427, 13: 0.9872122762148338, 14: 0.9923224568138196, 15: 0, 16: 0, 17: 0.8638743455497382, 18: 0.8181818181818182, 19: 0.8484848484848485, 20: 0}
f1_score = 
{1: 0.9976744186046512, 2: 0.9824729126832377, 3: 0.9589424176941655, 4: 0.9229540918163672, 5: 0.9208116358412642, 6: 0.8284561049445005, 7: 0.9709543568464729, 8: 0.9730619927067404, 9: 0.9824242424242424, 10: 0.8906337633202467, 11: 0.6833659491193738, 12: 0.9626047711154094, 13: 0.9897435897435897, 14: 0.9791666666666667, 15: 0, 16: 0, 17: 0.8753315649867373, 18: 0.9, 19: 0.9105691056910568, 20: 0}
总分数为
47.284972605746496
Epoch 5/10
184/184 [==============================] - 75s 405ms/step - loss: 3.5736 - sparse_accuracy: 0.9768

100%|██████████| 1970/1970 [03:17<00:00,  9.95it/s]

准确率 = 
{1: 0.9976726144297905, 2: 0.9850175326745298, 3: 0.9669992325402916, 4: 0.9108678655199375, 5: 0.9589299171062547, 6: 0.8481973434535104, 7: 0.9224806201550387, 8: 0.9744011272898074, 9: 0.99179580674567, 10: 0.8968921981223696, 11: 0.8295932678821879, 12: 0.9626769626769627, 13: 0.9923076923076923, 14: 0.9699812382739212, 15: 0, 16: 0, 17: 0.835, 18: 1.0, 19: 0.9824561403508771, 20: 0}
召回率 = 
{1: 0.9976726144297905, 2: 0.979708306911858, 3: 0.9601219202438405, 4: 0.9387590652699436, 5: 0.9224356650960492, 6: 0.8441926345609065, 7: 1.0, 8: 0.9732582688247713, 9: 0.9801801801801802, 10: 0.9364542842656751, 11: 0.7737083060824068, 12: 0.9752281616688396, 13: 0.989769820971867, 14: 0.9923224568138196, 15: 0, 16: 0, 17: 0.8743455497382199, 18: 0.9090909090909091, 19: 0.8484848484848485, 20: 0}
f1_score = 
{1: 0.9976726144297905, 2: 0.9823557463042442, 3: 0.9635483048687229, 4: 0.9246031746031746, 5: 0.9403288379826344, 6: 0.8461902508282064, 7: 0.9596774193548387, 8: 0.9738293627508509, 9: 0.9859537834164024, 10: 0.916246382802811, 11: 0.8006768189509307, 12: 0.9689119170984456, 13: 0.9910371318822023, 14: 0.9810246679316887, 15: 0, 16: 0, 17: 0.8542199488491049, 18: 0.9523809523809523, 19: 0.9105691056910568, 20: 0}
总分数为
47.87022897228883
Epoch 6/10
184/184 [==============================] - 75s 406ms/step - loss: 2.9267 - sparse_accuracy: 0.9809

100%|██████████| 1970/1970 [03:20<00:00,  9.83it/s]

准确率 = 
{1: 0.9976717112922002, 2: 0.9884578390509778, 3: 0.9654996166624074, 4: 0.9118329466357309, 5: 0.9658914728682171, 6: 0.8913738019169329, 7: 0.936, 8: 0.9811815150071462, 9: 0.9944012441679627, 10: 0.8569465648854961, 11: 0.8368263473053892, 12: 0.9211491442542787, 13: 0.9921465968586387, 14: 0.9591836734693877, 15: 0, 16: 0, 17: 0.8783068783068783, 18: 1.0, 19: 0.9821428571428571, 20: 0}
召回率 = 
{1: 0.9972847168347556, 2: 0.9774889029803424, 3: 0.9596139192278385, 4: 0.9500402900886382, 5: 0.9032258064516129, 6: 0.7903682719546742, 7: 0.9831932773109243, 8: 0.9662209711470795, 9: 0.9600600600600601, 10: 0.9486226128105459, 11: 0.7311968606932636, 12: 0.9823989569752282, 13: 0.969309462915601, 14: 0.9923224568138196, 15: 0, 16: 0, 17: 0.8691099476439791, 18: 1.0, 19: 0.8333333333333334, 20: 0}
f1_score = 
{1: 0.997478176527643, 2: 0.9829427706041767, 3: 0.9625477707006369, 4: 0.9305445935280189, 5: 0.9335081475931822, 6: 0.8378378378378378, 7: 0.959016393442623, 8: 0.9736437773312847, 9: 0.9769289533995417, 10: 0.9004572070265501, 11: 0.7804537521815009, 12: 0.9507886435331231, 13: 0.9805950840879688, 14: 0.9754716981132074, 15: 0, 16: 0, 17: 0.8736842105263158, 18: 1.0, 19: 0.9016393442622951, 20: 0}
总分数为
47.79034041776211
Epoch 7/10
184/184 [==============================] - 77s 417ms/step - loss: 2.4113 - sparse_accuracy: 0.9832

100%|██████████| 1970/1970 [03:24<00:00,  9.64it/s]

准确率 = 
{1: 0.9984478075281334, 2: 0.9757097791798107, 3: 0.9713616245769331, 4: 0.9151750972762646, 5: 0.9452662721893491, 6: 0.9036796536796536, 7: 0.952, 8: 0.9782813975448537, 9: 0.9900542495479204, 10: 0.8784617819275202, 11: 0.8153089887640449, 12: 0.9764089121887287, 13: 0.9922879177377892, 14: 0.9734848484848485, 15: 0, 16: 0, 17: 0.8434343434343434, 18: 1.0, 19: 0.9824561403508771, 20: 0}
召回率 = 
{1: 0.9980605120248255, 2: 0.9806594800253646, 3: 0.9476758953517908, 4: 0.9476228847703465, 5: 0.9264226168901776, 6: 0.7884796978281398, 7: 1.0, 8: 0.9720853858784894, 9: 0.9864864864864865, 10: 0.9381443298969072, 11: 0.7593198168737737, 12: 0.9713168187744459, 13: 0.9872122762148338, 14: 0.9865642994241842, 15: 0, 16: 0, 17: 0.8743455497382199, 18: 1.0, 19: 0.8484848484848485, 20: 0}
f1_score = 
{1: 0.9982541222114453, 2: 0.9781783681214422, 3: 0.9593725893545898, 4: 0.9311163895486935, 5: 0.9357495881383855, 6: 0.8421583459404943, 7: 0.9754098360655737, 8: 0.9751735498293919, 9: 0.9882671480144404, 10: 0.9073226544622426, 11: 0.7863189976295293, 12: 0.9738562091503268, 13: 0.9897435897435897, 14: 0.9799809342230695, 15: 0, 16: 0, 17: 0.858611825192802, 18: 1.0, 19: 0.9105691056910568, 20: 0}
总分数为
47.99478296639099
Epoch 8/10
184/184 [==============================] - 74s 405ms/step - loss: 2.1018 - sparse_accuracy: 0.9845

100%|██████████| 1970/1970 [03:21<00:00,  9.77it/s]

准确率 = 
{1: 0.9980605120248255, 2: 0.9881486226777707, 3: 0.9668198060234814, 4: 0.8965779467680608, 5: 0.9425749817117776, 6: 0.8398058252427184, 7: 0.9568965517241379, 8: 0.9710077156885667, 9: 0.9844124700239808, 10: 0.8609809434551703, 11: 0.8485947416137806, 12: 0.9754966887417219, 13: 0.9923273657289002, 14: 0.9718045112781954, 15: 0, 16: 0, 17: 0.8557213930348259, 18: 1.0, 19: 0.9821428571428571, 20: 0}
召回率 = 
{1: 0.9980605120248255, 2: 0.9781230183893469, 3: 0.9621539243078486, 4: 0.9500402900886382, 5: 0.9340340703153316, 6: 0.8168083097261567, 7: 0.9327731092436975, 8: 0.9741965751817968, 9: 0.9861861861861861, 10: 0.9315531519351022, 11: 0.6121648136036625, 12: 0.9602346805736637, 13: 0.9923273657289002, 14: 0.9923224568138196, 15: 0, 16: 0, 17: 0.900523560209424, 18: 1.0, 19: 0.8333333333333334, 20: 0}
f1_score = 
{1: 0.9980605120248255, 2: 0.9831102613129383, 3: 0.9644812221514958, 4: 0.9225352112676056, 5: 0.9382850901146914, 6: 0.8281474389660125, 7: 0.9446808510638298, 8: 0.9725995316159249, 9: 0.9852985298529853, 10: 0.8948778309927754, 11: 0.7112462006079028, 12: 0.9678055190538767, 13: 0.9923273657289002, 14: 0.9819563152896487, 15: 0, 16: 0, 17: 0.8775510204081632, 18: 1.0, 19: 0.9016393442622951, 20: 0}
总分数为
47.65081053525636
Epoch 9/10
184/184 [==============================] - 70s 381ms/step - loss: 1.8657 - sparse_accuracy: 0.9849

100%|██████████| 1970/1970 [03:17<00:00,  9.99it/s]

准确率 = 
{1: 0.9984484096198604, 2: 0.9822109275730623, 3: 0.968990261404408, 4: 0.9100775193798449, 5: 0.9518757104964002, 6: 0.8305243445692884, 7: 0.9444444444444444, 8: 0.9731308411214953, 9: 0.9850029994001199, 10: 0.8769009124379702, 11: 0.8332046332046332, 12: 0.967784352399737, 13: 0.9923273657289002, 14: 0.9788461538461538, 15: 0, 16: 0, 17: 0.8520408163265306, 18: 1.0, 19: 0.96875, 20: 0}
召回率 = 
{1: 0.9984484096198604, 2: 0.9803424223208624, 3: 0.9603759207518415, 4: 0.9460112812248187, 5: 0.9104748097136643, 6: 0.8375826251180358, 7: 1.0, 8: 0.9770114942528736, 9: 0.9861861861861861, 10: 0.9258069967889133, 11: 0.7056899934597777, 12: 0.9595827900912647, 13: 0.9923273657289002, 14: 0.9769673704414588, 15: 0, 16: 0, 17: 0.8743455497382199, 18: 1.0, 19: 0.9393939393939394, 20: 0}
f1_score = 
{1: 0.9984484096198604, 2: 0.9812757854649318, 3: 0.9646638601862483, 4: 0.927696562623469, 5: 0.9307150796591329, 6: 0.8340385519511049, 7: 0.9714285714285714, 8: 0.9750673065667799, 9: 0.9855942376950779, 10: 0.900690562315028, 11: 0.7641643059490084, 12: 0.9636661211129296, 13: 0.9923273657289002, 14: 0.9779058597502401, 15: 0, 16: 0, 17: 0.8630490956072352, 18: 1.0, 19: 0.9538461538461539, 20: 0}
总分数为
47.969684676288125
Epoch 10/10
184/184 [==============================] - 75s 409ms/step - loss: 1.6171 - sparse_accuracy: 0.9863

100%|██████████| 1970/1970 [03:19<00:00,  9.87it/s]

准确率 = 
{1: 0.9972868217054264, 2: 0.9675101530771634, 3: 0.9615384615384616, 4: 0.9108129439621152, 5: 0.9690353314807463, 6: 0.8630831643002028, 7: 0.9508196721311475, 8: 0.9762576398683592, 9: 0.9887946698970321, 10: 0.834731795025472, 11: 0.8486439195100612, 12: 0.9548678272082527, 13: 0.9923469387755102, 14: 0.9735349716446124, 15: 0, 16: 0, 17: 0.8870967741935484, 18: 1.0, 19: 0.9824561403508771, 20: 0}
召回率 = 
{1: 0.9980605120248255, 2: 0.9819277108433735, 3: 0.95250190500381, 4: 0.9298952457695407, 5: 0.8847408481333816, 6: 0.8035882908404155, 7: 0.9747899159663865, 8: 0.9741965751817968, 9: 0.9804804804804805, 10: 0.9415244211593713, 11: 0.6344015696533682, 12: 0.9654498044328553, 13: 0.9948849104859335, 14: 0.9884836852207294, 15: 0, 16: 0, 17: 0.8638743455497382, 18: 1.0, 19: 0.8484848484848485, 20: 0}
f1_score = 
{1: 0.9976735168670028, 2: 0.9746656176239181, 3: 0.9569988516013781, 4: 0.9202551834130782, 5: 0.9249715801439939, 6: 0.832273838630807, 7: 0.9626556016597511, 8: 0.975226018551133, 9: 0.9846200241254522, 10: 0.8849177984274481, 11: 0.7260479041916168, 12: 0.960129659643436, 13: 0.9936143039591314, 14: 0.980952380952381, 15: 0, 16: 0, 17: 0.8753315649867373, 18: 1.0, 19: 0.9105691056910568, 20: 0}
总分数为
47.637005244368176

首先我想分析的是第一波数据，

Epoch 1/10
184/184 [==============================] - 91s 411ms/step - loss: 69.9343 - sparse_accuracy: 0.6796
100%|██████████| 1970/1970 [03:19<00:00,  9.87it/s]
准确率 = 
{1: 1.0, 2: 0.9789741956036955, 3: 0.9465433050629658, 4: 0.9165302782324058, 5: 0.9289281598155974, 6: 0.7776752767527675, 7: 1.0, 8: 0.9415481832543444, 9: 0.9766816143497757, 10: 0.7871687587168759, 11: 0.8740458015267175, 12: 0.9035469819539514, 13: 0.9772079772079773, 14: 0.6666666666666666, 15: 0, 16: 0, 17: 0, 18: 0, 19: 0, 20: 0}
召回率 = 
{1: 0.9961210240496509, 2: 0.9743183259353202, 3: 0.9354838709677419, 4: 0.9024979854955681, 5: 0.8764044943820225, 6: 0.7960339943342776, 7: 0.1092436974789916, 8: 0.9786535303776683, 9: 0.981081081081081, 10: 0.9538617542673652, 11: 0.1497710922171354, 12: 0.9465449804432855, 13: 0.8772378516624041, 14: 0.003838771593090211, 15: 0, 16: 0, 17: 0.0, 18: 0.0, 19: 0.0, 20: 0}
f1_score = 
{1: 0.9980567431014381, 2: 0.9766407119021133, 3: 0.9409810935104751, 4: 0.9094600081201786, 5: 0.9019022752704214, 6: 0.7867475501633223, 7: 0.19696969696969696, 8: 0.959742351046699, 9: 0.9788764044943821, 10: 0.8625353404141514, 11: 0.25572305974316023, 12: 0.924546322827125, 13: 0.9245283018867925, 14: 0.0076335877862595426, 15: 0, 16: 0, 17: 0, 18: 0, 19: 0, 20: 0}
总分数为
33.780953100665556

1.在第一个epoch训练数据的过程中，之前做实验的时候有出现过第一次的训练效果很好，但是后续数据不收敛的情况，所以第一个epoch的训练效果与最终模型能够跑出来的效果并无必然的关系
而且这里我发现bert后面的网络层加的越多刚开始的效果越差，bert后面的网络层加的越少则刚开始的效果越好，这里的原因大概与bert的参数是之前预训练中上层都调好的对应参数，而加入的网络层越多则自己初始化的参数越多。
2.仔细观察下面训练过程中的一组数据
可以看出，这里的sparse_accuracy的数值上升的并不明显，由原先的0.9705上升到现在的0.9732，但是最终验证集合的总分数上升的却特别的明显，由原先的44.2861909256549上升到现在的47.284972605746496。
而后续的得分sparse_accuracy不断的上升过程之中，验证集合的分数仍然在有序地上升



因此训练的时候应该多跑几个epoch，有可能在sparse_accuracy上升不明显的时候，验证集合的分数有所上升。