在前面的陆续实验中已经将二分类minst0,1到二分类minst0,9这9个实验都做完了,并得到了各自网络的迭代次数与准确率公式,可以近似的估算预期准确率的网络训练时间。实验的具体过程以minst0,9为例如下
实验用minst数据集,将28*28的图片缩小到9*9,网络用一个3*3的卷积核,网络结构是81*49*30*2,画成图
这个网络由两部分组成,左右两边分别向1,0和0,1收敛,左边输入minst的0,右边输入minst的9,让左右两个网络的权重共享,由前面的实验表明这种效果相当于将两个弹簧并联,组成一个振子力学系统。
具体进样顺序 | |||
δ=0.5 | |||
初始化权重 | |||
迭代次数 | |||
minst 0-1 | 1 | 判断是否达到收敛 | |
minst 9-1 | 2 | 判断是否达到收敛 | |
梯度下降 | |||
minst 0-2 | 3 | 判断是否达到收敛 | |
minst 9-2 | 4 | 判断是否达到收敛 | |
梯度下降 | |||
…… | |||
minst 0-4999 | 9997 | 判断是否达到收敛 | |
minst 9-4999 | 9998 | 判断是否达到收敛 | |
梯度下降 | |||
…… | |||
如果4999图片内没有达到收敛标准再次从头循环 | |||
minst 0-1 | 9999 | 判断是否达到收敛 | |
minst 9-1 | 10000 | 判断是否达到收敛 | |
梯度下降 | |||
…… | |||
每当网路达到收敛标准记录迭代次数和对应的准确率测试结果 | |||
将这一过程重复199次 | |||
δ=0.4 | |||
… | |||
δ=2e-7 |
收敛条件是
if (Math.abs(f2[0]-y[0])< δ && Math.abs(f2[1]-y[1])< δ )
这个网络简写成
S(minst0)81-(con3*3)49-30-2-(1,0)
S(minst9)81-(con3*3)49-30-2-(0,1)
w=w,w1=w1,w2=w2
进一步简写成
d2(minst0,9)81-con(3*3)49-30-2-(2*k) ,k∈(0,1)
经实验表明网络的迭代次数n和准确率都可以用
这两个公式近似。
迭代次数的表格是
01 | 02 | 03 | 04 | 05 | 06 | 07 | 08 | 09 | |
δ | 迭代次数n | 迭代次数n | 迭代次数n | 迭代次数n | 迭代次数n | 迭代次数n | 迭代次数n | 迭代次数n | 迭代次数n |
0.1 | 2081.131 | 2785.725 | 2567.6482 | 2352.869 | 3508.41206 | 2763.37186 | 2232.146 | 2617.372 | 2362.905 |
1.00E-02 | 2850.236 | 3620.905 | 3501.4322 | 3239.744 | 4482.321608 | 3516.39698 | 3104.673 | 3487.045 | 3377.824 |
1.00E-03 | 4126.91 | 4846.435 | 4664.5226 | 4525.779 | 6103.939698 | 4851.05528 | 4367.447 | 4888.638 | 4638.894 |
1.00E-04 | 5887.709 | 7709.22 | 7099.9296 | 6481.905 | 8919.698492 | 7646.13065 | 6468.623 | 6770.578 | 7127.503 |
9*1e-5 | 5996.663 | 7951.16 | 7262.7085 | 6723.286 | 9349.020101 | 7773.49749 | 6437.633 | 7345.497 | 7156.286 |
8*1e-5 | 6169.337 | 8182.51 | 7505.9246 | 6901.276 | 9539.442211 | 7928.66834 | 6564.357 | 7308.02 | 7367.859 |
7*1e-5 | 6184.608 | 8193.28 | 7687.3618 | 6983.593 | 10203.43216 | 8338.36683 | 6779.578 | 7442.362 | 7518.492 |
6*1e-5 | 6469.729 | 8780.59 | 8094.2965 | 7224.774 | 9851.552764 | 8790.82412 | 6887.774 | 7950.407 | 7833.106 |
5*1e-5 | 6686.593 | 9227.095 | 8405.3869 | 7523.724 | 10868.83417 | 8774.35176 | 7299.528 | 7781.106 | 8203.98 |
4*1e-5 | 7160.337 | 9473.415 | 8815.392 | 7910.116 | 11182.1005 | 9638.64322 | 7553.477 | 8625.352 | 8463.402 |
3*1e-5 | 7711.472 | 10478 | 9679.4221 | 8599.352 | 12931.17085 | 10600.9447 | 8293.955 | 9775.462 | 9201.839 |
2*1e-5 | 8744.005 | 12060.84 | 10461.668 | 9137.015 | 14625.32161 | 11411.0804 | 8723.643 | 11628.49 | 10848.2 |
1.00E-05 | 9885.658 | 19757.36 | 12683.543 | 11235.33 | 20225.77387 | 14872.995 | 10555.44 | 14918.06 | 13375.71 |
9*1e-6 | 9949.095 | 22245.54 | 13059.99 | 11316.15 | 21326.76884 | 15159.2814 | 10476.27 | 14988.04 | 13466.41 |
8*1e-6 | 10597.78 | 22214.93 | 13171.085 | 11563.64 | 23468.94472 | 17935.0302 | 10925.47 | 16597.2 | 14629.72 |
7*1e-6 | 10781.61 | 28045.61 | 13862.523 | 12665.93 | 24229.21608 | 18620.4975 | 11219.57 | 17736.16 | 14828.54 |
6*1e-6 | 11409.87 | 28410.34 | 15417.608 | 13010.63 | 27358.97487 | 19321.8744 | 11748.35 | 20981.81 | 15859.35 |
5*1e-6 | 11777.72 | 33681.76 | 15919.558 | 13712.61 | 31394.18593 | 21230.9698 | 12474.75 | 21156.49 | 18927.24 |
4*1e-6 | 12539.73 | 37281.58 | 18205.724 | 14354.06 | 36071.50754 | 24558.2714 | 13049.01 | 24769.43 | 19663.64 |
3*1e-6 | 13767.38 | 45173.59 | 22269.518 | 16352.39 | 43770.62814 | 31304.9246 | 14324.43 | 32129.62 | 26072.95 |
2*1e-6 | 14645.3 | 60366.62 | 31163.588 | 18902.75 | 53362.9598 | 46862.8643 | 16918.78 | 53448.89 | 34811.55 |
1.00E-06 | 18080.93 | 90392.45 | 47298.698 | 29535.1 | 76472.82915 | 100355.884 | 21313.81 | 73646.55 | 70131.85 |
9*1e-7 | 18234.14 | 99247.65 | 50701.342 | 28357.42 | 86231.84925 | 103772.698 | 21287.42 | 81385.9 | 77841.01 |
8*1e-7 | 19182.81 | 95016.96 | 50896.834 | 32744.68 | 91895.44724 | 119839 | 24145.12 | 91615.03 | 108462 |
7*1e-7 | 20378.61 | 113411.5 | 62449.558 | 35204.73 | 94373.55276 | 129092.693 | 27625.46 | 109482.4 | 123232.6 |
6*1e-7 | 20348.53 | 116304.3 | 64837.91 | 39191.89 | 101428.5829 | 127953.05 | 29357.48 | 109426.8 | 140167.9 |
5*1e-7 | 22365.02 | 129507.3 | 77875.121 | 48544.11 | 95963.76884 | 156705.337 | 40684.06 | 124867.1 | 149534.4 |
4*1e-7 | 23351.5 | 135768.1 | 88745.734 | 60192.69 | 112533.3266 | 161217.764 | 40085.21 | 137533.3 | 164962.1 |
3*1e-7 | 27243.87 | 149701.4 | 114492.68 | 69731.63 | 120549.2513 | 205342.492 | 62320.85 | 159985.3 | 249506.1 |
2*1e-7 | 34178.87 | 155856.8 | 141850.72 | 99327.43 | 135646.7538 | 256312.372 | 74617.93 | 187551.5 | 289655.7 |
1.00E-07 | 38643.19 | 207402.7 | 1.82E+05 | 155931.3 | 159863.3467 | 318339.688 | 133071.7 | ||
将迭代次数画成图
对应同一个δ迭代次数n由少到多的顺序是1<7<4<3<5<2<8<6<9
或许可以理解成从造型上0和1的差别最大,0和9造型上差别最小。
2和8,6和9严重缠绕
3和5轻微缠绕
表明形态上2与8,6与9的外形的相似程度要大于3和5
对应这两个公式的系数表格
a | b | c | d | |
0-1 | 619.83644 | -0.242 | 0.99638 | 0.001 |
0-2 | 44.40443 | -0.54071 | 0.95803 | 0.01119 |
0-3 | 4.35078 | -0.67353 | 0.97375 | 0.00612 |
0-4 | 0.8431 | -0.755 | 0.95801 | 0.0135 |
0-5 | 440.68687 | -0.37058 | 0.93024 | 0.02092 |
0-6 | 77.8576 | -0.52061 | 0.95824 | 0.01032 |
0-7 | 2.55183 | -0.66474 | 0.97483 | 0.00749 |
0-8 | 14.32362 | -0.61749 | 0.93612 | 0.02125 |
0-9 | 1.7063 | -0.77752 | 0.9557 | 0.01331 |
用公式计算n
01 | 02 | 03 | 04 | 05 | 06 | 07 | 08 | 09 | |
δ | 计算n | 计算n | 计算n | 计算n | 计算n | 计算n | 计算n | 计算n | 计算n |
0.1 | 1082.124 | 154.2184378 | 20.51620942 | 4.795999 | 1034.451 | 258.1731 | 11.79212 | 59.36668 | 10.22293 |
1.00E-02 | 1889.196 | 535.6070682 | 96.74468691 | 27.28218 | 2428.232 | 856.0931 | 54.49187 | 246.0553 | 61.24849 |
1.00E-03 | 3298.201 | 1860.185692 | 456.2019355 | 155.1955 | 5699.938 | 2838.775 | 251.8093 | 1019.818 | 366.9572 |
1.00E-04 | 5758.072 | 6460.502511 | 2151.231376 | 882.8341 | 13379.82 | 9413.281 | 1163.622 | 4226.811 | 2198.545 |
0.00009 | 5906.775 | 6839.239814 | 2309.437248 | 955.9298 | 13912.56 | 9944.039 | 1248.04 | 4510.946 | 2386.232 |
0.00008 | 6077.561 | 7288.975874 | 2500.108891 | 1044.831 | 14533.26 | 10572.88 | 1349.683 | 4851.251 | 2615.079 |
0.00007 | 6277.163 | 7834.717443 | 2735.383611 | 1155.66 | 15270.51 | 11334.03 | 1474.963 | 5268.212 | 2901.181 |
0.00006 | 6515.751 | 8515.734976 | 3034.652115 | 1298.3 | 16168.24 | 12281.11 | 1634.118 | 5794.316 | 3270.598 |
0.00005 | 6809.675 | 9398.017049 | 3431.1512 | 1489.899 | 17298.39 | 13503.93 | 1844.669 | 6484.788 | 3768.706 |
0.00004 | 7187.512 | 10603.18751 | 3987.600644 | 1763.29 | 18789.65 | 15167.45 | 2139.629 | 7442.807 | 4482.721 |
0.00003 | 7705.73 | 12387.73955 | 4840.181471 | 2191.051 | 20903.48 | 17618.01 | 2590.539 | 8889.661 | 5606.4 |
0.00002 | 8500.175 | 15424.33221 | 6360.106537 | 2975.784 | 24292.62 | 21758.64 | 3391.916 | 11418.78 | 7684.198 |
1.00E-05 | 10052.57 | 22437.59474 | 10144.18413 | 5022.028 | 31407.26 | 31214.11 | 5377.145 | 17518.74 | 13172.11 |
0.000009 | 10312.18 | 23752.96519 | 10890.20778 | 5437.835 | 32657.8 | 32974.09 | 5767.247 | 18696.38 | 14296.6 |
0.000008 | 10610.34 | 25314.91729 | 11789.32457 | 5943.553 | 34114.82 | 35059.31 | 6236.942 | 20106.84 | 15667.69 |
0.000007 | 10958.81 | 27210.30052 | 12898.76827 | 6574.006 | 35845.42 | 37583.27 | 6815.87 | 21835 | 17381.8 |
0.000006 | 11375.34 | 29575.50283 | 14309.97621 | 7385.415 | 37952.71 | 40723.74 | 7551.33 | 24015.53 | 19595.09 |
0.000005 | 11888.48 | 32639.70527 | 16179.67734 | 8475.332 | 40605.59 | 44778.57 | 8524.294 | 26877.31 | 22579.39 |
0.000004 | 12548.12 | 36825.31256 | 18803.62829 | 10030.53 | 44106.11 | 50294.74 | 9887.315 | 30847.98 | 26857.26 |
0.000003 | 13452.83 | 43023.13626 | 22823.99402 | 12463.86 | 49068.02 | 58420.72 | 11970.99 | 36844.71 | 33589.54 |
0.000002 | 14839.79 | 53569.34924 | 29991.23781 | 16927.84 | 57023.57 | 72150.91 | 15674.19 | 47327.09 | 46038.22 |
1.00E-06 | 17550 | 77926.70255 | 47835.14816 | 28567.95 | 73724.21 | 103504.9 | 24848.02 | 72609.39 | 78917.88 |
0.0000009 | 18003.23 | 82495.03902 | 51353.04095 | 30933.28 | 76659.67 | 109340.9 | 26650.7 | 77490.34 | 85655.01 |
0.0000008 | 18523.77 | 87919.76381 | 55592.84815 | 33810.08 | 80079.81 | 116255.5 | 28821.18 | 83336.21 | 93869.58 |
0.0000007 | 19132.13 | 94502.50887 | 60824.45701 | 37396.42 | 84142.16 | 124624.8 | 31496.43 | 90498.88 | 104139.3 |
0.0000006 | 19859.33 | 102716.9551 | 67479.04255 | 42012.15 | 89088.74 | 135038.5 | 34895.02 | 99536.46 | 117399.8 |
0.0000005 | 20755.17 | 113359.0581 | 76295.6639 | 48212.18 | 95316 | 148484.2 | 39391.13 | 111397.6 | 135279.6 |
0.0000004 | 21906.78 | 127895.8468 | 88668.96875 | 57058.95 | 103533 | 166775.6 | 45689.71 | 127854.7 | 160909.5 |
0.0000003 | 23486.25 | 149421.147 | 107627.1016 | 70901.03 | 115180.4 | 193721.1 | 55318.47 | 152709.2 | 201244.5 |
0.0000002 | 25907.64 | 186048.5847 | 141424.4148 | 96294.49 | 133854.9 | 239249.9 | 72431.1 | 196155.2 | 275828.1 |
1.00E-07 | 30639.18 | 270642.6889 | 225567.8102 | 162509.6 | 173057.4 | 343218.8 | 114823.8 | 300942 | 472819.5 |
实测值/计算值 | |||||||||
δ | 01 | 02 | 03 | 04 | 05 | 06 | 07 | 08 | 09 |
0.1 | 1.92319 | 18.06350161 | 125.1521755 | 490.59 | 3.391568 | 10.70356 | 189.2914 | 44.08823 | 231.1377 |
1.00E-02 | 1.508703 | 6.760375684 | 36.19250083 | 118.7494 | 1.84592 | 4.107494 | 56.97498 | 14.17179 | 55.14951 |
1.00E-03 | 1.251261 | 2.605350111 | 10.22468835 | 29.16179 | 1.070878 | 1.708855 | 17.34427 | 4.793637 | 12.64151 |
1.00E-04 | 1.022514 | 1.193284886 | 3.30040261 | 7.342155 | 0.666653 | 0.812271 | 5.559044 | 1.601817 | 3.241918 |
9*1e-5 | 1.015218 | 1.162579499 | 3.144795794 | 7.033243 | 0.671984 | 0.781724 | 5.158194 | 1.628372 | 2.99899 |
8*1e-5 | 1.015101 | 1.122587061 | 3.002239083 | 6.605159 | 0.656387 | 0.749906 | 4.863629 | 1.50642 | 2.817452 |
7*1e-5 | 0.985255 | 1.045765857 | 2.8103414 | 6.042948 | 0.668179 | 0.735693 | 4.596438 | 1.412692 | 2.591529 |
6*1e-5 | 0.992937 | 1.03110184 | 2.667289751 | 5.564797 | 0.609315 | 0.7158 | 4.214979 | 1.372104 | 2.395007 |
5*1e-5 | 0.981925 | 0.981812967 | 2.449727932 | 5.049822 | 0.628315 | 0.649763 | 3.957094 | 1.199901 | 2.176869 |
4*1e-5 | 0.996219 | 0.893449728 | 2.210700806 | 4.485998 | 0.59512 | 0.635482 | 3.530275 | 1.158884 | 1.888006 |
3*1e-5 | 1.000745 | 0.845836317 | 1.999805621 | 3.924761 | 0.618613 | 0.601711 | 3.201633 | 1.099644 | 1.64131 |
2*1e-5 | 1.028685 | 0.781935635 | 1.644888852 | 3.070456 | 0.602048 | 0.524439 | 2.571893 | 1.018365 | 1.411754 |
1.00E-05 | 0.983396 | 0.880547146 | 1.250326547 | 2.23721 | 0.643984 | 0.476483 | 1.963019 | 0.851549 | 1.015457 |
9*1e-6 | 0.964791 | 0.936537389 | 1.199241576 | 2.081002 | 0.653038 | 0.459733 | 1.816512 | 0.801655 | 0.941931 |
8*1e-6 | 0.998817 | 0.877542863 | 1.117204412 | 1.945577 | 0.68794 | 0.511563 | 1.751735 | 0.82545 | 0.933751 |
7*1e-6 | 0.98383 | 1.030698098 | 1.074716773 | 1.926669 | 0.675936 | 0.495446 | 1.646095 | 0.812281 | 0.853107 |
6*1e-6 | 1.003035 | 0.960603617 | 1.077402772 | 1.761666 | 0.72087 | 0.474462 | 1.555798 | 0.873677 | 0.809353 |
5*1e-6 | 0.990684 | 1.031925985 | 0.98392307 | 1.617944 | 0.773149 | 0.474132 | 1.463435 | 0.787151 | 0.838253 |
4*1e-6 | 0.999332 | 1.012389914 | 0.968202697 | 1.431037 | 0.817835 | 0.488287 | 1.319772 | 0.802951 | 0.732154 |
3*1e-6 | 1.023381 | 1.049983542 | 0.975706424 | 1.311985 | 0.89204 | 0.535853 | 1.196595 | 0.872028 | 0.776222 |
2*1e-6 | 0.986894 | 1.126887219 | 1.039089755 | 1.116667 | 0.935805 | 0.649512 | 1.079404 | 1.129351 | 0.756144 |
1.00E-06 | 1.030253 | 1.159967534 | 0.98878545 | 1.033854 | 1.037282 | 0.969576 | 0.857767 | 1.014284 | 0.888669 |
9*1e-7 | 1.012826 | 1.203074102 | 0.987309432 | 0.916729 | 1.124866 | 0.949074 | 0.798757 | 1.050272 | 0.908774 |
8*1e-7 | 1.035578 | 1.080723502 | 0.915528451 | 0.968489 | 1.147548 | 1.030825 | 0.837756 | 1.099342 | 1.155455 |
7*1e-7 | 1.065151 | 1.200090255 | 1.026717884 | 0.941393 | 1.121596 | 1.035851 | 0.877098 | 1.209765 | 1.183343 |
6*1e-7 | 1.024633 | 1.132279377 | 0.960859951 | 0.93287 | 1.138512 | 0.94753 | 0.841308 | 1.099364 | 1.193937 |
5*1e-7 | 1.077564 | 1.142451933 | 1.020701789 | 1.006885 | 1.006796 | 1.055367 | 1.032823 | 1.120913 | 1.105373 |
4*1e-7 | 1.065948 | 1.061551672 | 1.000865747 | 1.054921 | 1.086932 | 0.966675 | 0.877336 | 1.0757 | 1.025186 |
3*1e-7 | 1.159992 | 1.00187529 | 1.063790456 | 0.983507 | 1.046613 | 1.05999 | 1.126583 | 1.047647 | 1.239815 |
2*1e-7 | 1.319258 | 0.837720831 | 1.003014393 | 1.031496 | 1.013386 | 1.071316 | 1.030192 | 0.956138 | 1.050131 |
1.00E-07 | 1.261235 | 0.766333873 | 0.808921687 | 0.95952 | 0.923759 | 0.927513 | 1.158921 | 0 | 0 |
可以看到这组表达式在δ∈[1e-7,1e-4]的区间上是相对精确的
δ越小越准确,当δ<1e-5时实测值/计算值<2。
将计算的n画成图
在这图里2与8,6与9缠绕,3与5交叉都有反应。
计算p-max
计算p-max | ||||||||||
δ | -lnδ | 01 | 02 | 03 | 04 | 05 | 06 | 07 | 08 | 09 |
0.1 | 2.302585 | 0.9972114 | 0.967013 | 0.978732998 | 0.968857609 | 0.946613 | 0.966523 | 0.980939 | 0.952859 | 0.966368 |
1.00E-02 | 4.60517 | 0.9979028 | 0.974543 | 0.982893661 | 0.977966231 | 0.96044 | 0.973462 | 0.986045 | 0.966998 | 0.975325 |
1.00E-03 | 6.907755 | 0.9983075 | 0.978974 | 0.985335688 | 0.98333408 | 0.968621 | 0.977544 | 0.989044 | 0.975366 | 0.980603 |
1.00E-04 | 9.21034 | 0.9985947 | 0.982131 | 0.987072012 | 0.987160488 | 0.974468 | 0.98045 | 0.991177 | 0.981347 | 0.984365 |
0.00009 | 9.315701 | 0.9986061 | 0.982256 | 0.987140726 | 0.987312083 | 0.9747 | 0.980565 | 0.991262 | 0.981584 | 0.984514 |
0.00008 | 9.433484 | 0.9986187 | 0.982394 | 0.987216633 | 0.987479562 | 0.974956 | 0.980693 | 0.991355 | 0.981846 | 0.984679 |
0.00007 | 9.567015 | 0.9986327 | 0.982549 | 0.987301559 | 0.987666958 | 0.975243 | 0.980835 | 0.991459 | 0.982139 | 0.984863 |
0.00006 | 9.721166 | 0.9986487 | 0.982724 | 0.987398145 | 0.987880107 | 0.975569 | 0.980997 | 0.991578 | 0.982473 | 0.985072 |
0.00005 | 9.903488 | 0.9986672 | 0.982929 | 0.987510437 | 0.988127947 | 0.975948 | 0.981185 | 0.991716 | 0.982861 | 0.985316 |
0.00004 | 10.12663 | 0.9986895 | 0.983174 | 0.987645107 | 0.988425224 | 0.976404 | 0.98141 | 0.991882 | 0.983326 | 0.985608 |
0.00003 | 10.41431 | 0.9987174 | 0.983482 | 0.987814439 | 0.988799085 | 0.976976 | 0.981694 | 0.99209 | 0.983912 | 0.985976 |
0.00002 | 10.81978 | 0.9987556 | 0.983902 | 0.988045369 | 0.989309069 | 0.977757 | 0.982081 | 0.992374 | 0.984711 | 0.986477 |
1.00E-05 | 11.51293 | 0.9988176 | 0.984586 | 0.988420916 | 0.990138732 | 0.979028 | 0.982711 | 0.992835 | 0.986011 | 0.987293 |
0.000009 | 11.61829 | 0.9988267 | 0.984687 | 0.988476025 | 0.99026051 | 0.979214 | 0.982803 | 0.992903 | 0.986202 | 0.987413 |
0.000008 | 11.73607 | 0.9988368 | 0.984798 | 0.988537046 | 0.990395363 | 0.979421 | 0.982906 | 0.992978 | 0.986413 | 0.987545 |
0.000007 | 11.8696 | 0.9988481 | 0.984922 | 0.988605494 | 0.990546641 | 0.979653 | 0.98302 | 0.993062 | 0.98665 | 0.987694 |
0.000006 | 12.02375 | 0.998861 | 0.985065 | 0.988683566 | 0.990719206 | 0.979917 | 0.983151 | 0.993158 | 0.986921 | 0.987863 |
0.000005 | 12.20607 | 0.998876 | 0.985231 | 0.988774632 | 0.990920511 | 0.980226 | 0.983304 | 0.99327 | 0.987237 | 0.988061 |
0.000004 | 12.42922 | 0.9988941 | 0.98543 | 0.988884265 | 0.991162889 | 0.980597 | 0.983488 | 0.993405 | 0.987617 | 0.9883 |
0.000003 | 12.7169 | 0.998917 | 0.985683 | 0.989022755 | 0.991469111 | 0.981067 | 0.98372 | 0.993575 | 0.988097 | 0.988601 |
0.000002 | 13.12236 | 0.9989483 | 0.986029 | 0.989212748 | 0.9918893 | 0.981711 | 0.984039 | 0.993809 | 0.988756 | 0.989014 |
1.00E-06 | 13.81551 | 0.9989997 | 0.986597 | 0.98952442 | 0.992578802 | 0.982769 | 0.984562 | 0.994192 | 0.989838 | 0.989692 |
9E-07 | 13.92087 | 0.9990073 | 0.986681 | 0.989570429 | 0.99268061 | 0.982925 | 0.984639 | 0.994249 | 0.989998 | 0.989792 |
8E-07 | 14.03865 | 0.9990157 | 0.986774 | 0.989621456 | 0.992793526 | 0.983099 | 0.984724 | 0.994311 | 0.990176 | 0.989903 |
7E-07 | 14.17219 | 0.9990252 | 0.986878 | 0.989678793 | 0.992920414 | 0.983293 | 0.984821 | 0.994382 | 0.990375 | 0.990027 |
6E-07 | 14.32634 | 0.999036 | 0.986998 | 0.989744319 | 0.993065437 | 0.983516 | 0.984931 | 0.994462 | 0.990602 | 0.99017 |
5E-07 | 14.50866 | 0.9990486 | 0.987138 | 0.989820922 | 0.993234989 | 0.983776 | 0.985059 | 0.994557 | 0.990869 | 0.990337 |
4E-07 | 14.7318 | 0.9990639 | 0.987306 | 0.989913385 | 0.993439666 | 0.98409 | 0.985214 | 0.99467 | 0.99119 | 0.990538 |
3E-07 | 15.01948 | 0.9990832 | 0.98752 | 0.990030557 | 0.993699073 | 0.984488 | 0.985411 | 0.994814 | 0.991598 | 0.990793 |
2E-07 | 15.42495 | 0.9991098 | 0.987814 | 0.990191969 | 0.994056484 | 0.985037 | 0.985682 | 0.995013 | 0.992159 | 0.991144 |
1.00E-07 | 16.1181 | 0.9991537 | 0.9883 | 0.99045838 | 0.994646543 | 0.985943 | 0.986129 | 0.99534 | 0.993086 | 0.991724 |
画成图
按δ=1e-7时的p-max大小排列
1>7>4>8>9>3>2>6>5
和迭代次数n顺序比较
1<7<4<3<5<2<8<6<9
1,7,4,3,5的顺序基本是规律的,2,8,6,9的相对顺序不规则。
迭代次数和识别难度的排序不一致的可能原因是与各个数据集本身的难度不同有关。
最后比较让网络的准确率p-max=0.999的计算耗时
计算δ | 计算n | 耗时min/199 | 耗时 天/199 | 耗时 年/199 | |
0-1 | 1.02E-06 | 1.75E+04 | 2.64E+01 | 0.018316985 | |
0-2 | 4.74E-19 | 3.59E+11 | 4.16E+08 | 288722.7808 | 791.0213174 |
0-3 | 3.74E-29 | 6.10E+19 | 6.15E+16 | 4.27E+13 | 1.17E+11 |
0-4 | 2.09E-10 | 17146949 | 16793.50574 | 11.66215676 | |
0-5 | 7.66E-14 | 31955366.8 | 31534.0064 | 21.89861559 | |
0-6 | 2.89E-25 | 4.64E+14 | 5.12E+11 | 355881000.7 | 975016.4404 |
0-7 | 3.90E-12 | 97869735 | 94623.68699 | 65.71089375 | |
0-8 | 5.50E-10 | 7479334.5 | 8967.287082 | 6.227282696 | |
0-9 | 7.67E-13 | 4484458556 | 5164961.655 | 3586.778927 | 9.826791581 |
意思是比如二分类0,1对应的网络
d2(minst0,1)81-con(3*3)49-30-2-(2*k) ,k∈(0,1)让这个网络的准确率等于0.999可以让收敛标准δ=1.75E+04可以在26.4min里收敛199次其中至少有一次的准确率可以达到0.999.或者让d2(minst0,1)81-con(3*3)49-30-2-(2*k) ,k∈(0,1)
的收敛标准δ=1.75E+04,准确率等于0.999的概率是5.025‰,估计耗时0.13min。
预期时间最长的二分类0,3的网络
d2(minst0,3)81-con(3*3)49-30-2-(2*k) ,k∈(0,1)
让这个网络的收敛标准δ= 3.74E-29收敛199次预期需要1170亿年其中至少有1次可以达到0.999,或者让δ= 3.74E-29收敛准确率等于0.999的概率为5.025‰,需要5.88亿年。
按照预期时间排序
1<8<4<5<7<9<2<6<3
按δ=1e-7时的p-max大小排列
1>7>4>8>9>3>2>6>5
迭代次数n顺序比较
1<7<4<3<5<2<8<6<9
对比表明迭代次数n大并不必然的导致更难分类。
关于调参
r学习率,
x权重分母,(0-1的随机数)/x
n迭代次数
p-ave 网络收敛199次的准确率的平均值
p-max网络收敛199次的准确率的最大值
δ | r | x | n | p-ave | p-max |
减小 | 不变 | 不变 | 增加 | 增加 | 增加 |
减小 | 不变 | 减小 | 增加 | 增加 | 增加 |
减小 | 减小 | 不变 | 增加 | 增加 | 增加 |
不变 | 减小 | 减小 | 增加 | 增加 | 小幅增加,几乎是定值 |
不变 | 不变 | 减小 | 增加 | 增加 | 几乎是定值 |
不变 | 减小 | 不变 | 增加 | 增加 | 几乎是定值 |
减小 | 减小 | 减小 | 增加 | 增加 | 增加 |
- δ,r或者x的减小都会使网络的平均性能p-ave和最大性能p-max增加
- 当δ保持不变并且非常小时,虽然r或者x减小同样会使最大性能p-max增加但是幅度相当小,几乎可以认为当δ保持不变并且非常小时最大性能p-max是定值
- 在网络的平均性能p-ave和收敛时间之间存在一种平衡,若想网络平均性能p-ave稳定就要增加收敛时间,或者希望网络收敛加快就会导致网络平均性能p-ave不稳定。
比如这个网络
d2(minst0,3)81-con(3*3)49-30-2-(2*k) ,k∈(0,1)
先让学习率r=1e-3,再次让学习率r=0.1,会导致网络看起来收敛速度快好多,但性能时好时坏。
因为r增大会导致迭代次数n减小,使得网络在更短的时间里可能达到更小的δ,而δ减小导致准确率增加;
r增大同时会导致平均性能下降,使网络性能不稳定。
实验数据 |
学习率 0.1 |
权重初始化方式 |
Random rand1 =new Random(); |
int ti1=rand1.nextInt(98)+1; |
int xx=1; |
if(ti1%2==0) |
{ xx=-1;} |
tw[a][b]=xx*((double)ti1/x); |
第一层第二层和卷积核的权重的初始化的x分别为1000,1000,200 |
http://www.qinms.com/webapp/curvefit/cf.aspx
具体数据位置
0-1 | 1个卷积核二分类0,1的特征频率曲线 | 2019/1/20 |
0-2 | 神经网络收敛标准与准确率之间的数学关系 | 2018/12/29 |
0-3 | 估算带卷积核二分类0,3的网络的收敛时间和迭代次数 | 2019/1/21 |
0-4 | 二分类0,4神经网络的收敛时间和准确率的估算表达式 | 2019/1/24 |
0-5 | 共振耦合二分类0,5神经网络迭代次数和准确率估算表达式 | 2019/1/24 |
0-6 | 二分类minst0,6收敛时间估算表达式 | 2019/1/26 |
0-7 | 神经网络训练时间计算实例:二分类minst0,7 | 2019/1/31 |
0-8 | 神经网络收敛精度计算实例:二分类minst0,8 | 2019/2/1 |
0-9 | 计算神经网络准确率实例二分类minst0,9 | 2019/2/8 |