摘要
通过分类mnist 0和2的实验证明神经网络的卷积核数量是有极大值的,超过极大值网络性能会下降。并且证实就是单核的网路的性能也比无核的网络性能要好。
实验过程
将mnist的图片变成9*9,先后分别用无卷积核网络和有1-12个卷积核的网络进行分类,停止收敛的标准是网络的输出函数与目标函数的差值小于δ,让δ等于1e-4到1e-6的20个值,每个δ统计199次,取平均值
得到数据
无核 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | |
平均准确率p-ave | 平均准确率p-ave | 平均准确率p-ave | 平均准确率p-ave | 平均准确率p-ave | 平均准确率p-ave | 平均准确率p-ave | 平均准确率p-ave | 平均准确率p-ave | 平均准确率p-ave | 平均准确率p-ave | 平均准确率p-ave | 平均准确率p-ave | |
1.00E-04 | 0.980931 | 0.9719622 | 0.9754188 | 0.9794774 | 0.9825319 | 0.9823871 | 0.9832263 | 0.9827917 | 0.9827617 | 0.9826443 | 0.9815729 | 0.9808411 | 0.9800618 |
9.00E-05 | 0.980409 | 0.9742924 | 0.9773045 | 0.9802542 | 0.9827318 | 0.982959 | 0.9828991 | 0.982462 | 0.9825045 | 0.9824645 | 0.9812732 | 0.9808536 | 0.9814555 |
8.00E-05 | 0.9805963 | 0.9734782 | 0.9782886 | 0.9811133 | 0.9827942 | 0.9827492 | 0.9822947 | 0.9828292 | 0.9817877 | 0.9818226 | 0.9807362 | 0.9808611 | 0.9805838 |
7.00E-05 | 0.9805189 | 0.9748269 | 0.9800194 | 0.9823421 | 0.9838731 | 0.9836184 | 0.9832512 | 0.9832762 | 0.98192 | 0.9825769 | 0.9813156 | 0.9803566 | 0.9805514 |
6.00E-05 | 0.9813631 | 0.9759159 | 0.9813256 | 0.9836334 | 0.9836509 | 0.9832887 | 0.983506 | 0.9827442 | 0.9826943 | 0.9818851 | 0.9809834 | 0.9814005 | 0.9821398 |
5.00E-05 | 0.9820649 | 0.9770622 | 0.9815554 | 0.9838731 | 0.9846124 | 0.9848622 | 0.9847923 | 0.9844576 | 0.9832487 | 0.9843552 | 0.9829865 | 0.9826793 | 0.9825569 |
4.00E-05 | 0.9827467 | 0.9787781 | 0.9825619 | 0.9839031 | 0.985022 | 0.9860685 | 0.9851694 | 0.9853767 | 0.984485 | 0.9842802 | 0.9844476 | 0.9833936 | 0.9833387 |
3.00E-05 | 0.9830564 | 0.9790029 | 0.9832737 | 0.9849821 | 0.986573 | 0.9864681 | 0.9861909 | 0.9859162 | 0.9856489 | 0.9854042 | 0.9852493 | 0.9853592 | 0.9847723 |
2.00E-05 | 0.983526 | 0.9790903 | 0.9848147 | 0.9860161 | 0.986628 | 0.9873223 | 0.9866704 | 0.9868528 | 0.9864057 | 0.9859361 | 0.9862134 | 0.9860086 | 0.9858188 |
1.00E-05 | 0.9821723 | 0.980966 | 0.9863482 | 0.9871974 | 0.988184 | 0.9875646 | 0.9877469 | 0.9880216 | 0.9876145 | 0.9869701 | 0.9874447 | 0.9866205 | 0.9870351 |
9.00E-06 | 0.9822897 | 0.9818076 | 0.9861859 | 0.9878218 | 0.9877769 | 0.9882389 | 0.988124 | 0.9884137 | 0.9875496 | 0.9869876 | 0.9871674 | 0.9869976 | 0.9870226 |
8.00E-06 | 0.9825819 | 0.9819875 | 0.986096 | 0.9873947 | 0.9883738 | 0.9886085 | 0.9884087 | 0.9877194 | 0.9878243 | 0.9875046 | 0.9879242 | 0.9867878 | 0.9869402 |
7.00E-06 | 0.9826743 | 0.9825819 | 0.9865256 | 0.9877794 | 0.9882589 | 0.9884712 | 0.9884937 | 0.9881964 | 0.9873673 | 0.9883038 | 0.9871425 | 0.987165 | 0.9872474 |
6.00E-06 | 0.9826368 | 0.9833062 | 0.9868428 | 0.9879966 | 0.9888408 | 0.9884187 | 0.9886235 | 0.9883838 | 0.9880766 | 0.9880366 | 0.9882289 | 0.9874447 | 0.9876195 |
5.00E-06 | 0.9826618 | 0.9830265 | 0.9868053 | 0.9883288 | 0.9883363 | 0.988144 | 0.9884462 | 0.9883188 | 0.9880041 | 0.987672 | 0.9875121 | 0.9874197 | 0.9877219 |
4.00E-06 | 0.9829016 | 0.9828242 | 0.98713 | 0.9884812 | 0.9894627 | 0.9890406 | 0.9889532 | 0.9882514 | 0.9885436 | 0.9883063 | 0.9882114 | 0.9879542 | 0.987587 |
3.00E-06 | 0.9834086 | 0.9839581 | 0.9874047 | 0.9888308 | 0.9893328 | 0.9890931 | 0.9892205 | 0.9887884 | 0.9885636 | 0.9882239 | 0.9882389 | 0.988124 | 0.9879692 |
2.00E-06 | 0.9839805 | 0.9847773 | 0.9881865 | 0.9889807 | 0.9895476 | 0.9900621 | 0.9892329 | 0.9895701 | 0.9892429 | 0.9891955 | 0.9887634 | 0.988094 | 0.988159 |
1.00E-06 | 0.9850945 | 0.9854841 | 0.9889957 | 0.9897649 | 0.9904767 | 0.9902844 | 0.9903394 | 0.9899123 | 0.9899897 | 0.9900821 | 0.9896076 | 0.9895801 | 0.9893054 |
这个数据很明显,这个网络的卷积核的极大值就是4个,超过4个以后网络的性能就是下降的。
与前面的实验相比改进了卷积核的写法后就是单卷积核的网络的性能也要好于无核的网络。
1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | |
δ | 最大值p-max | 最大值p-max | 最大值p-max | 最大值p-max | 最大值p-max | 最大值p-max | 最大值p-max | 最大值p-max | 最大值p-max | 最大值p-max | 最大值p-max | 最大值p-max |
1.00E-04 | 0.9870775 | 0.9900596 | 0.9900596 | 0.9900596 | 0.9895626 | 0.9890656 | 0.9905567 | 0.9900596 | 0.9905567 | 0.9905567 | 0.9905567 | 0.9895626 |
9.00E-05 | 0.9885686 | 0.9900596 | 0.9900596 | 0.9900596 | 0.9890656 | 0.9895626 | 0.9900596 | 0.9900596 | 0.9905567 | 0.9900596 | 0.9895626 | 0.9905567 |
8.00E-05 | 0.9870775 | 0.9895626 | 0.9895626 | 0.9900596 | 0.9895626 | 0.9900596 | 0.9900596 | 0.9895626 | 0.9905567 | 0.9905567 | 0.9900596 | 0.9905567 |
7.00E-05 | 0.9865805 | 0.9920477 | 0.9920477 | 0.9895626 | 0.9905567 | 0.9905567 | 0.9910537 | 0.9905567 | 0.9910537 | 0.9900596 | 0.9905567 | 0.9910537 |
6.00E-05 | 0.9875746 | 0.9905567 | 0.9905567 | 0.9905567 | 0.9905567 | 0.9905567 | 0.9910537 | 0.9910537 | 0.9910537 | 0.9905567 | 0.9925447 | 0.9920477 |
5.00E-05 | 0.9890656 | 0.9900596 | 0.9900596 | 0.9910537 | 0.9915507 | 0.9905567 | 0.9905567 | 0.9905567 | 0.9915507 | 0.9920477 | 0.9910537 | 0.9915507 |
4.00E-05 | 0.9870775 | 0.9910537 | 0.9910537 | 0.9905567 | 0.9915507 | 0.9905567 | 0.9910537 | 0.9910537 | 0.9910537 | 0.9920477 | 0.9900596 | 0.9915507 |
3.00E-05 | 0.9890656 | 0.9905567 | 0.9905567 | 0.9920477 | 0.9915507 | 0.9925447 | 0.9920477 | 0.9920477 | 0.9920477 | 0.9915507 | 0.9925447 | 0.9910537 |
2.00E-05 | 0.9890656 | 0.9915507 | 0.9915507 | 0.9925447 | 0.9925447 | 0.9930418 | 0.9930418 | 0.9925447 | 0.9925447 | 0.9920477 | 0.9920477 | 0.9925447 |
1.00E-05 | 0.9890656 | 0.9925447 | 0.9925447 | 0.9935388 | 0.9935388 | 0.9945328 | 0.9930418 | 0.9930418 | 0.9935388 | 0.9930418 | 0.9930418 | 0.9925447 |
9.00E-06 | 0.9905567 | 0.9930418 | 0.9930418 | 0.9945328 | 0.9930418 | 0.9945328 | 0.9935388 | 0.9940358 | 0.9935388 | 0.9935388 | 0.9935388 | 0.9930418 |
8.00E-06 | 0.9895626 | 0.9945328 | 0.9945328 | 0.9935388 | 0.9940358 | 0.9945328 | 0.9935388 | 0.9940358 | 0.9935388 | 0.9940358 | 0.9935388 | 0.9935388 |
7.00E-06 | 0.9905567 | 0.9930418 | 0.9930418 | 0.9935388 | 0.9935388 | 0.9945328 | 0.9940358 | 0.9930418 | 0.9935388 | 0.9940358 | 0.9925447 | 0.9930418 |
6.00E-06 | 0.9895626 | 0.9940358 | 0.9940358 | 0.9945328 | 0.9940358 | 0.9935388 | 0.9940358 | 0.9940358 | 0.9940358 | 0.9950298 | 0.9940358 | 0.9935388 |
5.00E-06 | 0.9895626 | 0.9935388 | 0.9935388 | 0.9950298 | 0.9945328 | 0.9940358 | 0.9955268 | 0.9940358 | 0.9940358 | 0.9940358 | 0.9955268 | 0.9940358 |
4.00E-06 | 0.9895626 | 0.9935388 | 0.9935388 | 0.9945328 | 0.9945328 | 0.9950298 | 0.9945328 | 0.9950298 | 0.9945328 | 0.9945328 | 0.9940358 | 0.9945328 |
3.00E-06 | 0.9905567 | 0.9945328 | 0.9945328 | 0.9940358 | 0.9940358 | 0.9955268 | 0.9945328 | 0.9945328 | 0.9940358 | 0.9945328 | 0.9945328 | 0.9945328 |
2.00E-06 | 0.9920477 | 0.9950298 | 0.9950298 | 0.9950298 | 0.9950298 | 0.9960239 | 0.9950298 | 0.9950298 | 0.9955268 | 0.9950298 | 0.9940358 | 0.9945328 |
1.00E-06 | 0.9920477 | 0.9945328 | 0.9945328 | 0.9950298 | 0.9950298 | 0.9955268 | 0.9960239 | 0.9945328 | 0.9950298 | 0.9955268 | 0.9960239 | 0.9950298 |
虽然随着核数量的增加网络性能下降但是最大值差异不大,但与无核的网络相比至少要大0.002,这个对比很显著。
无核 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | |
δ | 耗时 min/199 | 耗时 min/199 | 耗时 min/199 | 耗时 min/199 | 耗时 min/199 | 耗时 min/199 | 耗时 min/199 | 耗时 min/199 | 耗时 min/199 | 耗时 min/199 | 耗时 min/199 | 耗时 min/199 | 耗时 min/199 |
1.00E-04 | 0.2510167 | 1.2715 | 2.8217333 | 5.1089833 | 8.2244 | 11.63025 | 14.325867 | 18.464317 | 19.568783 | 24.883217 | 27.894917 | 30.5407 | 35.482333 |
9.00E-05 | 0.2691833 | 1.4081167 | 3.0315333 | 5.4479167 | 8.46165 | 11.841017 | 15.08195 | 18.796167 | 22.856683 | 26.815167 | 28.957083 | 32.827283 | 34.380633 |
8.00E-05 | 0.2690667 | 1.5663833 | 3.3068833 | 6.3013667 | 7.6072833 | 9.0804333 | 16.304467 | 20.220283 | 21.989017 | 25.436333 | 30.08465 | 32.092583 | 37.09115 |
7.00E-05 | 0.2797 | 1.6771667 | 3.6570833 | 6.3787167 | 10.332117 | 14.287483 | 17.20495 | 19.265317 | 24.540617 | 28.654333 | 31.4023 | 35.701783 | 38.670483 |
6.00E-05 | 0.29395 | 1.7820667 | 4.0756833 | 5.4802333 | 11.6074 | 15.76265 | 17.986717 | 21.974783 | 26.9867 | 29.22455 | 33.383133 | 37.477267 | 41.156033 |
5.00E-05 | 0.3057167 | 2.1413833 | 4.4190667 | 8.1032667 | 12.5821 | 16.450767 | 19.949167 | 22.1754 | 28.092133 | 32.563083 | 36.113567 | 38.653917 | 44.88355 |
4.00E-05 | 0.32555 | 2.39175 | 5.1381667 | 9.29685 | 13.4934 | 15.459617 | 22.3508 | 26.857917 | 31.82875 | 34.540633 | 38.003217 | 43.7331 | 46.465733 |
3.00E-05 | 0.3573333 | 2.8671 | 6.21925 | 10.574367 | 13.08 | 19.680767 | 25.4449 | 29.330183 | 34.697517 | 40.599 | 44.00425 | 48.46525 | 52.224633 |
2.00E-05 | 0.4487167 | 3.5389 | 7.6344833 | 12.621517 | 18.1334 | 23.220783 | 25.266033 | 33.831767 | 36.608567 | 42.10865 | 46.517267 | 50.2975 | 57.184533 |
1.00E-05 | 0.5657 | 5.16495 | 10.4814 | 16.757217 | 23.107117 | 29.435483 | 33.560983 | 39.054233 | 45.175417 | 54.56055 | 57.1836 | 64.459117 | 69.426083 |
9.00E-06 | 0.5779 | 5.4613667 | 11.08785 | 14.880367 | 23.517033 | 30.675617 | 38.538583 | 42.5816 | 49.113533 | 53.532183 | 59.51315 | 64.300083 | 69.5925 |
8.00E-06 | 0.6237333 | 5.8232 | 11.272 | 18.11145 | 23.764067 | 29.274283 | 35.182233 | 42.7501 | 48.519683 | 55.278533 | 60.619967 | 66.11955 | 73.005083 |
7.00E-06 | 0.6701667 | 6.1751 | 12.540717 | 19.626467 | 23.809333 | 33.355817 | 40.047483 | 44.642233 | 52.064817 | 57.1209 | 63.674367 | 68.816567 | 74.744667 |
6.00E-06 | -1.043133 | 6.4720833 | 13.542133 | 20.0948 | 26.614533 | 31.45445 | 38.972967 | 46.176583 | 52.081467 | 60.708967 | 64.057983 | 72.303883 | 76.163333 |
5.00E-06 | 0.8278667 | 7.2051167 | 14.259883 | 21.24865 | 28.258033 | 34.109517 | 41.226333 | 48.910033 | 54.673833 | 62.315317 | 66.795883 | 75.911267 | 81.254883 |
4.00E-06 | 0.92955 | 7.6314833 | 15.4772 | 22.509267 | 30.298767 | 39.369883 | 46.338433 | 51.8897 | 59.972433 | 66.741217 | 73.211083 | 80.869683 | 85.531033 |
3.00E-06 | 1.0805333 | 6.5921667 | 15.777033 | 24.25435 | 33.881 | 41.083117 | 47.96175 | 56.240533 | 63.19005 | 69.45185 | 78.905017 | 85.909833 | 94.3877 |
2.00E-06 | 1.4429333 | 9.5656 | 15.067417 | 27.935483 | 30.0696 | 46.009817 | 53.679467 | 62.338233 | 70.84785 | 78.75425 | 86.743033 | 96.4067 | 100.94462 |
1.00E-06 | 2.15025 | 11.655817 | 22.0313 | 30.766883 | 43.671067 | 54.981717 | 63.107367 | 72.584117 | 82.2587 | 92.99845 | 100.5198 | 111.2489 | 120.38415 |
由于改变了核的写法,网络收敛的时间几乎就是线性的,12个核的收敛时间是4个核的收敛时间的2.79倍,但是性能只有4个核的99.8%。也就是浪费了2.79倍的时间和算力但是换来的却是性能下降了0.2%。
由此可见增加卷积核虽然不会导致网络性能大幅下降,但是确实可能造成大量的浪费,因此优化下卷积核的数量是很有必要的。
无核 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | |
δ | 迭代次数n | 迭代次数n | 迭代次数n | 迭代次数n | 迭代次数n | 迭代次数n | 迭代次数n | 迭代次数n | 迭代次数n | 迭代次数n | 迭代次数n | 迭代次数n | 迭代次数n |
1.00E-04 | 3165.7085 | 6594.0653 | 7748.9648 | 9529.9246 | 11176.784 | 12379.889 | 13558.055 | 14096.704 | 15073.402 | 15596.523 | 15981.08 | 16323.141 | 16675.151 |
9.00E-05 | 3407.9095 | 7377.2462 | 8445.4271 | 10215.111 | 11433.447 | 12701.347 | 14308.377 | 14948.116 | 15857.749 | 16152.794 | 16621.407 | 16887.362 | 16965.899 |
8.00E-05 | 3486.7638 | 8280.9648 | 9263.9347 | 11330.613 | 12633.432 | 13926.367 | 15185.442 | 16021.583 | 16417.08 | 16968.116 | 17287.101 | 17275.548 | 17629.211 |
7.00E-05 | 3668.6834 | 8864.809 | 10286.09 | 11740.794 | 13858.935 | 15354.01 | 16078.221 | 16613.035 | 17118.744 | 17903.965 | 18037.789 | 18508.497 | 18403.266 |
6.00E-05 | 3962.5628 | 9461.8945 | 11461.08 | 12902.628 | 15540.714 | 17113.663 | 17103.623 | 17932.744 | 18677.93 | 18283.779 | 18910.442 | 19475.588 | 19564.789 |
5.00E-05 | 4167.6784 | 11471.121 | 12547.915 | 15065.08 | 16882.035 | 17916.347 | 18959.307 | 19536.06 | 20133.709 | 20395.01 | 20835.99 | 20913.201 | 20887.457 |
4.00E-05 | 4515.8291 | 12890.02 | 14653.377 | 17104.111 | 18521.171 | 19657.643 | 20620.779 | 21938.07 | 22008.899 | 22532.583 | 22735.92 | 22696.322 | 22777.518 |
3.00E-05 | 5059.8794 | 15370.442 | 17854.764 | 19685.779 | 21551.729 | 22471.181 | 23230.905 | 24183.859 | 24314.332 | 24922.613 | 24905.533 | 25087.668 | 24971.905 |
2.00E-05 | 6524.8744 | 19073.583 | 22028.106 | 23803.889 | 25042.492 | 26591.739 | 26606.236 | 26850.07 | 27237.663 | 27379.02 | 27449.276 | 27155.905 | 27362.648 |
1.00E-05 | 8527.8995 | 28067.92 | 30412.719 | 31097.427 | 32837.94 | 32943.824 | 33148.839 | 33615.513 | 33142.141 | 33984.995 | 33088.357 | 33560.442 | 33350.975 |
9.00E-06 | 8746.4221 | 29789.422 | 32222.844 | 32834.593 | 33713.864 | 34114.477 | 34996.824 | 34830.402 | 35206.653 | 34491.02 | 34649.281 | 34490.653 | 34142.261 |
8.00E-06 | 9474.0101 | 31697.523 | 32766.905 | 33801.07 | 34210.688 | 35002.668 | 36016.985 | 35084.824 | 35570.97 | 35643.171 | 35501.302 | 35080.271 | 35375.392 |
7.00E-06 | 10301.668 | 33712.196 | 35559.377 | 36198.317 | 36519.879 | 37369.322 | 36858.357 | 37080.543 | 37402.528 | 36598.528 | 36545.251 | 36462.261 | 36376.387 |
6.00E-06 | 11850.653 | 35417.709 | 37922.764 | 38082.186 | 38643.935 | 38723.136 | 38242.618 | 38353.578 | 38801.95 | 38738.503 | 38041.749 | 37771.367 | 37248.302 |
5.00E-06 | 13068.734 | 39449.126 | 39982.452 | 40338.633 | 41573.965 | 40671.688 | 40908.286 | 41188.02 | 40236.759 | 40241.07 | 39805.392 | 39794.266 | 39332.538 |
4.00E-06 | 14670.05 | 41822.709 | 43194.121 | 42763.352 | 41775.06 | 44011.492 | 43305.236 | 43111.573 | 43195.477 | 43185.925 | 42760.296 | 42320.879 | 41888.849 |
3.00E-06 | 17112.472 | 46296.513 | 46017.899 | 46075.472 | 45625.131 | 45659.955 | 46976.673 | 46795.719 | 46156.447 | 44958.789 | 46297.935 | 45931.382 | 45347.779 |
2.00E-06 | 23285.869 | 52644.849 | 51703.975 | 52201.01 | 52207.563 | 53319.307 | 51039.03 | 51098.784 | 50992.623 | 51048.387 | 50976.905 | 50405.407 | 49902.251 |
1.00E-06 | 35222.241 | 64157.402 | 63712.854 | 61751.06 | 62164.94 | 62810.206 | 61508.296 | 60616.573 | 60869.266 | 60364.543 | 59755.307 | 59271.693 | 58164.457 |
随着核的增加迭代次数是缓慢减少的。