2*2矩阵训练集比例对BP神经网络分类性能影响

 

r1	r2
<1	<1	吸引子	c
>1	>1	排斥子	p
>1	<1	鞍点	a
<1	>1	反鞍点	fa

构造一个网络

d2（ x(a||fa)   ||   y(a||c)  ）-4-4-2-（2*k）,k∈{0,1}  ，x+y=1

进样可能是x倍的a和fa组合,或者y倍的a和c组合，网络结构是4*4*2，让a向1，0收敛，让fa或c向0，1收敛。让x+y=1。

测试集在0，1位100%的p。

测试集是固定的，训练集的比例是变动的，观察x，y对网络分类性能的影响。

 

这个网络的收敛标准是

if (Math.abs(f2[0]-y[0])< δ  &&  Math.abs(f2[1]-y[1])< δ   )

尝试了δ从0.5到1e-6在内的26个值.

具体进样顺序
进样顺序	迭代次数
δ=0.5
x(a||fa)||y(a||c)	1，2		判断是否达到收敛
梯度下降
x(a||fa)||y(a||c)	3，4		判断是否达到收敛
梯度下降
……
达到收敛标准测量准确率,记录迭代次数，将这个过程重复199次
δ=0.4
…
δ=1e-6

 

得到的数据

	δ
a	fa	1	0.9	0.8	0.7	0.6	0.5	0.4	0.3	0.2	0.1	0
a	c	0	0.1	0.2	0.3	0.4	0.5	0.6	0.7	0.8	0.9	1
	0.5	0.44588	0.45022	0.32894	0.24683	0.11291	0.11512	0.03597	0.0311	0.01417	0.01011	0
	0.4	0.52028	0.52812	0.57773	0.56226	0.57428	0.56271	0.54622	0.53804	0.48635	0.47534	0.4654
p	0.3	0.49681	0.5514	0.58685	0.62591	0.64631	0.6751	0.67359	0.68461	0.66206	0.66526	0.7091
	0.2	0.50802	0.56838	0.63054	0.65494	0.70204	0.71924	0.73754	0.74053	0.755	0.76093	0.81201
	0.1	0.50178	0.59084	0.65361	0.6975	0.74633	0.76998	0.7966	0.81175	0.81593	0.82646	0.86962
	0.01	0.49582	0.67679	0.74343	0.7851	0.82071	0.84327	0.8629	0.8741	0.88365	0.88848	0.9292
	0.001	0.49319	0.73419	0.78728	0.83752	0.85759	0.87664	0.88819	0.89449	0.89978	0.90805	0.9449
	1E-04	0.50209	0.76548	0.81411	0.84136	0.86212	0.87197	0.89272	0.89352	0.90319	0.8986	0.93876
	9E-05	0.4932	0.7649	0.8205	0.85276	0.86098	0.88058	0.88292	0.89535	0.90353	0.90084	0.93174
	8E-05	0.49773	0.77015	0.81775	0.83731	0.86305	0.87634	0.89125	0.89529	0.9075	0.90114	0.93209
	7E-05	0.49551	0.7672	0.81861	0.84337	0.8618	0.87037	0.88483	0.8963	0.90331	0.90184	0.93454
	6E-05	0.50421	0.76373	0.81248	0.84394	0.85376	0.87508	0.88851	0.89856	0.90574	0.89944	0.93109
	5E-05	0.49854	0.77061	0.80627	0.84627	0.86156	0.87611	0.88957	0.89384	0.89862	0.90686	0.93405
	0.00004	0.48881	0.77312	0.80831	0.8332	0.85962	0.87353	0.8851	0.90074	0.90584	0.91051	0.93205
	3E-05	0.50209	0.76434	0.8174	0.8452	0.86985	0.87004	0.8886	0.89036	0.89856	0.89097	0.93109
	0.00002	0.5058	0.76266	0.80944	0.8311	0.85292	0.87174	0.88378	0.88588	0.9015	0.89695	0.93145
	0.00001	0.50192	0.76407	0.80132	0.8333	0.85989	0.86978	0.88523	0.88995	0.89636	0.90115	0.93569
	9E-06	0.50408	0.76325	0.80699	0.83221	0.85096	0.87095	0.87777	0.89079	0.89506	0.89588	0.93397
	8E-06	0.50188	0.75455	0.80558	0.83504	0.855	0.86698	0.87943	0.88833	0.89732	0.89039	0.9375
	7E-06	0.49782	0.75839	0.80228	0.83528	0.85845	0.8742	0.88515	0.89104	0.89008	0.89285	0.93388
	6E-06	0.49738	0.75785	0.80408	0.82737	0.85749	0.86809	0.88058	0.88697	0.89767	0.89526	0.93438
	5E-06	0.49136	0.75934	0.79688	0.82396	0.8476	0.86955	0.88131	0.89367	0.89572	0.88839	0.94185
	4E-06	0.49217	0.74927	0.79003	0.82463	0.8469	0.86969	0.88015	0.8876	0.894	0.88987	0.943
	3E-06	0.5033	0.74714	0.80018	0.82504	0.85333	0.86856	0.87966	0.88051	0.89376	0.88503	0.94163
	2E-06	0.50593	0.7406	0.79291	0.82347	0.84607	0.86327	0.87254	0.89193	0.89195	0.88841	0.94214
	1E-06	0.49536	0.75413	0.79126	0.82011	0.85432	0.85613	0.87883	0.88007	0.89488	0.88014	0.93318
	平均值	0.49912	0.75486	0.79915	0.82804	0.853	0.86772	0.88007	0.88808	0.89468	0.89074	0.93772

 

观察准确率的图像

在δ<1e-5以后是很稳定的，将1e-6<δ<1e-5的准确率取平均值得到图像

由《B*P神经网络分类22对角矩阵准确率数据汇总》得到的表格可以查到

 

		p	p
c	p	0	1	0.258402
c	p	1	0	0.740507
a	p	0	1	0.864001
a	p	1	0	0.136366
c	a	0	1	0.063246
c	a	1	0	0.93631
a	fa	0	1	0.49912
a	fa	1	0	0.500347
p	fa	0	1	0.135043
p	fa	1	0	0.865318
c	fa	0	1	0.064735
c	fa	1	0	0.935863

 

测试集a||fa对0，1位置的p的分类准确率是0.49912，a||c对0，1位置p的分类准确率是0.93631，但是得到的并不是一条直线，表明输入对象fa||c之间存在相互作用，对结果有影响。

a	fa	a	c	pb0	1	0.93631	0.49912	0.935863			1	0.49912	0.754857	0.799151	0.82804	0.852999	0.86772	0.880065	0.888085	0.894681	0.890737	0.937723
a	fa	a	c	pb1	2	0.063246	0.500347	0.064735			2	0.50088	0.245143	0.200849	0.17196	0.147001	0.13228	0.119935	0.111915	0.105319	0.109263	0.062277
a	fa	a	c	cb0	3	0.93874	0.500403	0.938718			3	0.500403	0.717344	0.768623	0.798632	0.819687	0.835527	0.849605	0.859516	0.867684	0.869758	0.93874
a	fa	a	c	cb1	4	0.061631	0.500046	0.060715			4	0.499597	0.282656	0.231377	0.201368	0.180313	0.164473	0.150395	0.140484	0.132316	0.130242	0.06126
a	fa	a	c	ab0	5	0.198454	0.000348	0			5	0.000348	0.033511	0.052534	0.067696	0.08233	0.094839	0.106371	0.113552	0.119355	0.122333	0.198454
a	fa	a	c	ab1	6	0.800701	0.999621	1			6	0.999652	0.966489	0.947466	0.932304	0.91767	0.905161	0.893629	0.886448	0.880645	0.877667	0.801546
a	fa	a	p	cb0	7	0.624975	0.500403	0.62563			7	0.499617	0.51692	0.522218	0.54357	0.560013	0.575717	0.587874	0.599967	0.608206	0.619354	0.624975
a	fa	a	p	cb1	8	0.373758	0.500046	0.374131			8	0.500383	0.48308	0.477782	0.45643	0.439987	0.424283	0.412126	0.400033	0.391794	0.380646	0.375025
a	fa	a	p	ab0	9	0.040855	0.000348	0			9	0.000394	0.005513	0.005128	0.009861	0.014368	0.019362	0.023502	0.027676	0.032474	0.037415	0.040855
a	fa	a	p	ab1	10	0.958847	0.999621	1			10	0.999606	0.994487	0.994872	0.990139	0.985632	0.980638	0.976498	0.972324	0.967526	0.962585	0.959145
a	fa	a	p	pb0	11	0.864001	0.49912	0.865318			11	0.50239	0.62861	0.682381	0.72875	0.762817	0.790262	0.810617	0.826737	0.842293	0.853794	0.864078
a	fa	a	p	pb1	12	0.136366	0.49912	0.135043			12	0.49761	0.37139	0.317619	0.27125	0.237183	0.209738	0.189383	0.173263	0.157707	0.146206	0.135922
c	fa	c	p	cb0	13	0	0.060715	0.374131			13	0.061631	0.000169	0	0	0	0	0	0	0	0	0
c	fa	c	p	cb1	14	1	0.938718	0.374131			14	0.938369	0.999831	1	1	1	1	1	1	1	1	1
c	fa	c	p	ab0	15	0	0	0			15	0	0	0	0	0	0	0	0	0	0	0
c	fa	c	p	ab1	16	0.999991	0.999991	0.999991			16	1	1	1	1	1	1	1	1	1	1	1
p	fa	p	c	ab0	17	0.999991	0	0.999991			17	0	0	0.251756	0.589507	0.674203	0.777887	0.681938	0.584354	0.249025	0	0.999981
p	fa	p	c	ab1	18	0	1	0			18	1	1	0.748244	0.410493	0.325797	0.222113	0.318062	0.415646	0.750975	1	1.91E-05
c	fa	c	a	ab0	19	0.800701	0	0.000348			19	0		0.097396	0.173459	0.235091	0.288908	0.342857	0.400415	0.489861		0.802727
c	fa	c	a	ab1	20	0.198454	1	0.999621			20	1		0.902604	0.826541	0.764909	0.711092	0.657143	0.599585	0.510139		0.197273
c	p	c	fa	pb0	21	0.258402	0.064735	0.865318			21	0.064883	0.258733	0.405405	0.502393	0.549853	0.554114	0.537257	0.523274	0.493078	0.452059	0.262191
p	fa	p	c	pb1	22	0.258402	0.865318	0.935863			22	0.864149	0.866174	0.842718	0.839909	0.825159	0.814363	0.799725	0.767323	0.733453	0.644103	0.255243
c	p	c	fa	pb1	23	0.740507	0.935863	0.135043			23	0.935117	0.741267	0.594595	0.497607	0.450147	0.445886	0.462743	0.476726	0.506922	0.547941	0.737809
p	fa	p	c	pb0	24	0.740507	0.135043	0.064735			24	0.135851	0.133826	0.157282	0.160091	0.174841	0.185637	0.200275	0.232677	0.266547	0.355897	0.744757
p	fa	p	c	cb0	25	1	0.374131	0.938718			25	0.374689	0.461328	0.70292	0.882162	0.957317	0.973871	0.983817	0.988585	0.994648	0.998053	1
p	fa	p	c	cb1	26	0	0.62563	0.060715			26	0.625311	0.538672	0.29708	0.117838	0.042683	0.026129	0.016183	0.011415	0.005352	0.001947	0

为了解准确率的表达式计算了26组不同的组合

具体数据在《B*P神经网络分类22对角矩阵准确率数据汇总》

其中的第21组是比较有代表性的

c

p

c

fa

pb0

21

0.258402

0.064735

0.865318

测试集c||p对0，1位的p的分类准确率是0.258，c||fa对0，1位的p的分类准确率是0.0647，但得到的准确率大于0.258，而p||fa对0，1位的p的分类准确率是0.86，表明两组训练集之间是有相互作用的。

假设准确率f（a,b,c）=xa+yb+zc ，让a和b分别是两个训试集准确率，c是相互作用项的分类准确率，求x，y，z。

解矩阵Ax=B

让A=

a	b	c
0.93631	0.49912	0.935863
0.063246	0.500347	0.064735
0.93874	0.500403	0.938718
0.061631	0.500046	0.060715
0.800701	0.999621	1
0.624975	0.500403	0.62563
0.373758	0.500046	0.374131
0.958847	0.999621	1
0.864001	0.49912	0.865318
0.136366	0.49912	0.135043
0.258402	0.064735	0.865318
0.258402	0.865318	0.935863
0.740507	0.935863	0.135043
0.740507	0.135043	0.064735
1	0.374131	0.938718

 

让B分别等于下面表格的第1-9列

	1	0.9	0.8	0.7	0.6	0.5	0.4	0.3	0.2	0.1	0
	0	0.1	0.2	0.3	0.4	0.5	0.6	0.7	0.8	0.9	1
1	0.49912	0.754857	0.799151	0.82804	0.852999	0.86772	0.880065	0.888085	0.894681	0.890737	0.937723
2	0.50088	0.245143	0.200849	0.17196	0.147001	0.13228	0.119935	0.111915	0.105319	0.109263	0.062277
3	0.500403	0.717344	0.768623	0.798632	0.819687	0.835527	0.849605	0.859516	0.867684	0.869758	0.93874
4	0.499597	0.282656	0.231377	0.201368	0.180313	0.164473	0.150395	0.140484	0.132316	0.130242	0.06126
6	0.999652	0.966489	0.947466	0.932304	0.91767	0.905161	0.893629	0.886448	0.880645	0.877667	0.801546
7	0.499617	0.51692	0.522218	0.54357	0.560013	0.575717	0.587874	0.599967	0.608206	0.619354	0.624975
8	0.500383	0.48308	0.477782	0.45643	0.439987	0.424283	0.412126	0.400033	0.391794	0.380646	0.375025
10	0.999606	0.994487	0.994872	0.990139	0.985632	0.980638	0.976498	0.972324	0.967526	0.962585	0.959145
11	0.50239	0.62861	0.682381	0.72875	0.762817	0.790262	0.810617	0.826737	0.842293	0.853794	0.864078
12	0.49761	0.37139	0.317619	0.27125	0.237183	0.209738	0.189383	0.173263	0.157707	0.146206	0.135922
21	0.064883	0.258733	0.405405	0.502393	0.549853	0.554114	0.537257	0.523274	0.493078	0.452059	0.262191
22	0.864149	0.866174	0.842718	0.839909	0.825159	0.814363	0.799725	0.767323	0.733453	0.644103	0.255243
23	0.935117	0.741267	0.594595	0.497607	0.450147	0.445886	0.462743	0.476726	0.506922	0.547941	0.737809
24	0.135851	0.133826	0.157282	0.160091	0.174841	0.185637	0.200275	0.232677	0.266547	0.355897	0.744757
25	0.374689	0.461328	0.70292	0.882162	0.957317	0.973871	0.983817	0.988585	0.994648	0.998053	1

 

可以解得

x	0.08595	0.13512	0.16272	0.19476	0.22421	0.26321	0.31324	0.37161	0.48366
y	0.64044	0.44095	0.30452	0.22251	0.18642	0.16607	0.14408	0.13176	0.1124
z	0.2732	0.4277	0.54119	0.59085	0.59716	0.58003	0.55157	0.50727	0.41328

 

比如第一组数据=0.93631*0.08595+0.49912*0.64044+0.935863*0.2732=0.65581

将x，y，z画成图

可以用曲线拟合x

R	0.1	0.2	0.3	0.4	0.5	0.6	0.7	0.8	0.9
X	0.08595	0.13512	0.16272	0.19476	0.22421	0.26321	0.31324	0.37161	0.48366

 

拟合方程式：x = a + b·r + c·r^2 + d·r^3

a = 0.0191981746031519

b = 0.787666859067049

c = -1.32153823953865

d = 1.12769360269386

相关系数 R2：0.998795559102663

R	0.1	0.2	0.3	0.4	0.5	0.6	0.7	0.8	0.9
y	0.64044	0.44095	0.30452	0.22251	0.18642	0.16607	0.14408	0.13176	0.1124

 

用同样的办法拟合y

拟合方程式：Y = a + b·r+ c·r^2 + d·r^3

a = 0.913671031746098

b = -3.17943332130888

c = 4.53483189033309

d = -2.21763468013543

相关系数 R2：0.999299411271458

因此准确率可以表达成

让r=0.1，0.2…0.9

拟合得到的数据

 

1	0.65581	0.74687	0.81083	0.84637	0.86184	0.87216	0.8814	0.88844	0.89573
2	0.34356	0.25686	0.19769	0.1619	0.14611	0.13729	0.12761	0.12227	0.11358
3	0.65762	0.74898	0.81316	0.84882	0.86432	0.87467	0.88392	0.89096	0.89823
4	0.34213	0.25479	0.19516	0.15914	0.14329	0.13448	0.12484	0.11959	0.11111
6	0.98222	0.97667	0.97588	0.96922	0.96303	0.95679	0.94641	0.93653	0.9129
7	0.54512	0.57268	0.59266	0.60272	0.60701	0.61049	0.61294	0.61554	0.61708
8	0.45459	0.43101	0.41557	0.40511	0.40043	0.39843	0.39548	0.39456	0.3916
10	0.99581	0.99804	1.00162	1.00002	0.99849	0.99842	0.99594	0.9953	0.98939
11	0.63032	0.70693	0.76088	0.7906	0.8035	0.81221	0.81984	0.82578	0.8316
12	0.36827	0.29627	0.24727	0.21741	0.20426	0.19711	0.18911	0.18494	0.17787
21	0.30007	0.43356	0.53006	0.576	0.58674	0.58067	0.56755	0.5435	0.48987
22	0.83207	0.81675	0.81203	0.79582	0.77811	0.75455	0.72181	0.68477	0.60901
23	0.6999	0.57048	0.47857	0.43225	0.42113	0.42866	0.44128	0.46699	0.51916
24	0.16782	0.18729	0.19665	0.21252	0.22986	0.25488	0.28712	0.32581	0.40009
25	0.58202	0.70158	0.78468	0.83265	0.85452	0.86983	0.88491	0.89709	0.91367

 

比较实测数据和拟合数据

实测	1	0.75486	0.79915	0.82804	0.853	0.86772	0.88007	0.88808	0.89468	0.89074
拟合	1	0.65581	0.74687	0.81083	0.84637	0.86184	0.87216	0.8814	0.88844	0.89573
实测	2	0.24514	0.20085	0.17196	0.147	0.13228	0.11993	0.11192	0.10532	0.10926
拟合	2	0.34356	0.25686	0.19769	0.1619	0.14611	0.13729	0.12761	0.12227	0.11358

 

用实测数据/拟合数据

1	1.15103	1.07	1.02123	1.00783	1.00683	1.00906	1.00759	1.00702	0.99443
2	0.71353	0.78194	0.86984	0.90798	0.90533	0.8736	0.87703	0.86139	0.96197
3	1.09082	1.02622	0.98214	0.96568	0.96668	0.97134	0.97239	0.97387	0.9683
4	0.82616	0.90811	1.0318	1.13303	1.1478	1.11834	1.1253	1.10643	1.17223
6	0.98399	0.97009	0.95534	0.94681	0.9399	0.93399	0.93665	0.94033	0.9614
7	0.94827	0.91188	0.91716	0.92915	0.94845	0.96296	0.97883	0.98808	1.00368
8	1.06268	1.10851	1.09833	1.08608	1.05955	1.03438	1.01151	0.99298	0.97203
10	0.99867	0.99682	0.98854	0.98561	0.98212	0.97805	0.97628	0.9721	0.9729
11	0.99728	0.96528	0.95777	0.96485	0.98353	0.99804	1.00842	1.01999	1.02669
12	1.00847	1.07206	1.097	1.09096	1.02681	0.9608	0.91618	0.85274	0.822
21	0.86223	0.93507	0.9478	0.9546	0.9444	0.92523	0.92198	0.90722	0.92281
22	1.04098	1.0318	1.03433	1.03686	1.04659	1.05988	1.06305	1.07109	1.05762
23	1.0591	1.04226	1.03978	1.0414	1.05877	1.07952	1.08032	1.0855	1.05545
24	0.79745	0.83977	0.81408	0.82271	0.80761	0.78575	0.81038	0.8181	0.88955
25	0.79264	1.00191	1.12424	1.14972	1.13967	1.13105	1.11715	1.10875	1.09236

 

从表达看到拟合精度还是有保障的。但这种办法只是对矩阵A中不含有0的元素的情况成立。

 

实验参数

学习率 0.1

权重初始化方式

Random rand1 =new Random();

int ti1=rand1.nextInt(98)+1;

tw[a][b]=xx*((double)ti1/100);

http://www.qinms.com/webapp/curvefit/cf.aspx