神经网络的来源
Neural Network:input layer,hidden layer,output layer
⎡⎣⎢⎢⎢x0x1x2x3⎤⎦⎥⎥⎥→⎡⎣⎢⎢⎢a(2)1a(2)2a(2)3⎤⎦⎥⎥⎥→hθ(x)
a(j)i="activation" of unit i in layer jΘ(j)=matrix of weights controlling function mapping from layer j to layer j+1
a(2)1=g(Θ(1)10x0+Θ(1)11x1+Θ(1)12x2+Θ(1)13x3)a(2)2=g(Θ(1)20x0+Θ(1)21x1+Θ(1)22x2+Θ(1)23x3)a(2)3=g(Θ(1)30x0+Θ(1)31x1+Θ(1)32x2+Θ(1)33x3)hΘ(x)=a(3)1=g(Θ(2)10a(2)0+Θ(2)11a(2)1+Θ(2)12a(2)2+Θ(2)13a(2)3)
If network has sj units in layer j and sj+1 units in layer j+1, then Θ(j) will be of dimension sj+1×(sj+1). (because of the bias unit)
向量化
z(2)k=Θ(1)k,0x0+Θ(1)k,1x1+⋯+Θ(1)k,nxn
a(2)1=g(z(2)1)a(2)2=g(z(2)2)a(2)3=g(z(2)3)
x=⎡⎣⎢⎢⎢x0x1⋯xn⎤⎦⎥⎥⎥/a(1)=⎡⎣⎢⎢⎢⎢⎢a(1)0a(1)1⋯a(1)n⎤⎦⎥⎥⎥⎥⎥z(j)=⎡⎣⎢⎢⎢⎢⎢z(j)1z(j)2⋯z(j)n⎤⎦⎥⎥⎥⎥⎥
z(2)=Θ(1)x=Θ(1)a(1)
a(2)=g(z(2))
Add
a(2)0=1
then
a(2)∈Rm,z(2)∈Rm;n=m+1.
hΘ(x)=a(3)=g(z(3))
a(j)=g(z(j))
z(j+1)=Θ(j)a(j)
hΘ(x)=a(j+1)=g(z(j+1))
Example
Example1
hΘ(x)=g(−30+20x1+20x2)x1=0 and x2=0 then g(−30)≈0x1=0 and x2=1 then g(−10)≈0x1=1 and x2=0 then g(−10)≈0x1=1 and x2=1 then g(10)≈1
Example2
AND:Θ(1)NOR:Θ(1)OR:Θ(1)=[−302020]=[10−20−20]=[−102020]
⎡⎣⎢x0x1x2⎤⎦⎥→⎡⎣a(2)1a(2)2⎤⎦→[a(3)]→hΘ(x)
Θ(1)=[−301020−2020−20]
Θ(2)=[−102020]
多分类问题