吴恩达·Machine Learning || chap9 Neural Network : Learning简记

9 Neural Network : Learning

9-1 Cost function

Neural Network(classification)

$(x^{(1)},y^{(1)}),(x^{(2)},y^{(2)}),\cdots ,(x^{(m)},y^{(m)})$

$L$ = total no. of layers in network

$s_l$ = no. of unit (not counting bias unit) in layer $l$

Compare between Binary | Multi-class(K classes) classification

Binary classification

​ $y=0or1$​​

​ 1 out put unit

Multi-class classification(K classes)

​ $y\in {\mathbb{R}}^K$

​ K output unit

Cost function

​ Logistic regression:

​ $$J(\Theta )=-\frac{1}{m}[\sum _{i=1}^m{y}^{(i)}\log h_{\theta }(x^{(i)})+(1-y^{(i)})\log (1-h_{\theta }(x^{(i)}))]+\frac{\lambda }{2m}\sum _{j=1}^n{\Theta }_j^2$$​​​

​ Neural network:

​ $h_{\Theta }(x)\in R^K(h_{\Theta }(x){)}_i=i^{th}output$

​ $J(\Theta )=-\frac{1}{m}[{\sum }_{i=11}^m{\sum }_{k=1}^K{y}_k^{(i)}\log (h_k(x^{(i)}){)}_k+(1-y_k^{(i)}\log (1-(h_{\Theta }(x^{(i)}{)}_k)]+\frac{\lambda }{2m}{\sum }_{l=1}^{L-1}{\sum }_{i=1}^{s_l}{\sum }_{j=1}^{s_l+1}({\Theta }_{ji}^{(l)}{)}^2$​​

9-2 Backpropagation algorithm

反向传播算法

Gradient computation

​ $mi{n}_{\Theta }J(\Theta )$

​ Need code to compute:

• $J(\Theta )$
• $\frac{\partial }{\partial {\Theta }_{ij}^{(l)}}J(\Theta )$​​

Given one training example (x,y):

​ Forward propagation:

​ $\begin{array}{l}{a}^{(1)}=x\\ z^{(2)}={\Theta }^{(1)}{a}^{(1)}\\ a^{(2)}=g(z^{(2)})(add{a}_0^{(2)})\\ z^{(3)}={\Theta }^{(2)}{a}^{(2)}\\ a^{(3)}=g(z^{(3)})(add{a}_0^{(3)})\\ z^{(4)}={\Theta }^{(3)}{a}^{(3)}\\ a^{(4)}=h_{\Theta }(x)=g(z^{(4)})\end{array}$​

Gradient computation: Backpropagation algorithm

Intuition: ${\delta }_j^{(l)}=$“error” of node $j$ in layer $l$

For each out put unit (layer L=4)

${\delta }_j^{(4)}=a_j^{(4)}-y_j$

$\begin{array}{l}{\delta }^{(3)}=({\Theta }^{(3)}{)}^T{\delta }^{(4)}.\ast g^{\prime }(z^{(3)})\\ {\delta }^{(2)}=({\Theta }^{(2)}{)}^T{\delta }^{(3)}.\ast g^{\prime }(z^{(2)})\end{array}$​​​​​

($no{\delta }^{(1)}$​​​)

Backpropagation algorithm

Training set: $(x^{(1)},y^{(1)}),(x^{(2)},y^{(2)}),\cdots ,(x^{(m)},y^{(m)})$​​

Set: ${△}_{ij}^{(l)}=0(foralll,i,j)$​​​

${△}_{ij}^{(l)}$​​​​ used to compute $\frac{\partial }{\partial {\Theta }_{ij}^{(l)}}J(\Theta )$​​​​​

For $i=1$​​ to $m$:

​ Set $a^{(1)}=x^{(i)}$​​

​ Perform forward propagation to compute $a^{(l)}$​ for $l=2,3,\cdots ,L$​​​

​ Using $y^{(i)}$​​​,compute $\delta (L)=a^{(L)}-y^{(i)}$​​​

​ Compute $\delta (L-1),{\delta }^{(L-2)},\cdots ,{\delta }^{(2)}$​​​

​ ${△}_{ij}^{(i)}:={△}_{ij}^{(l)}+a_j^{(l)}{\delta }_i^{(l+1)}$​​​

$\begin{array}{l}{D}_{ij}^{(l)}:=\frac{1}{m}{△}_{ij}^{(l)}+\lambda {\Theta }_{ij}^{(t)}ifj\ne 0\\ D_{ij}^{(l)}:=\frac{1}{m}{△}_{ij}^{(l)}ifj=0\end{array}$

$\frac{\partial }{\partial {\Theta }_{ij}^{(l)}}J(\Theta )=D_{ij}^{(l)}$​

9-3 Backpropagation intuition

Forward propagation

$x_1^{(i)},x_2^{(i)},\dots \stackrel{z^{(2)}={\Theta }^{(1)}{a}^{(1)}+\cdots }{\Rightarrow }{z}_1,z_2,\cdots \stackrel{a^{(2)}=g(z^{(2)})}{\Rightarrow }{a}_1,a_2,\cdots \Rightarrow \cdots$

​​​What is backpropagation doing?
$$J(\theta )=-\frac{1}{m}[\sum _{i=1}^m{y}^{(i)}\log (h_{\Theta }(x^{(i)}))+(1-y^{(i)})\log (1-(h_{\Theta }(x^{(i)}))]+\frac{\lambda }{2m}\sum _{l=1}^{L-1}\sum _{i=1}^{s_l}\sum _{j=1}^{s_l+1}({\Theta }_{ji}^{(l)}{)}^2$$
Focusing on a single example $x^{(i)},y^{(i)}$, the case of 1 output unit,and ignoring regularization ($\lambda =0$)

$$\mathrm{cost}(i)=y^{(i)}\log h_{\Theta }(x^{(i)})+(1-y^{(i)})\log h_{\Theta }(x^{(i)})$$​​

(Thinking of $\mathrm{cost}(i)\approx (h_{\Theta }(x^{(i)})-y^{(i)}{)}^2$​​)

I.e. how well is the network doing on example i?

Backpropagation

${\delta }_j^{(l)}=$“error” of cost for $a_j^{(l)}(unitjinlayerl).$

$$\begin{gathered} Formally,{\delta }_j^{(l)}=\frac{\partial }{\partial z_j^{(l)}}cost(i)(forj\ge 0),where \\ \mathrm{cost}(i)=y^{(i)}\log h_{\Theta }(x^{(i)})+(1-y^{(i)})\log h_{\Theta }(x^{(i)}) \end{gathered}$$​​

9-4 Implementation note: Unrolling parameters

Advanced optimization

function [jval,gradient] =costFunction(theta)
...
optTheta = fminunc(@costFunction,initialTheta,options)

Neural Network(L=4):

​ ${\Theta }^{(1)},{\Theta }^{(2)},{\Theta }^{(3)}-$​matrices(Theta1,Theta2,Theta3)

​ $D^{(1)},D^{(2)},D^{(3)}-$matrices(D1,D2,D3)

“unroll” into vectors

Example

$$\begin{array}{l}{s}_1=10,s_2=10,s_3=1\\ {\Theta }^{(1)}\in {\mathbb{R}}^{10\times 11},{\Theta }^{(2)}\in {\mathbb{R}}^{10\times 11},{\Theta }^{(3)}\in {\mathbb{R}}^{1\times 11}\\ D^{(1)}\in {\mathbb{R}}^{10\times 11},D^{(2)}\in {\mathbb{R}}^{10\times 11},D^{(3)}\in {\mathbb{R}}^{1\times 11}\end{array}$$​​​

thetaVec =[Theta1(:);Theta2(:);Theta3(:)];
Dvec =[D1(:),D2(:),D3(:)];

Theta1=reshape(thetaVec(1:110),10,11)
Theta2=reshape(thetaVec(111,220),10,11)
Theta1=reshape(thetaVec(221:231),1 ,11)

Learning Algorithm

Have initial parameters ${\Theta }^{(1)},{\Theta }^{(2)},{\Theta }^{(3)}$​

Unroll to get $initialTheta$​ to pass to

fminunc(@costFunction,initialTheta,options)

function[jval,gradientVec] = costFunction(thetaVec)

From $thetaVec$​, get ${\Theta }^{(1)},{\Theta }^{(2)},{\Theta }^{(3)}$​

Use forward prop/back prop to compute $D^{(1)},D^{(2)},D^{(3)}$ and $J(\theta )$

Unroll $D^{(1)},D^{(2)},D^{(3)}$ to get $gradientVec$

9-5 Gradient checking

Numerical estimation of gradient

$\frac{d}{d\theta }J(\theta )\approx \frac{J(\theta +ϵ)-J(\theta -ϵ)}{2ϵ}$​

Implement

gradApprox = (J(theta+EPSILON)-J(theta-EPSILON))/(2*EPSILON)

Parameter vector $\theta$

$\theta \in {\mathbb{R}}^n$ (E.g. $\theta$ is “unrolled” version of ${\Theta }^{(1)},{\Theta }^{(2)},{\Theta }^{(3)}$)

$\theta ={\theta }_1,{\theta }_2,{\theta }_3,\cdots ,{\theta }_n$

$\begin{array}{lc}& \\ \frac{\partial }{\partial {\theta }_1}J(\theta )\approx \frac{J({\theta }_1+ϵ,{\theta }_2,{\theta }_3\cdots ,{\theta }_n)-J({\theta }_1-ϵ,{\theta }_2,{\theta }_3\cdots ,{\theta }_n)}{2ϵ}& \\ \frac{\partial }{\partial {\theta }_1}J(\theta )\approx \frac{J({\theta }_1,{\theta }_2+ϵ,{\theta }_3\cdots ,{\theta }_n)-J({\theta }_1,{\theta }_2-ϵ,{\theta }_3\cdots ,{\theta }_n)}{2ϵ}& \\ ⋮& \\ \frac{\partial }{\partial {\theta }_1}J(\theta )\approx \frac{J({\theta }_1,{\theta }_2,{\theta }_3\cdots ,{\theta }_n+ϵ)-J({\theta }_1,{\theta }_2,{\theta }_3\cdots ,{\theta }_n-ϵ)}{2ϵ}& \end{array}$

for i=1:n,
	thetaPlus=theta;
	thetaPlus(i)=thetaPlus(i)+EPSILON;
	thetaMinus=theta;
	thetaMinus(i)=thetaMinus(i)-EPSILON;
	gradApprox(i)=(J(thetaPlus)-J(thetaMinus))/(2*EPSILON);
end;

Check that $gradApprox\approx Dvec$​

Implementation Note:

• Implement backprop to compute $Dvec$(unrolled $D^{(1)},D^{(2)},D^{(3)},$)
• Implement numerical gradient check to compute $gradApprox$
• Make sure they give similar values
• Turn off gradient checking Using backprop code for learning

Important:

• Be sure to disable your gradient checking code before training your classifier. If you run numerical gradient computation on every iteration of gradient descent(or in the inner loop of $costFunction(\cdots )$​​​) your code will be very slow.

9-6 Random initialization

Initial value of $\Theta$

For gradient descent and advanced optimization method, need initial value for $\Theta$

optTheta = fminunc(@costFunction,initialTheta,options)

Consider gradient descent

Set $initialTheta=zeros(n,1)$​? $\times$​

Zero initialization

${\Theta }_{ij}^{(l)}=0foralli,j,l.$

After each update, parameters corresponding to inputs going into each of two hidden units are identical

Random each ${\Theta }_{ij}^{(l)}$​ to a random value in $[-ϵ,ϵ]$​($i.e.-ϵ\le {\Theta }_{ij}^{(l)}\le ϵ$​)

$E.g.$

Theta1 = rand(10,11)*(2*INIT_EPSILON)-INIT_EPSILON;
Theta2 = rand(1,11)*(2*INIT_EPSILON)-INIT_EPSILON;

9-7 Putting it together

Training a neural network

Pick a network architecture( connectivity pattern between neurons)

• No of input units: Dimension of features $x^{(i)}$

• No output units: Number of classes

  $y^{(1)}=\left[\begin{array}{c}1\\ 0\\ 0\\ 0\\ 0\end{array}\right],y^{(2)}=\left[\begin{array}{c}0\\ 1\\ 0\\ 0\\ 0\end{array}\right],\cdots$​

• Reasonable default: 1 hidden layer, or if >1 hidden layer, have same no of hidden units in every layer(usually the more the better

1. Randomly initialize weights
2. Implement forward propagation to get $h_{\Theta }(x^{(i)})$ for any $x^{(i)}$
3. Implement code to compute cost function, $J(\Theta )$
4. Implement backprop to compute partial derivatives $\frac{\partial }{\partial {\Theta }_{jk}^{(l)}}J(\Theta )$​​

for i=1:m

​ Perform forward propagation and backpropagation using example $(x^{(i)},y^{(i)})$​
​ ( Get activations $a^{(l)}$​ and delta terms ${\delta }^{(l)}forl=2,\cdots ,L$​

5. Use gradient checking to compare $\frac{\partial }{\partial {\Theta }_{jk}^{(l)}}J(\Theta )$​ computed using backpropagation vs. using numerical estimate of gradient of $J(\Theta )$​.
   Then disable gradient checking code.
6. Use gradient descent or advanced optimization method withbackpropagation to try to minimize $J(\Theta )$​ as a function ofparameters $\Theta$​​

9-8 Autonomous driving example