Machine Learning week5 错题记录

1。第 1 个问题

You are training a three layer neural network and would like to use backpropagation to compute the gradient of the cost function. In the backpropagation algorithm, one of the steps is to update

Δ(2)ij:=Δ(2)ij+δ(3)i(a(2))j

for every i,j. Which of the following is a correct vectorization of this step?

Δ(2):=Δ(2)+δ(3)(a(2))T

Δ(2):=Δ(2)+(a(2))Tδ(3)

Δ(2):=Δ(2)+δ(2)(a(3))T

Δ(2):=Δ(2)+(a(2))Tδ(2)

第 2 个问题
1
point

2。第 2 个问题

Suppose Theta1 is a 5x3 matrix, and Theta2 is a 4x6 matrix. You set thetaVec=[Theta1(:);Theta2(:)]. Which of the following correctly recovers Theta2?

reshape(thetaVec(16:39),4,6)

reshape(thetaVec(15:38),4,6)

reshape(thetaVec(16:24),4,6)

reshape(thetaVec(15:39),4,6)

reshape(thetaVec(16:39),6,4)

第 3 个问题
1
point

3。第 3 个问题

Let J(θ)=2θ3+2. Let θ=1, and ϵ=0.01. Use the formula J(θ+ϵ)J(θϵ)2ϵ to numerically compute an approximation to the derivative at θ=1. What value do you get? (When θ=1, the true/exact derivati ve is dJ(θ)dθ=6.)

6

5.9998

8

6.0002

第 4 个问题
1
point

4。第 4 个问题

Which of the following statements are true? Check all that apply.

For computational efficiency, after we have performed gradient checking to

verify that our backpropagation code is correct, we usually disable gradient checking before using backpropagation to train the network.

Using gradient checking can help verify if one's implementation of backpropagation is bug-free.

Gradient checking is useful if we are using one of the advanced optimization methods (such as in fminunc) as our optimization algorithm. However, it serves little purpose if we are using gradient descent.

Computing the gradient of the cost function in a neural network has the same efficiency when we use backpropagation or when we numerically compute it using the method of gradient checking.

第 5 个问题
1
point

5。第 5 个问题

Which of the following statements are true? Check all that apply.

Suppose we are using gradient descent with learning rate α. For logistic regression and linear regression, J(θ) was a convex optimization problem and thus we did not want to choose a learning rate α that is too large. For a neural network however, J(Θ) may not be convex, and thus choosing a very large value of α can only speed up convergence.

Suppose that the parameter Θ(1) is a square matrix (meaning the number of rows equals the number of columns). If we replace Θ(1) with its transpose (Θ(1))T, then we have not changed the function that the network is computing.

Suppose we have a correct implementation of backpropagation, and are training a neural network using gradient descent. Suppose we plot J(Θ) as a function of the number of iterations, and find that it is increasing rather than decreasing. One possible cause of this is that the learning rate α is too large.

If we are training a neural network using gradient descent, one reasonable "debugging" step to make sure it is working is to plot J(Θ) as a function of the number of iterations, and make sure it is decreasing (or at least non-increasing) after each iteration.

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值