吴恩达深度学习学习笔记——C2W1——神经网络优化基础及正则化——作业3——梯度检验

这里主要梳理一下作业的主要内容和思路，完整作业文件可参考:

https://github.com/pandenghuang/Andrew-Ng-Deep-Learning-notes/tree/master/assignments/C2W1

作业完整截图，参考本文结尾：作业完整截图。

作业目录：

 

Gradient Checking（梯度检验）

Welcome to the final assignment for this week! In this assignment you will learn to implement and use gradient checking.

You are part of a team working to make mobile payments available globally, and are asked to build a deep learning model to detect fraud--whenever someone makes a payment, you want to see if the payment might be fraudulent, such as if the user's account has been taken over by a hacker.

But backpropagation is quite challenging to implement, and sometimes has bugs. Because this is a mission-critical application, your company's CEO wants to be really certain that your implementation of backpropagation is correct. Your CEO says, "Give me a proof that your backpropagation is actually working!" To give this reassurance, you are going to use "gradient checking".

...

 

1) How does gradient checking work? （梯度校验数学原理——数值法求导）

 

2) 1-dimensional gradient checking （1维梯度检验——一元函数求导）

...

...

 

3) N-dimensional gradient checking （N维梯度检验，多元函数求导）

...

...

 

Note

• Gradient Checking is slow! Approximating the gradient with ∂𝐽/∂𝜃≈(𝐽(𝜃+𝜀)−𝐽(𝜃−𝜀))/2𝜀 is computationally costly. For this reason, we don't run gradient checking at every iteration during training. Just a few times to check if the gradient is correct.
• Gradient Checking, at least as we've presented it, doesn't work with dropout. You would usually run the gradient check algorithm without dropout to make sure your backprop is correct, then add dropout.

Congrats, you can be confident that your deep learning model for fraud detection is working correctly! You can even use this to convince your CEO. :)

What you should remember from this notebook:
- Gradient checking verifies closeness between the gradients from backpropagation and the numerical approximation of the gradient (computed using forward propagation).
- Gradient checking is slow, so we don't run it in every iteration of training. You would usually run it only to make sure your code is correct, then turn it off and use backprop for the actual learning process.

 

作业完整截图：