Basics of Neural Network Programming - Computation Graph

最新推荐文章于 2024-06-24 16:20:40 发布

王彩旗 edwardwangcq.com

最新推荐文章于 2024-06-24 16:20:40 发布

阅读量88

点赞数

分类专栏：人工智能 # Neural Networks and Deep Learning

本文链接：https://blog.csdn.net/edward_wang1/article/details/118066832

版权

人工智能同时被 2 个专栏收录

142 篇文章 0 订阅

订阅专栏

Neural Networks and Deep Learning

32 篇文章 0 订阅

订阅专栏

这篇博客通过一个简单的例子解释了计算图在神经网络中的作用，特别是在前向传播和反向传播过程中的应用。计算图将神经网络的计算过程分为三个步骤，并以图形方式展示，便于理解。在前向传播中，从左到右计算输出值；而在反向传播中，通过右向左的路径计算梯度和导数。计算图对于优化如成本函数（J）等特定输出变量非常有用。

摘要由CSDN通过智能技术生成

This is the notes when studying the class Neural Networks & Deep Learning by Andrew Ng, section 2.7 computation graph. Share it with you and hope it helps.

The computations of a neural network are organized in terms of a forward propagation step in which we compute the output of the neural network followed by a backward propagation step which we use to compute gradients or derivatives. The computation graph explained why it's organized this way.

Let's use a simpler example than logistic regression to explain how computation graph works.

Suppose:

$J(a,b,c)=3(a+bc)$

Computing this function actually has 3 distinct steps:

$\\u=bc \\ v=a+u \\ J=3v$

We can take these 3 steps and draw them in a computation graph as below:

In addition, we've shown a concrete example in the computation graph by setting the variables $a,b,c$ to specific values.

The computation graph comes in handy when there is some special output variable, such as J in this case, you want to optimize. In the case of logistic regression, J is the cost function we try to optimize. In this little example, through a left-to-right pass, we can compute the value of J. In the next couple of classes, we'll see that in order to compute derivatives, it'll be a right-to-left path going in the opposite direction as the red arrows.

<end>