Neural Networks and Deep Learning 学习笔记(一)

1. 为什么假设$w \cdot x \equiv \sum_j w_j x_j$后，$w$与$x$就变成向量了？

The first change is to write $\sum_j w_j x_j$ as a dot product, $w \cdot x \equiv \sum_j w_j x_j$, where w and x are vectors whose components are the weights and inputs, respectively.

向量的点积为标量，两个同样维度的向量的点积恰好是每个维度积的累加，以上变换可用公式表示为

$$(w_1, w_2, w_3, ..., w_j) \cdot (x_1, x_2, x_3,...,x_j) = w_1x_1 +w_2x_2 + w_3x_3 + ... + w_jx_j = \sum_jw_jx_j$$

2. 用perceptron实现‘与非’、‘与’、‘或’门。

we have a perceptron with two inputs, each with weight −2, and an overall bias of 3. Then we see that input 00 produces output 1, since $(−2)∗0+(−2)∗0+3=3$ is positive. Here, I’ve introduced the ∗ symbol to make the multiplications explicit. Similar calculations show that the inputs 01 and 10 produce output 1. But the input 11 produces output 0, since $(−2)∗1+(−2)∗1+3=−1$ is negative. And so our perceptron implements a NAND gate!

以上是电子书中实现的与非门。以此类推实现与门的方式可以为bias设置为-3，$x_1$与$x_2$的权重设置为2，则输入00，计算为 $2 \times 0 + 2 \times 0 + (-3) = -3$，为负。输入01，计算为 $2 \times 0 \ + 2 \times 1 + (-3) = -1$ 为负。输入11，$2 \times 1 + 2 \times 1 + (-3) = 1$ 为正。实现与门。

实现或门的bias设置为-1，权重设置为2。则输入00，输出为负，输入01，输出为正，输入11，输出为正。

3. exp是啥？

exp是以$e$为底的自然对数，特点是对$e^x$求导还是$e^x$。

4. 这个偏导数是怎么得到的（未解决）?

$$\begin{eqnarray} \Delta \mbox{output} \approx \sum_j \frac{\partial \, \mbox{output}}{\partial w_j} \Delta w_j + \frac{\partial \, \mbox{output}}{\partial b} \Delta b \end{eqnarray}$$