机器学习单层感知机梯度推导

激活函数：

图示：

梯度求解：
$$f(x)=\frac{1}{1+e^{-x}}$$
$$f^{\prime }(x)=\frac{e^{-x}}{(1+e^{-x}{)}^2}$$
$$f^{\prime }(x)=\frac{(1+e^{-x})-1}{(1+e^{-x}{)}^2}$$
$$f^{\prime }(x)=\frac{1}{1+e^{-x}}-\frac{1}{(1+e^{-x}{)}^2}$$
$${\sigma }^{\prime }(x)=\sigma (x)(1-\sigma (x))$$

损失函数使用：MSE

$$\begin{gathered} E=\frac{1}{2}(O_0^1-t{)}^2 \\ (t是标签值，O_0^1是经过sigmoid激活函数后的输出值,1/2是为了求导时消掉常数项w_{j_0}表示第j个权值） \end{gathered}$$
$$\begin{gathered} \frac{{\phi }_E}{{\phi }_{w_{j_0}}}=(O_0^1-t)\frac{{\phi }_O}{{\phi }_{w_{j_0}}} \\ (这里为了便于书写，用O来代替O_0^1) \end{gathered}$$
$$\begin{gathered} O=\sigma (x) \\ \frac{{\phi }_E}{{\phi }_{w_{j_0}}}=(O-t)\frac{{\phi }_{\sigma (x_0^1)}}{{\phi }_{(w_{j_0})}} \end{gathered}$$
$$\frac{{\phi }_E}{{\phi }_{w_{j_0}}}=(O-t)\frac{{\phi }_{\sigma (x_0^1)}}{{\phi }_{(x_0^1)}}\frac{{\phi }_{(x_0^1)}}{{\phi }_{\sigma (w_{j_0})}}$$
$$\frac{{\phi }_E}{{\phi }_{w_{j_0}}}=(O-t)\sigma (x_0^1))(1-\sigma (x_0^1)))\frac{{\phi }_{(x_0^1)}}{{\phi }_{\sigma (w_{j_0})}}$$
$$\begin{gathered} (根据线性关系) \\ {x}_0^1=\Sigma w_{j_0}{x}_j^0 \\ \frac{{\phi }_E}{{\phi }_{w_{j_0}}}=(O-t)\sigma (x_0^1))(1-\sigma (x_0^1)))x_j^0 \\ (所有变量都为已知，求出梯度) \end{gathered}$$