【NTU_ML_Spring2020】4.DNN

台湾大学 机器学习（李宏毅） 2020年春

神经网络、反向传播算法(BP)、梯度消失问题、ReLU函数、Max-out网络、Adam学习率衰减、正则化、Dropout

深度网络的有效性理解（集成化、逻辑电路）

本文为本人学习过程中整理的学习笔记，想顺带学英语所以用英文呈现。发现错误还烦请指正。欢迎交流。

未经同意，请勿转载。

DNN

History

Perceptron

A kind of classifier like Logistic regression using sign function and square error instead of Sigmoid function and cross entropy error.

It’s also a linear model (which can not solve non-linearly separable problem).

Fully Connect Neural Network

• Designing a network structure is defining a function set.

  You need to decide the network structure to let a good function in your function set.

Matrix Operation

Multi-class Classifier Output Layer

Use sum of cross entropy error as loss function.

$D$ is the dimension of feature. $N$ is the scale of training data.
$$\begin{gathered} C(y,\hat{y})=-\sum _{i=1}^D{\hat{y}}_i\ln y_i \\ L=\sum _{n=1}^N=C^n \end{gathered}$$
$C^n$ means the n t