Deep Learning：深度前馈神经网络(一)

1. 深度前馈网络，也叫前馈神经网络或多层感知机（MLPs）。前馈网络的目标是近似某些函数$f^*$。例如，对于一个分类器，$y=f^*(x)$将$x$映射到一个分类$y$。前馈网络定义一个映射$y=f(x;\theta)$，并学习参数$\theta$的值，以找到最好的近似函数。
2. The model is associated with a directed acyclic graph describing how the functions are composed together。For example, we might have three functions f (1), f (2), and f (3) connected in a chain, to form f(x) = f (3)(f (2)(f(1)(x)))。The overall length of the chain gives the depth of the model.
3. It is best to think of feedforward networks as function approximation machines that are designed to achieve statistical generalization, occasionally drawing some insights from what we know about the brain, rather than as models of brain function.
4. To extend linear models to represent nonlinear functions of x, we can apply the linear model not to x itself but to a transformed input φ(x), where φ is a nonlinear transformation. The question is then how to choose the mapping φ:
   (1) One option is to use a very generic φ, such as the infinite-dimensional φ that is implicitly used by kernel machines based on the RBF kernel. If φ(x) is of high enough dimension, we can always have enough capacity to fit the training set, but generalization to the test set often remains poor. Very generic feature mappings are usually based only on the principle of local smoothness and do not encode enough prior information to solve advanced problems.
   (2) Another option is to manually engineer φ. Until the advent of deep learning, this was the dominant approach. This approach requires decades of human effort for each separate task, with practitioners specializing in different domains such as speech recognition or computer vision, and with little transfer between domains.
   (3) The strategy of deep learning is to learn φ. In this approach, we have a model y=f(x;θ,w)=φ(x;θ)Tw