蚊子爱牛牛的博客

深度前馈网络，也叫前馈神经网络或多层感知机（MLPs）。

Semi-Supervised Learning:In the paradigm of semi-supervised learning, both unlabeled examples from P(x) and labeled examples from P (x, y) are used to estimate P (y | x) or predict y from x.

Multi-Task Learning

Early Stopping

Parameter Tying and Parameter Sharing

Sparse Representations

Bagging and Other Ensemble Methods:Bagging (short for bootstrap aggregating) is a technique for reducing generalization error by combining several models.

Dropout

Adversarial Training

Tangent Distance, Tangent Prop, and Manifold Tangent ClassifierMany machine learning algorithms aim to overcome the curse of dimensionality by assuming that the data lies near a low-dimensional manifol

Noise Robustness:Dataset Augmentation has motivated the use of noise applied to the inputs as a dataset augmentation strategy.

Dataset Augmentation：The best way to make a machine learning model generalize better is to train it on more data.

Deep Learning : 深度前馈神经网路-Output Units

How Learning Differs from Pure Optimization

Hidden UnitsSo far we have focused our discussion on design choices for neural networks that are common to most parametric machine learning models trained with gradientbased optimization.

Architecture Design

Back-Propagation and Other Differentiation Algorithms

Many strategies used in machine learning are explicitly designed to reduce the test error, possibly at the expense of increased training error. These strategies are known collectively as regularization

We can minimize a function subject to constraints by constructing a generalized Lagrange function, consisting of the original objective function plus a set of penalties.

Regularization and Under-Constrained ProblemsMany linear models in machine learning, including linear regression and PCA, depend on inverting the matrix

Optimization for Training Deep Models

Deep Learning：Optimization for Training Deep Models