Deep Learning：正则化（十四）

Tangent Distance, Tangent Prop, and Manifold Tangent Classifier

Many machine learning algorithms aim to overcome the curse of dimensionality by assuming that the data lies near a low-dimensional manifold.
One of the early attempts to take advantage of the manifold hypothesis is the tangent distance algorithm.

• It is a non-parametric nearest-neighbor algorithm in which the metric used is not the generic Euclidean distance but one that is derived from knowledge of the manifolds near which probability concentrates.
• It is assumed that we are trying to classify examples and that examples on the same manifold share the same category.
• Since the classifier should be invariant to the local factors of variation that correspond to movement on the manifold, it would make sense to use as nearest-neighbor distance between points x1 and x2 the distance between the manifolds M1 and M2 to which they respectively belong.
• A cheap alternative that makes sense locally is to approximate Mi by its tangent plane at xi and measure the distance between the two tangents, or between a tangent plane and a point.