生成模型和判别模型直观理解

1.最直观理解

吴恩达老师CS229课程笔记内容：
http://cs229.stanford.edu/notes/cs229-notes2.pdf

Consider a classification problem in which we want to learn to distinguish between elephants (y = 1) and dogs (y = 0), based on some features of an animal. Given a training set, an algorithm like logistic regression or the perceptron algorithm (basically) tries to find a straight line—that is, a decision boundary—that separates the elephants and dogs. Then, to classify a new animal as either an elephant or a dog, it checks on which side of the decision boundary it falls, and makes its prediction accordingly.
Here’s a different approach. First, looking at elephants, we can build a model of what elephants look like. Then, looking at dogs, we can build a separate model of what dogs look like. Finally, to classify a new animal, we can match the new animal against the elephant model, and match it against the dog model, to see whether the new animal looks more like the elephants or more like the dogs we had seen in the training set.

简而言之：
  为了区分大象和狗，判别式模型根据训练集找到决策边界，生成式模型根据训练集分别对大象和狗建模，看测试样例更接近哪一个类别。这就是判别式（discrimitive model）和生成式模型（generative model）名称的由来（基于字面意思理解就对了）。显然，生成式模型在区分大象和狗的问题上，冗余较大，极有可能事倍功半。

Generative models can generate data.Discriminative models can discriminate data.
参见：https://www.zhihu.com/question/22374366/answer/155544744

2.对比一个经典例子

假设有4个samples：

	sample1