以前对这两个概念非常模糊,今天从网上找到了关于这两个概念的两种风格的解释,比较一下!
第一种风格:
For a data sample: x and it class lable:,y,
要检测图像中的淋巴结,有很多positive,很多的negetive。
Discrimitive model: p(y|x), 给定x,算y。
Generative model:和 discrimitive model 的区别:
Generative model focus 在自己的inclass 本身,不care 到底 decision boundary 在哪。 Generative model 实际上带的information 要比discrimitive model rich, 因为假设有generative model, 两类的,就完全得到了p(x|y),而discrimitive model 只care decision boundary。这里说的generative model 和 discrimitive model,在行业里,这个说法是通用的。 由Generative model 可以得到 discrimitive model, 但由discrimitive model 得不到 generative model。因为需要用到P(x), 如果只是label 的话,p(y)很简单,但为什么不能直接用Generative model 呢? 优缺 Discrimitive model: 相当于在图像上scan 一下,detection, 用一个path, 在不同的scale 上search, 每一词看probability, 在SVM上是positive 还是negetive。
Discrimitive 比较easy to learn, 给出正负例,给出lable, focus on discrimitive model marginal distribution。 某种意义上,比generativemodel 要简单,但power 是 limited, 可以告诉你的时1还是2,但没有办法把整个场景描述出来。 P(x|y) P(y) 当一个分类,没有negetive,研究single class,比discrimitive model flex 多,learning 和 computing 要比 discrimitive model 复杂得多
第二种解释 Discriminative Model是判别模型,又可以称为条件模型,或条件概率模型。 Generative Model是生成模型,又叫产生式模型。
二者的本质区别是 discriminative model 估计的是条件概率分布(conditional distribution)p(class|context) generative model 估计的是联合概率分布(joint probability distribution)p() 常见的Generative Model主要有:
– Gaussians, Naive Bayes, Mixtures of multinomials – Mixtures of Gaussians, Mixtures of experts, HMMs – Sigmoidal belief networks, Bayesian networks – Markov random fields 常见的Discriminative Model主要有:
– logistic regression
– SVMs
– traditional neural networks
– Nearest neighbor Successes of Generative Methods: NLP
– Traditional rule-based or Boolean logic systems Dialog and Lexis-Nexis) are giving way to statistical approaches (Markov models and stochastic context grammars) Medical Diagnosis
– QMR knowledge base, initially a heuristic expert systems for reasoning about diseases and symptoms been augmented with decision theoretic formulation Genomics and Bioinformatics
– Sequences represented as generative HMMs
主要应用Discriminative Model: Image and document classification Biosequence analysis Time series prediction Discriminative Model缺点: Lack elegance of generative
– Priors, structure, uncertainty Alternative notions of penalty functions, regularization, kernel functions Feel like black-boxes – Relationships between variables are not explicit and visualizable Bridging Generative and Discriminative: Can performance of SVMs be combined elegantly with flexible Bayesian statistics? Maximum Entropy Discrimination marries both methods – Solve over a distribution of parameters (a distribution over solutions)