xyqzki的专栏

RT

Given minf(a,b,Σ)\min f(a, b, \Sigma) s.t.a>0,b>0,Σ∈pds.t. a > 0, b>0, \Sigma \in pdwe can utilize the barrier function to prevent the a and b not to be negative, and Σ\Sigma to be pd.add the barri

General but ImportantGiven a matrix, ask yourself, is it square matrix? is it symmetric? eigenvectors are orthonormal for distinct eigenvalues can be decomposed? Eigendecomposed? SVD? How about is eige

ref: http://meroa.com/auc-awk-src/ http://alexkong.net/2013/06/introduction-to-auc-and-roc/ROC曲线分类器的评价与分类器本身同样重要。把分类器的性能映射到ROC(Receiver Operating Characteristic)空间，是一种常用的评价分类器的方法。ROC曲线横轴是FPR（False Pos

For random variable X and Y, to measure the mutual dependence, more general than the correlation.  MI is the expected value of the PMI

use the bayes ball methodhttp://www.cs.ubc.ca/~murphyk/Bayes/bnintro.htmlIn general, the conditional independence relationships encoded by a Bayes Net are best be explained by means of the “Bayes Ball”

什么时候用EM对概率模型做parameter estimation 的时候，通常可以用point estimation, MLE, MAP或者bayesian estimation(把parameter看成随机变量).Typically, 对于MLE或者MAP, 如果碰到了隐变量hidden variable, e.g. z in GMM, 就必须用EM, E步算Q M步算 新的参数 NoteEM

个人总结：统计中参数估计有四种方法点估计MLEMAP 加入参数的prior 信息，可以避免overfitting，还可以加入extra knowledge。称为Occam’razor Bayesian Inference这种方法不像MLE,MAP将参数看成未知的常量，而是看成随机变量，求出其后验分布的具体形式p(θ|X)p(\theta|X)，然后用E(θ|X)E(\theta|X)作为估计值。通常

variational EM: http://net.pku.edu.cn/~zhaoxin/vEMLDA.pdfEM: http://www.cnblogs.com/biyeymyhjob/archive/2012/07/21/2603020.html估计方法都是用的MLE, variational EM 估计α\alpha和β\beta(对应EM中的zz)，e而实际上也要求的是θ\theta

ref:eigen vectorhttp://www.quora.com/What-is-an-eigenvector-of-a-covariance-matrixgeometric interpretationhttp://www.visiondummy.com/2014/04/geometric-interpretation-covariance-matrix/好网站http://www.vis

参考 http://www.cnblogs.com/jerrylead/archive/2011/05/11/2043317.html

参考http://www.cnblogs.com/jerrylead/archive/2011/05/11/2043317.htmlhttp://blog.csdn.net/puqutogether/article/details/43055633http://blog.csdn.net/puqutogether/article/details/40889719regression problem

http://stats.stackexchange.com/questions/60680/kl-divergence-between-two-multivariate-gaussiansI give a detailed derivation process for the KL Divergence between two multivariate normal distributions.