线性判别分析 （LDA）

1. 不错的中文解释

    http://www.cnblogs.com/LeftNotEasy/archive/2011/01/08/lda-and-pca-machine-learning.html

    http://www.cnblogs.com/jerrylead/archive/2011/04/21/2024384.html

    总之一句话，优化目标是： 类间距离最大，类内距离最小.

    LDA本身不是分类器，是一种特征抽取的方法。其实是得到一个特征空间，在特征空间中符合类间距离最大，类内距离最小。             

2. LDA的实现实例

    http://www.eeprogrammer.com/tutorials/Matlab/discriminant_analyses.html

   matlab toolbox

   http://www.mathworks.co.uk/products/statistics/examples.html;jsessionid=64a02aac0c85404ecc97353f7f42?file=/products/demos/shipping/stats/classdemo.html

   

    http://www.mathworks.com/matlabcentral/fileexchange/30779-lda-linear-discriminant-analysis/content/LDA/LDA_example.m   

 

 c1=[0.4,0.58,0.089;-0.31,0.27,-0.04;0.38,0.055,-0.035;-0.15,0.53,0.011;-0.35,0.47,0.034;...
     0.17,0.69,0.1;-0.011,0.55,-0.18;-0.27,0.61,0.12;-0.065,0.49,0.0012;-0.12,0.054,-0.063];
% %
c2=[0.83,1.6,-0.014;1.1,1.6,0.48;-0.44,-0.41,0.32;0.047,-0.45,1.4;0.28,0.35,3.1;...
     -0.39,-0.48,0.11;0.34,-9.079,0.14;-0.3,-0.22,2.2;1.1,1.2,-0.46;0.18,-0.11,-0.49];



scatter(c1(:,1),c1(:,2),6,'r'),hold on;
scatter(c2(:,1),c2(:,2),6,'b');


% Number of observations of each class
n1=size(c1,1)
n2=size(c2,1)

%Mean of each class
mu1=mean(c1)
mu2=mean(c2)

% Average of the mean of all classes
%%%%% different from source code
mu=mean([c1;c2])

% Center the data (data-mean)
d1=c1-repmat(mu1,size(c1,1),1)
d2=c2-repmat(mu2,size(c2,1),1)


% Calculate the within class variance (SW)
s1=d1'*d1
s2=d2'*d2
sw=s1+s2
invsw=inv(sw)

% in case of two classes only use v
% v=invsw*(mu1-mu2)'

% if more than 2 classes calculate between class variance (SB)
sb1=n1*(mu1-mu)'*(mu1-mu)
sb2=n2*(mu2-mu)'*(mu2-mu)
SB=sb1+sb2
v=invsw*SB

% find eigne values and eigen vectors of the (v)
[evec,eval]=eig(v)

% Sort eigen vectors according to eigen values (descending order) and
% neglect eigen vectors according to small eigen values
% v=evec(greater eigen value)
% or use all the eigen vectors

% project the data of the first and second class respectively

y2=c2*evec
y1=c1*evec