机器学习（二）非参数估计matlab例程

机器学习（二）非参数估计matlab例程

2018/2/19
by ChenjingDing

---

问题描述：
分别使用K近邻和核函数的方法。为输入样本$\widehat{x}$估计概率密度函数。$\widehat{x}$ = linspace(-5, 5, 100);
假设$x_{trai}$ = random(‘norm’, 0, 1, 1, 100)服从高斯分布，则估计出的概率密度也应该服从高斯分布。

---

apply.m

close all;
clc;

%get fixed K of knn and fixed h of kde
p = parameters();

disp('Question: Kernel/K-Nearest Neighborhood Density Estimators');

% Produce the random samples
samples = random('norm', 0, 1, 1, 100);

% Compute the original normal distribution
realDensity = gauss1D(0, 1, 100, 5);

% Estimate the probability density using the KDE
estDensity = kde(samples, p.h);

% plot results
figure;
plot(estDensity(1, :), estDensity(2, :), 'r', 'LineWidth', 1.5);
hold on;
plot(realDensity(1, :), realDensity(2, :), 'b', 'LineWidth', 1.5);
legend('KDE Estimated Distribution', 'Real Distribution');
hold off;

% Estimate the probability density using KNN
estDensity = knn(samples, p.k);

% Plot the distributions
figure;
plot(estDensity(1, :), estDensity(2, :), 'r', 'LineWidth', 1.5);
hold on;
plot(realDensity(1, :), realDensity(2, :), 'b', 'LineWidth', 1.5);
legend('KNN Estimated Distribution', 'Real Distribution');
hold off;

knn.m

function estDensity = knn(samples, k)
    % compute density estimation from samples with KNN
    % Input
    %  samples    : DxN matrix of data points
    %  k          : number of neighbors
    % Output
    %  estDensity : estimated density in the range of [-5, 5]

    % Compute the number of the samples created
    N = length(samples);

    % Create a linearly spaced vector
    pos = linspace(-5, 5, 100);

    % Create two big matrices to avoid for loops
    x = repmat(pos, N, 1);
    samples = repmat(samples', 1, length(pos));

    % Sort the distances so that we can choose the k-th point
    dists = sort(abs(x-samples), 1);

    % Estimate the probability density using the k-NN density estimation
    % dists(k, :) = V/2;    
    res = (k/(2*N)) ./ dists(k, :);

    % Form the output variable
    estDensity = [pos; res];

end

kde.m

function estDensity = kde(samples, h)
    % compute density estimation from samples with KDE
    % Input
    %  samples    : DxN matrix of data points
    %  h          : (half) window size/radius of kernel
    % Output
    %  estDensity : estimated density in the range of [-5,5]

    % Compute the number of samples created
    N = length(samples);

    % Create a linearly spaced vector
    pos = linspace(-5, 5, 100);

    % Create two big matrices to avoid for loops
    x = repmat(pos, N, 1);
    samples = repmat(samples', 1, length(pos));

    % Estimate the density from the samples using a kernel density estimator
    % 参考机器学习（二）非参数估计核函数法 高斯函数一例
    res = sum(exp(-(x-samples).^2./(2*h^2)), 1) ./ (sqrt(2*pi)*h*N);

    % Form the output variable
    estDensity = [pos; res];

end

gauss1D.m

function [realDensity] = gauss1D(m, v, N, w)

    pos = (-w:(2*w/N):w-w/N);
    meanV = repmat(m,N,1)';

    aux = pos - meanV;
    insE = (aux.*aux)./(v^2)*(-0.5);
    norm = 1/(v*sqrt(2*pi));

    res = norm.*exp(insE);

    realDensity = [pos;res];

end

parameter.m

function p = parameters()
    p.k = 30; %knn neighbors
    p.h = 0.3; %kde windowsize/radius
end

---

结果如下：

图8 核函数估计概率密度结果（红色曲线为估计值，蓝色曲线为理想值）

图9 K近邻法估计概率密度结果（红色曲线为估计值，蓝色曲线为理想值）

---