止于至玄

混合因子分析（MFA），或曰概率主成分分析（PPCA），在数据科学中是一种有效的数据降维方法。但是其用途不啻如此。本文从实用的角度简单介绍下混合因子分析的基本原理。

本文介绍隐式半马尔可夫模型（HSMM）的基本概念与使用。

浅谈狄利克雷分布的概念与简单使用。

简要介绍隐式马尔可夫模型的一般概念，特征与相关问题，并给出常用的解决算法。

以下仅对机器学习领域常用的几种模型、距离进行介绍。

参数估计有两种形式：点估计与区间估计。本文选择几种常用的点估计方法作一些讨论。用于估计未知参数的统计量称为点估计（量）。参数 θθ\theta 的估计量常用 θ^=θ^(x1,x2,…,xn)θ^=θ^(x1,x2,…,xn)\hat{\theta} = \hat{\theta}(x_{1},x_{2}, \dots, x_{n}) 表示，参数 θθ\theta 的可能取值范围称为参数空间，记...

Thanks Cameron Davidson-Pilon for the great work of Bayesian Methods for Hackers: Probabilistic programming and Bayesian Inference.Popular Probability DistributionPoisson DistributonBinary Distr...

Monte Carlo 方法也即随机模拟方法的别称，它的基本原理是：当求解随机事件方式的概率或随机变量的数学期望时，通过设计某种实验，得出某个特定事件发生的频率，使用这个频率来近似表示这一事件发生的概率，从而得到问题的数值解。 可以看出，Monte Carlo 方法包含三个核心问题：构造概率过程、从已知概率分布中抽样、建立估计量。静态 Monte Carlo 方法MCMCMetropo...

当输入空间为欧氏空间火离散集合，特征空间是希尔伯特（Hilbert）空间时，核函数（Kernel function）表示将输入从输入空间映射到特征空间得到的特征向量之间的内积。 核函数的应用十分广泛，例如使用核函数可以学习非线性支持向量机，等价于隐式地在高维的特征空间中学习线性支持向量机，这样的方法称为核技巧。核方法（Kernel method）是一种比支持向量机更为一般的机器学习方法。...

Expectation-Maximization 算法简述高斯混合模型（Gaussian Mixture Model, GMM）用EM算法学习高斯混合模型Expectation-Maximization 算法简述高斯混合模型（Gaussian Mixture Model, GMM）用EM算法学习高斯混合模型感谢李航——《统计学习方法》清华大学出版...

Let’s talk about the Dynamic Movement Primitive (DMP) for robots learning from demonstration. In this article, we make an assumption that you readers all have the background of control theory and robo...

本文简单概括多元高斯分布的定义、性质与应用，内容会持续更新。

Supervised Dimension ReductionGreater dimensionality always brings about more difficult learning tasks. Here we introduce a supervised dimension reduction method based on linear dimension reduction as ...

Laplacian RegularizationIn Least Square learning methods, we calculate the Euclidean distance between sample points to find a classifier plane. However, here we calculate the minimum distance along the...

Logistic Regression & Least Square Probability Classification1. Logistic RegressionLikelihood function, as interpreted by wikipedia: https://en.wikipedia.org/wiki/Likelihood_functionplays one of the...

ℓ1\ell_{1} Constrained Least Squares In sparse learning, ℓ1\ell_{1} constrained LS, also known as Lasso Regression, is a common learning method:

Least Squaresℓ2ℓ2\ell_{2} constrained least squaresLeast SquaresLeast squares regression is a traditional ML algorithm that minimizes the total square error between samples and learning o...

Huber regression In least square learning methods, we make use of ℓ2\ell_{2} loss to make sure that we get a suitable outcome. However, in the robust point of view, it is always better to make use of

Unsupervised Dimension ReductionData with high dimension is always difficult to tackle. One hand is that it requires tremendous computation resource. On the other hand, it is not so objective as the on...

Let’s begin with ID3 decision tree:

聚类是一种常见的无监督学习方法，目的在于从原始无标记数据中提取出分类标记。最简单的代表是K-means聚类，下面给出一个简单例子...

Local Outlier FactorGiven local outlier factors, we can detect the outliers that are always away from most of the samples. In order to outline the algorithm, some concepts must go first:

Bagging (Bootstrap Aggregation)Pruning Classification is one of the simplest classification algorithms. It works just like if-then. However, when aggregating a lot of prunnings we are able to create a...