Introduction to Computational Applied Statistics

This course is provided from Mcgill University summer short program, Machine Learning program.
This is my own lecture notes.

---

Consider the two distributions in the figure just below. Within each graph, the values on the right side of the distribution taper differently from the values on the left side. These tapering sides are called tails, and they provide a visual means to determine which of the two kinds of skewness a distribution has:
negative skew: The left tail is longer; the mass of the distribution is concentrated on the right of the figure. The distribution is said to be left-skewed, left-tailed, or skewed to the left, despite the fact that the curve itself appears to be skewed or leaning to the right; left instead refers to the left tail being drawn out and, often, the mean being skewed to the left of a typical center of the data. A left-skewed distribution usually appears as a right-leaning curve.[1]

positive skew: The right tail is longer; the mass of the distribution is concentrated on the left of the figure. The distribution is said to be right-skewed, right-tailed, or skewed to the right, despite the fact that the curve itself appears to be skewed or leaning to the left; right instead refers to the right tail being drawn out and, often, the mean being skewed to the right of a typical center of the data. A right-skewed distribution usually appears as a left-leaning curve.[1]

Skewness in a data series may sometimes be observed not only graphically but by simple inspection of the values.

---

平均值(Mean)、方差(Variance)和标准差(Standard Deviation)

标准差n-1 本身n个自由度减去 均值X 一个自由度 共n-1个自由度

---

在概率论和统计学中，变异系数，又称“离散系数”（英文：coefficient of variation），是概率分布离散程度的一个归一化量度，其定义为标准差 σ 与平均值 μ 之比：

In probability theory and statistics, the coefficient of variation (CV), also known as relative standard deviation (RSD), is a standardized measure of dispersion of a probability distribution or frequency distribution. It is often expressed as a percentage, and is defined as the ratio of the standard deviation σ to the mean μ or its absolute value, | μ |

变异系数（coefficient of variation）只在平均值不为零时有定义，而且一般适用于平均值大于零的情况。变异系数也被称为标准离差率或单位风险。

变异系数只对由比率标量计算出来的数值有意义。举例来说，对于一个气温的分布，使用开尔文或摄氏度来计算的话并不会改变标准差的值，但是温度的平均值会改变，因此使用不同的温标的话得出的变异系数是不同的。也就是说，使用区间标量得到的变异系数是没有意义的。

---

Population variance (σ2) 总体方差
sample variance s2 样本方差
SD standard deviation

---

Measurement of Relationship

In mathematics and statistics, covariance is a measure of the relationship between two random variables. The metric evaluates how much – to what extent – the variables change together. In other words, it is essentially a measure of the variance between two variables (note that the variance of one variable equals the variance of the other variable). However, the metric does not assess the dependency between variables.
协方差（Covariance）在概率论和统计学中用于衡量两个变量的总体误差。而方差是协方差的一种特殊情况，即当两个变量是相同的情况。
协方差表示的是两个变量的总体的误差，这与只表示一个变量误差的方差不同。 如果两个变量的变化趋势一致，也就是说如果其中一个大于自身的期望值，另外一个也大于自身的期望值，那么两个变量之间的协方差就是正值。 如果两个变量的变化趋势相反，即其中一个大于自身的期望值，另外一个却小于自身的期望值，那么两个变量之间的协方差就是负值。



若两个随机变量X和Y相互独立，则E[(X-E(X))(Y-E(Y))]=0，因而若上述数学期望不为零，则X和Y必不是相互独立的，亦即它们之间存在着一定的关系。 [2]
协方差与方差之间有如下关系：
D(X+Y)=D(X)+D(Y)+2Cov(X，Y)
D(X-Y)=D(X)+D(Y)-2Cov(X，Y)
协方差与期望值有如下关系：
Cov(X，Y)=E(XY)-E(X)E(Y)。
协方差的性质：
（1）Cov(X，Y)=Cov(Y，X)；
（2）Cov(aX，bY)=abCov(X，Y)，（a，b是常数）；
（3）Cov(X1+X2，Y)=Cov(X1，Y)+Cov(X2，Y)。
由协方差定义，可以看出Cov(X，X)=D(X)，Cov(Y，Y)=D(Y)。

---

What is the Correlation Coefficient?

The correlation coefficient is a statistical measure that calculates the strength of the relationship between the relative movements of two variables. The values range between -1.0 and 1.0.

称为随机变量X和Y的(Pearson)相关系数。
在统计学中，皮尔逊积矩相关系数（英语：Pearson product-moment correlation coefficient，又称作 PPMCC或PCCs[1], 文章中常用r或Pearson’s r表示）用于度量两个变量X和Y之间的相关程度（线性相关），其值介于-1与1之间。

分别为m与n个标量元素的列向量随机变量X与Y，这两个变量之间的协方差定义为m×n矩阵.其中X包含变量X1.X2…Xm，Y包含变量Y1.Y2…Yn，假设X1的期望值为μ1，Y2的期望值为v2，那么在协方差矩阵中（1,2）的元素就是X1和Y2的协方差。
两个向量变量的协方差Cov(X,Y)与Cov(Y,X)互为转置矩阵。
协方差有时也称为是两个随机变量之间“线性独立性”的度量，但是这个含义与线性代数中严格的线性独立性线性独立不同。

如果协方差为正，说明X，Y同向变化，协方差越大说明同向程度越高；如果协方差为负，说明X，Y反向运动，协方差越小说明反向程度越高。

相关系数也可以看成协方差：一种剔除了两个变量量纲影响、标准化后的特殊协方差。既然是一种特殊的协方差，那它
1、也可以反映两个变量变化时是同向还是反向，如果同向变化就为正，反向变化就为负。
2、由于它是标准化后的协方差，因此更重要的特性来了：它消除了两个变量变化幅度的影响，而只是单纯反应两个变量每单位变化时的相似程度。

所以标准差描述了变量在整体变化过程中偏离均值的幅度。协方差除以标准差，也就是把协方差中变量变化幅度对协方差的影响剔除掉，这样协方差也就标准化了，它反应的就是两个变量每单位变化时的情况。