集体智慧编程-皮尔逊相关系数代码理解

最新推荐文章于 2024-01-24 18:02:42 发布

eye_water

最新推荐文章于 2024-01-24 18:02:42 发布

阅读量1.3k

点赞数 1

分类专栏：集体智慧编程代码详解

本文链接：https://blog.csdn.net/eye_water/article/details/78976659

版权

集体智慧编程代码详解专栏收录该内容

3 篇文章 0 订阅

订阅专栏

刚开始看关于皮尔逊相关系数计算的代码，把我看得是晕头转向，不过在学习完概率论的课程后，发现结合公式再来看代码就会比较简单了。

期望公式

E (x) = 1 n \sum i = 1 n x i

$E\left( x\right) =\dfrac {1}{n}\sum ^{n}_{i=1}x{i}$
方差公式

v a r (x) = E {[x - E (x)] 2}

$v_{ar}\left( x\right) =E\left\{ \left[ x-E\left( x\right) \right] ^{2}\right\}$

= E {x 2 - 2 x \times E (x) + [E (x)]}

$=E\left\{ x^{2}-2x\times E\left( x\right) +\left[ E\left( x\right) \right] \right\}$

= E (x 2) - 2 E (x) E (x) + [E (x)] 2

$=E\left( x^{2}\right) -2E\left( x\right) E\left( x\right) +\left[ E\left( x\right) \right] ^{2}$

= E (x 2) - [E (x)] 2

$=E\left( x^{2}\right) -\left[ E\left( x\right) \right] ^{2}$
期望性质

E (c) = c

$E\left( c\right) =c$
因此对于上诉方差推导公式

E {- 2 x \times E (x)} = - 2 E (x) E (E (x)) = - 2 E (x) E (x)

$E\left\{ -2x\times E\left( x\right) \right\}=-2E\left( x\right) E\left( E\left( x\right) \right) \\ =-2E\left( x\right) E\left( x\right)$
因为

E (x) = c

$E\left( x\right) =c$
即

E {E (x)} = E (x)

$E\left\{ E\left( x\right) \right\} =E\left( x\right)$
推导方差公式有利于下面皮尔逊相关系数的推导
皮尔逊相关系数计算公式

p x y = c o v ( x , y ) v a r ( x ) v a r ( y ) - - - - - - - - - - - \sqrt

$pxy=\dfrac {cov\left( x,y\right) }{\sqrt {var\left( x\right) var\left( y\right) }}$
协方差公式

c o v (x, y) = E {[x - E (x)] [y - E (y)]}

$cov\left( x,y\right) =E\left\{ \left[ x-E\left( x\right) \right] \left[ y-E\left( y\right) \right] \right\}$

= E {X Y - X E (Y) - Y E (X) + E (X) E (Y)}

$=E\left\{ XY-XE\left( Y\right) -YE\left( X\right) +E\left( X\right) E\left( Y\right) \right\}$

= E (X Y) - E (X) E (Y)

$=E\left( XY\right) -E\left( X\right) E\left( Y\right)$
书中第三章关于皮尔逊相关系数的代码如下

sum1 = sum(v1)
sum2 = sum(v2)
sum1sq = sum([pow(v, 2) for v in v1])
sum2sq = sum([pow(v, 2) for v in v2])
pSum = sum([v1[i]*v2[i] for i in range(len(v1))])
num = pSum - (sum1*sum2/len(v1))
den = sqrt((sum1sq - pow(sum1, 2)/len(v1)) * (sum2sq - pow(sum2, 2)/len(v)))
if den == 0:
    return 0
return num/den

变量的意义（i=j）为了公式推导方便，在下面的式子中会用i代替j

sum1

s u m 1 = \sum i = 1 n x i

$sum1=\sum ^{n}_{i=1}x_{i}$
sum2

s u m 2 = \sum j = 1 n y j

$sum2=\sum ^{n}_{j=1}y_{j}$
sum1sq

s u m 1 S q = \sum i = 1 n x 2 i

$sum1Sq=\sum ^{n}_{i=1}x^{2}_{i}$
sum2sq

s u m 2 s q = \sum j = 1 n y j 2

$sum2sq=\sum ^{n}_{j=1}yj^{2}$
psum

p s u m = \sum i = 1 n x i y i

$psum=\sum ^{n}_{i=1}x_{i}y_{i}$
num

n u m = \sum i = 1 n x i y i / n - \sum i = 1 n x i / n \times \sum i = 1 n y i / n

$num=\sum ^{n}_{i=1}x_{i}y_{i}/n-\sum ^{n}_{i=1}x_{i}/n\times \sum ^{n}_{i=1}y_{i}/n$

= E (X) E (y) - E (X) E (Y)

$=E\left( X\right) E\left( y\right) -E\left( X\right) E\left( Y\right)$

nden*nden

= ⎡ ⎣ \sum i = 1 n x i 2 - (\sum i = 1 n x i) 2 ⎤ ⎦ ⎡ ⎣ \sum i = 1 n y 2 i - (\sum i = 1 n y i) 2 ⎤ ⎦

$=\left[ \sum ^{n}_{i=1}xi^{2}-\left( \sum ^{n}_{i=1}x_{i}\right) ^{2}\right] \left[ \sum ^{n}_{i=1}y^{2}_{i}-\left( \sum ^{n}_{i=1}y_{i}\right) ^{2}\right]$

= n V a r (X) V a r (Y)

$=nVar\left( X\right) Var\left( Y\right)$
关于den这个变量因为公式太长不好展示只好先处理一下了

这样num/den就可以得到皮尔逊相关系数了

eye_water

关注

1
点赞
踩
4

收藏

觉得还不错? 一键收藏
2
评论
集体智慧编程-皮尔逊相关系数代码理解

刚开始看关于皮尔逊相关系数计算的代码，把我看得是晕头转向，不过在学习完概率论的课程后，发现结合公式再来看代码就会比较简单了。期望公式 E(x)=1n∑i=1nxiE\left( x\right) =\dfrac {1}{n}\sum ^{n}_{i=1}x{i} 方差公式 var(x)=E{[x−E(x)]2}v_{ar}\left( x\right) =E\left\{ \left[
复制链接

扫一扫