随机事件
A
∩
B
=
A
B
,
A
∪
B
=
A
+
B
P
(
A
+
B
)
=
P
(
A
)
+
P
(
B
)
−
P
(
A
B
)
P
(
A
−
B
)
=
P
(
A
B
‾
)
P
(
A
+
B
−
C
)
≠
P
(
A
−
C
+
B
)
,
P
(
A
+
B
−
C
)
≠
P
(
A
+
(
B
−
C
)
)
P
(
A
(
B
−
C
)
)
≠
P
(
A
B
−
A
C
)
\begin{aligned} & A\cap B=AB,A\cup B=A+B \\ & P(A+B)=P(A)+P(B)-P(AB) \\ & P(A-B)=P(A\overline{B}) \\ & P(A+B-C)\neq P(A-C+B),P(A+B-C)\neq P(A+(B-C)) \\ & P(A(B-C))\neq P(AB-AC) \\ \end{aligned}
A∩B=AB,A∪B=A+BP(A+B)=P(A)+P(B)−P(AB)P(A−B)=P(AB)P(A+B−C)=P(A−C+B),P(A+B−C)=P(A+(B−C))P(A(B−C))=P(AB−AC)
事件运算法则
A
+
A
B
=
A
,
A
−
B
=
A
B
‾
A
(
B
+
C
)
=
A
B
+
A
C
,
A
+
B
C
=
(
A
+
B
)
(
A
+
C
)
A
+
B
+
C
=
A
+
(
B
+
C
)
A
+
B
‾
=
A
‾
B
‾
,
A
B
‾
=
A
‾
+
B
‾
\begin{aligned} & A+AB=A,A-B=A\overline{B} \\ & A(B+C)=AB+AC,A+BC=(A+B)(A+C) \\ & A+B+C=A+(B+C) \\ & \overline{A+B}=\overline{A}\overline{B},\overline{AB}=\overline{A}+\overline{B} \\ \end{aligned}
A+AB=A,A−B=ABA(B+C)=AB+AC,A+BC=(A+B)(A+C)A+B+C=A+(B+C)A+B=AB,AB=A+B
独立
P
(
A
B
)
=
P
(
A
)
P
(
B
)
P(AB)=P(A)P(B)
P(AB)=P(A)P(B),不相容
P
(
A
+
B
)
=
P
(
A
)
+
P
(
B
)
P(A+B)=P(A)+P(B)
P(A+B)=P(A)+P(B),
A
B
=
∅
AB=\emptyset
AB=∅
一元线性回归
y
(
n
)
=
a
∗
x
(
n
)
+
b
∗
J
(
a
,
b
)
=
∑
n
=
1
N
(
y
(
n
)
−
a
x
(
n
)
−
b
)
2
\begin{aligned} & y(n)=a^*x(n)+b^* \\ & J(a,b)=\sum_{n=1}^N(y(n)-ax(n)-b)^2 \\ \end{aligned}
y(n)=a∗x(n)+b∗J(a,b)=n=1∑N(y(n)−ax(n)−b)2
∂
J
∂
b
=
−
2
∑
n
=
1
N
[
y
(
n
)
−
a
x
(
n
)
−
b
]
=
0
∑
n
=
1
N
y
(
n
)
−
a
∑
n
=
1
N
x
(
n
)
−
b
n
=
0
b
=
1
N
(
∑
n
=
1
N
y
(
n
)
−
a
∑
n
=
1
N
x
(
n
)
)
=
y
ˉ
−
a
x
ˉ
∂
J
∂
a
=
−
2
∑
n
=
1
N
x
(
n
)
[
y
(
n
)
−
a
x
(
n
)
−
b
]
=
0
∑
n
=
1
N
x
(
n
)
y
(
n
)
−
a
∑
n
=
1
N
x
2
(
n
)
−
b
∑
n
=
1
N
x
(
n
)
=
0
∑
n
=
1
N
x
y
−
a
∑
x
2
−
∑
x
(
∑
y
−
a
∑
x
)
=
0
∑
x
y
−
∑
x
∑
y
=
a
∑
x
2
−
a
∑
x
∑
x
{
a
=
∑
n
=
1
N
x
(
n
)
y
(
n
)
−
∑
n
=
1
N
x
(
n
)
∑
n
=
1
N
y
(
n
)
∑
n
=
1
N
x
2
(
n
)
−
(
∑
n
=
1
N
x
(
n
)
)
2
b
=
y
ˉ
−
a
x
ˉ
\begin{aligned} & \frac{\partial J}{\partial b} = -2\sum_{n=1}^N[y(n)-ax(n)-b]=0 \\ & \sum_{n=1}^Ny(n)-a\sum_{n=1}^Nx(n)-bn=0 \\ & b=\frac{1}{N}\left(\sum_{n=1}^Ny(n)-a\sum_{n=1}^Nx(n)\right)=\bar{y}-a\bar{x} \\ & \frac{\partial J}{\partial a} = -2\sum_{n=1}^Nx(n)[y(n)-ax(n)-b]=0 \\ & \sum_{n=1}^Nx(n)y(n)-a\sum_{n=1}^Nx^2(n)-b\sum_{n=1}^Nx(n)=0 \\ & \sum_{n=1}^Nxy-a\sum x^2-\sum x \left(\sum y-a\sum x\right)=0 \\ & \sum xy-\sum x\sum y =a\sum x^2-a\sum x\sum x \\ &\begin{cases} a=\frac{\displaystyle\sum_{n=1}^Nx(n)y(n)-\sum_{n=1}^Nx(n)\sum_{n=1}^Ny(n)} {\displaystyle\sum_{n=1}^Nx^2(n)-\left(\sum_{n=1}^Nx(n)\right)^2} \\ b=\bar{y}-a\bar{x} \end{cases} \end{aligned}
∂b∂J=−2n=1∑N[y(n)−ax(n)−b]=0n=1∑Ny(n)−an=1∑Nx(n)−bn=0b=N1(n=1∑Ny(n)−an=1∑Nx(n))=yˉ−axˉ∂a∂J=−2n=1∑Nx(n)[y(n)−ax(n)−b]=0n=1∑Nx(n)y(n)−an=1∑Nx2(n)−bn=1∑Nx(n)=0n=1∑Nxy−a∑x2−∑x(∑y−a∑x)=0∑xy−∑x∑y=a∑x2−a∑x∑x⎩
⎨
⎧a=n=1∑Nx2(n)−(n=1∑Nx(n))2n=1∑Nx(n)y(n)−n=1∑Nx(n)n=1∑Ny(n)b=yˉ−axˉ
分布律
二项分布
X
∼
B
(
n
,
p
)
X\sim B(n,p)
X∼B(n,p)
设事件X发生的概率为p,进行n次实验,事件X发生k次的概率为
P
(
X
=
k
)
=
C
n
k
p
k
(
1
−
p
)
n
−
k
P(X=k)=C_n^kp^k(1-p)^{n-k}
P(X=k)=Cnkpk(1−p)n−k
E
(
X
)
=
n
p
,
D
X
=
n
p
(
1
−
p
)
E(X)=np,DX=np(1-p)
E(X)=np,DX=np(1−p)
几何分布
X
∼
G
(
p
)
X\sim G(p)
X∼G(p)
设事件X发生的概率为p,事件X首次发生时的实验次数k满足
P
(
X
=
k
)
=
p
(
1
−
p
)
k
−
1
P(X=k)=p(1-p)^{k-1}
P(X=k)=p(1−p)k−1
超几何分布
X
∼
H
(
n
,
M
,
N
)
X\sim H(n,M,N)
X∼H(n,M,N)
N件产品中有m件次品,从中任取n件产品不放回,有X件次品的概率为
P
(
X
=
k
)
=
C
M
k
⋅
C
N
−
M
n
−
k
C
N
n
P(X=k)=\frac{C_M^k\cdot C_{N-M}^{n-k}}{C_N^n}
P(X=k)=CNnCMk⋅CN−Mn−k
泊松分布
X
∼
P
(
λ
)
X\sim P(\lambda)
X∼P(λ)
P
(
X
=
k
)
=
λ
k
e
−
λ
k
!
lim
N
→
+
∞
C
M
k
⋅
C
N
−
M
n
−
k
C
N
n
=
C
n
k
p
k
(
1
−
p
)
n
−
k
lim
n
→
+
∞
C
n
k
p
k
(
1
−
p
)
n
−
k
=
λ
k
e
−
λ
k
!
\begin{aligned} & P(X=k)=\frac{\lambda^k\text{e}^{-\lambda}}{k!} \\ & \lim_{N\to+\infty}\frac{C_M^k\cdot C_{N-M}^{n-k}}{C_N^n}=C_n^kp^k(1-p)^{n-k} \\ & \lim_{n\to+\infty}C_n^kp^k(1-p)^{n-k}=\frac{\lambda^k\text{e}^{-\lambda}}{k!} \\ \end{aligned}
P(X=k)=k!λke−λN→+∞limCNnCMk⋅CN−Mn−k=Cnkpk(1−p)n−kn→+∞limCnkpk(1−p)n−k=k!λke−λ
证明如下
P
(
X
=
k
)
=
C
n
k
p
k
(
1
−
p
)
n
−
k
E
(
x
)
=
n
p
=
λ
lim
n
→
+
∞
C
n
k
p
k
(
1
−
p
)
n
−
k
=
lim
n
→
+
∞
n
(
n
−
1
)
(
n
−
2
)
⋯
(
n
−
k
+
1
)
k
!
(
λ
n
)
k
(
1
−
λ
n
)
n
−
k
=
λ
k
k
!
lim
n
→
+
∞
1
(
1
−
1
n
)
(
1
−
2
n
)
⋯
(
1
−
k
n
)
(
1
−
λ
n
)
n
(
1
−
λ
n
)
−
k
=
λ
k
k
!
lim
n
→
+
∞
(
1
−
λ
n
)
n
=
λ
k
e
−
λ
k
!
\begin{aligned} & P(X=k)=\text{C}_n^kp^k(1-p)^{n-k} \\ & \text{E}(x)=np=\lambda \\ &\lim_{n\rightarrow+\infty}\text{C}_n^kp^k(1-p)^{n-k} \\ =&\lim_{n\rightarrow+\infty}\frac{n(n-1)(n-2)\cdots(n-k+1)}{k!}\left(\frac{\lambda}{n}\right)^k\left(1-\frac{\lambda}{n}\right)^{n-k} \\ =&\frac{\lambda^k}{k!}\lim_{n\rightarrow+\infty}1(1-\frac{1}{n})(1-\frac{2}{n})\cdots(1-\frac{k}{n})\left(1-\frac{\lambda}{n}\right)^n\left(1-\frac{\lambda}{n}\right)^{-k} \\ =&\frac{\lambda^k}{k!}\lim_{n\rightarrow+\infty}\left(1-\frac{\lambda}{n}\right)^n \\ =&\frac{\lambda^k\text{e}^{-\lambda}}{k!} \end{aligned}
====P(X=k)=Cnkpk(1−p)n−kE(x)=np=λn→+∞limCnkpk(1−p)n−kn→+∞limk!n(n−1)(n−2)⋯(n−k+1)(nλ)k(1−nλ)n−kk!λkn→+∞lim1(1−n1)(1−n2)⋯(1−nk)(1−nλ)n(1−nλ)−kk!λkn→+∞lim(1−nλ)nk!λke−λ
均匀分布
X ∼ U [ a , b ] X\sim U[a,b] X∼U[a,b]
指数分布
X
∼
e
(
λ
)
X\sim e(\lambda)
X∼e(λ)
f
(
x
)
=
{
λ
e
−
λ
x
,
x
≥
0
0
,
x
<
0
f(x)=\begin{cases} \lambda\text{e}^{-\lambda x} &, x\ge 0 \\ 0 &, x<0 \end{cases}
f(x)={λe−λx0,x≥0,x<0
指数分布与泊松分布的关系
P
(
Y
>
t
)
=
P
(
X
=
0
,
t
)
=
(
λ
t
)
0
0
!
e
−
λ
t
=
e
−
λ
t
,
t
≥
0
F
(
x
)
=
1
−
e
−
λ
x
,
x
≥
0
\begin{aligned} & P(Y>t)=P(X=0,t)=\frac{\left(\lambda t\right)^0}{0!}e^{-\lambda t}=e^{-\lambda t},\quad t\ge 0 \\ & F(x)=1-\text{e}^{-\lambda x},x\ge 0 \\ \end{aligned}
P(Y>t)=P(X=0,t)=0!(λt)0e−λt=e−λt,t≥0F(x)=1−e−λx,x≥0
正态分布
X ∼ N ( μ , σ ) f ( x ) = 1 2 π σ exp { − ( x − μ ) 2 2 σ 2 } Φ ( x ) = ∫ − ∞ x 1 2 π e − t 2 2 d t X\sim N(\mu,\sigma) \\ f(x)=\frac{1}{\sqrt{2\pi}\sigma} \text{exp}\left\{-\frac{(x-\mu)^2}{2\sigma^2}\right\} \\ \Phi(x)=\int_{-\infty}^x\frac{1}{\sqrt{2\pi}}\text{e}^{-\frac{t^2}{2}}\text{d}t X∼N(μ,σ)f(x)=2πσ1exp{−2σ2(x−μ)2}Φ(x)=∫−∞x2π1e−2t2dt
数学期望和方差
Cov ( X , X ) = D X Cov ( X , Y ) = E [ ( X − E X ) ( Y − E Y ) ] = E ( X Y ) − E X ⋅ E Y ρ x y = Cov ( X , Y ) D X D Y E ( X ± Y ) = E X ± E Y D ( X ± Y ) = D X + D Y ± 2 Cov ( X , Y ) Cov ( X , Y 1 ± Y 2 ) = Cov ( X , Y 1 ) ± Cov ( X , Y 2 ) Cov ( X , k Y ) = k Cov ( X , Y ) \begin{aligned} \text{Cov}(X,X) &= \text{D}X \\ \text{Cov}(X,Y) &= \text{E}[(X-\text{E}X)(Y-\text{E}Y)] =\text{E}(XY)-\text{E}X\cdot\text{E}Y \\ \rho_{xy} &= \frac{\operatorname{Cov}(X, Y)}{\sqrt{\text{D} X} \sqrt{\text{D} Y}} \\ \text{E}(X\pm Y) &= \text{E}X\pm \text{E}Y \\ \text{D}(X\pm Y) &= \text{D}X+\text{D}Y\pm 2\text{Cov}(X,Y) \\ \text{Cov}(X,Y_1\pm Y_2) &= \text{Cov}(X,Y_1)\pm\text{Cov}(X,Y_2) \\ \text{Cov}(X,kY) &= k\text{Cov}(X,Y) \\ \end{aligned} Cov(X,X)Cov(X,Y)ρxyE(X±Y)D(X±Y)Cov(X,Y1±Y2)Cov(X,kY)=DX=E[(X−EX)(Y−EY)]=E(XY)−EX⋅EY=DXDYCov(X,Y)=EX±EY=DX+DY±2Cov(X,Y)=Cov(X,Y1)±Cov(X,Y2)=kCov(X,Y)
切比雪夫不等式
马尔可夫不等式
P
{
X
>
ε
}
≤
E
X
ε
P\{X>\varepsilon\}\le\frac{E X}{\varepsilon}
P{X>ε}≤εEX
其中
X
≥
0
X\ge0
X≥0
E
x
=
∫
0
∞
x
f
(
x
)
d
x
⩾
∫
ε
∞
x
f
(
x
)
d
x
≥
ε
∫
ε
∞
f
(
x
)
d
x
=
P
{
x
⩾
ε
}
\begin{aligned} E x &=\int_{0}^{\infty} x f(x) d x \\ & \geqslant \int_{\varepsilon}^{\infty} x f(x) d x \\ &\ge\varepsilon\int_{\varepsilon}^{\infty} f(x) d x \\ &=P\{x \geqslant \varepsilon\} \end{aligned}
Ex=∫0∞xf(x)dx⩾∫ε∞xf(x)dx≥ε∫ε∞f(x)dx=P{x⩾ε}
切比雪夫不等式
P
{
∣
X
−
E
X
∣
⩾
ε
}
⩽
D
X
ε
2
P\left\{|X-EX|\geqslant \varepsilon\right\} \leqslant\frac{DX}{\varepsilon^{2}}
P{∣X−EX∣⩾ε}⩽ε2DX
样本方差的无偏估计
1 n − 1 ∑ i = 1 n ( x i − x ˉ ) 2 = 1 n − 1 ∑ i = 1 n [ ( x i − μ ) − ( x ˉ − μ ) ] 2 = 1 n − 1 ∑ i = 1 n ( x i − μ ) 2 − 2 n − 1 ∑ i = 1 n ( x i − μ ) ( x ˉ − μ ) + 1 n − 1 ∑ i = 1 n ( x ˉ − μ ) 2 = 1 n − 1 ∑ i = 1 n ( x i − μ ) 2 − 2 ( x ˉ − μ ) n − 1 ( n x ˉ − n μ ) + n n − 1 ( x ˉ − μ ) 2 = 1 n − 1 ∑ i = 1 n ( x i − μ ) 2 − n n − 1 ( x ˉ − μ ) 2 E [ 1 n − 1 ∑ i = 1 n ( x i − x ˉ ) 2 ] = E [ 1 n − 1 ∑ i = 1 n ( x i − μ ) 2 − n n − 1 ( x ˉ − μ ) 2 ] = n n − 1 D x − n n − 1 ⋅ D x n = D x \begin{aligned} \frac{1}{n-1}\sum_{i=1}^{n}(x_{i}-\bar{x})^{2} &= \frac{1}{n-1}\sum_{i=1}^{n}[(x_{i}-\mu)-(\bar{x}-\mu)]^{2} \\ &=\frac{1}{n-1} \sum_{i=1}^{n}(x_{i}-\mu)^{2}-\frac{2}{n-1}\sum_{i=1}^{n}(x_{i}-\mu)(\bar{x}-\mu) +\frac{1}{n-1}\sum_{i=1}^{n}(\bar{x}-\mu)^{2} \\ &= \frac{1}{n-1} \sum_{i=1}^{n}(x_{i}-\mu)^{2}-\frac{2(\bar{x}-\mu)}{n-1}(n \bar{x}-n \mu)+\frac{n}{n-1}(\bar{x}-\mu)^{2} \\ &= \frac{1}{n-1} \sum_{i=1}^{n}(x_{i}-\mu)^{2}-\frac{n}{n-1}(\bar{x}-\mu)^{2} \\ \text{E}\left[\frac{1}{n-1}\sum_{i=1}^{n}(x_{i}-\bar{x})^{2}\right] &= \text{E}\left[\frac{1}{n-1} \sum_{i=1}^{n}(x_{i}-\mu)^{2}-\frac{n}{n-1}(\bar{x}-\mu)^{2}\right] \\ &=\frac{n}{n-1}\text{D}x-\frac{n}{n-1}\cdot\frac{\text{D}x}{n}=\text{D}x \end{aligned} n−11i=1∑n(xi−xˉ)2E[n−11i=1∑n(xi−xˉ)2]=n−11i=1∑n[(xi−μ)−(xˉ−μ)]2=n−11i=1∑n(xi−μ)2−n−12i=1∑n(xi−μ)(xˉ−μ)+n−11i=1∑n(xˉ−μ)2=n−11i=1∑n(xi−μ)2−n−12(xˉ−μ)(nxˉ−nμ)+n−1n(xˉ−μ)2=n−11i=1∑n(xi−μ)2−n−1n(xˉ−μ)2=E[n−11i=1∑n(xi−μ)2−n−1n(xˉ−μ)2]=n−1nDx−n−1n⋅nDx=Dx
n维正态随机变量
二维正态随机变量
(
X
1
,
X
2
)
(X_1,X_2)
(X1,X2)的概率密度为
f
(
x
1
,
x
2
)
=
1
2
π
σ
1
σ
2
1
−
ρ
2
exp
{
−
1
2
(
1
−
ρ
2
)
[
(
x
1
−
μ
1
)
2
σ
1
2
−
2
ρ
(
x
1
−
μ
1
)
(
x
2
−
μ
2
)
σ
1
σ
2
+
(
x
2
−
μ
2
)
2
σ
2
2
]
}
=
1
2
π
det
1
/
2
C
exp
{
−
1
2
[
x
1
−
μ
1
x
2
−
μ
2
]
⊤
C
−
1
[
x
1
−
μ
1
x
2
−
μ
2
]
}
\begin{aligned} f(x_1,x_2)=& \frac{1}{2\pi\sigma_1\sigma_2\sqrt{1-\rho^2}}\exp\left\{\frac{-1}{2(1-\rho^{2})}\left[\frac{(x_1-\mu_1)^2}{\sigma_1^2} -2 \rho \frac{\left(x_{1}-\mu_{1}\right)\left(x_{2}-\mu_{2}\right)}{\sigma_{1} \sigma_{2}}+\frac{\left(x_{2}-\mu_{2}\right)^{2}}{\sigma_{2}^{2}}\right]\right\} \\ =& \frac{1}{2\pi\text{det}^{1/2}\mathbf{C}}\exp\left\{-\frac{1}{2} \left[\begin{matrix} x_1-\mu_1 \\ x_2-\mu_2 \end{matrix}\right]^\top\mathbf{C}^{-1}\left[\begin{matrix} x_1-\mu_1 \\ x_2-\mu_2 \end{matrix}\right]\right\} \end{aligned}
f(x1,x2)==2πσ1σ21−ρ21exp{2(1−ρ2)−1[σ12(x1−μ1)2−2ρσ1σ2(x1−μ1)(x2−μ2)+σ22(x2−μ2)2]}2πdet1/2C1exp{−21[x1−μ1x2−μ2]⊤C−1[x1−μ1x2−μ2]}
其中的协方差矩阵
C
\mathbf{C}
C为
C
=
[
σ
1
2
ρ
σ
1
σ
2
ρ
σ
1
σ
2
σ
2
2
]
=
[
D
X
1
Cov
(
X
1
,
X
2
)
Cov
(
X
1
,
X
2
)
D
X
2
]
\mathbf{C}=\left[\begin{matrix} \sigma_1^2 & \rho\sigma_1\sigma_2 \\ \rho\sigma_1\sigma_2 & \sigma_2^2 \\ \end{matrix}\right] =\left[\begin{matrix} \text{D}X_1 & \text{Cov}(X_1,X_2) \\ \text{Cov}(X_1,X_2) & \text{D}X_2 \\ \end{matrix}\right]
C=[σ12ρσ1σ2ρσ1σ2σ22]=[DX1Cov(X1,X2)Cov(X1,X2)DX2]
n维正态随机变量的概率密度为
f
(
x
)
=
1
(
2
π
)
n
/
2
det
1
/
2
C
exp
{
−
1
2
(
x
−
μ
)
⊤
C
−
1
(
x
−
μ
)
}
f(\boldsymbol{x})=\frac{1}{(2 \pi)^{n/2}\text{det}^{1/2}\mathbf{C}}\exp \left\{-\frac{1}{2}(\boldsymbol{x}-\boldsymbol{\mu})^{\top}\mathbf{C}^{-1}(\boldsymbol{x}-\boldsymbol{\mu})\right\}
f(x)=(2π)n/2det1/2C1exp{−21(x−μ)⊤C−1(x−μ)}