自用的概率论与统计笔记

随机事件

$$\begin{array}{rl}& A\cap B=AB,A\cup B=A+B\\ & P(A+B)=P(A)+P(B)-P(AB)\\ & P(A-B)=P(A\overline{B})\\ & P(A+B-C)\ne P(A-C+B),P(A+B-C)\ne P(A+(B-C))\\ & P(A(B-C))\ne P(AB-AC)\end{array}$$
事件运算法则
$$\begin{array}{rl}& A+AB=A,A-B=A\overline{B}\\ & A(B+C)=AB+AC,A+BC=(A+B)(A+C)\\ & A+B+C=A+(B+C)\\ & \overline{A+B}=\overline{A}\overline{B},\overline{AB}=\overline{A}+\overline{B}\end{array}$$
独立$P(AB)=P(A)P(B)$，不相容$P(A+B)=P(A)+P(B)$，$AB=\varnothing$

一元线性回归

$$\begin{array}{rl}& y(n)=a^{\ast }x(n)+b^{\ast }\\ & J(a,b)=\sum _{n=1}^N(y(n)-ax(n)-b{)}^2\end{array}$$
$$\begin{array}{rl}& \frac{\partial J}{\partial b}=-2\sum _{n=1}^N[y(n)-ax(n)-b]=0\\ & \sum _{n=1}^{N}y(n)-a\sum _{n=1}^{N}x(n)-bn=0\\ & b=\frac{1}{N}\left(\sum _{n=1}^{N}y(n)-a\sum _{n=1}^{N}x(n)\right)=\bar{y}-a\bar{x}\\ & \frac{\partial J}{\partial a}=-2\sum _{n=1}^{N}x(n)[y(n)-ax(n)-b]=0\\ & \sum _{n=1}^{N}x(n)y(n)-a\sum _{n=1}^N{x}^2(n)-b\sum _{n=1}^{N}x(n)=0\\ & \sum _{n=1}^{N}xy-a\sum x^2-\sum x\left(\sum y-a\sum x\right)=0\\ & \sum xy-\sum x\sum y=a\sum x^2-a\sum x\sum x\\ & \{\begin{array}{l}a=\frac{\sum _{n=1}^{N}x(n)y(n)-\sum _{n=1}^{N}x(n)\sum _{n=1}^{N}y(n)}{\sum _{n=1}^N{x}^2(n)-{\left(\sum _{n=1}^{N}x(n)\right)}^2}\\ b=\bar{y}-a\bar{x}\end{array}\end{array}$$

分布律

二项分布

$X\sim B(n,p)$
设事件X发生的概率为p，进行n次实验，事件X发生k次的概率为$P(X=k)=C_n^k{p}^k(1-p{)}^{n-k}$
$E(X)=np,DX=np(1-p)$

几何分布

$$X\sim G(p)$$
设事件X发生的概率为p，事件X首次发生时的实验次数k满足$P(X=k)=p(1-p{)}^{k-1}$

超几何分布

$$X\sim H(n,M,N)$$
N件产品中有m件次品，从中任取n件产品不放回，有X件次品的概率为
$$P(X=k)=\frac{C_M^k\cdot C_{N-M}^{n-k}}{C_N^n}$$

泊松分布

$$X\sim P(\lambda )$$
$$\begin{array}{rl}& P(X=k)=\frac{{\lambda }^k{\text{e}}^{-\lambda }}{k!}\\ & \underset{N\to +\infty }{\lim }\frac{C_M^k\cdot C_{N-M}^{n-k}}{C_N^n}=C_n^k{p}^k(1-p{)}^{n-k}\\ & \underset{n\to +\infty }{\lim }{C}_n^k{p}^k(1-p{)}^{n-k}=\frac{{\lambda }^k{\text{e}}^{-\lambda }}{k!}\end{array}$$
证明如下
$$\begin{array}{rl}& P(X=k)={\text{C}}_n^k{p}^k(1-p{)}^{n-k}\\ & \text{E}(x)=np=\lambda \\ & \underset{n\to +\infty }{\lim }{\text{C}}_n^k{p}^k(1-p{)}^{n-k}\\ =& \underset{n\to +\infty }{\lim }\frac{n(n-1)(n-2)\cdots (n-k+1)}{k!}{\left(\frac{\lambda }{n}\right)}^k{\left(1-\frac{\lambda }{n}\right)}^{n-k}\\ =& \frac{{\lambda }^k}{k!}\underset{n\to +\infty }{\lim }1(1-\frac{1}{n})(1-\frac{2}{n})\cdots (1-\frac{k}{n}){\left(1-\frac{\lambda }{n}\right)}^n{\left(1-\frac{\lambda }{n}\right)}^{-k}\\ =& \frac{{\lambda }^k}{k!}\underset{n\to +\infty }{\lim }{\left(1-\frac{\lambda }{n}\right)}^n\\ =& \frac{{\lambda }^k{\text{e}}^{-\lambda }}{k!}\end{array}$$

均匀分布

$$X\sim U[a,b]$$

指数分布

$$X\sim e(\lambda )$$
$$f(x)=\{\begin{array}{ll}\lambda {\text{e}}^{-\lambda x}& ,x\ge 0\\ 0& ,x<0\end{array}$$
指数分布与泊松分布的关系
$$\begin{array}{rl}& P(Y>t)=P(X=0,t)=\frac{{\left(\lambda t\right)}^0}{0!}{e}^{-\lambda t}=e^{-\lambda t},t\ge 0\\ & F(x)=1-{\text{e}}^{-\lambda x},x\ge 0\end{array}$$

正态分布

$$\begin{gathered} X\sim N(\mu ,\sigma ) \\ f(x)=\frac{1}{\sqrt{2\pi }\sigma }\text{exp}\left\{-\frac{(x-\mu {)}^2}{2{\sigma }^2}\right\} \\ \Phi (x)={\int }_{-\infty }^x\frac{1}{\sqrt{2\pi }}{\text{e}}^{-\frac{t^2}{2}}\text{d}t \end{gathered}$$

数学期望和方差

$$\begin{array}{rl}\text{Cov}(X,X)& =\text{D}X\\ \text{Cov}(X,Y)& =\text{E}[(X-\text{E}X)(Y-\text{E}Y)]=\text{E}(XY)-\text{E}X\cdot \text{E}Y\\ {\rho }_{xy}& =\frac{\mathrm{Cov}(X,Y)}{\sqrt{\text{D}X}\sqrt{\text{D}Y}}\\ \text{E}(X\pm Y)& =\text{E}X\pm \text{E}Y\\ \text{D}(X\pm Y)& =\text{D}X+\text{D}Y\pm 2\text{Cov}(X,Y)\\ \text{Cov}(X,Y_1\pm Y_2)& =\text{Cov}(X,Y_1)\pm \text{Cov}(X,Y_2)\\ \text{Cov}(X,kY)& =k\text{Cov}(X,Y)\end{array}$$

切比雪夫不等式

马尔可夫不等式
P { X > ε } ≤ E X ε P\{X>\varepsilon\}\le\frac{E X}{\varepsilon} P{X>ε}≤εEX​
其中$X\ge 0$
E x = ∫ 0 ∞ x f ( x ) d x ⩾ ∫ ε ∞ x f ( x ) d x ≥ ε ∫ ε ∞ f ( x ) d x = P { x ⩾ ε } \begin{aligned} E x &=\int_{0}^{\infty} x f(x) d x \\ & \geqslant \int_{\varepsilon}^{\infty} x f(x) d x \\ &\ge\varepsilon\int_{\varepsilon}^{\infty} f(x) d x \\ &=P\{x \geqslant \varepsilon\} \end{aligned} Ex​=∫0∞​xf(x)dx⩾∫ε∞​xf(x)dx≥ε∫ε∞​f(x)dx=P{x⩾ε}​
切比雪夫不等式
P { ∣ X − E X ∣ ⩾ ε } ⩽ D X ε 2 P\left\{|X-EX|\geqslant \varepsilon\right\} \leqslant\frac{DX}{\varepsilon^{2}} P{∣X−EX∣⩾ε}⩽ε2DX​

样本方差的无偏估计

$$\begin{array}{rl}\frac{1}{n-1}\sum _{i=1}^n(x_i-\bar{x}{)}^2& =\frac{1}{n-1}\sum _{i=1}^n[(x_i-\mu )-(\bar{x}-\mu ){]}^2\\ & =\frac{1}{n-1}\sum _{i=1}^n(x_i-\mu {)}^2-\frac{2}{n-1}\sum _{i=1}^n(x_i-\mu )(\bar{x}-\mu )+\frac{1}{n-1}\sum _{i=1}^n(\bar{x}-\mu {)}^2\\ & =\frac{1}{n-1}\sum _{i=1}^n(x_i-\mu {)}^2-\frac{2(\bar{x}-\mu )}{n-1}(n\bar{x}-n\mu )+\frac{n}{n-1}(\bar{x}-\mu {)}^2\\ & =\frac{1}{n-1}\sum _{i=1}^n(x_i-\mu {)}^2-\frac{n}{n-1}(\bar{x}-\mu {)}^2\\ \text{E}\left[\frac{1}{n-1}\sum _{i=1}^n(x_i-\bar{x}{)}^2\right]& =\text{E}\left[\frac{1}{n-1}\sum _{i=1}^n(x_i-\mu {)}^2-\frac{n}{n-1}(\bar{x}-\mu {)}^2\right]\\ & =\frac{n}{n-1}\text{D}x-\frac{n}{n-1}\cdot \frac{\text{D}x}{n}=\text{D}x\end{array}$$

n维正态随机变量

二维正态随机变量$(X_1,X_2)$的概率密度为
$$\begin{array}{rl}f(x_1,x_2)=& \frac{1}{2\pi {\sigma }_1{\sigma }_2\sqrt{1-{\rho }^2}}\exp \left\{\frac{-1}{2(1-{\rho }^2)}\left[\frac{(x_1-{\mu }_1{)}^2}{{\sigma }_1^2}-2\rho \frac{\left(x_1-{\mu }_1\right)\left(x_2-{\mu }_2\right)}{{\sigma }_1{\sigma }_2}+\frac{{\left(x_2-{\mu }_2\right)}^2}{{\sigma }_2^2}\right]\right\}\\ =& \frac{1}{2\pi {\text{det}}^{1/2}\mathbf{C}}\exp \left\{-\frac{1}{2}{\left[\begin{array}{c}{x}_1-{\mu }_1\\ x_2-{\mu }_2\end{array}\right]}^{\top }{\mathbf{C}}^{-1}\left[\begin{array}{c}{x}_1-{\mu }_1\\ x_2-{\mu }_2\end{array}\right]\right\}\end{array}$$
其中的协方差矩阵$\mathbf{C}$为
$$\mathbf{C}=\left[\begin{array}{cc}{\sigma }_1^2& \rho {\sigma }_1{\sigma }_2\\ \rho {\sigma }_1{\sigma }_2& {\sigma }_2^2\end{array}\right]=\left[\begin{array}{cc}\text{D}{X}_1& \text{Cov}(X_1,X_2)\\ \text{Cov}(X_1,X_2)& \text{D}{X}_2\end{array}\right]$$
n维正态随机变量的概率密度为
$$f(\mathbf{x})=\frac{1}{(2\pi {)}^{n/2}{\text{det}}^{1/2}\mathbf{C}}\exp \left\{-\frac{1}{2}(\mathbf{x}-\mathbf{\mu }{)}^{\top }{\mathbf{C}}^{-1}(\mathbf{x}-\mathbf{\mu })\right\}$$