概率论与数理统计总结

课程分数构成

• 日常表现：25%
• Closed-book final exam 75%

chapter 1 introduction to probability

experiments and Events

• experiment: any process, real or hypothetical, in which the possible outcomes can be identified ahead of time.
• Sample space: a set of all possible outcomes.
  • exhausive: all possible outcomes should be included.
  • mutually exlucsive(互斥的)
• All event: any subset of the sample space

Set Theory

• Member Relation: elements s belongs to Set S $$s\in S$$
• Containment$$\begin{gathered} A\subset B \\ B\supset A \end{gathered}$$
• empty set $$A=\Phi$$
• Countable; Uncountable
• complement A c = { s ∈ S ∣ s ∉ A } A^c = \{s\in S\mid s\notin A\} Ac={s∈S∣s∈/​A}
• union of Sets A ∪ B = { s ∈ S ∣ s ∈ A   o r s ∈ B } A\cup B=\{ s\in S \mid s\in A\ or s\in B\} A∪B={s∈S∣s∈A ors∈B}
  • union of many sets ${\cup }_{i=1}^n{A}_i$
• intersection of Sets $A\cap B$
• difference of Sets$$A-B=A\cap B^c=A-A\cap B$$
• disjoint/mutually exclusive$A\cap B=\varphi$
• Theorem 1.4.11 Partition a Set $$\begin{gathered} A=(A\cap B)\cup (A\cap B^c) \\ B=(A\cap B)\cup (A^C\cap B) \end{gathered}$$
• properties of set operations
  • commutative laws
  • associative laws
  • distributive laws$$\begin{gathered} A\cup (B\cap C)=(A\cup B)\cap (A\cup C) \\ A\cap (B\cup C)=(A\cap B)\cup (A\cap C) \end{gathered}$$
  • morgan’s laws$$\begin{gathered} (A\cup B{)}^c=A^c\cap B^c \\ (A\cap B{)}^c=A^c\cup B^c \end{gathered}$$
  • must occur必然事件

The definition of probability: Axioms and Theorems

• probability: A probabilistic measure, or simply a probability, on a sample space S is a specification of numbers Pr(A) for all event A that satisfy Axioms 1, 2, and 3.
• Not disjoint(share common elements): $$Pr(E\cup F)=Pr(E)+Pr(F)-Pr(E\cap F)$$

finite sample space

• Finite sample space: a sample space with a finite number of possible outcomes 有限样本空间$$Pr(A)=\frac{m}{n}$$
• m is the number of outcomes in an event
• n is the number of outcomes in the sample space

Counting Methods

• Multiplication Rule
• Permutations
  • sampling without replacement没有放回的抽样
  • $$\begin{gathered} P_{n,k}=n(n-1)\cdots (n-k+1)=\frac{n!}{(n-k)!} \\ p=\frac{P_{n,k}}{n^k} \end{gathered}$$
• Combinational Methods
  • $$C_{n,k}=\left(\genfrac{}{}{0ex}{}{n}{k}\right)=\frac{n!}{k!(n-k)!}$$
  • 只关心数目k，不关心抽出k的内部次序。
  • Unordered Sampling with replacement: blood Types and selecting baked goods$$C_{n+k-1,k}$$

Multinomial Coefficients

$$\left(\genfrac{}{}{0ex}{}{n}{n_1{n}_2\cdots n_k}\right)=\frac{n!}{n_1!n_2!n_3!\cdots n_k!}$$

The probability of a Union of Events

• for 3 events$$Pr(A\cup B\cup C)=Pr(A)+Pr(B)+Pr(C)-Pr(A\cap B)-Pr(A\cap C)-Pr(B\cap C)+Pr(A\cap B\cap C)$$

Chapter 2 conditional probability

definition of the conditional probability

$$Pr(A\mid B)=\frac{Pr(A\cap B)}{Pr(B)}$$
Mutiplication tule:
$$Pr(A\cap B)=Pr(A\mid B)Pr(B)$$
More terms:
$$Pr(A\cap B\cap C\cap D)=Pr(A)Pr(B\mid A)Pr(C\mid A\cap B)Pr(D\mid A\cap B\cap C)$$
for complement:
$$\begin{gathered} Pr(A^c\mid B)=1-Pr(A\mid B) \\ Pr(A^c\mid B\cap C)=1-Pr(A\mid B\cap C) \end{gathered}$$

conditional version of Muliplication Rule

$$Pr(A_1\cap A_2\cap \cdots \cap A_n\mid B)=Pr(A_1\mid B)Pr(A_2\mid A_1\cap B)Pr(A_3\mid A_1\cap A_2\cap B)\cdots Pr(A_n\mid A_1\cap A_2\cap \cdots \cap A_{n-1}\cap B)$$

partitions

$$Pr(A)=Pr(B)Pr(A\mid B)+Pr(B^c)Pr(A\mid B^c)$$

Augmented Experiment

• if desired, any experiment can be augmented to include the potential or hypothetical observation of as much additional information as we would find useful to help us calculate any probabilities that we desire.

independent events

• the occurrence of $B$ do not affect $A$, then we say $A$ and $B$ are independent.
• $A$ and $B$ are not independent, but they would be independent if $C$ occur, $A$ and $B$ are conditionally independent given $C$.
• Two events $A$ and $B$ are independent if $$Pr(A\mid B)=Pr(A)$$
• Theorem 2.2.1
  If 2 events $A$ and $B$ are independent, then the events $A$ and $B^c$ Are also independent.
• Mutually Independent Events
• Mutually Exclusive $\ne$ Mutually Independence
• Theorem 2.2.3
  Let $n>1$ and let $A_1,\cdots ,A_n$ be events that are mutually exclusive. The events are also mutually independent if and only if all the events except possibly one of them has probability 0.
  $$Pr(A_i\cap A_j)=Pr(A_i)Pr(A_j)=0$$
  Summary:

1. A collection of events is independent if and only if that some of them occur does not change the probability that any combination of the rest of them occurs.
2. Equivalently, a collection of events is independent if and only if the probability of the intersection of every sub collection is the product of the individual probabilities. $$Pr(A\cap B)=Pr(A)Pr(B)$$
3. The concept of independence has a version conditional on another event.

Bayes’ Theorem

$$Pr(B_i\mid A)=\frac{Pr(A\cap B_i)}{A}=\frac{Pr(B_i)Pr(A\mid B_i)}{\sum _{j=1}^{k}Pr(B_j)Pr(A\mid B_j)}$$

• Prior and posterior probabilities

summary:

1. Definition of Conditional Probability
2. Law of Total Probability
3. Independent Events
4. Bayes’ Theorem

Chapter 3 Random Variables And Distributions

definition of a Random Variable

• Let $S$ be the sample sample space for an experiment. A real-valued function that is defined on $S$ is called a random variable.

The distribution of a random variable

• Let $X$ be a random variable. The distribution of $X$ is the collection of all probabilities of the form $Pr(x\in C)$ for all sets $C$ of real numbers such that {$X\in C$} is an event.
  某随机变量的分布是该随机变量能取到某些值的一系列概率的集合。
• Discrete Distribution / Random Variable
  A dandom variable $X$ is called discreate if it takes a finite number $k$ of different values:$x_1,\cdots ,x_k$, or, ar most, an infinite sequence of different values $x_1,x_2,\cdots$
• Probability function:$$f(x)=Pr(X=x)$$also called probability mass function(p.m.f)
  • The colsure of the set {x:f(x)>0} is called the support of (the distribution of) $X$
• important properties:
  $f(x)\ge 0$
  $\sum _{i=1}^{\infty }f(x_i)=1$
  $Pr(x\in C)=\sum _{x_i\in C}f(x_i)$

Typical Discreate Distributions

• Bernolli Distribution
  • a random variable $Z$ takes only the values 0 and 1 with $Pr(Z=1)=p$
  • $$f(x\mid p)=\{\begin{array}{ll}{p}^x(1-p{)}^{1-x}& \text{x=0,1}\\ 0& otherwise\end{array}$$
  • It is also called 2 points distribution.
• Bernoulli Trials/Process
  • independent and indentically distributed(i.d.d)
  • Every randon variable $X_i$ is Bernoulli distribution
  • e.g.

Continuous Distribution/Random Variable

$$Pr(a\le X\le b)=\underset{a}{\overset{b}{\int }}f(x)dx$$

• Uniformly distribution on intervals$$f(x)=\{\begin{array}{ll}\frac{1}{b-a}& a\le x\le b\\ 0& otherwise\end{array}$$$X\sim U(a,b)$

The cumulative distribution function

The distribution function or cumulative distribution function ($c.d.f$) F of a random variable $X$ is the function$$F(x)=Pr(X\le x)-\infty <x<\infty$$
Properties:

1. None decreasing function
2. A random variable that only takes real numbers.
3. Continuous from the right. $F(x)=F(x^{+})$

Bernoulli c.d.f:$$F(x)=\{\begin{array}{ll}0& x<0\\ 1-p& 0\le x<1\\ 1& x\ge 1\end{array}$$

Bimonial c.d.f:$$F(x)=\{\begin{array}{ll}0& x<0\\ \sum _{i=0}^{sub(x)}\left(\genfrac{}{}{0ex}{}{n}{i}\right)p^i(1-p{)}^{n-i}& 0\le x<n\\ 1& x\ge n\end{array}$$

uniform c.d.f: have the uniform distribution on the interval $[a,b]$$$F(x)=\{\begin{array}{ll}0& x\le a\\ {\int }_a^x\frac{1}{b-a}dy=\frac{x-a}{b-a}& a<x\le b\\ 1& x>b\end{array}$$

Standard Normal c.d.f:
$$F(x)=\Phi (x)=\underset{-\infty }{\overset{x}{\int }}\frac{1}{\sqrt{2\pi }}{e}^{-\frac{t^2}{2}}dt$$
Theorem:
$$\Phi (-x)=1-\Phi (x)$$$$Pr(x>x)=1-F(x)$$$$Pr(x_1<X\le x_2)=F(x_2)-F(x_1)$$$$Pr(X=x)=F(x)-F(x^{-})$$
c.d.f of a Discrete Distribution:
$F(x)$ will have a jump of magnitude $f(x)$ at each possible value $x_i$ of $X$, and $F(x)$ will be constant between every pair of successive jumps.
The c.d.f of a continuous distribution:
$$\begin{gathered} F(x)={\int }_{-\infty }^{x}f(t)dt \\ \frac{dF(x)}{dx}=f(x) \end{gathered}$$