Probability and Information Theory
1 Random Variables
take on different values randomly
2 Proability distribution
a description of how likely a random variable is to take on each of its possible states
3 Discrete variables(离散型变量)and Proability Mass Functions(概率质量函数,PMF)
4 Continuous Variables(连续型变量) and Proability Density Functions(概率密度函数)
5 Marginal Proability(边缘概率)
6 Conditional Proability(条件概率)
7 Independence
8 Conditional Independence
9 Expectation(期望)
linear:
10 Variance(方差) and Covariance(协方差)
11 Covariance matrix
12 Useful Properties of Common Functions
logisitic:
softplus:
some properties:
13 Common Proability Distributions
- Bernoulli Distribution
- Multinoulli Distribution
- normal Distribution
- Exponential and Laplace Distributions
- The Dirac Distribution and Empirical Distribution