MLaPP: 02 Probability (univariate)

probability interpretation:

• frequentist interpretation: probabilities represent long run frequencies of events. for example: if we flip the even coin many times, we expect it to land heads about 1/2 the time.
• Bayesian interpretation: probabilities are used to quantify out uncertainty about something, it is fundamentally related to information rather than repeated trials. for example: we believe the coin is equally likely to land heads or tails on the next toss.

 

define: , then:

common DISCRETE distributions:

binomial and Bernoulli distributions:

multi-nomial and multi-noulli distributions:

bi-nomial distribution is to model the cases for two outcomes. to model K-outcome cases, the multinomial is used. for example, for a K-sided die, let  be a random vector, where  is the number of times side j of the die occurs. then x follows:

if n=1, then the distribution follows the following and called: multi-noulli distribution.

summary:

Poisson distribution:

empirical distribution:

common CONTINUOUS distributions:

Gaussian (normal) distribution:

Degenerate pdf:

if the variance of Gaussian distribution goes to 0, the Gaussian becomes infinitely tall and thin spike, centered at the mean:

a more robust "Gaussian" distribution is called Student t distribution:

Laplace distribution (double sided exponential):

Gamma distribution:

Beta distribution: