Algorithms - Lecture 12 - Randomized Algorithms

1 Linearity of Expectation

1.1 Expectation $E(x)={\sum }_{x}xP(x)$

1.2 $E(X+Y)=E(X)+E(Y)$ for all $X,Y$

1.3 Ex 1, Expected waiting time for heads

• Let the random variable $X$ denote the number of flips of a p-biased coin until we get the first “heads”.
• Indicator variable $X_i=1$ if the number of flips is at least $i$, $0$ otherwise. $E(X_i)=(1-p{)}^{i-1}$, $X_i$s are dependent.
• $X={\sum }_i{X}_i$
• $E(X)={\sum }_i(1-p_i{)}^{i-1}=\frac{1}{p}$

1.4 Ex 2, A coupon collector process

• Suppose we repeatedly draw a uniformly random number from { 1 , … , n } \{1,\dots,n\} {1,…,n} until we have drawn each number at least once.
• What is the expected number of draws?
• Partition into $n$ phases,
  • Phase $i$ begins once $i-1$ distinct integers have been drawn.
  • $X_i$ denote the number of draws in Phase $i$.
  • $X$ denotes ${\sum }_{1\le i\le n}{X}_i$, $X_i$ like flip a $\frac{n-(i-1)}{n}$ biased coin, $E(X_i)=\frac{n}{n-(i-1)}$
  • $E(X)={\sum }_{i\le i\le n}E(X_i)={\sum }_{1\le i\le n}\frac{1}{i}=n{H}_n\approx n\ln n$

2 Markov and Chebyshev inequalities

2.1 Markov’s inequality (Nonnegative random variable $X$)

For any nonnegative random variable $X$ and any $a>0$.
$$P_r(X\ge a)\le \frac{E(x)}{a}$$

2.2 $E(XY)=E(X)E(Y)$ if $X$ and $Y$ are independent

2.3 $Var(x)=E(X^2)-E(X{)}^2$

Ex, variance of a p-biased coin flip, $Var(X)=p-p^2$

2.4 Variance of a sum of independent random variables $Var(X+Y)=Var(X)+Var(Y)$

Ex, variance of a series of p-biased coin flips, $Var(\sum X_i)=np(1-p)$

2.5 Chebyshev’s inequality (any random variable $X$)

$$P_r(|X-E(X)|\ge a)\le \frac{Var(x)}{a^2}$$

3 Chernoff bounds

3.1 Binomial distribution

For $1\le i\le n$, let $p_i\in [0,1]$, let $X_i$ be a 0-1 random variable such that $P_r(X_i=1)=p_i$, denote $p=\frac{1}{n}{\sum }_{1\le i\le n}{p}_i$, $X=\sum X_i$, thus $E(X)=np$.

3.2 Lower bound

$$P_r(X\le (1-\delta )np)\le e^{-{\delta }^{2}np/2}$$
$\delta \in [0,1)$

3.3 Upper bound

$$P_r(X\ge (1+\delta )np)\le e^{-{\delta }^{2}np/3}$$
$\delta \in [0,1)$

3.4 When $p=\frac{1}{2}$

$$P_r(X\le (1-\delta )n/2)\le e^{-{\delta }^{2}n/2}$$
$$P_r(X\ge (1+\delta )n/2)\le e^{-{\delta }^{2}n/2}$$
$\delta \in [0,1)$

4 Balls and bins

4.1 Balls and bins problems

Suppose we throw a series of balls independently and uniformly at random into $n$ bins.

4.2 Upper bounding the load of a specific bin

Assume $n$ balls into $n$ bins uniformly and independently. $X_i$ denotes indicator variable that $=1$ when ball $i$ land into bin 1. $X={\sum }_i{X}_i$ denotes the load of bin 1 $X\sim B(n,1/n)$.

$$P_r(X\ge c^{\prime }\frac{\ln n}{\ln \ln n})\le n^{-c}$$

4.3 Upper bounding maximum bin load

$E_i$ denotes bin $i$ exceeds $c^{\prime }f(n)$, where $f(n)=\frac{\ln n}{\ln \ln n}$, we see $P_r(E_i)\le n^{-c}$, by union bound, the probability of bad events is $\le n^{1-c}$, the maximum load of any bin is $O(f(n))$ with high probability.

4.4 Lower bounding the load of a specific bin

$$P_r(X\ge {\epsilon }^{\prime }f(n))\ge n^{-\epsilon }$$

4.5 Lower bounding the minimum bin load

Minimum less than ${\epsilon }^{\prime }f(n)$ balls, $(1-n^{-\epsilon }{)}^n$