Randomized Algorithms: Median Finding

More Divide-and-Conquer

In General Divide-and-Conquer:
$$T(n)=qT(n/p)+O(n)$$ 其中 $q$ 是 # of calls for problem, $p$ 是每一次 size reduce factor。

由 recursion tree 可知，# of levels= ${\log }_{p}n$
因此，Total time
$$T(n)\le \sum _{k=0}^{{\log }_{p}n}{q}^k\cdot c\cdot \frac{n}{p^k}=cn\sum _{k=0}^{{\log }_{p}n}(\frac{q}{p}{)}^k$$ 这里，$q^k$ 是 # of subproblem at level $k$，$c\frac{n}{p^k}$ 是 time spend at level $k$.

Case 1:
\quad $q=1,p=2$ (binary search)
$$T(n)\le cn\sum _{k=1}^{\infty }\frac{1}{2^k}\le 2cn=O(n)$$ \quad In General: if $q<p$, then $T(n)=O(n)$**

Case 2:
\quad $q=2,p=2$ (merge sort)
$$T(n)\le cn\sum _{k=0}^{{\log }_{2}n}{1}^k=O(n\log n)$$ \quad In General: if $p=q$, then $T(n)=O(n\log n)$

Case 3:
\quad $q>p$ (multiplication)
$$T(n)\le cn\sum _{k=0}^{{\log }_{p}n}(\frac{q}{p}{)}^k\le \frac{q/p}{q/p-1}\cdot cn\cdot (\frac{q}{p}{)}^{{\log }_{p}n}=\frac{q/p}{q/p-1}\cdot c\cdot q^{{\log }_{p}n}=O(q^{{\log }_{p}n})=O(p^{{\log }_{p}q\cdot {\log }_{p}n})=O(n^{{\log }_{p}q})$$

The Problem - Finding the Median

Suppose we are given a set of $n$ numbers S = { a 1 , a 2 , . . . , a n } S=\{a_1, a_2, ..., a_n\} S={a1​,a2​,...,an​}. The median is the number that would be in the middle position if we were to sort them. 但是如果 $n$ 是偶数，就没有middle position了。
定义：The median of S = { a 1 , a 2 , . . . , a n } S=\{a_1,a_2,...,a_n\} S={a1​,a2​,...,an​} is equal to the $k^{th}$ largest element in $S$.

• $n$ 是奇数：$k=(n+1)/2$
• $n$ 是偶数：$k=n/2$

如果先 sort the numbers，需要 $O(n\log n)$
这里我们展示如何用一个基于 divide-and-conquer 的 randomized approach 用 $O(n)$ 得到median。

Design the Algorithm

基于 Splitters 的简单算法

我们先不考虑 median-finding，而考虑 selection 问题：Given a set of $n$ numbers $S$ and a number $k$ between $1$ and $n$, consider the function $Select(S,k)$ that returns the $k^{th}$ largest element in $S$.
目的：$Select(S,k)$ runs in expected time $O(n)$.

Algorithm Structure:

• Choose an element $a_i\in S$ as the splitter.
• Form sets
  • S − = { a j : a j < a i } S^- = \{a_j : a_j<a_i\} S−={aj​:aj​<ai​}
  • S + = { a j : a j > a i } S^+=\{a_j: a_j>a_i\} S+={aj​:aj​>ai​}
• We can then determine which of $S^{-}$ or $S^{+}$ contains the $k^{th}$ largest element, and iterate only on this one.

---

$Select(S,k)$
\quad Choose a splitter $a_i\in S$
\quad For each element $a_j$ of $S$
\quad \quad Put $a_j$ in $S^{-}$ if $a_j<a_i$
\quad \quad Put $a_j$ in $S^{+}$ if $a_j>a_i$
\quad EndFor
\quad If $|S^{-}|=k-1$ then
\quad \quad The splitter $a_i$ was in fact the desired answer
\quad Else If $|S^{-}|\ge k$ then
\quad \quad The $k^{th}$ largest element lies in $S^{-}$
\quad \quad Recursively call $Select(S^{-},k)$
\quad Else suppose $|S^{-}|=l<k-1$
\quad \quad The $k^{th}$ largest element lies in $S^{+}$
\quad \quad Recursively call $Select(S^{+},k-1-l)$
\quad EndIf

---

Also, observe that if $|S|=1$, then we must have $k=1$, and indeed the single element in $S$ will be returned by the algorithm.

定理 (13.17): Regardless of how the splitter is chosen, the algorithm above returns the $k^{th}$ largest element of $S$.

Choosing a Good Splitter

Essentially, it’s important that the splitter significantly reduce the size of the set being considered, so that we don’t keep making passes through large sets of numbers many times. So a good choice of splitter should produce sets $S^{-}$ and $S^{+}$ that are approximately equal in size.

如果 medians 是 splitter，那么$T(n)\le T(n/2)+cn=O(n)$。然而我们就是要找 median。但是，我们可以证明任意一个 well-centered element 都可以成为一个 good splitter。

Well-Centered Splitter : Choose a splitter $a_i$ such that there were at least $\epsilon n$ both larger and smaller than $a_i$, for any fixed constant $\epsilon >0$.

这样 the size of the sets in the recursive call would shrink by a factor of at least $(1-\epsilon )$ each time，即有
$$T(n)\le T((1-\epsilon )n)+cn$$ If we unroll the recurrence for any $\epsilon >0$, we get
$$T(n)\le cn+(1-\epsilon )cn+(1-\epsilon {)}^{2}cn+...=[1+(1-\epsilon )+(1-\epsilon {)}^2+....]\cdot cn\le \frac{1}{\epsilon }cn$$

Analyzing the Algorithm

定义：algorithm is in phase $j$ when the size of the set under consideration is at most $n(\frac{3}{4}{)}^j$ but greater than $n(\frac{3}{4}{)}^{j+1}$.

In a given iteration of the algorithm, we say that an element of the set under consideration is central if

• at least a quarter of the elements are smaller than it
• at least a quarter of the elements are larger than it.
  

# of subproblems $q=1$
max size of a subproblem $\frac{3n}{4}=\frac{n}{p}$, thus $p=\frac{4}{3}$
$\therefore T(n)\le T(3n/4)+O(n)$, 其中 $3n/4$ 是 max size recursive call, $O(n)$ 是 expected time till recursive call。

在一次迭代中，the Probability that our random choice of splitter produces a central element is $\frac{1}{2}$.

更新算法：

---

$QuickFind(S,k)$
\quad While $i\notin [n/4,3n/4]$
\quad \quad Select $x\in S$ randomly
\quad \quad S − = { y : y < x } , S + = { y : y > x } S^-=\{y: y<x\}, S^+ =\{y:y>x\} S−={y:y<x},S+={y:y>x}
\quad \quad $i=|S^{-}|+1$ \quad \quad [$x$'s position]
\quad EndWhile
\quad If $i=k$ \quad Return $x$
\quad If $i>k$ \quad $QuickFind(S^{-},k)$
\quad If $i<k$ \quad $QuickFind(S^{+},k-i)$

---

Running Time: $O(n)\cdot$ # iterations on while loop

Claim: The expected number of iterations before a central element is found is $2$; and so the expected number of iterations spent in phase $j$, for any $j$, is at most $2$.

证明：
Let $X$ be be a random variable equal to the number of repeats until $i\in [n/4,3n/4]$.
$\because X$ is nonnegative R.V.
$\therefore E[X]={\sum }_{i=0}^{\infty }P(X>i)$
$$E[X]=1+P(X>1)+P(X>2)+...$$$$=1+(1-p)+(1-p{)}^2+...$$$$=\frac{1}{1-(1-p)}=2$$

定理 (13.18): The expected running time of $Select(n,k)$ is $O(n)$