Algorithmic Toolbox 算法基础学习笔记（第二周）

Algorithmic Toolbox 算法基础学习笔记

【第二周】Fibonacci sequence, GCD and Big O notation

---

---

学习目标

1. Estimate the running time of an algorithm
2. Practice implementing efficient solutions
3. Practice solving programming challenges Implement programs that are
4. several orders of magnitude faster than straightforward programs

---

一、斐波那契数列（Fibonacci sequence）

1. 定义

1.2. Lemma

Rapid Growth
$$F_n\ge 2^{n/2},\forall n\ge 6$$
Proof:

利用数学归纳法证明如下：

• Base case: $n=6$ (By direct induction)
• Inductive step:
  $$F_n=F_{n-1}+F_{n-2}\ge 2^{(n-1)/2}+2^{(n-2)/2}\ge 2\ast 2^{(n-2)/2}=2^{n/2}$$

1.3. Formula

$$F_n=\frac{1}{\sqrt{5}}\left({\left(\frac{1+\sqrt{5}}{2}\right)}^n-{\left(\frac{1-\sqrt{5}}{2}\right)}^n\right)$$

Example:
$F_{20}=6765$
$F_{50}=12586269025$
$F_{100}=354224848179261915075$

2. Naive Algorithm

FibRecurs(n):
if n<= 1:
	return n
else:
	return FibRecurs(n-1)+FibRecurs(n-2)

Too Slow!

3. Efficient Algorithm

FibList(n):
create an array F[0,...,n]
F[0] = 0
F[1] = 1
for i from 2 to n:
	F[i] = F[i-1] + F[i-2]
return F[n]

• T(n) = 2n + 2
• Easy to compute

二、最大公约数（Greatest Common Divisors）

1.定义

1.1. Definition：

For integers, a and b, their greatest common divisor or gcd(a, b) is the largest integer d so that d divides both a and b.

1.2. Key Lemma:

Let $a^{\prime }$ be the remainder when $a$ is divided by $b$, then
$$gcd(a,b)=gcd(a^{\prime },b)=gcd(b,a^{\prime })$$.
Proof:

• a = a′ + bq for some q
• d divides a and b if and only if it divides a′
  and b

2. Naive Algorithm:

int NaiveGCD(int a, int b){
	int best = 0;
	for (i=1, i <= a+b, i++){
		if d|a and d|b:
			best = d;
		}
	return best;
}

• Runtime approximately a + b.
• Very slow for 20 digit numbers.

3. Efficient Algorithm

Euclidean Algorithm:

int EuclidGCD(int a, int b){
	if (b == 0)
		return a;
	a = a % b;
	return EuclidGCD(b, a);
}

• Produces correct result by Lemma.
• Each step reduces the size of numbers by about a factor of 2.
• Takes about log(ab) steps.

3.1 Euclidean Algorithm 的时间复杂度推导:

时间复杂度：$O(log(min(a,b))$
证明如下
Proof：
假设 a 和 b 是两个整数，且a >b。那么根据欧几里德算法：

gcd(a, b) = gcd(b, a%b)

重复使用上述公式若干次，直到到达b为0，此时结果将是两个整数的GCD，数值等于a。通过观察可发现该算法的时间复杂度与将b减少到0所需的步数成正比。
让我们假设，使用该算法使b降低至0所需的步骤的数目是N。

gcd(a, b) ------> N steps

Lemma:
现在，如果gdc(a,b)通过N步欧几里德算法减少到gdc(1, 0)，那么a应该至少是f (N + 2)且b应该至少是f (N + 1)。

gcd(a, b) ——> N steps
Then, a >= f(N + 2) and b >= f(N + 1)
where, fN is the Nth term in the Fibonacci series(0, 1, 1, 2, 3, …) and N >= 0.

上述定理可用数学归纳法证明：

Base Case (gdc(2,1)满足上述定理):

• Let’s assume a = 2 and b = 1. Then, gcd(2, 1) will reduce to gcd(1, 0) in 1 step, i.e., N = 1.
• This means 2 should be at least f3 and 1 should be at least f2 and f3 = 2 and f2 = 1.
• This implies, a is at least f(N + 2) and b is at least f(N + 1).
• It can be concluded that the statement holds true for the Base Case.

Inductive Step (假设gcd(b, a%b)满足上述定理，并推导gcd(a, b)也满足):

• Assume that the statement holds true for the $(N–1{)}^{th}$ Step. So, below are the steps to prove it for the $N^{th}$ Step:

gcd(b, a%b) ——> (N – 1) steps
Then,  
b >= f(N – 1 + 2) i.e., b >= f(N + 1)
a%b >= f(N – 1 + 1) i.e., a%b >= fN

a可写作：

a = floor(a/b)*b + a%b

• Now, (a/b) would always be greater than 1（因为a>b）. So, from the above result, it is concluded that:

a >= b + (a%b)
This implies, a >= f(N + 1) + fN = f(N+2)

由上述假设推导出了a >= f (N + 2)和 b >= f (N + 1)，故lemma成立。
我们知道，斐波那契数列第N项可表示为（见1.3）：
$$F_n=\frac{1}{\sqrt{5}}\left({\left(\frac{1+\sqrt{5}}{2}\right)}^n-{\left(\frac{1-\sqrt{5}}{2}\right)}^n\right)$$
or
$$F_n={\varnothing }^N$$
其中$\varnothing \approx 1.618$，为黄金比例。
现在，已经说明时间复杂度将与 N 成正比，即将b减少到0所需的步骤数。
因此，为了证明时间复杂度，已知：
$$f_N={\varnothing }^N,N\approx lo{g}_{\varnothing }(f_N)$$
我们还证明了如果欧氏算法进行N步能使gcd(a, b)变换为gcd($a^{,}$, 0)，那么a至少应该是f (N+2)并且b应至少为f (N + 1)。

从以上两个结果可以得出结论：
$$f_N=min(a,b),N=lo{g}_{\varnothing }(min(a,b))$$
故欧式算法的时间复杂度为:
$$O(log(min(a,b)))=O(log(a+b))=O(log(ab))$$
证毕。

三、Big O Notation

1. Definition

Big O Notatoion:
$f(n)=O(g(n))$ (f is Big-O of g) or f ⪯ g
if there exist constants N and c so that for
all n ≥ N, $f(n)\le c\cdot g(n)$.
Other Notations:
For functions $f,g:N\to R^{+}$ we say that:

• $f(n)=\Omega (g(n))$ or $f⪰g$ if for some c, $f(n)\ge c\cdot g(n)$ (f grows
  no slower than g).
• $f(n)=\Theta (g(n))$ or $f\asymp g$ if $f=O(g)$ and $f=\Omega (g)$ (f grows at the same rate as g).
• $f(n)=o(g(n))$ or $f\prec g$ if $f(n)/g(n)\to 0$ as $n\to \infty$ (f grows slower than g).

2. Common Rules

1. Multiplicative constants can be omitted:
   $$7n^3=O(n^3),\frac{n^3}{3}=O(n^3)$$
2. $n^a\prec n^b$ for 0 < a < b:
   $$n=O(n^2),\sqrt{n}=O(n)$$
3. $n^a\prec b^n$ (a > 0, b > 1):
   $$n^5=O({\sqrt{2}}^n),n^{100}=O(1.1^n)$$
4. $log(n{)}^a\prec n^b$ (a, b > 0):
   $$(logn{)}^3=O(\sqrt{n}),nlogn=O(n^2)$$
5. Smaller terms can be omitted :
   $$n^2+n=O(n^2)$$

3. Practice

3.1 Fibonacci number

下面估计上述斐波那契优化算法的复杂度：
注意：F[i] = F[i-1] + F[i-2] 的复杂度为$O(n)$。通常情况下，加法运算的复杂度应为常数，但由于斐波那契数列很大（第n项包含n/5位数），故复杂度位$O(n)$。

总结

Asymptotic Notation

1. Lets us ignore messy details in analysis.
2. Produces clean answers.
3. Throws away a lot of practically useful information.