Introduction to Algorithm 3rd Edition

Our first algorithm, insertion sort, solves the sorting problem:
Input: A sequence of n numbers
$a_{0},a_{1},a_{2},a_{3}.......a_{n}$
Output:
A pernutation (reordering)
$a{'}_{1},a{'}_{2},a{'}_{3}......a{'}_{n}$
of the input sequence such that
$a{'}_{1}\leq a{'}_{2}\leq a{'}_{3}......\leq a{'}_{n}$
The numbers that we wish to sort are also known as the keys. The input comes to us in the form of an array with n elements.
Insertion-Sort(A)

for j=2 to A.length
    key=A[j]
    //Insert A[j] into the sorted sequence A[1...j-1].
    i=j-1
    while i>0 and A[i]>key
        A[i+1]=A[i]
        i=i-1
        A[i+1]=key

(In the later chapter, the code I will use is Python as an example to run the program.)

We use loop invariants to help us understand why an algorithm is correct. We must show three things about a loop invariant:
Initialization: It is true prior to the first iteration of the loop.
Maintenance: If it is true before an iteration of the loop, it remains true before the next iteration.
Termination: When the loop terminates, the invariant gives us a useful property that helps us show that the algorithm is correct.
对于插入排序：
初始化：
首先证明在第一次循环迭代之前（当j=2时），循环不变是成立。所以子数组A[1..j-1]仅由单个元素A[1]组成，实际上就是A[1]中原来的元素。而且该子数组是排序好的（当然很平凡）。这表明第一次循环迭代之前循环不变式成立。
保持：
其次处理第二条性质：证明每次迭代保持循环不变式。有些通俗地讲，for循环体的第4~7行将A[j-1]、A[j-2]、A[j-3]等向右移动一个位置，直到找到A[j]的适当位置，第8行将A[j]的值插入该位置。这时子数组A[1..j]由原来在A[1..j]中的元素组成，但已按序排列。那么对for循环的下一次迭代增加j将保持循环不变式。
终止：
导致for循环终止的条件是j>A.length=n。因此每次循环迭代j增加1，那么必有j=n+1。在循环不变式的表述中将j用n+1代替，我们有：子数组A[1..n]由原来的A[1..n]中的元素组成，但已按序排列。注意到，子数组A[1..n]就是整数组，我们推测出整个数组已经排序。
因此算法正确。

Logarithms
We shall use following notations:
$lgn=log_{2}n$(binary logarithm)
$lnn=log_{e}n$(natural logarithm)
$lg^{k}n=(lgn)^{k}$(exponentiation logarithm)
$lglgn=lg(lgn)$(composition)
For all real a>0,b>0,c>0,and n,
$a=b^{log_{b}a}$
$log_{c}(ab)=log_{c}a+log_{c}b$
$log_{b}a^{n}=nlog_{b}a$
$log_{b}(1/a)=-log_{b}a$
$log_{b}a=1/log_{a}b$
$a^{log_{b}c}=c^(log_{b}a)$
where, in each equation above, logarithm bases are not 1.

More recurrence examples
a. $T(n)=4T(n/3)+nlgn$
b.$T(n)=3T(n/3)+nlng$
c.$T(n)=4T(n/2)+n^2\sqrt n$
d.$T(n)=3T(n/3-2)+n/2$
e.$T(n)=2T(n/2)+n/lgn$
f.$T(n)=T(n/2)+T(n/4)+T(n/8)+n$
g.$T(n)=T(n-1)+1/n$
h.$T(n)=T(n-1)+lgn$
i.$T(n)=T(n-2)+1/lgn$
j.$T(n)=\sqrt n T(\sqrt n)+n$