Algorithm

Sort

Insertion Sort

void insertSort(int a[], int length){
        for(int i = 1;i < length;i++){
            int key = a[i];
            int j = i - 1;
            while(j >= 0 && a[j] > key){
                a[j+1] = a[j];
                j--;
            }
            a[j+1] = key; 
        }
}

Running time: $T(n)=O(n^2)$

---

Merge Sort

import java.util.Arrays;

public class MergeSort {
    public static void sort(int[] a, int p, int r){
        if(p < r){
            int q = (p+r)/2;
            sort(a, p, q);
            sort(a, q+1, r);
            merge(a, p, q, r);
        }
    }

    public static void sort(int[] a) {
        sort(a, 0, a.length-1);
    }

    private static void merge(int[] a, int p, int q, int r) {
        int n1 = q-p+1;
        int n2 = r-q; 
        int[] leftArray = new int[n1];
        int[] rigthArray = new int[n2];
        for(int i = 0;i < n1;i++){
            leftArray[i] = a[p+i];
        }
        for(int i = 0;i < n2;i++){
            rigthArray[i] = a[q+1+i];
        }
        int i = 0, j = 0;
        for(int k = p;k <= r;k++){
            if(i == n1){
                while(j < n2){
                    a[k++] = rigthArray[j++];
                }
                break;
            } else if(j == n2) {
                while(i < n1){
                    a[k++] = leftArray[i++];
                }
                break;
            }
            if(leftArray[i] < rigthArray[j]){
                a[k] = leftArray[i++];
            } else {
                a[k] = rigthArray[j++];
            }
        }
    }


    public static void main(String[] args) {
        //Test a normal situation and several special situations
//      int[] testArray = {5, 2, 4, 7, 1, 3, 2, 6};
//      int[] testArray = {5};
//      int[] testArray = {0, 0};
        int[] testArray = {};
        System.out.println(Arrays.toString(testArray));
        MergeSort.sort(testArray);
        System.out.println(Arrays.toString(testArray));
    }

}

此例所有方程满足条件$p <= q < r$

Running time: $T(n) = O(nlgn)$

详情参考Introduction to Algorithm 2.3.1

---

Quick Sort

Implementation

import java.util.Arrays;
import java.util.Random;

public class QuickSort {
    public static void swap(int[] nums, int i, int j){
        int tmp = nums[i];
        nums[i] = nums[j];
        nums[j] = tmp;
    }

    public static int partition(int[] nums, int p, int r){
        int i = p;
        swap(nums, i, r);
        for(int j = p;j < r;j++){
            if(nums[j] < nums[r]){
                swap(nums, i++, j);
            }
        }
        swap(nums, i, r);
        return i;
    }

    public static int randomPartition(int[] nums, int p, int r){
        Random random = new Random(System.currentTimeMillis());
        int i = p + random.nextInt(r-p+1);
        swap(nums, i, r);
        return partition(nums, p, r);
    }

    public static void quickSort(int[] nums, int p, int r){
        if(p < r){
            int q = randomPartition(nums, p, r);
            quickSort(nums, p, q-1);
            quickSort(nums, q+1, r);
        }
    }

    public static void quickSort(int[] nums){
        quickSort(nums, 0, nums.length-1);
    }

    public static void main(String[] args) {
        int[] nums = {2,3,1};
        System.out.println(Arrays.toString(nums));
        quickSort(nums);
        System.out.println(Arrays.toString(nums));
    }

}

Properties：

Worst-case partitioning:
The worst-case behavior for quicksort occurs when the partitioning routine produces one subproblem with $n-1$ elements and one with $0$ elements. Let us assume that this unbalanced partitioning arises
in each recursive call. The partitioning costs ‚$\Theta (n)$ time. Since the recursive call on an array of size $0$ just returns, $T(0)= \Theta(1)$, and the recurrence for the running time is
$T(n)=T(n-1)+T(0)+\Theta(n)=T(n-1)+\Theta(n)$
so$T(n) = \Theta(n^2)$

Best-case partitioning:
In the most even possible split, $PARTITION$ produces two subproblems, each of size no more than $n/2$, since one is of size $\lfloor n/2 \rfloor$ and one of size $\lceil n/2 \rceil -1$. In this case, quicksort runs much faster. The recurrence for the running time is then
$T(n)=2T(n/2)+\Theta(n)$
so $T(n)=\Theta (nlgn)$

Balanced partitioning:
The average-case running time of quicksort is much closer to the best case than to the worst case.

---

Divide and Conquer

Problem:

Maximum subarray

public class FindMaxSubArray {

    private class SubArray {
        private int maxLeft;
        private int maxRight;
        private int sum;
        public void setMaxLeft(int maxLeft) {
            this.maxLeft = maxLeft;
        }
        public void setMaxRight(int maxRight) {
            this.maxRight = maxRight;
        }
        public int getSum() {
            return sum;
        }
        public void setSum(int sum) {
            this.sum = sum;
        }
        public String toString(){
            return "["+maxLeft+" "+maxRight+" "+sum+"]";
        }
        public String showMaxSubArray(int a[]){
            String result = "[";
            for(int i = maxLeft;i < maxRight;i++){
                result += (a[i] + " ");
            }
            result += (a[maxRight] + "]");
            return result;
        }
    }
    //Using Divide and Conquer Method
    public SubArray findMaxCrossingSubArray(int[] a, int low, int mid, int high) {
        SubArray subArray = new SubArray();
        int leftSum = Integer.MIN_VALUE;
        int sum = 0;
        for(int i = mid;i >= low;i--){
            sum += a[i];
            if(leftSum < sum){
                leftSum = sum;
                subArray.setMaxLeft(i);
            }
        }
        int rightSum = Integer.MIN_VALUE;
        sum = 0;
        for(int i = mid+1;i <= high;i++){
            sum += a[i];
            if(rightSum < sum){
                rightSum = sum;
                subArray.setMaxRight(i);
            }
        }
        subArray.setSum(leftSum+rightSum);
        return subArray;
    }

    public SubArray findMaximumSubArray(int[] a, int low, int high){
        SubArray subArray = new SubArray();
        if(high == low) { //base case only one element
            subArray.setMaxLeft(low);
            subArray.setMaxRight(high);
            subArray.setSum(a[low]);
            return subArray;
        } else {
            int mid = (low+high)/2;
            SubArray subArrayFromLeft = findMaximumSubArray(a, low, mid);
            int leftSum = subArrayFromLeft.getSum();
            SubArray subArrayFromRight = findMaximumSubArray(a, mid+1, high);
            int rightSum = subArrayFromRight.getSum();
            SubArray subArrayFromCross = findMaxCrossingSubArray(a, low, mid, high);
            int crossSum = subArrayFromCross.getSum();
            if(leftSum >= rightSum && leftSum >= crossSum){
                return subArrayFromLeft;
            } else if (rightSum >= leftSum && rightSum > crossSum) {
                return subArrayFromRight;
            } else {
                return subArrayFromCross;
            }
        }
    }

    public SubArray findMaximumSubArray(int[] a) {
        return findMaximumSubArray(a, 0, a.length-1);
    }

    //Linear Time Consumption Realization
    public SubArray findMaximumSubArrayInLinearTime(int[] a) {
        SubArray subArray = new SubArray();
        int leftBound = 0;
        int rightBound = 0;
        int maxSum = a[0];
        for(int i = 1;i < a.length;i++){
            int sum = a[i];
            for(int j = i-1;j >= leftBound;j--){
                sum += a[j];
                if(sum > maxSum){
                    maxSum = sum;
                    rightBound = i;
                    leftBound = j;
                }
            }
        }
        subArray.setMaxLeft(leftBound);
        subArray.setMaxRight(rightBound);
        subArray.setSum(maxSum);
        return subArray;
    }


    public static void main (String args[]) {
        int[] a = {13,-3, -25, 20, -3, -16, -23, 18, 20, -7, 12, -5, -22, 15, -4, 7};
        FindMaxSubArray f = new FindMaxSubArray();
        SubArray subArray = f.findMaximumSubArray(a);
        System.out.println(subArray.showMaxSubArray(a));
        subArray = f.findMaximumSubArrayInLinearTime(a);
        System.out.println(subArray.showMaxSubArray(a));
    }
}

此例给出了一个分治算法的解决方案和蠕动算法的解决方案。
分支算分
Running-time: $T(n) = O(nlgn)$
详情参考Introduction to Algorithm 4.1

The Master Theorem

The theorem is used to solve the recurrences of the form

$$T(n)=aT(n/b)+f(n)$$ where $a\ge1$ and $a\gt1$ are constants and $f(n)$ is an asymptotically positive function.
Let $T(n)$ be defined on the nonegative integers by the recurrence, we interpret $n/b$ to mean either $\lfloor n/b \rfloor$ or $\lceil n/b \rceil$. Then $T(n)$ has the following asymptotic bound.

1. If $f(n)=O(n^{log_ba-\epsilon})$ for some constant $\epsilon>0$, then $T(n)=\Theta(n^{log_ba})$.
2. If $f(n)=\Theta(n^{log_ba})$, then $T(n)=\Theta(n^{log_ba}lgn)$.
3. If $f(n)=O(n^{log_ba+\epsilon})$ for some constant $\epsilon>0$, and if $af(n/b) \le cf(n)$ for some constant $c < 1$ and all sufficiently large n, then $T(n)=\Theta(f(n))$

---

Dynamic Programming

CutRod

public class CutRod {
    public static int memorizedCutRod(int[] p, int n){
        int[] r = new int[n+1];
        for(int i = 0;i <= n;i++){
            r[i] = Integer.MIN_VALUE;
        }
        return memorizedCutRodAux(p, n, r);
    }

    private static int memorizedCutRodAux(int[] p, int n, int[] r){
        if(r[n] >= 0){
            return r[n];
        }
        int q = Integer.MIN_VALUE;
        if(n == 0){
            q = 0;
        } else {
            for(int i = 1;i <= n;i++){
                int temp = p[i]+memorizedCutRodAux(p, n-i, r);
                if(temp > q){
                    q = temp;
                }
            }
            r[n] = q;
        }
        return q;
    }

    private static int bottomUpCutRod(int[] p, int n){
        int[] r = new int[n+1];
        for(int i = 0;i <= n;i++){
            r[i] = Integer.MIN_VALUE;
        }
        r[0] = 0;
        for(int i = 1;i <= n;i++){
            int q = Integer.MIN_VALUE;
            for(int j = 0;j < i;j++){
                int temp = r[j] + p[i-j];
                if(temp > q) q = temp;
            }
            r[i] = q;
        }
        return r[n];
    }

    private static int extendedBottomUpCutRod(int[] p, int n, int[] s){
        int[] r = new int[n+1];
        for(int i = 0;i <= n;i++){
            r[i] = Integer.MIN_VALUE;
        }
        r[0] = 0;
        s[0] = 0;
        for(int i = 1;i <= n;i++){
            int q = Integer.MIN_VALUE;
            for(int j = 0;j < i;j++){
                int temp = r[j] + p[i-j];
                if(temp > q){
                    q = temp;
                    s[i] = i-j;
                }
            }
            r[i] = q;
        }
        return r[n];
    }

    private static void printCutRodSolution(int[] p, int n, int[] s){
        System.out.print("[");
        System.out.print(extendedBottomUpCutRod(p, n, s));
        while(n > 0){
            System.out.print(" "+s[n]);
            n -= s[n];
        }
        System.out.println("]");
    }

    public static void main(String[] args) {
        int[] p = {0, 1, 5, 8, 9, 10, 17, 17, 20, 24, 30};
        int n = 4;
        System.out.println(bottomUpCutRod(p, n));
        System.out.println(memorizedCutRod(p, n));
        int[] s = new int[n+1];
        printCutRodSolution(p, n, s);
    }

}

暴力求解方式 $T(n)=2^n$
动态规划方式$T(n)=\Theta(n^2)$

---

Matrix Chain Order

class ChainOrderSolution{
    private int[][] m;
    private int[][] s;

    public int[][] getM() {
        return m;
    }
    public void setM(int[][] m) {
        this.m = m;
    }
    public int[][] getS() {
        return s;
    }
    public void setS(int[][] s) {
        this.s = s;
    }
}

public class ChainOrder {

    public static ChainOrderSolution matrixChainOrder(int[] p){//The length of p is n+1
        int n = p.length-1;
        ChainOrderSolution solution = new ChainOrderSolution();
        int[][] m = new int[n][n];
        int[][] s = new int[n][n];

        //m[i,j] when i==j
        for(int i = 0;i < n;i++){
            m[i][i] = 0;
        }

        for(int l = 2; l <= n;l++){//l is the chain length
            for(int i = 0;i < n-l+1;i++){
                int j = i+l-1;
                m[i][j] = Integer.MAX_VALUE;
                for(int k = i;k <j;k++){
                    int q = m[i][k] + m[k+1][j] + p[i]*p[k+1]*p[j+1];
                    if(q < m[i][j]){
                        m[i][j] = q;
                        s[i][j] = k;
                    }
                }
            }
        }
        solution.setM(m);
        solution.setS(s);
        return solution;
    }

    public static void printOptimalParens(int[][] s, int i, int j){
        if(i == j){
            System.out.print("A"+(i+1));
        } else {
            System.out.print("(");
            printOptimalParens(s, i, s[i][j]);
            printOptimalParens(s, s[i][j]+1, j);
            System.out.print(")");
        }
    }

    public static void printOptimalParens(int[][] s){
        printOptimalParens(s, 0, s.length-1);
    }

    public static void main(String[] args) {
        int[] p = {30, 35, 15, 5, 10, 20, 25};
        int n = p.length-1;
        ChainOrderSolution solution = ChainOrder.matrixChainOrder(p);
        ChainOrder.printOptimalParens(solution.getS());
        System.out.println(" ");
        int[][] m = solution.getM();
        System.out.println(m[0][n-1]);
    }

}

暴力求解方式 $T(n)=\Omega(2^n)$
用动态规划的方式 $T(n)=\Omega(n^3)$

---

Principle

Optimal Substructure

• Concept: If an optimal solution to the problem contains within it optimal solutions to subproblems.
• Model:
  
  • Making a choice to divide the problem into subproblems
  • Every choice lead to the optimal solution
  • Given this choice, you determine which subproblems ensue and how to best characterize the resulting space of subproblems.
  • prove the solution by Cut-and-Paste proof
• Independence: Using resources in one subproblem renders them unavailable for the other subproblems.

Overlapping subproblems

The recursive algorithm for the problem solves the sane subproblems over and over, rather than always gerating new subproblems. Here we compare Dynamic Programming with Divide-and-Conquer. The subproblems in Divide-and-Conquer do not repeat, but after each division a new subproblem occured.

---

Back Tracking

Concept

Solution Space Tree

1. Each node in the Tree is a problem state
2. The path from root nodes to the other nodes are the state space
3. Solution states is problem state S from root to which is a tuple in the state space.
4. Answer states are the solution states S from root to which is a solution to the problem.
5. The solution space tree construction si called state space tree.
6. The Backtracking method is actually a deep first search(DFS)

8 Queens Problem

import java.util.Arrays;

public class RecursiveNQueens {
    //r represent the row of the board, j represent the column and i is the row before the present row
    public static void recursiveArray(int[] q, int r){
        int n = q.length;
        if(r == n){
            System.out.println(Arrays.toString(q));
        } else {
            for(int j = 0;j < n;j++){
                boolean legal = true;
                //To make sure that no two queens are in the same column or diagonal
                for(int i = 0;i < r;i++){
                    if((q[i] == j)||(q[i] == j+r-i)||(q[i]==j-r+i)){ 
                        legal = false;
                    }
                }
                if(legal){
                    q[r] = j;
                    recursiveArray(q, r+1);
                }
            }
        }
    }

    public static void recursiveArray(int[] q){
        recursiveArray(q, 0);
    }

    public static void main(String[] args) {
        int[] q = new int[4];
        RecursiveNQueens.recursiveArray(q);
    }

}

---

Greedy Algorithm

How to Design Greedy Algorithm:

1. Cast the optimization problem as one in which we make a choice and are left with one subproblem to solve.
2. Prove that there is always an optimal solution to the original problem that makes the greedy choice, so that the greedy choice is always safe.
3. Demonstrate optimal substructure by showing that, having made the greedy choice, what remains is a subproblem with the property that if we combine an optimal solution to the subproblem with the greedy choice we have made, we arrive at an optimal solution to the original problem.

The difference between Dynamic Programming and Greedy Algorithm:

In dynamic programming, we make a choice at each step, but the choice usually depends on the solutions to subproblems. Consequently, we typically solve dynamic-programming problems in a bottom-up manner, progressing from smaller subproblems to larger subproblems. (Alternatively, we can solve them top down, but memoizing. Of course, even though the code works top down, we still must solve the subproblems before making a choice.) In a greedy algorithm, we make whatever choice seems best at the moment and then solve the subproblem that remains. The choice made by a greedy algorithm may depend on choices so far, but it cannot depend on any future choices or on the solutions to subproblems. Thus, unlike dynamic programming, which solves the subproblems before making the first choice, a greedy algorithm makes its first choice before solving any subproblems. A dynamic programming algorithm proceeds bottom up, whereas a greedy strategy usually progresses in a top-down fashion, making one greedy choice after another, reducing each given problem instance to a smaller one.