算法学习笔记--3.Binary Search & Algorithmic Complexity

1. Binary Search

很久前就看过二分法。wikipedia-Binary search algorithm中介绍的更加详细:

1. 
   
2. In computer science, binary search is a search algorithm that finds the position of a target value within a sorted array.
   
3. Binary search compares the target value to the middle element of the array; if they are unequal, the half in which the target cannot lie is eliminated and the search continues on the remaining half until it is successful. If the search ends with the remaining half being empty, the target is not in the array.
   
4. Binary search runs in at worst logarithmic time, making O(log n) comparisons, where n is the number of elements in the array, the O is Big O notation, and log is the logarithm.

二分法查找在python中的实现：
（这里查找到的位置从1开始，python列表中计数从0开始）

In [36]: def bisearch(A,target,Left=0,Right=None):
    ...:     if Right is None:
    ...:         Right = len(A) + 1
    ...:         
    ...:     while Left < Right:
    ...:         mid = int((Left+Right)/2)
    ...:         mid_val = A[mid-1]
    ...:         if mid_val < target:
    ...:             Left = mid + 1
    ...:         elif mid_val > target:
    ...:             Right = mid + 1
    ...:         else:
    ...:             return mid
    ...:     return -1
    ...: 

In [37]: A = sorted([2,3,1,5,4,6])

In [38]: print(A)
[1, 2, 3, 4, 5, 6]

In [39]: bisearch(A, 2)
Out[39]: 2

In [40]: bisearch(A, 6)
Out[40]: 6

In [41]: bisearch(A, 30)
Out[41]: -1

2. Algorithmic Complexity

上面提到了二分法最坏的情况下，运行时间为O(log n)
O理解成上限，就是最坏情况下代码运行的时间。

John V. Guttag–Introduction to Computional and Programming Using Python–Chapter 8 中有算法复杂度的介绍。

2.1 Computational Complexity

We could run the program on some input and time it. The result would depend upon:

1. The speed of the computer on which it run.
2. The efficiency of the Python implementation on that machine.
3. The value of the input.

For the first two issues, We measure time in terms of the number of basic steps executed by the program instead of measuring time in milliseconds.
(A step is an opeartion that takes a fixed amount of time, one step for a time.)

Now that we have a more abstract way to think about the meaning of time, we deal with that by moving away from expressing time complexity as a single number and instead relating it to the sizes of the input.

Of course, the actual running time of an algorithm depends not only on the sizes of the inputs but also upon their values.

In general，there are three broad cases to think about :

1. The best-case running time is the running time of the algorithm when the inputs are as favorable as possible.
2. The worst-case running time is the maximum running time over all the possible inputs of a given size.
3. The average-case running time is the average running time over all possible inputs of a given size.

(The worst case provides an upper bound on the running time. )

2.2 Asymptotic Notation

We use asymptotic notation to provide a formal way to talk about the relationship between the running time of an algorithm and the size of its inputs.

Asymptotic notation dascribes the complexity of an algorithm as the size of its inputs approach infinity.

这里用中文解释一下：

用f(x)来表示程序运行时间和输入数据的关系，x是输入的数。
$f(x) = 2x^2 +x + 1000$ 表示输入数为x时，程序需要运行$2x^2 +x + 1000$ 步。
高数学过极限，x趋于无穷大时，忽略一次项和0次项，这里$x^2$前面的系数也忽略掉，变成了$f(x)=x^2$ 。然后就到了$O(x^2)$。

The most commonly used asympotic notation is called “Big O” notation. Big O notation is used to give a upper bound on the asymptotic growth (the order of growth) of a function.

The $f(x)=O(x^2)$ means that the function f grows no faster than the quadratic polynomial $x^2$, in an asymptotic sense.

We say the complexity of f(x) is $O(x^2)$ means that in the worst case f will take $O(x^2)$ steps to run.

2.3 Complexity Classes

Some of the most common instances of Big O are listed below:
(n is a measure of the size of the inputs to the function. k, c are constants.)

• 
  
• O(1) denotes constant running time.
  
• O(n) denotes linear running time.
  
• O($log_2n$) denotes logarithmic running time.
  
• O($nlog_2n$) denotes log-linear running time.
  
• O($n^k$) denotes polynomial running time.
  
• O($c^n$) denotes exponential running time.