Big O notation：算法的时间复杂度

The Definition of Time Complexity

Definition

How can we analyze the runtime of an algorithm as the size of the inputs increases?
OR
A way of showing how the runtime(operations) of a function(or a particular piece of code) increases as the size of the input increases.

• 时间复杂度是用来表示一个方法或者一个代码片段的运行时间（这边的运行时间用为得到结果所需要的操作次数来表示。即Operations）是怎么随着输入（Input）的增加而增加的。

我们可以用x轴表示Input，用Y轴表示Operations

简化版

Big O notation

We say that an algorithm’s time complexity is O(f(n)) if the number of simple operations the computer has to do is eventually less than a constant times f(n), as n increases
我们说如果随着n的增加，计算机必须执行的简单操作的数量最终小于常数f（n）的算法的时间复杂度为O（f（n））

I. O(1) - Constant Time

Definition

when the time taken to complete a function does not increase at all as the size of the input increases. The time stays as constant.

• usually read as either “big-oh of 1” or simply “oh of 1”
  

II. O(logⁿ)

e.g. binary search’s time complexity is log₂ⁿ

What is binary search?

• Use to search an ordered list for a particular value
• Divide and conquer approach
• Target compared with middle value, then half of the list is discarded, repeatedly, until the target is found.
• Very efficient for large sorted lists

Time complexity of Binary Search

• Double the data requires only one extra chop (特点是：排序好的数组，当它的长度每*2的时候，只要多切一次即可）
• This makes the binary search very efficient for very larg data sets, if the data is already sorted
• The big O time complexity is Logarithmic
  
  切第一次，发现要找的在后半段，就把前半段直接舍弃；反之，把前半段直接舍弃。

Time complexity of binary search algorithm is O(log2n) which is very efficient. Auxiliary space used by it is O(1) for iterative implementation and O(log2n) for recursive implementation due to call stack.

III. O(n) - Linear Time

Definition

as the size of the input increases, the execution time of the function increases linearly.

• n is usually the size of the input, in a particular case, it’s the number of elements in the given array.
• usually read as either “big-oh of n” or simply “oh of n”
  

IV. O(nlogⁿ)

If the content of an iteration loop is a binary search, then the time complexity is O(nlogⁿ)

V. O(n²) - Quadratic Time

Definition

when the time it takes to complete a function increases like a quadratic function二次函数

• usually read as either “big-oh of n squared” or simply “oh of n squared”
  

VI. O(2ⁿ)

Algorithms with running time O(2^N) are often recursive algorithms that solve a problem of size N by recursively solving two smaller problems of size N-1.

This program, for instance prints out all the moves necessary to solve the famous “Towers of Hanoi” problem for N disks in pseudo-code

void solve_hanoi(int N, string from_peg, string to_peg, string spare_peg)
{
    if (N<1) {
        return;
    }
    if (N>1) {
        solve_hanoi(N-1, from_peg, spare_peg, to_peg);
    }
    print "move from " + from_peg + " to " + to_peg;
    if (N>1) {
        solve_hanoi(N-1, spare_peg, to_peg, from_peg);
    }
}

Let T(N) be the time it takes for N disks.

We have:
T(1) = O(1)
and
T(N) = O(1) + 2*T(N-1) when N>1

If you repeatedly expand the last term, you get:

T(N) = 3O(1) + 4T(N-2)
T(N) = 7O(1) + 8T(N-3)
…
T(N) = (2^(N-1) - 1)*O(1) + (2^(N-1))*T(1)
T(N) = (2^N - 1)*O(1)
T(N) = O(2^N)

To actually figure this out, you just have to know that certain patterns in the recurrence relation lead to exponential results. Generally T(N) = … + C*T(N-1) with C > 1means O(x^N). See:
参考：维基百科 - Recurrence_relation
来自stackoverflow

VII. O(n!)

• brute-force search
• Permutations of a string

getPermutations('a') // => [ 'a']
getPermutations('ab') // =>  [ 'ab', 'ba']
getPermutations('abc') // => [ 'abc', 'acb', 'bac', 'bca', 'cab', 'cba' ]

How would you solve that?

A straightforward way will be to check if the string has a length of 1 if so, return that string since you can’t arrange it differently.

For strings with a length bigger than 1, we could use recursion to divide the problem into smaller problems until we get to the length 1 case. We can take out the first character and solve the problem for the remainder of the string until we have a length of 1.

function getPermutations(string, prefix = '') {
  if(string.length <= 1) {
    return [prefix + string];
  }

  return Array.from(string).reduce((result, char, index) => {
    const reminder = string.slice(0, index) + string.slice(index+1);
    result = result.concat(getPermutations(reminder, prefix + char));
    return result;
  }, []);
}

the output would be something like this:

getPermutations('ab') // ab, ba...
// n = 2, f(n) = 2;
getPermutations('abc') // abc, acb, bac, bca, cab, cba...
// n = 3, f(n) = 6;
getPermutations('abcd') // abcd, abdc, acbd, acdb, adbc, adcb, bacd...
// n = 4, f(n) = 24;
getPermutations('abcde') // abcde, abced, abdce, abdec, abecd, abedc, acbde...
// n = 5, f(n) = 120;

Stay away if possible from algorithms that have this running time!

How to analyze Time Complexity with Big O

Just changing what’s inside the Big O e expression. you are able to express different types of time complexity.

那么我们怎么知道线性时间是O(n)…How to understand.

Two Steps to Find Out Big O

• Linear time 的时间表达式就应该是下面这种方程：
  • T= an+b (a&b are two constants, n is the size of the input)
  • How do we know this function is O(n)? to find out , there are two steps you need to follow"
    1. to find the fastest growing term in the given function, an and b , b does not grow at all as n increases, so the fastest growing term here is an.
    2. take out the coefficient out of that fastest growing term. coefficient(系数) here is a ,cause a is a constant, and n is not. So you take out the coefficient a, the left is just n.
  • and that’s is the inside the big O expression , O(n)

同样可以用以上方法来判断别的时间复杂度方程：

​ T= cn^2+dn+e…

• 第一步，e不是增加最快的，dn也不是，那么就剩下cn^2; 当n足够大的时候，cn^2肯定是可以超过dn的

• 第二步，去掉系数c，那么剩下就是n^2

• One convinient feature of time complexity and big O notation give you a rough idea about how your function scales as the input to that function gets larger and larger.

在计算机科学中，我们更着眼于大的input， 因为在input足够小的时候，function不会耗费太多时间。

How environment affect Time Complexity

• another feature is that it doesn’t depend on your particular environment.
  • 就算是电脑速度比较慢，仍然是一个linear function，仍然满足T=an+b
  • as long as you are using the same algorithm or same steps for solving the problem.

you still get the same time complexity or same big O.

conventionally (By convention) we use O(1) to indicate the constant complexity.

约定俗成使用O(1)

• 尽管在input的元素比较少的时候O(1)时间复杂度的function耗时有可能大于O(n)的，但是当元素足够多的时候，O(1)还是要好于O(n)的。
• The Big O notation and time complexity allow you to quickly compare multiple functions in terms of their performance. - in this particular example, if you can achieve the same thing, constant time is better then linear time.
• time complexity is not the only factor you should consider. when you trying to decide between multiple potential solutions for a single problem.
  • for example ,you should consider how easy it is to read your code; or how much memory each of your potential functions consumes

我们现在采用的方法是，将不同长度的input跑很多遍，在不同长度上去取他们耗时的平均值，然后再将每个耗时的平均值的点画在xy坐标轴上。但是这样工作量很大。

那么有什么办法不用做实验就可以得到时间复杂度和big O呢？

我们可以问自己，运行每一行代码需要多少时间呢？

第一行和第二行的运行时间都和给的input无关，所以都是O(1)的常数的时间复杂度。

然后你再将所有这些代码的时间加起来去得到整个function所需要耗费的时间。

所以，T = O(1) + O(1) = C1+C2 = C3 = C3 * 1 = O(1)

通常情况下，O(1) = O(1) + O(1) 也是成立的。

O(n)

Two-dimensional array (number of columns = number of rows = n)

so we have n squred elements in that two-dimensional array. qaudratic time. O(n^2)



big O is about time complexity or speed

The Problem with time

• Different machines will record different times.
• The same machine will record different times!
• For fast algorithms, speed measurements may not be precise enough?

IF NOT TIME, Then what?
Rather than counting seconds, which are so variable…
Let’s count the numbe of simple operations the computer has to perform!

Counting Operations

function addUpTo(n){
	return n * (n + 1) / 2;
}

不管n再怎么大，所有操作都是 乘法， 加法，除法。。。O(1)

T = 5n + 2 = O(n)

Examples

We always care about the big picture. 2n or 1000n doesn’t matter …

Just O(n)

n squared call the quadratic time

Some Rules

Constants doesn’t matter

Smaller terms don’t matter

Big O Shorthands

• Analyzing complexity with big O can get complicated
• There are several rules of thumb that can help
• These rules won’t always work, but are helpful starting point

Rules

1. Arithemetic operations are constant
2. Veriable assignment is constant
3. Accessing elements in an array (by index) or object (by key) is constant
4. In a loop, the complexity is the length of the loop times the complexity of whatever happens inside of the loop

More Examples

上面最小5，下面最大5是一个常数

Space Complexity

We can also use big O notation to analyze space complexity:
HOW much additional memory do we need to allocate in order to run the code in our algorithm?
空间复杂度指要用的内存，也可以用big O去分析，当input增加到无穷的时候，需要的内存会怎么变化。