Python Algorithms Learning Notes（1）--Asymptotic Notations

Lately , I’ve been reading《Python Algorithms》by Magnus Lie Hetland.
To understand the contents better , I’ll keep notes about it through the whole process.
It’s my first time to try to make notes in English only , hope I’ll make progress in both writing and algorithms after fininshing this book.

Here we go.
Today, I finished reading the first chapter and half of the chapter 2.

Chapter 1 Introduction

1. Write down the problem.
2. Think real hard.
3. Write down the solution.
   　　　　　　　　　　　　　　　　　“The Feynman Algorithm”
   　　　　　　　　　　　　　as described by Murray Gell-Mann

For Chapter 1 , it’s introduction. This book seems to mainly focus on algorithms which are more high-level than those basic data structures like linked list. And I’m happy to know that many important concepts like sorting, searching and hashing are already available in Python or its standard libraries. Personally I feel I don’t have enough time to start from scratch ,so I’m quite satisfied with this book’s contents.

Chapter 2 The basics

Tracey: I didn’t know you were out there.
Zoe: Sort of the point. Stealth—you may have heard of it.
Tracey: I don’t think they covered that in basic.
　　　　　　　　　　　　
　　　　　　　　　　　　　　From “The Message,” episode 14 of Firefly

For Chapter 2, it tells the basic concepts and terminologies we need to know to describe and evalute an algorithm.

• First problem is how we define “algorithm”?
  There is a very important concept —”Infinite State Machine”
  Infinite State Machine isn’t new to me ,I’ve learnt about it in several classes.These machines are simple yet surprisingly enough to implement any possible form of computation.
  Infinite State Machine is called Turing Machine , which enables us to depict all kinds of algorithm workflow in one model.
  With progress made in memories , the model is now called “Random Access Machine”，for we use big chunk of directly accessible memories now.

• Now we have hardware we can run algorithms on ,the problem is how to note the problem .

A problem is a relation between input and output.
A relation (in the mathematical sense) is a set of pairs
In our case, which outputs are acceptable for which inputs.
By specifying this relation, we’ve got our problem nailed down.
The elements of input is problem instance , the relation is the actual problem

• Asymptotic Notation
  To evaluate the performance of an algorithm , we have notions for running time complexity as $O(g)$ , $\Omega(g)$ and $\theta(g)$
  The running time of an algorithm depends on two aspects: the size of input and the hardware condition.
  The “good” algorithms will increase at a reasonable rate when the input size grows . Considering the various hardware conditions, we choose a description system regardless of the hardware.
  We let the running time be the number of times a certain basic operation is performed

$O(g)$ is actually the notation in calculus for high order infinetesiaml ,we use it to describe the upper bound of running time.

For example,for a simple program like:

sum = 0
for i in range(n)
    sum = sum + i

There are n times of addtions implemented inside the loop and one initialization step outside the loop ,so the running time is n+1 ,easy to see , $n+1 \in O(n)$ , so we say it complexity is $O(n)$
we leave out all the unimportant part of the formula for simplisity.

$\Omega(g)$ is for lower bound
$\theta(g)$ . By supplying an upper bound and a lower bound at the same time, the operator is the most informative of the three.

Below is the chart of common examples of asymptotic running Times , I think it’s really useful.

Complexity	Name	Examples,Comments
$\theta(1)$	Constant	Hash table lookup and modification
$\theta(\lg n)$	Logarithmic	Binary search,Logarithm base unimportant.
$\theta(n)$	Linear	Iterating over a list.
$\theta(n\lg n)$	Loglinear	Optimal sorting of arbitrary values Same as (lg n!).
$\theta(n^2)$	Quadratic	Comparing n objects to each other
$\theta(n^3)$	Cubic	Floyd and Warshall’s algorithms
$\theta(n^k)$	Polynomial	k nested for loops over n (if k is pos. integer). For any constant k > 0.
$\Omega(k^n)$	Exponential	Producing every subset of n items (k = 2). Any k > 1.
$\theta(n!)$	Factorial	Producing every ordering of n values

That’s all for the first article , later on , I’ll learn about implementing graphs and trees.

I deeply realize how poor my English is writing this. Hope I’ll get better day by day.

Reference
《Python Algorithms》by Magnus Lie Hetland.