Introduction and Algorithm Analysis
Basic Definitions
-
Type: a collection of values
Boolean type: collection of true or false value
Integer type: collection of 1,2,3… value
Data type: a type and a collection of operations that manipulate the type
Integer data type: Integer type and + - x ÷ operations -
Data Item: a piece of information of a record drawn from a data type
-
Atomic Data Type: contain no subparts
e.g. Integer, Boolean -
Structure Data Type: contain several pieces of information
e.g. Array, String -
Abstract Data Type: a definition for a data type solely in terms of a set of values and a set of operations on that data type. ADT operation is defined by its inputs and outputs Hide implementation details (Encapsulation)
-
Data Structure: the physical implementation of an ADT Operations associated with the ADT are implemented by subroutines (functions)
Usually refers to an organization for data in main memory -
File Structure: an organization for data on peripheral storage
e.g. Disk drive
Logical vs Physical Form
-
Logical Form : Definition of the data item within an ADT
e.g. Integers in mathematical sense: +, - -
Physical Form : Implementation of the data item within a data structure
e.g. 16/32 bit integers: overflow
Problem, Algorithm and Program
- Problem: A task to be performed
- Algorithm: a method to solve a problem
Correct
No ambiguity
A series of concrete steps
A finite number of steps
Terminate
- Program: an instance for an algorithm in some programming languages
Algorithm Evaluation
- Running Time T(n) : Some function T on input size n
Examples :
sum=0
T ( n ) = c T(n)=c T(n)=c
c : the running time to assign a value to a variable
for i=1 to n for j=1 to n sum=sum+1 end end
T ( n ) = c + c n 2 T(n)=c+cn^2 T(n)=c+cn2
The c not related to n is not so important
s1=1; s2=1; s3=1;
T ( n ) = c T(n)=c T(n)=c
for a=1 ro n/2 s1=s1+s1; end
T ( n ) = c n / 2 T(n)=cn/2 T(n)=cn/2
for b=1 to n*n s2=s2*s2 end
T ( n ) = c n 2 T(n)=cn^2 T(n)=cn2
for i=1 to n for j=1 ro log(n) s3=s1+s2+s3; end end
T ( n ) = c n l o g n T(n)=cnlog n T(n)=cnlogn
s1=1; s2=1; s3=1; for a=1 ro n/2 s1=s1+s1; end for b=1 to n*n s2=s2*s2 end for i=1 to n for j=1 ro log(n) s3=s1+s2+s3; end end
T ( n ) = c + c n / 2 + c n 2 + c n l o g n T(n)=c+cn/2 +cn^2 +cnlog n T(n)=c+cn/2+cn2+cnlogn
- Best, Worst, Average Cases
- The Change of The Complexity
- Big-Oh
- Indicates the upper bound of a growth rate
- For T(n) a non-negatively valued function, T(n) is in the set
O
(
f
(
n
)
)
O(f (n))
O(f(n)) if there exist two positive constants
c
c
c and
n
0
n_0
n0such that
T
(
n
)
≤
c
f
(
n
)
T(n) ≤ cf (n)
T(n)≤cf(n) for all
n
>
n
0
n > n_0
n>n0
- n 0 n_0 n0 is the smallest value of n for which the claim of an upper bound holds true
- Actually value of c c c is irrelevant
- Algorithm Analysis: Big-Oh
O ( 1 ) : c o n s t a n t O(1): constant O(1):constant
O ( l o g 2 n ) : l o g a r i t h m i c O(log2^n): logarithmic O(log2n):logarithmic
O ( O( O( log 2 2 n \log_22^n log22n ) : ): ): l o g s q u a r e d log squared logsquared
O ( n ) : l i n e a r O(n): linear O(n):linear
O ( n l o g 2 n ) : n l o g n O(nlog_2n): n log n O(nlog2n):nlogn
O ( n 2 ) : q u a d r a t i c O(n^2): quadratic O(n2):quadratic
O ( n 3 ) : c u b i c O(n^3): cubic O(n3):cubic
O ( 2 n ) : e x p o n e n t i a l O(2^n): exponential O(2n):exponential
- Big-Oh
Examples
T ( n ) = 3 n 2 T(n)=3n^2 T(n)=3n2
T ( n ) T(n) T(n) is in O ( n 2 ) O(n^2) O(n2)
T ( n ) = 3 n 2 + n T(n)=3n^2+n T(n)=3n2+n
T ( n ) T(n) T(n) is in O ( n 2 ) O(n^2) O(n2)
T ( n ) = c + c n / 2 + c n 2 + c n l o g n T(n)=c+cn/2+cn^2+cnlog n T(n)=c+cn/2+cn2+cnlogn
ps. n > l o g n , a s n → ∞ n>log n, as\ n\rightarrow \infty n>logn,as n→∞
T ( n ) T(n) T(n) is in O ( n 2 ) O(n^2) O(n2)
- Big-Omega
- Indicates the lower bound of a growth rate
- For
T
(
n
)
T(n)
T(n) a non-negatively valued function,
T
(
n
)
T(n)
T(n) is in the set
Ω
(
g
(
n
)
)
Ω(g(n))
Ω(g(n)) if there exist two positive constants
c
c
c and
n
0
n_0
n0 such that
T
(
n
)
≥
c
g
(
n
)
T(n) ≥ c g(n)
T(n)≥cg(n) for all
n
>
n
0
n > n_0
n>n0
- n 0 n_0 n0 is the smallest value of n for which the claim of an upper bound holds true
- The actually value of c is irrelevant
Examples
T ( n ) = 3 n 2 T(n)=3n^2 T(n)=3n2
T ( n ) T(n) T(n) is in Ω ( n 2 ) Ω(n^2) Ω(n2)
T ( n ) = 3 n 2 + n T(n) = 3n^2 + n T(n)=3n2+n
T ( n ) T(n) T(n) is in Ω ( n 2 ) Ω(n^2) Ω(n2)
(与之前的区别是系数不一样)
- Big-Theta
- An algorithm is said to be Θ ( h ( n ) ) Θ(h(n)) Θ(h(n)) if it is in O ( h ( n ) ) O(h(n)) O(h(n)) and Ω ( h ( n ) ) Ω(h(n)) Ω(h(n))
Example
What is the Big-O, Big-Omega and Big-Theta of the following program?
sum1 = 0; for(i=1;i<=n;i++) //first double loop for(j=1;j<=n;j++) //do n times sum1++; sum2 = 0; for(i=1;i<=n;i++) //second double loop for(j=1;j<=i;j++) //do i times sum2++;
T ( n ) = c 1 + c 2 n 2 + c 1 + c 2 n ( n + 1 ) / 2 T(n) = c_1 + c_2n^2 + c_1 + c_2n(n+1)/2 T(n)=c1+c2n2+c1+c2n(n+1)/2
(注意第二个循环)
Ω ( n 2 ) , O ( n 2 ) , Θ ( n 2 ) Ω(n^2), O(n^2), Θ(n^2) Ω(n2),O(n2),Θ(n2)
How many elements are examined in worst case?(二分查找)
// Return position of element in sorted // array of size n with value K. int binary(int array[], int n, int K) { int l = -1; int r = n; // l, r are beyond array bounds while (l+1 != r) { // Stop when l, r meet int i = (l+r)/2; // Check middle if (K < array[i]) r = i; // Left half if (K == array[i]) return i; // Found it if (K > array[i]) l = i; // Right half } return n; // Search value not in array }
T ( n ) = T ( n / 2 ) + 1 , w h e r e n > 1 a n d T ( 1 ) = 1 T(n) = T(n/2) + 1,where \ n>1\ and\ T(1) = 1 T(n)=T(n/2)+1,where n>1 and T(1)=1 (比少一半多了一次‘二分’过程)
Therefore, T ( n ) = l o g 2 n + 1 T(n) = log_2 n + 1 T(n)=log2n+1
Cost is Θ ( l o g 2 n ) Θ(log_2 n) Θ(log2n)
Other Control Statements
- While loop : Analyze like a for loop
- If statement : Take greater complexity of then/else clauses
- Switch statement : Take complexity of most expensive case
- Subroutine call : Complexity of the subroutine
- Reduce time but sacrifice space
- Encoding or packing information
- Table lookup