Algorithms - Week 1-1 Analysis of Algorithms

Introduction

Primary practical reason: avoid performance bugs

A framework for predicting performance and comparing algorithms.

Scientific method.

• Observe some feature of the natural world
• Hypothesize a model that is consistent with the observations
• Predict events using the hypothesis
• Verify the predictions by making further observations
• Validate by repeating until the hypothesis and observations agree.

Principles

• Experiments must be reproducible.
• Hypotheses must be falsifiable.

Observations

Mathematical Models

cost model, tilde notation

How to estimate a discrete sum?

• A.1 Take discrete mathematics course.
• A.2 Replace the sum with an integral, and use calculus!

Order-of-Growth Classifications

Theory of Algorithms

Best case. Lower bound on cost. Determined by “easiest” input. Provides a goal for all inputs.

Worst case. Upper bound on cost. Determined by “most difficult” input. Provides a guarantee for all inputs.

Average case. Expected cost for random input. Need a model for “random” input. Provides a way to predict performance.

Goals.

• Establish “difficult” of a problem
• Develop “optimal” algorithms.

Upper bound. A specific algorithm.

Lower bound. Proof that no algorithm can do better.

Optimal algorithm. Lower bound equals upper bound (to within a constant factor).

Algorithm design approach

Start.

• Develop an algorithm.
• Prove a lower bound.

Gap?

• Lower the upper bound (discover a new algorithm).
• Raise the lower bound (more difficult).

Memory

summary

Empirical analysis.

• Execute program to perform experiments.
• Assume power law and formulate a hypothesis for running time.
• Model enables us to make predictions.

Mathematical analysis.

• Analyze algorithm to count frequency of operations.
• Use tilde notation to simplify analysis.
• Model enables us to explain behavior.

Scienfic method.

• Mathematical model is independent of a particular system; applies no machines not yet built.
• Empirical analysis is necessary to validate mathematical models and to make predictions.