1.1 Randomized Algorithms

1 Different Categories of Algorithms

1.1 Different Categories of Algorithms distinguished by design paradigm

• Divide and conquer algorithms
  • quicksort algorithm
  • divide the problem to smaller problems and tackle them separately
  • 分而治之，大化小，然后一个个解决。
• Dynamic programming algorithms
• Mathematical programming algorithms
  • linear programming for certain optimisation problem
• Search and enumeration algorithms
  • Brute force algorithms, enumerating all possible candidate solutions and check 把所有可能列出来然后逐个check
  • Improved brute force algorithms, e.g. branch and bound algorithms
  • Heuristic algorithm
    • deterministic algorithms (Local search), e.g. greedy search
    • Randomized algorithms, which include Evolutionary Computation

📒 Heuristic algorithm在搜索和列举算法下，包括deterministic （比如local search）和Randomized。Randomized里面包括EV。
📒 Heuristic代表‘I found something’

1.2 Heuristic Algorithms

Definition

• a (usually simple) algorithm that produces a good enough solution for a problem in a reasonable time frame

Features

• Heuristic: ‘find’ or ‘discover’
• solutions: (usually) not optimal but satisfactory
• compared to brute force, it is faster
• there exists a trade off among optimality, completeness, accuracy, or precision for speed
• target problem: difficult for other algorithms, e.g. brute force algorithms
• members: deterministic (local search) and randomised algorithms

2 Randomized Algorithms

Definition

• Anhrutistic algorithm that makes random choices during execution to produce a result

Two different ways

• Use some random number generators to make random choices
• Some external disturbances lead to some executions or behaviors can form different ways of doing things

Randomized vs deterministic algorithm

• structure: input -> algorithm -> output
• randomized algorithm adds ‘random number’ to the algorithm

Two categories of randomized algorithms

• Using random numbers to help find a solution to a problem
  Las Vegas algorithm
  Monte Carlo algorithm: 对比Las Vegas多了一个maximum length of trials
  
• Using random numbers to improve a solution to a problem

Monte Carlo vs Las Vegas

• Las Vegas always gives correct results, but need to think about the running time. it won’t stop until the correct results are found.
• Monte Carlo的running time是deterministic的，its results may be incorrect with a certain (typically small) probability. it runs in a fixed number of steps.

📒 Monte Carlo最明显的标志就是，除了常规结束条件，还有一个counter

Advantages/disadvantages of randomized algorithms
Advantages

• Simplicity: easy to implement
• Performance: produce near-optimum solutions with high probability (usually)

Disadvantages

• getting incorrect answer with a finite probability
• find a better solution by repeating the algorithm
• running time is difficult to analyse
• exists a probability of getting an incorrectsolution
• getting truly random numbers is impossible

1.5 Further reading

• Probabilistic Analysis and Randomized Quicksort
• Matching nuts and bolts
• An introduction to randomized algorithms