CMU Computer Systems: Program Optimization

Optimization

• Overview
• Generally Useful Optimizations
  • Code motion/precomputation
  • Strength reduction
  • Sharing of common subexpressions
  • Removing unnecessary procedure calls
• Optimization Blockers
  • Procedure calls
  • Memory aliasing
• Exploiting Instruction-Level Parallism
• Dealing with Conditionals

Performance Realties

• There’s more to performance than asymptotic complexity
• Constant factors matter tool
  • Easily see 10:1 performance range depending on how code is written
  • Must optimize at multiple levels:
    • algorithm, data representations, procedures, and loops
• Must understand system to optimize performance
  • How programs are compiled and executed
  • How modern processors + memory systems operate
  • How to measure program performance and identify bottlenecks
  • How to improve performance without destroying code modular generality

Optimizing Compilers

• Provide efficient mapping of program to machine
• Don’t (usually) improve asymptotic efficiency
• Have difficulty overcoming “optimization blockers”

Limitations of Optimizing Compilers

• Operate under fundamental constraint
  • Must not cause any change in program behavior
  • Often prevents it from making optimizations that would only affect behavior under pathological conditions
• Behavior that may be obvious to the programmer can be obfuscated by languages and coding styles
• Most analysis is performed only within procedures
  • Whole-program analysis is too expensive in most cases
  • Newer versions of GCC do interprocedural analysis within individual files
• Most analysis is based on static information
• When in doubt, the compiler must be conservative

Generally Useful Optimizations

• Optimizations that you or the compiler should do regardless of processor / compiler
• Code Motion
  • Reduce frequency with which computation performed
    • If it will always procedure same result
    • Especially moving code out of loop
  • Reduction in Strength
    • Replace costly operation with simpler one
    • Shift, and instead of multiply or divide
    • Recognize sequence of products
  • Share Common Subexpressions
    • Reuse portions of expressions
    • GCC will do this with -O1

Optimization Blocker #1: Procedure Calls

• Why couldn’t compiler move strlen out of inner loop
  • Procedure may have side effects
    • Alters global state each time called
  • Function may not return same value for given arguments
    • Depends on other parts of global state
    • Procedure lower could interact with strlen
• Warning
  • Compiler treats procedure call as a black box
  • Weak optimizations near them
• Remedies
  • Use of inline functions
  • Do your own code motion

Optimization Blocker #2: Memory Aliasing

• Aliasing
  • Two different memory references specify single location
  • Easy to have happen in C
    • Since allowed to do address arithmetic
    • Direct access to storage structures
  • Get in habit of introducing local variables
    • Accumulating within loops
    • Your way of telling compiler not to check for aliasing

Exploiting Instruction-Level Parallelism

• Need general understanding of modern
  • Hardware can execute multiple instructions in parallel
• Performance limited by data dependencies
• Simple transformations can yield dramatic performance improvement
  • Compilers often cannot make these transformations
  • Lack of associativity and distributivity in floating-point arithmetic

Cycles Per Element (CPE)

• Convenient way to express performance of program that operates on vectors in lists
• Length = n
• In our case: CPE = cycles per OP
• T = CPE*n + Overhead
  • CPE is slope of line

Superscalar Processor

• Definition
  • A superscalar processor can issue and execute multiple instructions in one cycle. The instructions are retrieved from a sequential instruction stream and are usually scheduled dynamically.
• Benefit
  • without programming effort, superscalar processor can take advantage of instruction level parallelism that most programs have

Pipelined Functional Units

• Divide computation into stages
• Pass partial computations from stage to stage
• Stage i can start on new computation once values passed to i+1

Unrolling & Accumulating

• Idea
  • Can unroll to any degree L
  • Can accumulate K results in parallel
  • L must be multiple of K
• Limitations
  • Diminishing returns
    • Cannot go beyond throughput limitations of execution units
  • Large overhead for short lengths
    • Finish off iterations sequentially

Using Vector Instructions

• Make use of AVX Instructions
  • Parallel operations on multiple data elements
  • See Web Aside OPT: SIMD on CS: APP web page

Branch Prediction

• Idea
  • Guess which way branch will go
  • Begin executing instructions at predicted position
    -But don’t actually modify register or memory data

Branch Misprediction Recovery

• Performance Cost
  • Multiple clock cycles on modern processor
  • Can be a major performance limiter

Getting High Performance

• Good compiler and flags
• Don’t do anything stupid
  • Watch out for hidden algorithmic inefficiencies
  • Write compiler-friendly code
    • Watch out for optimization blockers
  • Look carefully at innermost loops
• Turn code for machine
  • Exploit instruction-level parallelism
  • Avoid unpredictable branches
  • Make code cache friendly (Covered later in course)