CMU Computer Systems: Machine-Level Programming (Advanced)

Advanced

• Memory Layout
• Buffer Overflow
  • Vulnerability
  • Protection
• Unions

x86-64 Linux Memory Layout

• Stack
  • Runtime stack (8MB limit)
  • E.g., local variables
• Heap
  • Dynamically allocated as needed
  • When call malloc(), calloc(), new()
• Data
  • Statistically allocated data
  • E.g., global vars, static vars, string constants
• Text / Shared Libraries
  • Executable machine instructions
  • Read-only

A big deal

• Generally called a “buffer overflow”
  • when exceeding the memory size allocated for an array
• Why a big deal?
  • It’s the #1 technical cause of security vulnerabilities
• Most common form
  • Unchecked lengths on string inputs
  • Particularly for bounded character arrays on the stack
    • sometimes referred to as stack smashing

Code Injection Attacks

• Input string contains byte representation of executable code
• Overwrite return address A with address of buffer B
• When original program execute ret, will jump to exploit code

Aside: Worms and Viruses

• Worm: A program that
  • Can run by itself
  • Can propagate a fully working version of itself to other computers
• Virus: Code that
  • Adds itself to other programs
  • Does not run independently
• Both are (usually) designed to spread among computers and to wreak havoc

What to do about buffe overflow attacks

• Avoid overflow vulnerabilities
  • fgets insead of gets
  • strncpy instead of strcp
  • don’t use scanf with %s conversion specification
• Employ system-level protections
  • Randomized stack offsets
    • At start of program, allocate random amount of space on stack
    • Shifts stack addresses for entire program
    • Makes it difficult for hacker to predict beginning of inserting code
  • Nonexecutable code segments
    • traditional x86 can execute anything readable
    • x86-64 added explicit “execute” permission
    • Stack marked as non-executable
• Have compiler use “stack canaries”
  • Place special value (“canary”) on stack just beyond buffer
  • Check for corruption before exiting function
  • GCC now is default

Return-Oriented Programming Attacks

• Challenge (for hackers)
  • Stack randomization makes it hard to predict buffer location
  • Marking stack nonexecutable makes it hard to insert binary code
• Alternative Strategy
  • Use existing code
  • String together fragments to achieve overall desired outcomde
  • Does not overcome stack canaries
• Construct program from gadgets
  • Sequence of instructions ending in ret
    • Encoding by single byte 0xc3
  • Code positions fixed from runt to run
  • Code is executable

ROP Execution

• Trigger with ret instruction
  • Will start executing gadget 1,2,3,…
• Final ret in each gadget will start next one

Union Allocation

• Allocate according to largest element
• Can only use one field at a time

Summary of Compound Types in C

• Arrays
  • Contiguous allocation of memory
  • Aligned to satisfy every element’s alignment requirement
  • Pointer to first element
  • No bounds checking
• Structures
  • Allocate bytes in order declared
  • Pad in middle and at end to satisfy alignment
• Unions
  • Overlay declarations
  • Way to circumvent type system