CMU Computer Systems: The Memory Hierarchy

Memory

• Storage technologies and trends
• Locality of reference
• Caching in the memory hierarchy

Random-Access Memory (RAM)

• Key features
  • RAM is traditionally packaged as a chip
  • Basic storage unit is normally a cell (one bit per cell)
  • Multiple RAM chips form a memory
• RAM comes in two varieties
  • SRAM (Static RAM)
  • DRAM (Dynamic RAM)

Nonvolatile Memories

• DRAM and SRAM are volatile memories
  • Lose information if powered off
• Nonvolatile memories retain value even if powered off
  • Read-only memory (ROM): programmed during production
  • Programmable ROM (PROM): can be programmed once
  • Erasable PROM (EPROM): can be bulk erased
  • Electrically erasable PROM (EEPROM): electronic erase capability
  • Flash memory: EEPROMs. with partial (block-level) erase capability
    • Wears out after about 100,000 erasings
• Uses for Nonvolatile Memories
  • Firmware programs stored in a ROM
  • Solid state disks
  • Disk caches

Traditional Bus Structure Connecting CPU and Memory

• A bus is collection of parallel wires that carry address
• Buses are typically shared by multiple devices

Solid State Disks (SSDs)

• Pages and Blocks
• Data read/written in units of pages
• Page can be written only after its block has been erased
• A block wares out after about 100,000 repeated write

SSD Performance Characteristics

• Sequential access faster than random access
• Random writes are somewhat slower

Locality

• The key to bridging the CPU-Memory gap is a fundamental property of computer programs.
• Principle
  • Programs tend to use data and instructions with address near or equal to those they have used recently
• Temporal locality
  • Recently referenced items are likely to be referenced again in the near future
• Spatial locality
  • Items with nearby addresses tend to be referenced close together in time

Qualitative Estimates of Locality

Claim: Being able to look at code and get a qualitative sense of its locality is a key skill for a professional programmer

Memory Hierarchy

• Regs: CPU registers hold words retrieved from the L1
• L1 cache: hold cache lines retrieved from the L2
• L2 cache: hold cache lines retrieved from the L3
• L3 cache: hold cache lines retrieved from main memory
• Main memory (DRAM)
• Local secondary storage
• Remote secondary storage

Caches

• A smaller, faster storage device that acts as a staging area for a subset of the data in a larger, slower device
• Fundamental idea
  • For each k, the faster, smaller device at level k serves as a cache for the larger, slower device at level k+1

Types of Cache Misses

• Cold (compulsory) miss
  • the cache is empty
• Conflict miss
  • Most caches limit blocks at level k+1 to a small subset
• Capacity miss
  • the set of active cache blocks is larger than the cache

Summary

• The speed gap between CPU, memory and mass storage continues to widen,
• Well-written programs exhibit a property called locality.
• Memory hierarchies based on caching close the gap by exploiting locality.