CMU Computer Systems: Dynamic Memory Allocation (Basic Concepts)

Dynamic Memory Allocation

• Programmers use dynamic memory allocators (such as malloc) to acquire VM at run time
  • For data structures whose size is only known at runtime
• Dynamic memory allocators manage an area of process virtual memory known as the heap
• Allocator maintains heap as collection of variable sized blocks, which are either allocated or free
• Types of allocators
  • Explicit allocator: application allocates and frees space
  • Implicit allocator: application allocates, but does not free space
• Will discuss simple explicit memory allocation today

Constraints

• Applications
  • Can issue arbitrary sequence of malloc and free requests
  • free request must be to a malloc’d block
• Allocators
  • Can’t control number or size of allocated blocks
  • Must respond immediately to malloc requests
  • Must allocate blocks from free memory
  • Must align blocks so they satisfy all alignment requirements
  • Can manipulate and modify only free memory
  • Can’t move the allocated blocks once they are malloc’d

Performance Goal: Throughput

• Given some sequence of malloc and free requests
  • $R_0,R_1,\dots ,R_k,\dots ,R_{n-1}$
• Goals: maximize throughput and peak memory utilization
  • These goals are often conflicting
• Throughput
  • Number of completed requests per unit time

Performance Goal: Peak Memory Utilization

• Given some sequence of malloc and free requests
  • $R_0,R_1,\dots ,R_k,\dots ,R_{n-1}$
• Def: Aggregate payload $P_k$
  • malloc (p) results in a block with a payload of p bytes
  • After request $R_k$ has complted, the aggregate payload $P_k$ is the sum of currently allocated payloads
• Def: Current heap size $H_k$
  • Assume $H_k$ is monotonically nondecreasing
• Def: Peak memory utilization after k+1 requests
  • $U_k$ = ($ma{x}_{i\le k}$ $P_i)/H_k$

Fragmentation

• Poor memory utilization caused by fragmentation
  • internal fragmentation
  • external fragmentation
• Internal Fragmentation
  • For a given block, internal fragmentation occurs if payload is smaller than block size
• Caused by
  • Overhead of maintaining heap data structures
  • Padding for alignment purposes
  • Explicit policy decisions
• Depends only on the pattern of previous requests
  • Easy to measure

External Fragmentation

• Occurs when there is enough aggregate heap memory, but no single free block is large enough
• Depends on the pattern of future requests
  • Difficult to measure

Knowing How Much to Free

• Standard method
  • Keep the length of a block in the word preceding the block
    • This word is often called the header field or header
  • Requires an extra word for every allocated block

Keeping Track of Free Blocks

• Method 1: Implicit list using length – links all blocks
• Method 2: Explicit list among the free blocks using pointers
• Method 3: Segregated free list
  • Different free lists for different size classes
• Method 4: Blocks sorted by size
  • Can use a balanced tree (e.g. Red-Black tree) with pointers with free block, and the length used as a key

Implicit List

• For each block we need both size and allocation status
  • Could store this information in two words: wasteful
• Standard trick
  • If blocks are aligned, some low-order address bits are always 0
  • Instead of storing an always-0 bit, use it as a allocated/free flag
  • When reading size word, must mask out this bit

Implicit List: Finding a Free Block

• First fit
  • Search list from beginning, choose first free block that fits
  • Can take linear time in total number of blocks
  • In practice it can cause “splinters” at beginning of list
• Next fit
  • Like first fir, but search list starting where previous search finished
  • Should often be faster than first fit: avoids re-scanning unhelpful blocks
  • Some research suggests that fragmentation is worse
• Best fit
  • Search the list, choose the best free block: fits, with fewest bytes left
  • Keeps fragments small – usually improves memory utilization
  • Will typically run slower than first fit

Implicit List: Allocating in Free Block

• Allocating in a free block: splitting
  • Since allocated space might be smaller than free space, we might want to split the block

Implicit List: Freeing a Block

• Need only clear the “allocated” flag
• But can lead to “false fragmentation”

Implicit List: Coalescing

• Join (coalesce) with next/previous blocks, if they are free
  • Coalescing with next block

Implicit List: Bidirectional Coalescing

• Boundary tags
  • Replicate size/allocated word at “bottom” (end) of free blocks
  • Allows us to traverse the “list” backwards, but requires extra space
  • Important and general technique

Summary of Key Allocator Policies

• Placement policy
  • First-fit, next-fit, best-fit, etc.
  • Trades off lower throughput for less fragmentation
  • Interesting observation: segregated free lists (next lecture) approximate a best fit placement policy without having to search entire free list
• Splitting policy
  • When do we go ahead and split free blocks
  • How much internal fragmentation are we willing to tolerate
• Coalescing policy
  • Immediate coalescing: coalesce each time free is called
  • Deferred coalescing: try to improve performance of free by coalescing until needed

Implicit Lists: Summary

• Implementation: simple
• Allocate cost
  • linear time worst case
• Free cost
  • constant time worst case
  • even with coalescing
• Memory usage
  • will depend on placement policy
  • First-fit, next-fit or best-fit
• Not used in practice for malloc/free because of linear-time allocation
  • used in many special purpose applications
• However, the concepts of splitting and boundary tag coalescing are general to all allocators