Chapter 6-03

Please indicate the source: http://blog.csdn.net/gaoxiangnumber1
6.4.5 Issues with Writes
Suppose we write a word w that is already cached (a write hit). After the cache updates its copy of w, what does it do about updating the copy of w in the next lower level of the hierarchy?
The simplest approach, write-through, is to immediately write w’s cache block to the next lower level. Write-through has the disadvantage of causing bus traffic with every write.
Another approach, write-back, defers the update as long as possible by writing the updated block to the next lower level only when it is evicted from the cache by the replacement algorithm. Because of locality, write-back can significantly reduce the amount of bus traffic, but it has the disadvantage of additional complexity. The cache must maintain an additional dirty bit for each cache line that indicates whether or not the cache block has been modified.
Another issue is how to deal with write misses.
One approach, known as write-allocate, loads the corresponding block from the next lower level into the cache and then updates the cache block. Write-allocate tries to exploit spatial locality of writes, but it has the disadvantage that every miss results in a block transfer from the next lower level to cache.
The alternative, known as no-write-allocate, bypasses the cache and writes the word directly to the next lower level. Write-through caches are typically no-write-allocate. Write-back caches are typically write-allocate.
6.4.6 Anatomy of a Real Cache Hierarchy
A cache that holds instructions only is called an i-cache. A cache that holds program data only is called a d-cache. A cache that holds both instructions and data is known as a unified cache.
Modern processors include separate i-caches and d-caches. Reasons for this: With two separate caches, the processor can read an instruction word and a data word at the same time. I-caches are typically read-only, and thus simpler. The two caches are often optimized to different access patterns and can have different block sizes, associativities, and capacities. Also, having separate caches ensures that data accesses do not create conflict misses with instruction accesses, and vice versa, at the cost of a potential increase in capacity misses.

Each CPU chip has four cores. Each core has its own private L1 i-cache, L1 d-cache, and L2 unified cache. All of the cores share an on-chip L3 unified cache.

6.4.7 Performance Impact of Cache Parameters
Cache performance is evaluated with a number of metrics:
*Miss rate. The fraction of memory references during the execution of a program, or a part of a program, that miss. It is computed as misses/references.
*Hit rate. The fraction of memory references that hit. It is computed as 1 − miss rate.
*Hit time. The time to deliver a word in the cache to the CPU, including the time for set selection, line identification, and word selection.
*Miss penalty. Any additional time required because of a miss.
On the one hand, a larger cache will tend to increase the hit rate. On the other hand, it is always harder to make large memories run faster. As a result, larger caches tend to increase the hit time. This is especially important for on-chip L1 caches that must have a short hit time.
Large blocks are a mixed blessing. On the one hand, larger blocks can help increase the hit rate by exploiting any spatial locality that might exist in a program. However, for a given cache size, larger blocks imply a smaller number of cache lines, which can hurt the hit rate in programs with more temporal locality than spatial locality. Larger blocks also have a negative impact on the miss penalty, since larger blocks cause larger transfer times. Modern systems usually compromise with cache blocks that contain 32 to 64 bytes.
Associativity: The issue here is the impact of the choice of the parameter E, the number of cache lines per set. The advantage of higher associativity (i.e., larger values of E) is that it decreases the drawback of the cache to thrashing due to conflict misses. However, higher associativity comes at a significant cost. Higher associativity is expensive to implement and hard to make fast. Higher associativity can increase hit time, because of the increased complexity, and it can also increase the miss penalty because of the increased complexity of choosing a victim line.
Write-through caches are simpler to implement and can use a write buffer that works independently of the cache to update memory. Furthermore, read misses are less expensive because they do not trigger a memory write. On the other hand, write-back caches result in fewer transfers, which allows more bandwidth to memory for I/O devices that perform DMA. Further, reducing the number of transfers becomes increasingly important as we move down the hierarchy and the transfer times increase. In general, caches further down the hierarchy are more likely to use write-back than write-through.
Distinction between cache lines, sets, and blocks.
*A block is a fixed-sized packet of information that moves back and forth between a cache and main memory (or a lower-level cache).
*A line is a container in a cache that stores a block, as well as other information such as the valid bit and the tag bits.
*A set is a collection of one or more lines. Sets in direct-mapped caches consist of a single line. Sets in set associative and fully associative caches consist of multiple lines.
6.5 Writing Cache-friendly Code
The basic approach try to ensure that our code is cache friendly:
1.Make the common case go fast. Programs often spend most of their time in a few core functions. These functions often spend most of their time in a few loops. So focus on the inner loops of the core functions and ignore the rest.
2.Minimize the number of cache misses in each inner loop. All other things being equal, such as the total number of loads and stores, loops with better miss rates will run faster.

In general, if a cache has a block size of B bytes, then a stride-k reference pattern (where k is expressed in words) results in an average of min (1, (word_size × k) / B) misses per loop iteration. This is minimized for k = 1, so the stride-1 references to v are indeed cache friendly.
Suppose that v is block aligned, words are 4 bytes, cache blocks are 4 words, and the cache is initially empty (a cold cache). Then, regardless of the cache organization, the references to v will result in the following pattern of hits and misses:

The reference to v[0] misses and the corresponding block, which contains v[0]–v[3], is loaded into the cache from memory. Thus, the next three references are all hits. The reference to v[4] causes another miss as a new block is loaded into the cache, the next three references are hits, and so on.
Two important points about writing cache-friendly code:
1.Repeated references to local variables are good because the compiler can cache them in the register file (temporal locality).
2.Stride-1 reference patterns are good because caches at all levels of the memory hierarchy store data as contiguous blocks (spatial locality).

Since C stores arrays in row-major order, the inner loop of this function has the same desirable stride-1 access pattern as sumvec. For example, suppose we make the same assumptions about the cache as for sumvec. Then the references to the array a will result in the following pattern of hits and misses:


If the entire array fits in the cache, then we will enjoy the same miss rate of 1/4. However, if the array is larger than the cache (the more likely case), then each and every access of a[i][j] will miss!

6.6 Putting It Together: The Impact of Caches on Program Performance
6.6.3 Exploiting Locality in Your Programs
Techniques to write more efficient programs
1.Focus your attention on the inner loops, where the bulk of the computations and memory accesses occur.
2.Try to maximize the spatial locality in your programs by reading data objects sequentially, with stride 1, in the order they are stored in memory.
3.Try to maximize the temporal locality in your programs by using a data object as often as possible once it has been read from memory.
Please indicate the source: http://blog.csdn.net/gaoxiangnumber1.