Exhibiting Good Locality in Your Programs

Well written computer programs tend to exhibit good locality. That is, they tend to reference data items that are near other recently referenced data items, or that were recently referenced themselves. This tendency, known  as the principle of locality, is an enduring concept that has enormous impact on the design and performance of hardware and software system.

      Locality is typically described as having two distinct forms: temporal locality and spatial locality. In a program with good temporal locality, a memory location that is referenced once is likely to be referenced again multiple times in the near future. In a program with good spatial locality, if  a memory location is referenced once, then the program is likely to reference a nearby memory location in the near future.

      Consider the sumarrayrows function below:

int sumarrayrows(int a[M][N])
{
    int sum = 0;
    for (int i=0; i!=M; ++i)
        for (int j=0; j!=N; ++j)
            sum += a[i][j];
    return sum;
}

The doubly nested loop reads the elements of the array in row-major order. That is, the inner loop reads the elements of the first row, then the second row, and so on. The sumarrayrows enjoys good spatial locality because it references the array in the same row-major order that the array is stored.

      Seemingly trivial changes to a program can have a big impact on its locality. For example, the following sumarraycols function computes

int sumarraycols(int a[M][N])
{
    int sum = 0;
    for (int j=0; j!=N; ++j)
        for (int i=0; i!=M; ++i)
            sum += a[i][j];
    return sum;
}

the same result as the sumarrayrows function. The only difference is that we have interchanged the i and j loops. And the sumarraycols function suffers from poor spatial locality because it scans the array column-wise instead of row-size. Since C arrays are laid out in memory row-wise, the result is to visit every Nth element of the contiguous array a.