Gain 30% Linux Disk Performance with noatime, nodiratime, and relatime

This is post #8 in my December 2013 series about Linux Virtual Machine Performance Tuning. For more, please see the tag “Linux VM Performance Tuning.”

According to Red Hat’s Enterprise Linux 6 Power Management guide, the POSIX family of standards for operating systems “requires that operating systems maintain file system metadata that records when each file was last accessed.” This is called “atime” (a is for access) and is one of three timestamps Linux filesystems keep for their files (the other two are mtime and ctime, modify and change times). You can see these with the ‘stat’ command:

$ stat /bin/ls
 File: `/bin/ls'
 Size: 109208 Blocks: 216 IO Block: 4096 regular file
Device: fd00h/64768d Inode: 519 Links: 1
Access: (0755/-rwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2013-12-07 21:29:08.068475219 -0600
Modify: 2013-05-23 08:10:31.000000000 -0500
Change: 2013-06-11 19:12:14.577018287 -0500

On the surface this sounds like a good thing to track, but like many decisions it has unintended consequences: every read operation on a filesystem isn’t just a read operation, it’s a write operation, too, since the atime needs to be updated.

Reads are easy for systems to do, and they’re easy for systems to cache. Writes are hard to do, as they need to be done in a timely fashion, in a guaranteed order, journaled, etc. Doing a write for every read is a sure way to sap performance, especially since many executables need to dynamically load system libraries. Each time a library is accessed, even from cache, there are writes being made to update atime. Back in 2007, Ingo Molnar noted that with atime enabled “[Linux gives] Windows a 20%-30% performance edge, for almost nothing.”

How do I get my 30% disk performance back?

Simply add “noatime” to your filesystem mount options in /etc/fstab to prevent atime from being updated on file and directory accesses:

/dev/mapper/Volume00-root       /       ext4    defaults,noatime        1       1

You can also use Puppet’s mount type to control this:

mount { '/':
 options => 'defaults,noatime',
 pass => '1',
 dump => '1',
}

Be careful, though. Some applications, like tmpwatch, some backup programs, and some things having to do with email (mutt, pine, sendmail, etc.) care about atime. In those cases you have two options. First, commenter Mike Lowe pointed out that noatime implies nodiratime. If you have an application that cares about file atimes you might gain some performance with using just nodiratime instead of noatime to cover the directories but not the files.

Second, “relatime” is potentially a better option that only updates atime if the file or directory has been modified since the last atime update. You might choose to use ‘noatime’ on most of your filesystems but leave /var/spool and /tmp as ‘relatime':

mount { '/var/spool':
 options => 'defaults,relatime',
 pass => '2',
 dump => '1',
}

You could also switch tmpwatch to use mtime or ctime, or eliminate it completely and use Puppet’s ‘tidy’ directive. A little testing will go a long way here.

Linux kernel 2.6.30 and newer, as well as the kernels with Red Hat Enterprise Linux 6, mount all filesystems with relatime by default, providing yet another example of why you should use the most recent OS you can. I’d still suggest that, no matter which distribution or kernel you use, you explicitly add the option for the behavior you want.

Reference: https://lonesysadmin.net/2013/12/08/gain-30-linux-disk-performance-noatime-nodiratime-relatime/

List all mounts

cat /proc/mounts

List NFS mounts

nfsstat -m