XFS文件系统在SUSE Linux下最佳调优方案

https://www.suse.com/communities/conversations/practical-xfs-suse-linux-enterprise/

As discussed in this excellent Conversations article posted a few months ago, XFS has a lot to offer as an enterprise grade file system. SUSE has held to this position for years, and with XFS chosen as the default file system on RHEL 7 Beta, it appears that the engineers at Red Hat now agree.

I will admit that I had some misconceptions about XFS when I was tasked to rebuild a heavily used file/web server. Much of that misconception was based on outdated information and benchmarks that did not represent realistic workloads. After getting to know XFS a little better, I found that it was an excellent file system for the “real life” enterprise server. In this article, I will talk about my project, some conservative XFS tuning that I applied, and what I learned.
The System

The server in question is an HP ProLiant with 3TB x 12 HDs in RAID 6 configuration (effective 30 TB capacity). 27 TB of the system is partitioned for content. This partition is currently comprised of 7 million regular files within 2 million directories, utilizing about 13 TB of storage. These files range from 1KB to 6GB in size. In addition, multiple layers of symbolic links are used throughout the server. The number of files and the overall storage are projected to grow 10-20% per year.

The system is used for the following:

150 rsync transmissions and 50 rsync receives per day, covering different content areas. Each content area is scanned between 8 and 24 times, and a typical day may see anywhere from a few dozen to a couple of thousand files transferred.
3.5 million HTTP hits per day
10-100 FTP uploads and downloads daily
File management thread, which constantly scans for outdated files, broken symbolic links, etc.

XFS Tuning

Many XFS tuning guides are outdated and designed for benchmarks rather than actual workloads. Most Linux and XFS documentation state that the default XFS configuration is sufficient for most applications, and I completely agree. However, I found that some conservative tuning was beneficial in my case. While this is in no way intended to be a comprehensive tuning guide, here are some options that worked for me:
Format Options

-d su=64k,sw=10

The su/sw options are for defining the stripe geometry of the partition. This is only necessary when using hardware RAID controllers that do not export this information. The su value should be set to the RAID chunk size (most common is 64k, but check the manual) and the sw value should be a multiple of the number of data disks in your array. If you are using hardware RAID and are not sure if this information will be available to mkfs, then it’s a good idea to define it.

It used to be a good idea to manually define a larger log size and agcount, but I have found that mkfs currently does a good job of picking reasonable values based on your partition size. In my case, it defaulted to 32 allocation groups.
Mount Options

nobarrier,noatime,logbufs=8

Since this server’s RAID controller has a backup for the write cache, disabling the write barrier for XFS was a safe and highly useful optimization. This can also be safely disabled if you do not have write cache available. The noatime option does not offer much improvement over the default relatime, but it doesn’t hurt anything in my case.

The logbufs option tells the system how much journaling information to keep in memory. The default is 2, and the maximum is 8. Setting this to the maximum of 8 will only use an extra 192K of memory and will vastly improve the performance of mass transactions, particularly file deletions. You can also increase the size of the buffer from the default of 32k, but I did not find it to be necessary.
What I learned

Unfortunately, I did not record any objective measurements of XFS vs ext3 or of any of the above optimizations. However, I have noticed that many operations were noticeably faster when using XFS instead of ext3. This was particularly true of find and rsync operations, which ran anywhere from 10% to 50% faster. The performance difference was more noticeable when running these operations against a large set of files (more that 200k). Applying the above optimizations improved performance further, by a small but noticeable amount.

In the end, I have found XFS to be a very capable file system for a heavily used file server. It certainly is a major improvement over ext3, and I find it difficult to imagine any existing file system providing significant additional performance over XFS. I have also found that some conservative tuning provides a noticeable improvement in my case, although these optimizations would likely be negligible on a less busy system.

XFS has changed a lot over the last few years. If you have not taken a look at it lately, you may be surprised what it can do for you.