Fio is short for Flexible I/O, a versatile IO workload generator. It was original developed by Jens Axboe, the backbone behind and author of the IO stack in the Linux kernel.
- It can be used to collect all sorts of I/O performance information, including complete IO latencies and percentiles.
- It supports different types of I/O engines (e.g., sync, mmap, libaio, posixaio, network, etc), rate I/O, forked or threaded jobs.
- It provides many options to define different workloads (e.g., random and sequential IO mix, or read/write mix).
Install and Run fio
- Make sure libaio-dev / libaio-devel / libaio is installed.
- Use apt-get to install fio on Ubuntu; or download source code of the latest version from GitHub (Link), and build & install it.
-
There are two ways to run fio: 1) passing a config file specifying workload options; 2) passing workload options as command line parameters. The following shows a simple example, where we create a config file random_read.fio.
12345678#Job name between brackets
[random_read]
#Random Read test
rw=randread
#Transfer 1024MB data
size=1024m
-
Run testing using this configure file, which will test the I/O performance for randomly reading 1024MB data under current directory (other unspecified options remains default)
fio ./random_read.fio
This is the same as the following, where options are specified in command line.
fio --name=rand_read --rw=randread --size=1024m
-
The output will be shown as follows. It displays plenty of performance information (e.g., bw=20972KB/s, iops=5242). The interpretation of output can be found here.
random-read: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=1
fio-2.1.5-19-gdc63
Starting 1 process
random-read: Laying out IO file(s) (1 file(s) / 1024MB)
Jobs: 1 (f=1): [r] [100.0% done] [23092KB/0KB/0KB /s] [5773/0/0 iops] [eta 00m:00s]
random-read: (groupid=0, jobs=1): err= 0: pid=17458: Mon Feb 24 17:31:03 2014
read : io=1024.0MB, bw=20972KB/s, iops=5242, runt= 50000msec
clat (usec): min=104, max=6026, avg=186.75, stdev=68.49
lat (usec): min=104, max=6026, avg=187.07, stdev=68.49
clat percentiles (usec):
| 1.00th=[ 125], 5.00th=[ 161], 10.00th=[ 167], 20.00th=[ 169],
| 30.00th=[ 177], 40.00th=[ 181], 50.00th=[ 183], 60.00th=[ 187],
| 70.00th=[ 191], 80.00th=[ 195], 90.00th=[ 203], 95.00th=[ 207],
| 99.00th=[ 258], 99.50th=[ 330], 99.90th=[ 1080], 99.95th=[ 1512],
| 99.99th=[ 2480]
bw (KB /s): min=11712, max=22840, per=99.87%, avg=20943.90, stdev=1858.80
lat (usec) : 250=98.69%, 500=0.95%, 750=0.13%, 1000=0.11%
lat (msec) : 2=0.09%, 4=0.03%, 10=0.01%
cpu : usr=2.74%, sys=16.74%, ctx=263863, majf=0, minf=22
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=262144/w=0/d=0, short=r=0/w=0/d=0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: io=1024.0MB, aggrb=20971KB/s, minb=20971KB/s, maxb=20971KB/s, mint=50000msec, maxt=50000msec
-
The following config file includes some more useful options. Much more options can be found in the man page, and fio source code also provides several examples.
# job name between brackets (except when value is "global" )
[random_read]
#rw=
# read Sequential reads
# write Sequential writes
# randwrite Random writes
# randread Random reads
# rw Sequential mixed reads and writes
# randrw Random mixed reads and writes
rw=randread
# ioengine=
# sync Basic read(2) or write(2) io. lseek(2) is
# used to position the io location.
# psync Basic pread(2) or pwrite(2) io.
# vsync Basic readv(2) or writev(2) IO.
# libaio Linux native asynchronous io.
# posixaio glibc posix asynchronous io.
# solarisaio Solaris native asynchronous io.
# windowsaio Windows native asynchronous io.
ioengine=libaio
#iodepth Number of I/O units to keep in flight against the file.
iodepth=32
# direct If value is true, use non-buffered io. This is usually
# O_DIRECT. Note that ZFS on Solaris doesn't support direct io.
direct=1
#invalidate Invalidate buffer-cache for the file prior to starting I/O. Default: true.
invalidate=1
# bs The block size used for the io units. Defaults to 4k.
bs=4k
#directory Prefix filenames with this directory. Used to place files in a location other than './'.
directory=.
/data
#nrfiles= Number of files to use for this job. Defaults to 1.
nrfiles=1
#size Total size of I/O for this job.
size=200m
#numjobs Number of clones (processes/threads performing the same workload) of this job. Default: 1.
numjobs=4
Preconditioning
Preconditioning is important to get steady performance data, especially for write. When the SSD drive is in a clean state, the write operation is fast (even higher than read performance). However, when garbage collector starts to work due to low available capacity, the write performance will go down, and reach a steady state. We can see an example here (The graph of interest is the "preconditioning curve"). (Refer to SNIA Solid State Storage Performance Test Specification, discussion-1, discussion-2)