fio

本文转载自：http://freecode.com/projects/fio

fio

fio is an I/O tool meant to be used both for benchmark and stress/hardware verification. It has support for 19 different types of I/O engines (sync, mmap, libaio, posixaio, SG v3, splice, null, network, syslet, guasi, solarisaio, and more), I/O priorities (for newer Linux kernels), rate I/O, forked or threaded jobs, and much more. It can work on block devices as well as files. fio accepts job descriptions in a simple-to-understand text format. Several example job files are included. fio displays all sorts of I/O performance information, including complete IO latencies and percentiles. Fio is in wide use in many places, for both benchmarking, QA, and verification purposes. It supports Linux, FreeBSD, NetBSD, OpenBSD, OS X, OpenSolaris, AIX, HP-UX, Android, and Windows.

Tags	Benchmark Filesystems
Licenses	GPLv2
Operating Systems	POSIX Linux BSD Solaris HP-UX AIX OS X AndroidWindows
Implementation	C

Website

Download

Tweet

• All releases

Recent releases

• 
• 2.1.10

• 
•  10 Jun 2014 19:26
• 

Release Notes: A sync engine fix for Windows, support for the Ceph rados block device and Gluster IO engine, improvements to the replay of iologs or blktrace files, support for ARM 64, improvements for S390, and parallellized writeout of IO logs, making that part much faster. There were lots of other bugfixes and improvements.

• 
• 2.1.7

• 
•  31 Mar 2014 15:16
• 

Release Notes: Adds support for explicit file appending. More accurate implementation of number_ios. Increases the job description field length. More accurate data compression settings. Windows build improvements. A SPARC64 SIGBUS bug has been fixed. Various little bugfixes and improvements.

• 
• 2.1.6.1

• 
•  06 Mar 2014 16:27
• 

Release Notes: Full support for IPv6 for the net I/O engine. Windows threading fixes. Support for ranges in the 'startdelay' option. Adds rbd (rados block device) engine support. Various fixes for blktrace playback on Linux. Adds OpenBSD support. An option added to manually seed random generators. Various client/server fixes. A fix for a crash on some file locking options. Adds support for fast software-based xxhash, and support for testing checksum speeds. A fix for not always honoring the 'openfiles' setting. Support for setting policy on how to distribute CPUs set for jobs. Support for data retention testing with verify.

• 
• 2.1.4

• 
•  18 Nov 2013 19:28
• 

Release Notes: Adds JSON output for client/server. A fix for Windows build breakage in the previous release. A fix for ioengine specific options. Adds support for O_ATOMIC atomic writes on Linux. A failsafe check for realloc() failure during logging. A fix for a Solaris thread problem. Various man page and documentation updates. A fix for a crash on PowerPC without the mfspr instruction.

• 
• 2.1.2

• 
•  06 Aug 2013 15:52
• 

Release Notes: This release adds basic support for an ACT (aerospike) profile, fixes atomic write support, adds various parsing fixes, makes a big revamp of the included graphing support, makes the IOPS log includes bandwidth, adds support for specific randomness settings for both reads and writes, adds support for random vs. sequential blocksize settings, fixes a potential verification failure, and adds multicast support for the net ioengine.

• All comments

Recent comments

29 May 2014 04:25 paragkamble

Kindly help me to get FIO for solaris 10/Solaris11

07 Oct 2013 13:56 diatta

I have written a script using fio the benchmark filesystems that I would like to share:

fio.pl

#!/usr/bin/perl 
# Program name : my_fio_script_generator.pl 
# Author : Mamadou Lamine Diatta 
# Email: diatta at post dot harvard dot edu 
# Purpose : Generate job files used by Linux command fio. 
# ==========================================================================
# Here few example how to use fio at the command line without a job file. 
# 
#Storage subsystem: 
# IBM EXP300 - 14 Disk Fiber Expansion, 17G - 15K RPMS 
# Host: AMD Barcelona, 2 socket, 8G RAM 
# HBA: QLogic Corp. ISP2312-based 2Gb Fibre Channel to PCI-X HBA (rev 02) 
# 
#Benchmark[1]: 
# fio --name=guestrun --filename=/dev/mapper/volumes-fibre \ 
# --rw=randwrite --bs=16k --ioengine=libaio --direct=1 \ 
# --norandommap --runtime=120 --time_based --numjobs=1 \ 
# --group_reporting --thread --size=25g --write_lat_log \ 
# --write_bw_log --iodepth=74 
# 
#Qemu parameters: 
# -m 1024 \ 
# -drive file=/images/npt2-guest-virtio.qcow2,if=ide,boot=on,snapshot=off \ 
# -drive file=/dev/mapper/volumes-fibre,if=virtio,cache=(on|off) \ 
# -drive file=/dev/mapper/volumes-npt2--dom1,if=virtio,cache=off 
# -net nic,macaddr=00:FF:FF:00:00:01,model=rtl8139 -net tap -vnc :123 \ 
# -monitor stdio 
# 
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 
# Few functions here. 
# *************************************************************************** 
sub usage { 
print<<END;

-------------------------------------------------------------------- 
Usage: $0 <directory> 
Where: <directory> is a full path access read and write directory 
NOTE: If the directory doesn't exist, it will be created if possible 
i.e: $0 /asm01/fio/io1 
--------------------------------------------------------------------

END 
} 
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 
sub generate_filename { 
my $range = 36; 
my $minimum = 0; 
@ALPHA=('q','w','e','r','0','t','y','1','u','i','2','o','p','3','a','s','4','d','f','5','g','h','j','6','k','l','7','z','x','8','c','v','9','b','n','m');
$filenamesize=10; 
$line=1; 
$filename=""; 
while(1){ 
my $random_number = int(rand($range)) + $minimum; 
$filename=$filename . $ALPHA[$random_number]; 
if ($line == $filenamesize) { 
return $filename; 
} 
$line++; 
} 
} 
# *************************************************************************** 
sub generate_seconds { 
my $range = 60; 
my $minimum = 1; 
my $seconds = int(rand($range)) + $minimum; 
return $seconds; 
} 
# *************************************************************************** 
sub generate_percent { 
my $range = 100; 
my $minimum = 1; 
my $percent = int(rand($range)) + $minimum; 
return $percent; 
} 
# *************************************************************************** 
sub generate_numjobs { 
my $range = 10; 
my $minimum = 0; 
my $numjobs = int(rand($range)) + $minimum; 
return $numjobs; 
} 
# *************************************************************************** 
sub generate_runtime { 
my $range = 3600; 
# Run between 1s or 1 hour = 60*60 seconds 
my $minimum = 0; 
my $runtime = int(rand($range)) + $minimum; 
return $runtime; 
} 
# *************************************************************************** 
sub generate_nrfiles { 
my $range = 5000; 
my $minimum = 1; 
my $ioload = int(rand($range)) + $minimum; 
return $ioload; 
} 
# *************************************************************************** 
sub generate_filesize { 
my $rangesize = 9; 
my $rangebyte = 3; 
my $minimum = 0; 
@BYTE=('k','m','g'); 
@SIZEK=('1024','2048','4092','4096','5120','6144','7168','8192','9216'); 
@SIZEM=('100','200','300','400','500','600','700','800','900'); 
@SIZEG=('1','2','3','4','5','6','7','8','9'); 
$filesize="";

my $random_number = int(rand($rangebyte)) + $minimum; 
my $byte=$BYTE[$random_number]; 
if ( $byte eq "k" ){ 
my $random_number = int(rand($rangesize)) + $minimum; 
$filesize=$SIZEK[$random_number] . $byte; 
} 
elsif ( $byte eq "m" ){ 
my $random_number = int(rand($rangesize)) + $minimum; 
$filesize=$SIZEM[$random_number] . $byte; 
} 
else { 
my $random_number = int(rand($rangesize)) + $minimum; 
$filesize=$SIZEG[$random_number] . $byte; 
} 
return $filesize; 
} 
# *************************************************************************** 
sub generate_write_read_options { 
my $rangerw = 4; 
my $minimum = 0; 
@RW=('read','randread','write','randwrite'); 
my $random_number = int(rand($rangerw)) + $minimum; 
$rw=$RW[$random_number]; 
return $rw; 
} 
# *************************************************************************** 
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 
# Main function 
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 
#print "#ARGV=$#ARGV\n"; 
if ( $#ARGV != 0) { 
&usage; 
exit 1; 
} 
my $workingdir=$ARGV[0]; 
if ( ! -d "$workingdir" ) { 
mkdir ($workingdir, 0755) || 
die "Can't make directory: $workingdir: $!\n"; 
} 
else { 
print "Directory: $workingdir already exist!\n"; 
} 
while(1) { 
$FIO="/usr/local/bin/fio"; 
$RM="/bin/rm -fr"; 
$CAT="/bin/cat"; 
my $scriptdir="/tmp";

my $fio_script= &generate_filename . ".fio"; 
open(F ,">$scriptdir/$fio_script") || 
die "Can't open file : $scriptdir/$fio_script : $!\n"; 

my $rw= &generate_write_read_options; 
my $filename= &generate_filename; 
my $filesize= &generate_filesize; 
my $rwmixread= &generate_percent;
my $rwmixwrite= 100 - $rwmixread; 
# Note: rwmixread and rwmixwrite are related and must 
# add up to 100%.

my $nrfiles= &generate_nrfiles; 
my $runtime= &generate_runtime; 
my $numjobs= &generate_numjobs; 
print F "; $rw $filesize with rwmixread=$rwmixread and rwmixwrite=$rwmixwrite\n"; 
print F "[${rw}-${rwmixread}-${rwmixwrite}]\n"; 
print F "rw=$rw\n"; 
print F "directory=$workingdir\n"; 
print F "filename=$filename\n"; 
print F "size=$filesize\n"; 
print F "rwmixread=$rwmixread\n"; 
print F "rwmixwrite=$rwmixwrite\n"; 
print F "nrfiles=$nrfiles\n"; 
print F "runtime=$runtime\n"; 
print F "description=${rw}_${filename}_${filesize}_${rwmixread}_${rwmixwrite}_${nrfiles}_${runtime}\n";
print F "unlink=1\n"; 
# If set unlink files when it's done. 
print F "numjobs=$numjobs\n"; 
print F "write_bw_log\n"; 
# To write bandwidth log. 
print F "write_lat_log\n"; 
# To write latency log. 
print F "; ----------------------------------------------------\n\n\n"; 
close(F); 
print "---------------------- Running $FIO $scriptdir/$fio_script\n"; 
open(R, "$scriptdir/$fio_script") || 
die "Can't open $scriptdir/$fio_script : $!\n"; 
while(<R>){ 
chop $_; 
print "$_\n"; 
} 
print "\n"; 
print "\n"; 
close(R); 
`cat $scriptdir/$fio_script > ${fio_script}.log`; 
`$FIO $scriptdir/$fio_script >> ${fio_script}.log 2>&1`; 
`$RM $scriptdir/$fio_script`; 
sleep 10; 
} 
# *********************************************************************************** 
# #######################################################

30 Nov 2012 18:10 bslade

Getting 'stat64 already defined' type errors when trying to "make" fio on RedHat/Centos 6.3 (linux kernel 2.6.18-308). Nothing in the http://www.spinics.net/lists/fio mailing list archives about it. No response to my signup request to the fio@vger.kernel.org mailing list, so I'm posting this problem here.

Has anybody else seen a problem like this?

> make 
DEP .depend 
CC gettime.o 
: Assembler messages: 
:354: Error: symbol `fstatat64' is already defined 
:388: Error: symbol `fstat64' is already defined 
:418: Error: symbol `lstat64' is already defined 
:448: Error: symbol `stat64' is already defined 
:1704: Error: symbol `pread64' is already defined 
make: *** [gettime.o] Error 1

25 Oct 2012 16:44 dschulz

Id appreciate it if anyone here could let me know how to find that information.

25 Oct 2012 16:43 dschulz

When was the switch made to version 3 from version 2 of the headers for --minimal output and was something other than the order of the FIO Version/Jobname/Groupid/Error changed?

moving between 1.5.8 and 2.0.10 it went from: 
fio 1.50
2 job 0 0

to

3 fio-2.0.10 job 0 0

注：近期参加MySQL运维学习，老师推荐该文章作为学习和技术提高的扩展阅读，先记录到自己的博客中，随后慢慢消化、学习、提高。本文与MySQL数据库 “压力测试”主题有关。