- 我们可以用’Fio’检查 IOPS(I/O 每秒)来查看磁盘性能
- 本文与磁盘的读写数量无关. 本文讨论的问题是数据延迟同步也就是磁盘性能.
运行环境
- Openshift Container Platform 4
- RHCOS
通过读取一个随机 2Gb 文件来执行一个随机的读取测试.例子如下:
[root@toolbox /]# fio --name=randread --ioengine=libaio --iodepth=16 --rw=randread --bs=4k --direct=0 --size=512M --numjobs=4 --runtime=240 --group_reporting
randread: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=16
...
fio-3.14
Starting 4 processes
randread: Laying out IO file (1 file / 512MiB)
randread: Laying out IO file (1 file / 512MiB)
randread: Laying out IO file (1 file / 512MiB)
randread: Laying out IO file (1 file / 512MiB)
Jobs: 2 (f=2): [r(2),_(2)][100.0%][r=56.2MiB/s][r=14.4k IOPS][eta 00m:00s]
randread: (groupid=0, jobs=4): err= 0: pid=107: Thu Feb 20 11:50:10 2020
read: IOPS=14.9k, BW=58.0MiB/s (60.9MB/s)(2048MiB/35287msec)
slat (usec): min=39, max=32963, avg=254.11, stdev=252.78
clat (usec): min=5, max=46011, avg=3962.81, stdev=1233.22
lat (usec): min=93, max=46274, avg=4218.60, stdev=1282.91
clat percentiles (usec):
| 1.00th=[ 1795], 5.00th=[ 2343], 10.00th=[ 2966], 20.00th=[ 3458],
| 30.00th=[ 3621], 40.00th=[ 3752], 50.00th=[ 3884], 60.00th=[ 3982],
| 70.00th=[ 4146], 80.00th=[ 4359], 90.00th=[ 4817], 95.00th=[ 5276],
| 99.00th=[ 8717], 99.50th=[11076], 99.90th=[16188], 99.95th=[18744],
| 99.99th=[26084]
bw ( KiB/s): min=45488, max=89732, per=100.00%, avg=60015.20, stdev=1759.90, samples=275
iops : min=11372, max=22432, avg=15003.65, stdev=439.94, samples=275
lat (usec) : 10=0.01%, 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%
lat (usec) : 1000=0.01%
lat (msec) : 2=2.54%, 4=57.78%, 10=39.01%, 20=0.62%, 50=0.04%
cpu : usr=4.47%, sys=11.27%, ctx=526355, majf=0, minf=99
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=524288,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
READ: bw=58.0MiB/s (60.9MB/s), 58.0MiB/s-58.0MiB/s (60.9MB/s-60.9MB/s), io=2048MiB (2147MB), run=35287-35287msec
通过写入一个 4GB (4jobs x 512MB=2GB)文件,每次运行两个进程来执行一个随意的写入测试.
[root@toolbox /]# fio --name=randwrite --ioengine=libaio --iodepth=1 --rw=randwrite --bs=4k --direct=0 --size=512M --numjobs=2 --runtime=240 --group_reporting
randwrite: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=1
...
fio-3.14
Starting 2 processes
randwrite: Laying out IO file (1 file / 512MiB)
randwrite: Laying out IO file (1 file / 512MiB)
Jobs: 2 (f=2)
randwrite: (groupid=0, jobs=2): err= 0: pid=113: Thu Feb 20 11:52:57 2020
write: IOPS=186k, BW=726MiB/s (762MB/s)(1024MiB/1410msec); 0 zone resets
slat (usec): min=3, max=1056, avg= 5.56, stdev= 4.92
clat (nsec): min=1200, max=206300, avg=1641.67, stdev=1786.82
lat (usec): min=5, max=1577, avg= 8.04, stdev= 6.79
clat percentiles (nsec):
| 1.00th=[ 1304], 5.00th=[ 1400], 10.00th=[ 1400], 20.00th=[ 1400],
| 30.00th=[ 1400], 40.00th=[ 1400], 50.00th=[ 1496], 60.00th=[ 1496],
| 70.00th=[ 1496], 80.00th=[ 1496], 90.00th=[ 1896], 95.00th=[ 2008],
| 99.00th=[ 2704], 99.50th=[10944], 99.90th=[24960], 99.95th=[35072],
| 99.99th=[64256]
bw ( KiB/s): min=736973, max=762848, per=100.00%, avg=749910.50, stdev=7496.79, samples=4
iops : min=184242, max=190712, avg=187477.00, stdev=1874.56, samples=4
lat (usec) : 2=94.36%, 4=4.77%, 10=0.23%, 20=0.49%, 50=0.13%
lat (usec) : 100=0.02%, 250=0.01%
cpu : usr=38.68%, sys=59.93%, ctx=33, majf=0, minf=17
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,262144,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=726MiB/s (762MB/s), 726MiB/s-726MiB/s (762MB/s-762MB/s), io=1024MiB (1074MB), run=1410-1410msec
可以执行读写性能测试以便同时检查随机的读写值,以检查存储单元的性能集。以下流程会创建一个 4GB大小的文件,该文件以 4KB的比例(bs=4k)来进行读取. 写入操作姜维使用三次读取进行测试比率为 75%/25%(–rwmixread75)所以这是一个一次性测试 64operators (I/O) of iodepth=64)基准性能默认值的随机测试.可以调整此比率以模拟其他环境,因为这只是一个随机示例.
[root@toolbox /]# fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=random_read_write.fio --bs=4k --iodepth=64 --size=4G --readwrite=randrw --rwmixread=75
test: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=64
fio-3.14
Starting 1 process
Jobs: 1 (f=1): [m(1)][100.0%][r=3107KiB/s,w=1069KiB/s][r=776,w=267 IOPS][eta 00m:00s]
test: (groupid=0, jobs=1): err= 0: pid=30: Thu Feb 20 12:30:54 2020
read: IOPS=769, BW=3077KiB/s (3151kB/s)(3070MiB/1021632msec)
bw ( KiB/s): min= 2104, max= 4024, per=99.98%, avg=3076.26, stdev=269.58, samples=2043
iops : min= 526, max= 1006, avg=769.03, stdev=67.41, samples=2043
write: IOPS=257, BW=1028KiB/s (1053kB/s)(1026MiB/1021632msec); 0 zone resets
bw ( KiB/s): min= 768, max= 1184, per=100.00%, avg=1028.09, stdev=52.21, samples=2043
iops : min= 192, max= 296, avg=256.99, stdev=13.06, samples=2043
cpu : usr=0.87%, sys=3.12%, ctx=1048852, majf=0, minf=7
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=785920,262656,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=3077KiB/s (3151kB/s), 3077KiB/s-3077KiB/s (3151kB/s-3151kB/s), io=3070MiB (3219MB), run=1021632-1021632msec
WRITE: bw=1028KiB/s (1053kB/s), 1028KiB/s-1028KiB/s (1053kB/s-1053kB/s), io=1026MiB (1076MB), run=1021632-1021632msec
FIO需要先在 RHCOS 上安装.
以下快速安装指导
core@ ~$ sudo podman run -it --rm docker.io/library/fedora:latest /bin/bash
root@5969d1894047 /# yum install fio -y
root@5969d1894047 /# fio
或者打开一个命令框创建以下文件去运行 toolbox
[root@master-1 ~]$ cat .toolboxrc
REGISTRY=docker.io
IMAGE=fedora:latest
[root@master-1 ~]$ toolbox
打开另一个命令框用以下命令安装 FIO
[root@master-1 ~]# podman ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
7d0288079ec9 docker.io/library/fedora:latest /bin/bash 6 seconds ago Up 4 seconds ago toolbox-root
[root@master-1 ~]# sudo podman attach toolbox-root
[root@toolbox /]# sudo yum install fio -y
Complete!
[root@toolbox /] fio --version
fio-3.14
本文经过作者 David 同意后翻译