如何在 OCP4 通过 IOPS测试磁盘性能

• 我们可以用’Fio’检查 IOPS(I/O 每秒)来查看磁盘性能
• 本文与磁盘的读写数量无关. 本文讨论的问题是数据延迟同步也就是磁盘性能.

运行环境

• Openshift Container Platform 4
• RHCOS

通过读取一个随机 2Gb 文件来执行一个随机的读取测试.例子如下:

[root@toolbox /]# fio --name=randread --ioengine=libaio --iodepth=16 --rw=randread --bs=4k --direct=0 --size=512M --numjobs=4 --runtime=240 --group_reporting
randread: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=16
...
fio-3.14
Starting 4 processes
randread: Laying out IO file (1 file / 512MiB)
randread: Laying out IO file (1 file / 512MiB)
randread: Laying out IO file (1 file / 512MiB)
randread: Laying out IO file (1 file / 512MiB)
Jobs: 2 (f=2): [r(2),_(2)][100.0%][r=56.2MiB/s][r=14.4k IOPS][eta 00m:00s]
randread: (groupid=0, jobs=4): err= 0: pid=107: Thu Feb 20 11:50:10 2020
  read: IOPS=14.9k, BW=58.0MiB/s (60.9MB/s)(2048MiB/35287msec)
    slat (usec): min=39, max=32963, avg=254.11, stdev=252.78
    clat (usec): min=5, max=46011, avg=3962.81, stdev=1233.22
     lat (usec): min=93, max=46274, avg=4218.60, stdev=1282.91
    clat percentiles (usec):
     |  1.00th=[ 1795],  5.00th=[ 2343], 10.00th=[ 2966], 20.00th=[ 3458],
     | 30.00th=[ 3621], 40.00th=[ 3752], 50.00th=[ 3884], 60.00th=[ 3982],
     | 70.00th=[ 4146], 80.00th=[ 4359], 90.00th=[ 4817], 95.00th=[ 5276],
     | 99.00th=[ 8717], 99.50th=[11076], 99.90th=[16188], 99.95th=[18744],
     | 99.99th=[26084]
   bw (  KiB/s): min=45488, max=89732, per=100.00%, avg=60015.20, stdev=1759.90, samples=275
   iops        : min=11372, max=22432, avg=15003.65, stdev=439.94, samples=275
  lat (usec)   : 10=0.01%, 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%
  lat (usec)   : 1000=0.01%
  lat (msec)   : 2=2.54%, 4=57.78%, 10=39.01%, 20=0.62%, 50=0.04%
  cpu          : usr=4.47%, sys=11.27%, ctx=526355, majf=0, minf=99
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=524288,0,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=58.0MiB/s (60.9MB/s), 58.0MiB/s-58.0MiB/s (60.9MB/s-60.9MB/s), io=2048MiB (2147MB), run=35287-35287msec

通过写入一个 4GB (4jobs x 512MB=2GB)文件,每次运行两个进程来执行一个随意的写入测试.

[root@toolbox /]# fio --name=randwrite --ioengine=libaio --iodepth=1 --rw=randwrite --bs=4k --direct=0 --size=512M --numjobs=2 --runtime=240 --group_reporting
randwrite: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=1
...
fio-3.14
Starting 2 processes
randwrite: Laying out IO file (1 file / 512MiB)
randwrite: Laying out IO file (1 file / 512MiB)
Jobs: 2 (f=2)
randwrite: (groupid=0, jobs=2): err= 0: pid=113: Thu Feb 20 11:52:57 2020
  write: IOPS=186k, BW=726MiB/s (762MB/s)(1024MiB/1410msec); 0 zone resets
    slat (usec): min=3, max=1056, avg= 5.56, stdev= 4.92
    clat (nsec): min=1200, max=206300, avg=1641.67, stdev=1786.82
     lat (usec): min=5, max=1577, avg= 8.04, stdev= 6.79
    clat percentiles (nsec):
     |  1.00th=[ 1304],  5.00th=[ 1400], 10.00th=[ 1400], 20.00th=[ 1400],
     | 30.00th=[ 1400], 40.00th=[ 1400], 50.00th=[ 1496], 60.00th=[ 1496],
     | 70.00th=[ 1496], 80.00th=[ 1496], 90.00th=[ 1896], 95.00th=[ 2008],
     | 99.00th=[ 2704], 99.50th=[10944], 99.90th=[24960], 99.95th=[35072],
     | 99.99th=[64256]
   bw (  KiB/s): min=736973, max=762848, per=100.00%, avg=749910.50, stdev=7496.79, samples=4
   iops        : min=184242, max=190712, avg=187477.00, stdev=1874.56, samples=4
  lat (usec)   : 2=94.36%, 4=4.77%, 10=0.23%, 20=0.49%, 50=0.13%
  lat (usec)   : 100=0.02%, 250=0.01%
  cpu          : usr=38.68%, sys=59.93%, ctx=33, majf=0, minf=17
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=0,262144,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
  WRITE: bw=726MiB/s (762MB/s), 726MiB/s-726MiB/s (762MB/s-762MB/s), io=1024MiB (1074MB), run=1410-1410msec

可以执行读写性能测试以便同时检查随机的读写值，以检查存储单元的性能集。以下流程会创建一个 4GB大小的文件,该文件以 4KB的比例(bs=4k)来进行读取. 写入操作姜维使用三次读取进行测试比率为 75%/25%(–rwmixread75)所以这是一个一次性测试 64operators (I/O) of iodepth=64)基准性能默认值的随机测试.可以调整此比率以模拟其他环境，因为这只是一个随机示例.

[root@toolbox /]# fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=random_read_write.fio --bs=4k --iodepth=64 --size=4G --readwrite=randrw --rwmixread=75
test: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=64
fio-3.14
Starting 1 process
Jobs: 1 (f=1): [m(1)][100.0%][r=3107KiB/s,w=1069KiB/s][r=776,w=267 IOPS][eta 00m:00s]
test: (groupid=0, jobs=1): err= 0: pid=30: Thu Feb 20 12:30:54 2020
  read: IOPS=769, BW=3077KiB/s (3151kB/s)(3070MiB/1021632msec)
   bw (  KiB/s): min= 2104, max= 4024, per=99.98%, avg=3076.26, stdev=269.58, samples=2043
   iops        : min=  526, max= 1006, avg=769.03, stdev=67.41, samples=2043
  write: IOPS=257, BW=1028KiB/s (1053kB/s)(1026MiB/1021632msec); 0 zone resets
   bw (  KiB/s): min=  768, max= 1184, per=100.00%, avg=1028.09, stdev=52.21, samples=2043
   iops        : min=  192, max=  296, avg=256.99, stdev=13.06, samples=2043
  cpu          : usr=0.87%, sys=3.12%, ctx=1048852, majf=0, minf=7
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued rwts: total=785920,262656,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: bw=3077KiB/s (3151kB/s), 3077KiB/s-3077KiB/s (3151kB/s-3151kB/s), io=3070MiB (3219MB), run=1021632-1021632msec
  WRITE: bw=1028KiB/s (1053kB/s), 1028KiB/s-1028KiB/s (1053kB/s-1053kB/s), io=1026MiB (1076MB), run=1021632-1021632msec

FIO需要先在 RHCOS 上安装.
以下快速安装指导

core@ ~$ sudo podman run -it --rm docker.io/library/fedora:latest /bin/bash
root@5969d1894047 /# yum install fio -y
root@5969d1894047 /# fio

或者打开一个命令框创建以下文件去运行 toolbox

[root@master-1 ~]$ cat .toolboxrc 
REGISTRY=docker.io
IMAGE=fedora:latest

[root@master-1 ~]$ toolbox

打开另一个命令框用以下命令安装 FIO

[root@master-1 ~]# podman ps
CONTAINER ID  IMAGE                            COMMAND    CREATED        STATUS            PORTS  NAMES
7d0288079ec9  docker.io/library/fedora:latest  /bin/bash  6 seconds ago  Up 4 seconds ago         toolbox-root
[root@master-1 ~]# sudo podman attach toolbox-root
[root@toolbox /]# sudo yum install fio -y
Complete!
[root@toolbox /] fio --version
fio-3.14

本文经过作者 David 同意后翻译