本次评估是基于高通660 平台,Arm 64 位处理器,其中有 8 个核,4个小核 1.8G 频率,4个大核 2.2G 频率 ,主要对比 zram 的两个主流的压缩算法的性能差别。由于前段时间在看工作中发现这个平台默认只使用 lzo,但在社区上早就有了 lz4 的算法,且性能要比 lzo 算法在读方面要好 3 倍,于是就搜了一下 lz4 的压缩算法在 arm 平台上的实际表现,可惜资料很少,并且还有一些评估报告说 lz4 没有达到预期那么好,所以我确定自己着手实践一下,到底为什么这个平台默认只开 lzo 算法。
使能 LZ4
需要打开以下几个内核配置项
+CONFIG_ZRAM_LZ4_COMPRESS=y
+CONFIG_LZ4_DECOMPRESS=y
+CONFIG_LZ4_COMPRESS=y
重新编译后启动系统,可以通过往 sys/block/zram0/comp_algorithm 节点上写 lz4 来切换zram 的压缩算法
注意: 需要先 reset zram 设备后才能切换压缩算法,正确的切换方法为
echo 1 > /sys/class/block/zram0/reset
echo lz4 > sys/block/zram0/comp_algorithm
测试方法
由于 zram 被设计为 block 设备,所以可以通过块设备读写来评估 zram 的性能,在不挂载任何文件系统的情况下测试。
为了确保去运算能力的一致性,把 cpu 的频率都锁定到最高点测试。
由于sdm660 没有动态 cpu hotplug 的功能,因此 8 核全部在线测试。
分别测试单线程和多线程(8个)读写两种情况下的 lz4 和 lzo 性能。
以下是初始化命令。
swapoff /dev/block/zram0
echo 1 > /sys/class/block/zram0/reset
echo lzo/lz4 > /sys/class/block/zram0/comp_algorithm
echo 8 > /sys/class/block/zram0/max_comp_streams # 设置最大压缩流为 8 个
echo 1610612736 > /sys/class/block/zram0/disksize # 设置块设备大小为1.5G
echo 1843200 > /sys/devices/system/cpu/cpufreq/policy0/scaling_max_freq
echo 1843200 > /sys/devices/system/cpu/cpufreq/policy0/scaling_min_freq
echo 2208000 > /sys/devices/system/cpu/cpufreq/policy4/scaling_max_freq
echo 2208000 > /sys/devices/system/cpu/cpufreq/policy4/scaling_min_freq
然后,通过 fio 工具进行 io 测试,总 io 量为 500M,单次 io 的大小为 32 M。
fio -filename=/dev/block/zram0 -thread -rw=write -bs=32468k -size=500M -group_reporting -numjobs=1 -name=mytest
-rw :指定读/写
-numjobs :指定线程数
LZO 测试结果
单线程
写:
第一次测试结果:
mytest: (g=0): rw=write, bs=32468K-32468K/32468K-32468K/32468K-32468K, ioengine=sync, iodepth=1
fio-2.2.6
Starting 1 thread
Jobs: 1 (f=1)
mytest: (groupid=0, jobs=1): err= 0: pid=5149: Thu Dec 28 16:32:03 2017
write: io=519488KB, bw=660926KB/s, iops=20, runt= 786msec
clat (msec): min=17, max=81, avg=46.84, stdev=28.01
lat (msec): min=20, max=84, avg=49.17, stdev=27.91
clat percentiles (usec):
| 1.00th=[17536], 5.00th=[17536], 10.00th=[17792], 20.00th=[17792],
| 30.00th=[18304], 40.00th=[18816], 50.00th=[43264], 60.00th=[71168],
| 70.00th=[74240], 80.00th=[74240], 90.00th=[81408], 95.00th=[81408],
| 99.00th=[81408], 99.50th=[81408], 99.90th=[81408], 99.95th=[81408],
| 99.99th=[81408]
bw (KB /s): min=777318, max=777318, per=100.00%, avg=777318.00, stdev= 0.00
lat (msec) : 20=43.75%, 50=12.50%, 100=43.75%
cpu : usr=6.35%, sys=60.99%, ctx=106, majf=0, minf=3
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=0/w=16/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: io=519488KB, aggrb=660926KB/s, minb=660926KB/s, maxb=660926KB/s, mint=786msec, maxt=786msec
Disk stats (read/write):
zram0: ios=0/53289, merge=0/0, ticks=0/360, in_queue=360, util=48.78%
第二次测试结果:
mytest: (g=0): rw=write, bs=32468K-32468K/32468K-32468K/32468K-32468K, ioengine=sync, iodepth=1
fio-2.2.6
Starting 1 thread
Jobs: 1 (f=1): [W(1)] [-.-% done] [0KB/0KB/0KB /s] [0/0/0 iops] [eta 00m:00s]
mytest: (groupid=0, jobs=1): err= 0: pid=5212: Thu Dec 28 16:34:49 2017
write: io=519488KB, bw=619914KB/s, iops=19, runt= 838msec
clat (msec): min=17, max=89, avg=50.01, stdev=31.28
lat (msec): min=19, max=92, avg=52.40, stdev=31.06
clat percentiles (usec):
| 1.00th=[17024], 5.00th=[17024], 10.00th=[17280], 20.00th=[17536],
| 30.00th=[17536], 40.00th=[18048], 50.00th=[43776], 60.00th=[72192],
| 70.00th=[79360], 80.00th=[79360], 90.00th=[87552], 95.00th=[89600],
| 99.00th=[89600], 99.50th=[89600], 99.90th=[89600], 99.95th=[89600],
| 99.99th=[89600]
bw (KB /s): min=728987, max=728987, per=100.00%, avg=728987.00, stdev= 0.00
lat (msec) : 20=43.75%, 50=6.25%, 100=50.00%
cpu : usr=21.48%, sys=41.77%, ctx=71, majf=0, minf=3
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=0/w=16/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: io=519488KB, aggrb=619914KB/s, minb=619914KB/s, maxb=619914KB/s, mint=838msec, maxt=838msec
Disk stats (read/write):
zram0: ios=0/50866, merge=0/0, ticks=0/400, in_queue=400, util=53.98%
读:
第一次测试结果:
mytest: (g=0): rw=read, bs=32468K-32468K/32468K-32468K/32468K-32468K, ioengine=sync, iodepth=1
fio-2.2.6
Starting 1 thread
mytest: (groupid=0, jobs=1): err= 0: pid=5139: Thu Dec 28 16:49:24 2017
read : io=519488KB, bw=628160KB/s, iops=19, runt= 827msec
clat (msec): min=36, max=78, avg=51.71, stdev= 9.23
lat (msec): min=36, max=78, avg=51.71, stdev= 9.23
clat percentiles (usec):
| 1.00th=[36608], 5.00th=[36608], 10.00th=[39168], 20.00th=[47360],
| 30.00th=[49920], 40.00th=[51968], 50.00th=[52480], 60.00th=[52992],
| 70.00th=[54016], 80.00th=[55040], 90.00th=[56064], 95.00th=[78336],
| 99.00th=[78336], 99.50th=[78336], 99.90th=[78336], 99.95th=[78336],
| 99.99th=[78336]
bw (KB /s): min=617262, max=617262, per=98.27%, avg=617262.00, stdev= 0.00
lat (msec) : 50=31.25%, 100=68.75%
cpu : usr=0.00%, sys=97.94%, ctx=14, majf=0, minf=8121
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=16/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: io=519488KB, aggrb=628159KB/s, minb=628159KB/s, maxb=628159KB/s, mint=827msec, maxt=827msec
Disk stats (read/write):
zram0: ios=97677/0, merge=0/0, ticks=330/0, in_queue=340, util=45.88%
第二次测试结果:
mytest: (g=0): rw=read, bs=32468K-32468K/32468K-32468K/32468K-32468K, ioengine=sync, iodepth=1
fio-2.2.6
Starting 1 thread
mytest: (groupid=0, jobs=1): err= 0: pid=5204: Thu Dec 28 16:34:16 2017
read : io=519488KB, bw=698237KB/s, iops=21, runt= 744msec
clat (msec): min=36, max=71, avg=46.52, stdev=10.11
lat (msec): min=36, max=71, avg=46.52, stdev=10.11
clat percentiles (usec):
| 1.00th=[37120], 5.00th=[37120], 10.00th=[38144], 20.00th=[38656],
| 30.00th=[38656], 40.00th=[39168], 50.00th=[39680], 60.00th=[52992],
| 70.00th=[54016], 80.00th=[54016], 90.00th=[55552], 95.00th=[71168],
| 99.00th=[71168], 99.50th=[71168], 99.90th=[71168], 99.95th=[71168],
| 99.99th=[71168]
bw (KB /s): min=737909, max=737909, per=100.00%, avg=737909.00, stdev= 0.00
lat (msec) : 50=56.25%, 100=43.75%
cpu : usr=0.00%, sys=98.12%, ctx=10, majf=0, minf=8121
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=16/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: io=519488KB, aggrb=698236KB/s, minb=698236KB/s, maxb=698236KB/s, mint=744msec, maxt=744msec
Disk stats (read/write):
zram0: ios=114357/0, merge=0/0, ticks=410/0, in_queue=410, util=55.11%
压缩率
sdm660:/ # cat /sys/block/zram0/mm_stat
531955712 41390528 43917312 0 43917312 0 0
8 线程
写:
第一次测试结果:
mytest: (g=0): rw=write, bs=32468K-32468K/32468K-32468K/32468K-32468K, ioengine=sync, iodepth=1
...
fio-2.2.6
Starting 8 threads
Jobs: 1 (f=1): [_(3),W(1),_(4)] [-.-% done] [0KB/1802MB/0KB /s] [0/56/0 iops] [eta 00m:00s]
mytest: (groupid=0, jobs=8): err= 0: pid=5574: Thu Dec 28 17:31:41 2017
write: io=4058.6MB, bw=3008.6MB/s, iops=94, runt= 1349msec
clat (msec): min=11, max=143, avg=65.31, stdev=26.37
lat (msec): min=13, max=165, avg=72.88, stdev=28.55
clat percentiles (msec):
| 1.00th=[ 18], 5.00th=[ 26], 10.00th=[ 29], 20.00th=[ 48],
| 30.00th=[ 52], 40.00th=[ 56], 50.00th=[ 64], 60.00th=[ 71],
| 70.00th=[ 76], 80.00th=[ 86], 90.00th=[ 103], 95.00th=[ 111],
| 99.00th=[ 139], 99.50th=[ 143], 99.90th=[ 143], 99.95th=[ 143],
| 99.99th=[ 143]
bw (KB /s): min=173625, max=559793, per=12.47%, avg=384234.75, stdev=113215.08
lat (msec) : 20=1.56%, 50=26.56%, 100=60.94%, 250=10.94%
cpu : usr=9.32%, sys=72.94%, ctx=10109, majf=0, minf=3
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=0/w=128/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: io=4058.6MB, aggrb=3008.6MB/s, minb=3008.6MB/s, maxb=3008.6MB/s, mint=1349msec, maxt=1349msec
Disk stats (read/write):
zram0: ios=0/60263, merge=0/0, ticks=0/900, in_queue=900, util=60.32%
第二次测试结果:
mytest: (g=0): rw=write, bs=32468K-32468K/32468K-32468K/32468K-32468K, ioengine=sync, iodepth=1
...
fio-2.2.6
Starting 8 threads
Jobs: 1 (f=1): [W(1),_(7)] [-.-% done] [0KB/1614MB/0KB /s] [0/50/0 iops] [eta 00m:00s]
mytest: (groupid=0, jobs=8): err= 0: pid=5600: Thu Dec 28 17:32:15 2017
write: io=4058.6MB, bw=3146.2MB/s, iops=99, runt= 1290msec
clat (msec): min=25, max=166, avg=68.87, stdev=16.38
lat (msec): min=34, max=202, avg=78.04, stdev=18.45
clat percentiles (msec):
| 1.00th=[ 32], 5.00th=[ 48], 10.00th=[ 52], 20.00th=[ 57],
| 30.00th=[ 62], 40.00th=[ 68], 50.00th=[ 70], 60.00th=[ 73],
| 70.00th=[ 74], 80.00th=[ 78], 90.00th=[ 85], 95.00th=[ 93],
| 99.00th=[ 103], 99.50th=[ 167], 99.90th=[ 167], 99.95th=[ 167],
| 99.99th=[ 167]
bw (KB /s): min=314003, max=458911, per=12.45%, avg=401197.88, stdev=36107.38
lat (msec) : 50=7.81%, 100=89.06%, 250=3.12%
cpu : usr=11.15%, sys=74.17%, ctx=2878, majf=0, minf=5
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=0/w=128/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: io=4058.6MB, aggrb=3146.2MB/s, minb=3146.2MB/s, maxb=3146.2MB/s, mint=1290msec, maxt=1290msec
Disk stats (read/write):
zram0: ios=0/36094, merge=0/0, ticks=0/610, in_queue=610, util=49.15%
读:
第一次测试结果:
mytest: (g=0): rw=read, bs=32468K-32468K/32468K-32468K/32468K-32468K, ioengine=sync, iodepth=1
...
fio-2.2.6
Starting 8 threads
Jobs: 8 (f=8)
mytest: (groupid=0, jobs=8): err= 0: pid=5379: Thu Dec 28 17:10:16 2017
read : io=4058.6MB, bw=3986.8MB/s, iops=125, runt= 1018msec
clat (msec): min=34, max=138, avg=63.37, stdev=13.64
lat (msec): min=34, max=138, avg=63.38, stdev=13.64
clat percentiles (msec):
| 1.00th=[ 40], 5.00th=[ 55], 10.00th=[ 56], 20.00th=[ 59],
| 30.00th=[ 60], 40.00th=[ 60], 50.00th=[ 61], 60.00th=[ 61],
| 70.00th=[ 62], 80.00th=[ 65], 90.00th=[ 69], 95.00th=[ 87],
| 99.00th=[ 135], 99.50th=[ 139], 99.90th=[ 139], 99.95th=[ 139],
| 99.99th=[ 139]
bw (KB /s): min=454552, max=566302, per=12.05%, avg=491748.33, stdev=29880.69
lat (msec) : 50=1.56%, 100=93.75%, 250=4.69%
cpu : usr=0.00%, sys=85.58%, ctx=5283, majf=0, minf=64942
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=128/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: io=4058.6MB, aggrb=3986.8MB/s, minb=3986.8MB/s, maxb=3986.8MB/s, mint=1018msec, maxt=1018msec
Disk stats (read/write):
zram0: ios=112713/0, merge=0/0, ticks=890/0, in_queue=910, util=57.52%
第二次测试结果:
mytest: (g=0): rw=read, bs=32468K-32468K/32468K-32468K/32468K-32468K, ioengine=sync, iodepth=1
...
fio-2.2.6
Starting 8 threads
Jobs: 8 (f=8)
mytest: (groupid=0, jobs=8): err= 0: pid=5562: Thu Dec 28 17:31:20 2017
read : io=4058.6MB, bw=3975.3MB/s, iops=125, runt= 1021msec
clat (msec): min=49, max=132, avg=63.26, stdev=12.85
lat (msec): min=49, max=132, avg=63.26, stdev=12.85
clat percentiles (msec):
| 1.00th=[ 50], 5.00th=[ 54], 10.00th=[ 55], 20.00th=[ 57],
| 30.00th=[ 60], 40.00th=[ 61], 50.00th=[ 61], 60.00th=[ 62],
| 70.00th=[ 62], 80.00th=[ 65], 90.00th=[ 72], 95.00th=[ 87],
| 99.00th=[ 127], 99.50th=[ 133], 99.90th=[ 133], 99.95th=[ 133],
| 99.99th=[ 133]
bw (KB /s): min=475721, max=488240, per=11.92%, avg=485303.88, stdev=3954.66
lat (msec) : 50=0.78%, 100=95.31%, 250=3.91%
cpu : usr=0.37%, sys=84.56%, ctx=4863, majf=0, minf=64942
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=128/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: io=4058.6MB, aggrb=3975.3MB/s, minb=3975.3MB/s, maxb=3975.3MB/s, mint=1021msec, maxt=1021msec
Disk stats (read/write):
zram0: ios=113421/0, merge=0/0, ticks=840/0, in_queue=900, util=57.46%
mytest: (g=0): rw=read, bs=32468K-32468K/32468K-32468K/32468K-32468K, ioengine=sync, iodepth=1
压缩率
sdm660:/ # cat sys/block/zram0/mm_stat
531955712 42136643 47136768 0 49504256 0 0
LZ4 测试结果
单线程
写:
第一次测试结果:
mytest: (g=0): rw=write, bs=32468K-32468K/32468K-32468K/32468K-32468K, ioengine=sync, iodepth=1
fio-2.2.6
Starting 1 thread
Jobs: 1 (f=1)
mytest: (groupid=0, jobs=1): err= 0: pid=5252: Thu Dec 28 16:35:49 2017
write: io=519488KB, bw=829853KB/s, iops=25, runt= 626msec
clat (msec): min=17, max=67, avg=36.56, stdev=19.18
lat (msec): min=19, max=69, avg=39.15, stdev=19.10
clat percentiles (usec):
| 1.00th=[17536], 5.00th=[17536], 10.00th=[18048], 20.00th=[18304],
| 30.00th=[19072], 40.00th=[19584], 50.00th=[21632], 60.00th=[43776],
| 70.00th=[55552], 80.00th=[56064], 90.00th=[62208], 95.00th=[67072],
| 99.00th=[67072], 99.50th=[67072], 99.90th=[67072], 99.95th=[67072],
| 99.99th=[67072]
bw (KB /s): min=875935, max=875935, per=100.00%, avg=875935.00, stdev= 0.00
lat (msec) : 20=43.75%, 50=25.00%, 100=31.25%
cpu : usr=7.99%, sys=73.48%, ctx=71, majf=0, minf=3
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=0/w=16/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: io=519488KB, aggrb=829853KB/s, minb=829853KB/s, maxb=829853KB/s, mint=626msec, maxt=626msec
Disk stats (read/write):
zram0: ios=0/70534, merge=0/0, ticks=0/380, in_queue=380, util=51.28%
第二次测试结果:
mytest: (g=0): rw=write, bs=32468K-32468K/32468K-32468K/32468K-32468K, ioengine=sync, iodepth=1
fio-2.2.6
Starting 1 thread
Jobs: 1 (f=1)
mytest: (groupid=0, jobs=1): err= 0: pid=5172: Thu Dec 28 16:51:17 2017
write: io=519488KB, bw=825895KB/s, iops=25, runt= 629msec
clat (msec): min=17, max=63, avg=36.88, stdev=19.06
lat (msec): min=18, max=66, avg=39.31, stdev=19.28
clat percentiles (usec):
| 1.00th=[17280], 5.00th=[17280], 10.00th=[18048], 20.00th=[18048],
| 30.00th=[19328], 40.00th=[19584], 50.00th=[21632], 60.00th=[47360],
| 70.00th=[56576], 80.00th=[57088], 90.00th=[59136], 95.00th=[63232],
| 99.00th=[63232], 99.50th=[63232], 99.90th=[63232], 99.95th=[63232],
| 99.99th=[63232]
bw (KB /s): min=907289, max=907289, per=100.00%, avg=907289.00, stdev= 0.00
lat (msec) : 20=43.75%, 50=18.75%, 100=37.50%
cpu : usr=6.36%, sys=74.72%, ctx=53, majf=0, minf=3
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=0/w=16/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: io=519488KB, aggrb=825895KB/s, minb=825895KB/s, maxb=825895KB/s, mint=629msec, maxt=629msec
Disk stats (read/write):
zram0: ios=0/70067, merge=0/0, ticks=0/420, in_queue=430, util=58.03%
读:
第一次测试结果:
mytest: (g=0): rw=read, bs=32468K-32468K/32468K-32468K/32468K-32468K, ioengine=sync, iodepth=1
fio-2.2.6
Starting 1 thread
mytest: (groupid=0, jobs=1): err= 0: pid=5285: Thu Dec 28 16:37:29 2017
read : io=519488KB, bw=733740KB/s, iops=22, runt= 708msec
clat (msec): min=34, max=69, avg=44.24, stdev= 8.35
lat (msec): min=34, max=69, avg=44.24, stdev= 8.35
clat percentiles (usec):
| 1.00th=[34560], 5.00th=[34560], 10.00th=[35072], 20.00th=[35584],
| 30.00th=[42240], 40.00th=[43776], 50.00th=[43776], 60.00th=[44800],
| 70.00th=[46336], 80.00th=[46848], 90.00th=[50432], 95.00th=[70144],
| 99.00th=[70144], 99.50th=[70144], 99.90th=[70144], 99.95th=[70144],
| 99.99th=[70144]
bw (KB /s): min=753609, max=753609, per=100.00%, avg=753609.00, stdev= 0.00
lat (msec) : 50=87.50%, 100=12.50%
cpu : usr=0.00%, sys=97.46%, ctx=10, majf=0, minf=8121
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=16/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: io=519488KB, aggrb=733740KB/s, minb=733740KB/s, maxb=733740KB/s, mint=708msec, maxt=708msec
Disk stats (read/write):
zram0: ios=118616/0, merge=0/0, ticks=340/0, in_queue=340, util=45.76%
第二次测试结果:
mytest: (g=0): rw=read, bs=32468K-32468K/32468K-32468K/32468K-32468K, ioengine=sync, iodepth=1
fio-2.2.6
Starting 1 thread
mytest: (groupid=0, jobs=1): err= 0: pid=5230: Thu Dec 28 16:56:36 2017
read : io=519488KB, bw=751792KB/s, iops=23, runt= 691msec
clat (msec): min=34, max=74, avg=43.22, stdev= 9.95
lat (msec): min=34, max=74, avg=43.22, stdev= 9.95
clat percentiles (usec):
| 1.00th=[34048], 5.00th=[34048], 10.00th=[34048], 20.00th=[34560],
| 30.00th=[35072], 40.00th=[43264], 50.00th=[43776], 60.00th=[45312],
| 70.00th=[45824], 80.00th=[46336], 90.00th=[47872], 95.00th=[74240],
| 99.00th=[74240], 99.50th=[74240], 99.90th=[74240], 99.95th=[74240],
| 99.99th=[74240]
bw (KB /s): min=771516, max=771516, per=100.00%, avg=771516.00, stdev= 0.00
lat (msec) : 50=93.75%, 100=6.25%
cpu : usr=0.00%, sys=98.41%, ctx=13, majf=0, minf=8121
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=16/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: io=519488KB, aggrb=751791KB/s, minb=751791KB/s, maxb=751791KB/s, mint=691msec, maxt=691msec
Disk stats (read/write):
zram0: ios=120736/0, merge=0/0, ticks=350/0, in_queue=350, util=47.17%
压缩率
sdm660:/ # cat sys/block/zram0/mm_stat
531955712 41000087 43372544 0 51195904 0 0
8 线程
写:
第一次测试结果:
mytest: (g=0): rw=write, bs=32468K-32468K/32468K-32468K/32468K-32468K, ioengine=sync, iodepth=1
...
fio-2.2.6
Starting 8 threads
Jobs: 1 (f=1): [_(5),W(1),_(2)] [-.-% done] [0KB/1639MB/0KB /s] [0/51/0 iops] [eta 00m:00s]
mytest: (groupid=0, jobs=8): err= 0: pid=5647: Thu Dec 28 17:34:11 2017
write: io=4058.6MB, bw=3107.6MB/s, iops=98, runt= 1306msec
clat (msec): min=11, max=142, avg=68.53, stdev=16.41
lat (msec): min=14, max=149, avg=77.46, stdev=17.37
clat percentiles (msec):
| 1.00th=[ 19], 5.00th=[ 46], 10.00th=[ 51], 20.00th=[ 61],
| 30.00th=[ 65], 40.00th=[ 68], 50.00th=[ 70], 60.00th=[ 72],
| 70.00th=[ 75], 80.00th=[ 77], 90.00th=[ 85], 95.00th=[ 92],
| 99.00th=[ 125], 99.50th=[ 143], 99.90th=[ 143], 99.95th=[ 143],
| 99.99th=[ 143]
bw (KB /s): min=326311, max=456492, per=12.49%, avg=397464.25, stdev=36325.79
lat (msec) : 20=2.34%, 50=7.03%, 100=88.28%, 250=2.34%
cpu : usr=9.94%, sys=74.54%, ctx=2923, majf=0, minf=3
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=0/w=128/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: io=4058.6MB, aggrb=3107.6MB/s, minb=3107.6MB/s, maxb=3107.6MB/s, mint=1306msec, maxt=1306msec
Disk stats (read/write):
zram0: ios=0/68515, merge=0/0, ticks=0/620, in_queue=620, util=49.80%
第二次测试结果:
mytest: (g=0): rw=write, bs=32468K-32468K/32468K-32468K/32468K-32468K, ioengine=sync, iodepth=1
...
fio-2.2.6
Starting 8 threads
Jobs: 1 (f=1): [_(5),W(1),_(2)] [-.-% done] [0KB/1789MB/0KB /s] [0/56/0 iops] [eta 00m:00s]
mytest: (groupid=0, jobs=8): err= 0: pid=5659: Thu Dec 28 17:34:30 2017
write: io=4058.6MB, bw=3024.3MB/s, iops=95, runt= 1342msec
clat (msec): min=16, max=164, avg=65.75, stdev=22.80
lat (msec): min=20, max=166, avg=73.42, stdev=23.86
clat percentiles (msec):
| 1.00th=[ 18], 5.00th=[ 26], 10.00th=[ 38], 20.00th=[ 49],
| 30.00th=[ 60], 40.00th=[ 67], 50.00th=[ 68], 60.00th=[ 70],
| 70.00th=[ 74], 80.00th=[ 77], 90.00th=[ 88], 95.00th=[ 102],
| 99.00th=[ 141], 99.50th=[ 165], 99.90th=[ 165], 99.95th=[ 165],
| 99.99th=[ 165]
bw (KB /s): min=277504, max=510860, per=12.35%, avg=382469.31, stdev=72763.75
lat (msec) : 20=3.91%, 50=17.97%, 100=72.66%, 250=5.47%
cpu : usr=9.40%, sys=74.13%, ctx=5562, majf=0, minf=3
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=0/w=128/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: io=4058.6MB, aggrb=3024.3MB/s, minb=3024.3MB/s, maxb=3024.3MB/s, mint=1342msec, maxt=1342msec
Disk stats (read/write):
zram0: ios=0/85372, merge=0/0, ticks=0/920, in_queue=930, util=61.92%
读:
第一次测试结果:
mytest: (g=0): rw=read, bs=32468K-32468K/32468K-32468K/32468K-32468K, ioengine=sync, iodepth=1
...
fio-2.2.6
Starting 8 threads
Jobs: 8 (f=8)
mytest: (groupid=0, jobs=8): err= 0: pid=5686: Thu Dec 28 17:35:19 2017
read : io=4058.6MB, bw=3967.3MB/s, iops=125, runt= 1023msec
clat (msec): min=49, max=129, avg=63.64, stdev=13.01
lat (msec): min=49, max=129, avg=63.64, stdev=13.01
clat percentiles (msec):
| 1.00th=[ 50], 5.00th=[ 52], 10.00th=[ 56], 20.00th=[ 59],
| 30.00th=[ 61], 40.00th=[ 61], 50.00th=[ 62], 60.00th=[ 62],
| 70.00th=[ 63], 80.00th=[ 64], 90.00th=[ 67], 95.00th=[ 93],
| 99.00th=[ 120], 99.50th=[ 130], 99.90th=[ 130], 99.95th=[ 130],
| 99.99th=[ 130]
bw (KB /s): min=488240, max=492872, per=12.10%, avg=491479.50, stdev=1854.08
lat (msec) : 50=2.34%, 100=92.97%, 250=4.69%
cpu : usr=0.24%, sys=82.04%, ctx=6451, majf=0, minf=64944
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=128/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: io=4058.6MB, aggrb=3967.3MB/s, minb=3967.3MB/s, maxb=3967.3MB/s, mint=1023msec, maxt=1023msec
Disk stats (read/write):
zram0: ios=114639/0, merge=0/0, ticks=810/0, in_queue=850, util=53.48%
第二次测试结果:
mytest: (g=0): rw=read, bs=32468K-32468K/32468K-32468K/32468K-32468K, ioengine=sync, iodepth=1
...
fio-2.2.6
Starting 8 threads
Jobs: 8 (f=8)
mytest: (groupid=0, jobs=8): err= 0: pid=5708: Thu Dec 28 17:35:39 2017
read : io=4058.6MB, bw=4002.5MB/s, iops=126, runt= 1014msec
clat (msec): min=41, max=128, avg=63.14, stdev=11.99
lat (msec): min=41, max=128, avg=63.14, stdev=11.99
clat percentiles (msec):
| 1.00th=[ 51], 5.00th=[ 52], 10.00th=[ 52], 20.00th=[ 59],
| 30.00th=[ 61], 40.00th=[ 62], 50.00th=[ 62], 60.00th=[ 63],
| 70.00th=[ 63], 80.00th=[ 64], 90.00th=[ 68], 95.00th=[ 97],
| 99.00th=[ 123], 99.50th=[ 129], 99.90th=[ 129], 99.95th=[ 129],
| 99.99th=[ 129]
bw (KB /s): min=481007, max=506323, per=12.26%, avg=502299.00, stdev=8682.96
lat (msec) : 50=0.78%, 100=96.88%, 250=2.34%
cpu : usr=0.86%, sys=82.86%, ctx=6763, majf=0, minf=64943
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=128/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: io=4058.6MB, aggrb=4002.5MB/s, minb=4002.5MB/s, maxb=4002.5MB/s, mint=1014msec, maxt=1014msec
Disk stats (read/write):
zram0: ios=113066/0, merge=0/0, ticks=1010/0, in_queue=1030, util=63.57%
压缩率
sdm660:/ # cat sys/block/zram0/mm_stat
531955712 42835978 45449216 0 49844224 0 0
结论
测试 | LZO | LZ4 | delta ( lz4 - lz0 ) |
---|---|---|---|
单线程写 | 640M/s | 828M/s | 188M/s (29%) |
单线程读 | 663M/s | 743M/s | 80M/s (12%) |
8 线程写 | 3077M/s | 3065M/s | -12M/s |
8 线程读 | 3980M/s | 3984M/s | 4M/s |
压缩率 | 12.85倍 | 12.97倍 | 0.12倍 |
上表的压缩率的计算方法:末压缩的内存除以压缩后的内存占用。
上表对比的是两次测试结果的平均读写速度,从单线来看,lz4 的比 lz0 要快,写可快 29%,读可以快 12%,但压缩仍然可以达到 lzo 的水平。
从多线程来看,两者的速度差不多。
在读方面 lz4 只比 lzo 快了 12 %,这与社区所说的3倍速度是有差别的,可能这与测试的读写量有关,也或许是在整个读流程中,压缩算法只占一小部分时间,即使这部分时间快3倍,但放在整个读 io 过程中,这个时间优化的影响就被缩小了,或许增大每一笔 io 的请求量可突出这部分时间吧,大家有兴趣可以尝试一下。