一、说明
红帽官方版本已经升级到7.2,该版本的标准文件系统已经由EXT4升级到XFS。从操作系统角度来讲,为了获取更好的支持和服务,此次测试EXT4和XFS的性能区别。
Ext4的文件系统容量达到1EB,而文件容量则达到16TB,这是一个非常大的数字了。对一般的台式机和服务器而言,这可能并不重要,但对于大型磁盘阵列的用户而言,这就非常重要了。
XFS是一个64位文件系统,最大支持8EB减1字节的单个文件系统,实际部署时取决于宿主操作系统的最大块限制。对于一个32位Linux系统,文件和文件系统的大小会被限制在16TB。
二、测试模型
我们对文件系统的消耗主要是在MySQL这一块,MySQL数据库对磁盘的读写压力大,因此不同的文件系统和文件系统挂载参数对MySQL的性能会有一定的影响。
本次测试主要测试使用sysbench测试同样主机磁盘情况下的IOPS。
2.1 XFS磁盘挂载参数
/dev/mapper/datavg-mysqldata on /mysqldatatype xfs (rw,noatime,nodiratime,seclabel,attr2,nobarrier,inode64,noquota)
2.2 EXT4磁盘挂载参数
/dev/mapper/datavg-mysqldata on /mysqldatatype ext4 (rw,noatime,nodiratime,seclabel,nobarrier,data=ordered)
三、测试步骤
3.1 生成测试文件
sysbench --test=fileio --file-num=16--file-block-size=16384 --file-total-size=2G prepare
3.2 测试顺序写
sysbench --test=fileio --file-num=16--file-block-size=16384 --file-total-size=2G --num-threads=4--max-requests=100000000 --max-time=180 --file-test-mode=seqwr--file-extra-flags=direct run
3.2.1 XFS顺序写
[root@host01 mysqldata]# sysbench--test=fileio --file-num=16 --file-block-size=16384 --file-total-size=2G--num-threads=4 --max-requests=100000000 --max-time=180 --file-test-mode=seqwr--file-extra-flags=direct run
sysbench 0.5: multi-threaded system evaluation benchmark
Running the test with following options:
Number of threads: 4
Random number generator seed is 0 and willbe ignored
Extra file open flags: 3
16 files, 128Mb each
2Gb total file size
Block size 16Kb
Periodic FSYNC enabled, calling fsync()each 100 requests.
Calling fsync() at the end of test,Enabled.
Using synchronous I/O mode
Doing sequential write (creation) test
Initializing worker threads...
Threads started!
Operations performed: 0 reads, 9710025 writes, 1553600 Other =11263625 Total
Read 0b Written 148.16Gb Totaltransferred 148.16Gb (842.88Mb/sec)
53944.57 Requests/sec executed
General statistics:
total time: 180.0000s
total number of events: 9710025
total time taken by event execution: 678.2973s
response time:
min: 0.04ms
avg: 0.07ms
max: 90.93ms
approx. 95 percentile: 0.10ms
Threads fairness:
events (avg/stddev): 2427506.2500/6365.48
execution time (avg/stddev): 169.5743/0.22
3.2.2 EXT4顺序写
[root@host02 mysqldata]# sysbench--test=fileio --file-num=16 --file-block-size=16384 --file-total-size=2G--num-threads=4 --max-requests=100000000 --max-time=180 --file-test-mode=seqwr--file-extra-flags=direct run
sysbench 0.5: multi-threaded system evaluation benchmark
Running the test with following options:
Number of threads: 4
Random number generator seed is 0 and willbe ignored
Extra file open flags: 3
16 files, 128Mb each
2Gb total file size
Block size 16Kb
Periodic FSYNC enabled, calling fsync()each 100 requests.
Calling fsync() at the end of test,Enabled.
Using synchronous I/O mode
Doing sequential write (creation) test
Initializing worker threads...
Threads started!
Operations performed: 0 reads, 4014480 writes, 642304 Other =4656784 Total
Read 0b Written 61.256Gb Totaltransferred 61.256Gb (348.48Mb/sec)
22302.64 Requests/sec executed
General statistics:
total time: 180.0002s
total number of events: 4014480
total time taken by event execution: 712.0619s
response time:
min: 0.04ms
avg: 0.18ms
max: 123.86ms
approx. 95 percentile: 0.23ms
Threads fairness:
events (avg/stddev): 1003620.0000/19916.58
execution time(avg/stddev): 178.0155/0.07
结果:XFS的顺序写比EXT4快一倍。
3.3测试顺序读
sysbench--test=fileio --file-num=16 --file-block-size=16384 --file-total-size=2G--num-threads=4 --max-requests=100000000 --max-time=180 --file-test-mode=seqrd--file-extra-flags=direct run
3.3.1 XFS顺序读
[root@host02 mysqldata]# sysbench--test=fileio --file-num=16 --file-block-size=16384 --file-total-size=2G--num-threads=4 --max-requests=100000000 --max-time=180 --file-test-mode=seqrd--file-extra-flags=direct run
sysbench0.5: multi-threaded system evaluationbenchmark
Running the testwith following options:
Number ofthreads: 4
Random numbergenerator seed is 0 and will be ignored
Extra file openflags: 3
16 files, 128Mbeach
2Gb total filesize
Block size 16Kb
Periodic FSYNCenabled, calling fsync() each 100 requests.
Calling fsync()at the end of test, Enabled.
Usingsynchronous I/O mode
Doing sequentialread test
Initializingworker threads...
Threads started!
Operationsperformed: 10222146 reads, 0 writes, 0Other = 10222146 Total
Read155.98Gb Written 0b Total transferred 155.98Gb (887.34Mb/sec)
56789.67Requests/sec executed
Generalstatistics:
total time: 180.0001s
total number of events: 10222146
total time taken by event execution:714.2186s
response time:
min: 0.04ms
avg: 0.07ms
max: 144.94ms
approx. 95 percentile: 0.10ms
Threadsfairness:
events (avg/stddev): 2555536.5000/124.69
execution time (avg/stddev): 178.5547/0.05
3.3.2 EXT4顺序读
[root@host01 mysqldata]# sysbench--test=fileio --file-num=16 --file-block-size=16384 --file-total-size=2G--num-threads=4 --max-requests=100000000 --max-time=180 --file-test-mode=seqrd--file-extra-flags=direct run
sysbench 0.5: multi-threaded system evaluation benchmark
Running the test with following options:
Number of threads: 4
Random number generator seed is 0 and willbe ignored
Extra file open flags: 3
16 files, 128Mb each
2Gb total file size
Block size 16Kb
Periodic FSYNC enabled, calling fsync()each 100 requests.
Calling fsync() at the end of test,Enabled.
Using synchronous I/O mode
Doing sequential read test
Initializing worker threads...
Threads started!
Operations performed: 9961497 reads, 0 writes, 0 Other = 9961497Total
Read 152Gb Written 0b Total transferred152Gb (864.7Mb/sec)
55341.00 Requests/sec executed
General statistics:
total time: 180.0021s
total number of events: 9961497
total time taken by event execution: 713.3962s
response time:
min: 0.04ms
avg: 0.07ms
max: 143.99ms
approx. 95 percentile: 0.10ms
Threads fairness:
events (avg/stddev): 2490374.2500/436.98
execution time(avg/stddev): 178.3491/0.18
结果:XFS的顺序读比EXT4快20M/S
3.4 测试随机写
sysbench --test=fileio --file-num=16--file-block-size=16384 --file-total-size=2G --num-threads=4--max-requests=100000000 --max-time=180 --file-test-mode=rndwr--file-extra-flags=direct run
3.4.1 XFS随机写
[root@host02 mysqldata]# sysbench--test=fileio --file-num=16 --file-block-size=16384 --file-total-size=2G--num-threads=4 --max-requests=100000000 --max-time=180 --file-test-mode=rndwr--file-extra-flags=direct run
sysbench 0.5: multi-threaded system evaluation benchmark
Running the test with following options:
Number of threads: 4
Random number generator seed is 0 and willbe ignored
Extra file open flags: 3
16 files, 128Mb each
2Gb total file size
Block size 16Kb
Number of IO requests: 100000000
Read/Write ratio for combined random IOtest: 1.50
Periodic FSYNC enabled, calling fsync()each 100 requests.
Calling fsync() at the end of test,Enabled.
Using synchronous I/O mode
Doing random write test
Initializing worker threads...
Threads started!
Operationsperformed: 0 reads, 186364 writes, 29808Other = 216172 Total
Read 0b Written 2.8437Gb Total transferred 2.8437Gb (16.171Mb/sec)
1034.95 Requests/sec executed
General statistics:
total time: 180.0709s
total number of events: 186364
total time taken by event execution: 523.4992s
response time:
min: 0.04ms
avg: 2.81ms
max: 772.01ms
approx. 95 percentile: 0.12ms
Threads fairness:
events (avg/stddev): 46591.0000/26773.96
execution time(avg/stddev): 130.8748/75.55
3.4.2 EXT4随机写
[root@host01 mysqldata]# sysbench--test=fileio --file-num=16 --file-block-size=16384 --file-total-size=2G--num-threads=4 --max-requests=100000000 --max-time=180 --file-test-mode=rndwr--file-extra-flags=direct run
sysbench 0.5: multi-threaded system evaluation benchmark
Running the test with following options:
Number of threads: 4
Random number generator seed is 0 and willbe ignored
Extra file open flags: 3
16 files, 128Mb each
2Gb total file size
Block size 16Kb
Number of IO requests: 100000000
Read/Write ratio for combined random IOtest: 1.50
Periodic FSYNC enabled, calling fsync()each 100 requests.
Calling fsync() at the end of test,Enabled.
Using synchronous I/O mode
Doing random write test
Initializing worker threads...
Threads started!
Operationsperformed: 0 reads, 194208 writes, 31072Other = 225280 Total
Read 0b Written 2.9634Gb Total transferred 2.9634Gb (16.855Mb/sec)
1078.70 Requests/sec executed
General statistics:
total time: 180.0396s
total number of events: 194208
total time taken by event execution: 694.4098s
response time:
min: 0.04ms
avg: 3.58ms
max: 349.96ms
approx. 95 percentile: 1.74ms
Threads fairness:
events (avg/stddev): 48552.0000/559.95
execution time(avg/stddev): 173.6025/0.45
结论: EXT4的随机写比XFS快0.7M/S
3.5 测试随机读
sysbench --test=fileio --file-num=16--file-block-size=16384 --file-total-size=2G --num-threads=4--max-requests=100000000 --max-time=180 --file-test-mode=rndrd--file-extra-flags=direct run
3.5.1 XFS随机读
[root@host02 mysqldata]# sysbench--test=fileio --file-num=16 --file-block-size=16384 --file-total-size=2G--num-threads=4 --max-requests=100000000 --max-time=180 --file-test-mode=rndrd--file-extra-flags=direct run
sysbench 0.5: multi-threaded system evaluation benchmark
Running the test with following options:
Number of threads: 4
Random number generator seed is 0 and willbe ignored
Extra file open flags: 3
16 files, 128Mb each
2Gb total file size
Block size 16Kb
Number of IO requests: 100000000
Read/Write ratio for combined random IOtest: 1.50
Periodic FSYNC enabled, calling fsync()each 100 requests.
Calling fsync() at the end of test,Enabled.
Using synchronous I/O mode
Doing random read test
Initializing worker threads...
Threads started!
Operations performed: 164839 reads, 0 writes, 0 Other = 164839Total
Read 2.5152Gb Written 0b Total transferred 2.5152Gb (14.308Mb/sec)
915.73 Requests/sec executed
General statistics:
total time: 180.0092s
total number of events: 164839
total time taken by event execution: 719.9114s
response time:
min: 0.04ms
avg: 4.37ms
max: 48.49ms
approx. 95 percentile: 10.48ms
Threads fairness:
events (avg/stddev): 41209.7500/51.29
execution time(avg/stddev): 179.9779/0.00
3.5.2 EXT4随机读
[root@host01 mysqldata]# sysbench--test=fileio --file-num=16 --file-block-size=16384 --file-total-size=2G--num-threads=4 --max-requests=100000000 --max-time=180 --file-test-mode=rndrd--file-extra-flags=direct run
sysbench 0.5: multi-threaded system evaluation benchmark
Running the test with following options:
Number of threads: 4
Random number generator seed is 0 and willbe ignored
Extra file open flags: 3
16 files, 128Mb each
2Gb total file size
Block size 16Kb
Number of IO requests: 100000000
Read/Write ratio for combined random IOtest: 1.50
Periodic FSYNC enabled, calling fsync()each 100 requests.
Calling fsync() at the end of test,Enabled.
Using synchronous I/O mode
Doing random read test
Initializing worker threads...
Threads started!
Operations performed: 208723 reads, 0 writes, 0 Other = 208723Total
Read 3.1849Gb Written 0b Total transferred 3.1849Gb (18.118Mb/sec)
1159.54 Requests/sec executed
General statistics:
total time: 180.0045s
total number of events: 208723
total time taken by event execution: 719.8451s
response time:
min: 0.04ms
avg: 3.45ms
max: 37.33ms
approx. 95 percentile: 8.16ms
Threads fairness:
events (avg/stddev): 52180.7500/160.49
execution time(avg/stddev): 179.9613/0.00
结论:EXT4的随机读比XFS的随机读快4M/S