MySQL 压测工具 sysbench 安装及使用

最新推荐文章于 2024-05-18 14:35:41 发布

weixin_34050519

最新推荐文章于 2024-05-18 14:35:41 发布

阅读量229

点赞数

文章标签：数据库 lua git

原文链接：https://my.oschina.net/peakfang/blog/2875114

版权

2019独角兽企业重金招聘Python工程师标准>>>

sysbench是一款开源的多线程性能测试工具，可以执行CPU/内存/线程/IO/数据库等方面的性能测试。数据库目前支持MySQL/Oracle/PostgreSQL，后续的版本oltp测试主要结合了lua脚本，不需要修改源码，通过自定义lua脚本就可以实现不同业务类型的测试；

总结起来sysbench的缺点就是，模拟的表结构太简单，不像tpcc-mysql那样完整的事务系统。但对于性能压测对比还是很有用的，因为sysbench使用的环境参数限制是一样的。

安装

通过git上的源码进行编译安装过程（需要联网）

yum install -y git

yum install -y gcc gcc-c++ automake make libtool

克隆sysbench源码

进入一个目录

git clone https://github.com/akopytov/sysbench.git

cd sysbench

./autogen.sh

#Centos7的mysql-client-libs目录在/var/lib64/mysql目录下

./configure --prefix=/usr/local/sysbench1.0 --with-mysql-includes=/usr/include/mysql --with-mysql-libs=/usr/lib64/mysql

make && make install

/usr/local/sysbench1.0/bin/sysbench  --help
[root@server01 sysbench0.5]# ./bin/sysbench  --help
Usage:
  sysbench [options]... [testname] [command]

Commands implemented by most tests: prepare run cleanup help

General options:
  --threads=N                     number of threads to use [1]
  --events=N                      limit for total number of events [0]
  --time=N                        limit for total execution time in seconds [10]
  --warmup-time=N                 execute events for this many seconds with statistics disabled before the actual benchmark run with statistics enabled [0]
  --forced-shutdown=STRING        number of seconds to wait after the --time limit before forcing shutdown, or 'off' to disable [off]
  --thread-stack-size=SIZE        size of stack per thread [64K]
  --thread-init-timeout=N         wait time in seconds for worker threads to initialize [30]
  --rate=N                        average transactions rate. 0 for unlimited rate [0]
  --report-interval=N             periodically report intermediate statistics with a specified interval in seconds. 0 disables intermediate reports [0]
  --report-checkpoints=[LIST,...] dump full statistics and reset all counters at specified points in time. The argument is a list of comma-separated values representing the amount of time in seconds elapsed from start of test when report checkpoint(s) must be performed. Report checkpoints are off by default. []
  --debug[=on|off]                print more debugging info [off]
  --validate[=on|off]             perform validation checks where possible [off]
  --help[=on|off]                 print help and exit [off]
  --version[=on|off]              print version and exit [off]
  --config-file=FILENAME          File containing command line options
  --luajit-cmd=STRING             perform LuaJIT control command. This option is equivalent to 'luajit -j'. See LuaJIT documentation for more information
  --tx-rate=N                     deprecated alias for --rate [0]
  --max-requests=N                deprecated alias for --events [0]
  --max-time=N                    deprecated alias for --time [0]
  --num-threads=N                 deprecated alias for --threads [1]

Pseudo-Random Numbers Generator options:
  --rand-type=STRING   random numbers distribution {uniform, gaussian, special, pareto, zipfian} to use by default [special]
  --rand-seed=N        seed for random number generator. When 0, the current time is used as an RNG seed. [0]
  --rand-spec-iter=N   number of iterations for the special distribution [12]
  --rand-spec-pct=N    percentage of the entire range where 'special' values will fall in the special distribution [1]
  --rand-spec-res=N    percentage of 'special' values to use for the special distribution [75]
  --rand-pareto-h=N    shape parameter for the Pareto distribution [0.2]
  --rand-zipfian-exp=N shape parameter (exponent, theta) for the Zipfian distribution [0.8]

Log options:
  --verbosity=N verbosity level {5 - debug, 0 - only critical messages} [3]

  --percentile=N       percentile to calculate in latency statistics (1-100). Use the special value of 0 to disable percentile calculations [95]
  --histogram[=on|off] print latency histogram in report [off]

General database options:

  --db-driver=STRING  specifies database driver to use ('help' to get list of available drivers)
  --db-ps-mode=STRING prepared statements usage mode {auto, disable} [auto]
  --db-debug[=on|off] print database-specific debug information [off]


Compiled-in database drivers:
  mysql - MySQL driver

mysql options:
  --mysql-host=[LIST,...]          MySQL server host [localhost]
  --mysql-port=[LIST,...]          MySQL server port [3306]
  --mysql-socket=[LIST,...]        MySQL socket
  --mysql-user=STRING              MySQL user [sbtest]
  --mysql-password=STRING          MySQL password []
  --mysql-db=STRING                MySQL database name [sbtest]
  --mysql-ssl[=on|off]             use SSL connections, if available in the client library [off]
  --mysql-ssl-cipher=STRING        use specific cipher for SSL connections []
  --mysql-compression[=on|off]     use compression, if available in the client library [off]
  --mysql-debug[=on|off]           trace all client library calls [off]
  --mysql-ignore-errors=[LIST,...] list of errors to ignore, or "all" [1213,1020,1205]
  --mysql-dry-run[=on|off]         Dry run, pretend that all MySQL client API calls are successful without executing them [off]

Compiled-in tests:
  fileio - File I/O test
  cpu - CPU performance test
  memory - Memory functions speed test
  threads - Threads subsystem performance test
  mutex - Mutex performance test

See 'sysbench <testname> help' for a list of options for each test.

测试CPU
Centos7CPU信息查询：
# 总核数 = 物理CPU个数 X 每颗物理CPU的核数 
# 总逻辑CPU数 = 物理CPU个数 X 每颗物理CPU的核数 X 超线程数
# 查看物理CPU个数
cat /proc/cpuinfo| grep "physical id"| sort| uniq| wc -l
# 查看每个物理CPU中core的个数(即核数)
cat /proc/cpuinfo| grep "cpu cores"| uniq
# 查看逻辑CPU的个数
cat /proc/cpuinfo| grep "processor"| wc -l

#CPU测试通过计算指定范围内最大素数所需时间
[root@server01 sysbench-0.5]# ./bin/sysbench  cpu  help
sysbench 1.1.0-651e7fd (using bundled LuaJIT 2.1.0-beta3)

cpu options:
  --cpu-max-prime=N upper limit for primes generator [10000]

#测试实例，最大素数为5000，线程数等于服务器逻辑CPU个数，预热时间30秒，执行时间60秒，报告输出频率5秒
[root@server01 sysbench-0.5]# ./bin/sysbench cpu --cpu-max-prime=5000 --threads=2 --warmup-time=30  --time=60 --report-interval=5  run
sysbench 1.1.0-651e7fd (using bundled LuaJIT 2.1.0-beta3)

Running the test with following options:
Number of threads: 2
Warmup time: 30s
Report intermediate results every 5 second(s)
Initializing random number generator from current time


Prime numbers limit: 5000

Initializing worker threads...

Threads started!

Warming up for 30 seconds...

[ 5s ] thds: 2 eps: 5325.41 lat (ms,95%): 0.53
[ 10s ] thds: 2 eps: 5340.05 lat (ms,95%): 0.53
[ 15s ] thds: 2 eps: 5438.63 lat (ms,95%): 0.53
[ 20s ] thds: 2 eps: 5289.38 lat (ms,95%): 0.54
[ 25s ] thds: 2 eps: 5286.74 lat (ms,95%): 0.53
[ 30s ] thds: 2 eps: 5452.40 lat (ms,95%): 0.52
[ 35s ] thds: 2 eps: 5378.61 lat (ms,95%): 0.52
[ 40s ] thds: 2 eps: 5295.81 lat (ms,95%): 0.54
[ 45s ] thds: 2 eps: 5318.48 lat (ms,95%): 0.52
[ 50s ] thds: 2 eps: 5362.93 lat (ms,95%): 0.52
[ 55s ] thds: 2 eps: 5385.05 lat (ms,95%): 0.52
CPU speed:
    events per second:  5352.08

Throughput:
    events/s (eps):                      5352.0840
    time elapsed:                        60.0013s
    total number of events:              321122

Latency (ms):                                                                 #ms毫秒，us微秒，ns纳秒
         min:                                    0.29
         avg:                                    0.37
         max:                                   36.06
         95th percentile:                        0.53
         sum:                               119806.45

Threads fairness:
    events (avg/stddev):           160559.0000/1453.00
    execution time (avg/stddev):   59.9032/0.00

文件I/O基准测试
#查看操作系统的默认块大小
[root@server01 sysbench-0.5]# stat /boot/|grep "IO Block"
  Size: 4096            Blocks: 8          IO Block: 4096   directory

#测试系统在不同I/O负载下的性能
[root@server01 sysbench-0.5]# ./bin/sysbench  fileio help
sysbench 1.1.0-651e7fd (using bundled LuaJIT 2.1.0-beta3)

fileio options:
  --file-num=N                  number of files to create [128]
  --file-block-size=N           block size to use in all IO operations [16384]                  #innodb块16k,OS的默认块大小4k
  --file-total-size=SIZE        total size of files to create [2G]
  --file-test-mode=STRING       test mode {seqwr顺序写, seqrewr顺序读写, seqrd顺序读, rndrd随机读, rndwr随机写, rndrw随机读写}
  --file-io-mode=STRING         file operations mode {sync同步,async异步,mmap} [sync]
  --file-extra-flags=[LIST,...] list of additional flags to use to open files {sync,dsync,direct} []
  --file-fsync-freq=N           do fsync() after this number of requests (0 - don't use fsync()) [100]
  --file-fsync-all[=on|off]     do fsync() after each write operation [off]
  --file-fsync-end[=on|off]     do fsync() at the end of test [on]
  --file-fsync-mode=STRING      which method to use for synchronization {fsync, fdatasync} [fsync]
  --file-merged-requests=N      merge at most this number of IO requests if possible (0 - don't merge) [0]
  --file-rw-ratio=N             reads/writes ratio for combined test [1.5]
#准备阶段
生成测试用的数据文件，要求生成的数据文件至少要比内存大，否则数据由于被操作系统缓存而无法体现 I/O 密集型工作负载。
[root@server01 sysbench-0.5]# ./bin/sysbench  fileio --file-total-size=5G  --file-test-mode=rndrw  --file-block-size=16384  prepare
sysbench 1.1.0-651e7fd (using bundled LuaJIT 2.1.0-beta3)

128 files, 40960Kb each, 5120Mb total
Creating files for the test...
Extra file open flags: (none)
Creating file test_file.0
......
Creating file test_file.127
5368709120 bytes written in 345.57 seconds (14.82 MiB/sec).
[root@server01 sysbench-0.5]# ls -lh
total 5.0G
drwxr-xr-x. 2 root root  21 Mar 16 17:57 bin
drwxr-xr-x. 3 root root  21 Mar 16 17:57 share
-rw-------. 1 root root 40M Mar 19 10:18 test_file.0

#运行阶段，利用前一步生成的文件，运行测试
[root@server01 sysbench-0.5]# ./bin/sysbench  fileio --file-total-size=5G  --file-test-mode=rndrw  --file-block-size=16384 --threads=2 --warmup-time=30  --time=60 --report-interval=5  run
sysbench 1.1.0-651e7fd (using bundled LuaJIT 2.1.0-beta3)

Running the test with following options:
Number of threads: 2
Warmup time: 30s
Report intermediate results every 5 second(s)
Initializing random number generator from current time


Extra file open flags: (none)
128 files, 40MiB each
5GiB total file size
Block size 16KiB
Number of IO requests: 0
Read/Write ratio for combined random IO test: 1.50
Periodic FSYNC enabled, calling fsync() each 100 requests.
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random r/w test
Initializing worker threads...

Threads started!

Warming up for 30 seconds...

[ 5s ] reads: 0.73 MiB/s writes: 0.49 MiB/s fsyncs: 98.73/s latency (ms,95%): 58.923
[ 10s ] reads: 0.73 MiB/s writes: 0.48 MiB/s fsyncs: 102.40/s latency (ms,95%): 38.942
[ 15s ] reads: 0.92 MiB/s writes: 0.61 MiB/s fsyncs: 127.00/s latency (ms,95%): 34.330
[ 20s ] reads: 1.12 MiB/s writes: 0.75 MiB/s fsyncs: 129.62/s latency (ms,95%): 34.330
[ 25s ] reads: 1.13 MiB/s writes: 0.75 MiB/s fsyncs: 178.58/s latency (ms,95%): 34.954
[ 30s ] reads: 0.95 MiB/s writes: 0.63 MiB/s fsyncs: 128.00/s latency (ms,95%): 33.116
[ 35s ] reads: 1.08 MiB/s writes: 0.72 MiB/s fsyncs: 127.99/s latency (ms,95%): 33.718
[ 40s ] reads: 0.83 MiB/s writes: 0.56 MiB/s fsyncs: 128.00/s latency (ms,95%): 32.525
[ 45s ] reads: 0.91 MiB/s writes: 0.61 MiB/s fsyncs: 128.01/s latency (ms,95%): 35.589
[ 50s ] reads: 0.77 MiB/s writes: 0.52 MiB/s fsyncs: 102.40/s latency (ms,95%): 41.851
[ 55s ] reads: 0.85 MiB/s writes: 0.57 MiB/s fsyncs: 102.40/s latency (ms,95%): 32.525
[ 60s ] reads: 0.98 MiB/s writes: 0.65 MiB/s fsyncs: 139.60/s latency (ms,95%): 35.589

Throughput:
         read:  IOPS=57.12 0.89 MiB/s (0.94 MB/s)
         write: IOPS=38.06 0.59 MiB/s (0.62 MB/s)
         fsync: IOPS=122.58

Latency (ms):
         min:                                  0.00
         avg:                                  9.19
         max:                                948.38
         95th percentile:                     36.24
         sum:                             120152.38

#清除阶段
删除测试过程中的文件
[root@server01 sysbench-0.5]# ./bin/sysbench  fileio --file-total-size=5G   cleanup                                                                                     sysbench 1.1.0-651e7fd (using bundled LuaJIT 2.1.0-beta3)

Removing test files...
[root@server01 sysbench-0.5]# ls -lh
total 0
drwxr-xr-x. 2 root root 21 Mar 16 17:57 bin
drwxr-xr-x. 3 root root 21 Mar 16 17:57 share

内存测试

[root@server01 sysbench-0.5]# ./bin/sysbench  memory  help
sysbench 1.1.0-651e7fd (using bundled LuaJIT 2.1.0-beta3)

memory options:
  --memory-block-size=SIZE    size of memory block for test [1K]
  --memory-total-size=SIZE    total size of data to transfer [100G]
  --memory-scope=STRING       memory access scope {global,local} [global]
  --memory-hugetlb[=on|off]   allocate memory from HugeTLB pool [off]
  --memory-oper=STRING        type of memory operations {read, write, none} [write]
  --memory-access-mode=STRING memory access mode {seq,rnd} [seq]
[root@server01 sysbench-0.5]# ./bin/sysbench  memory --memory-block-size=4k  --memory-total-size=1G  --memory-access-mode=rnd run
sysbench 1.1.0-651e7fd (using bundled LuaJIT 2.1.0-beta3)

Running the test with following options:
Number of threads: 1
Initializing random number generator from current time


Running memory speed test with the following options:
  block size: 4KiB
  total size: 1024MiB
  operation: write
  scope: global

Initializing worker threads...

Threads started!

Total operations: 262144 (598832.43 per second)

1024.00 MiB transferred (2339.19 MiB/sec)


Throughput:
    events/s (eps):                      598832.4339
    time elapsed:                        0.4378s
    total number of events:              262144

Latency (ms):
         min:                                    0.00
         avg:                                    0.00
         max:                                    0.17
         95th percentile:                        0.00
         sum:                                  396.48

Threads fairness:
    events (avg/stddev):           262144.0000/0.00
    execution time (avg/stddev):   0.3965/0.00

线程测试
[root@server01 sysbench-0.5]# ./bin/sysbench  threads  help
sysbench 1.1.0-651e7fd (using bundled LuaJIT 2.1.0-beta3)

threads options:
  --thread-yields=N number of yields to do per request [1000]       #请求产生的线程
  --thread-locks=N  number of locks per thread [8]                  #每个线程的锁
[root@server01 sysbench-0.5]# ./bin/sysbench  threads  --threads=2 --time=60  --warmup-time=30  run
sysbench 1.1.0-651e7fd (using bundled LuaJIT 2.1.0-beta3)

Running the test with following options:
Number of threads: 2
Warmup time: 30s
Initializing random number generator from current time


Initializing worker threads...

Threads started!

Warming up for 30 seconds...


Throughput:
    events/s (eps):                      5041.7009
    time elapsed:                        60.0002s
    total number of events:              302499

Latency (ms):
         min:                                    0.28
         avg:                                    0.40
         max:                                   18.76
         95th percentile:                        0.77
         sum:                               119842.00

Threads fairness:
    events (avg/stddev):           151247.5000/30.50
    execution time (avg/stddev):   59.9210/0.00

OLTP测试
对于mysql的OLTP测试，和file一样，同样需要经历prepare,run,cleanup三个阶段。prepare阶段会在数据库中指定张和指定行数的表，默认表在sbtest架构下，表名为sbtestN，数据库sbtest需要手动在数据库中创建否则会报错。
通过调用不同的lua脚本测试不同业务模式下的数据库性能，也可自定义lua脚本进行测试。
[root@server01 sysbench0.5]# ./bin/sysbench share/sysbench/oltp_read_write.lua --mysql-host=localhost --mysql-port=3306 --mysql-user=root --mysql-password='Aa123456789' --mysql-socket=/data/mysqldata3306/sock/mysql.sock --mysql-db=sbtest --db-driver=mysql --tables=10 --table-size=1000000 --report-interval=10 --threads=4 --time=120  prepare
sysbench 1.1.0-651e7fd (using bundled LuaJIT 2.1.0-beta3)

Initializing worker threads...

Creating table 'sbtest3'...
......
Creating a secondary index on 'sbtest10'...

[root@server01 sysbench0.5]# ./bin/sysbench share/sysbench/oltp_read_write.lua --mysql-host=localhost --mysql-port=3306 --mysql-user=root --mysql-password='Aa123456789' --mysql-socket=/data/mysqldata3306/sock/mysql.sock --mysql-db=sbtest --db-driver=mysql --tables=10 --table-size=1000000 --report-interval=10 --threads=4 --time=120  run
sysbench 1.1.0-651e7fd (using bundled LuaJIT 2.1.0-beta3)

Running the test with following options:
Number of threads: 4
Report intermediate results every 10 second(s)
Initializing random number generator from current time
......
SQL statistics:
    queries performed:
        read:                            4340
        write:                           1240
        other:                           620
        total:                           6200
    transactions:                        310    (2.55 per sec.)                         #TPS
    queries:                             6200   (51.05 per sec.)                        #QPS
    ignored errors:                      0      (0.00 per sec.)
    reconnects:                          0      (0.00 per sec.)

Throughput:
    events/s (eps):                      2.5523
    time elapsed:                        121.4602s
    total number of events:              310

Latency (ms):
         min:                                  312.48
         avg:                                 1564.91                                   #平均响应时间
         max:                                 3674.74
         95th percentile:                     3040.14                                   #95%执行响应时间
         sum:                               485121.69

Threads fairness:
    events (avg/stddev):           77.5000/1.12
    execution time (avg/stddev):   121.2804/0.14

转载于:https://my.oschina.net/peakfang/blog/2875114