MySQL优化之mysqldumpslow及mysqlsla

1.mysqldumpslog
在my.cnf文件中加入以下的2行，加到[mysqld]部分，例如我的CentOS上的my.cnf文件位置为/usr/local/webserver/mysql
log-slow-queries=/tmp/mysqlslow.log
long-query-time=0

前者是让mysql记录slow query；long-query-time设置为0的含义是把所有的SQL都记录在slow query log中。如果不打补丁，那么long-query-time最小只能设为1(其实设为0也可以的，只不过不打补丁的时候无法记录微秒级别的时间)，即1秒，即只有超过1秒的SQL才会记录。打了补丁则没有这个限制了。

然后重启MySQL服务，重启后文件/tmp/mysqlslow.log会自动生成

还可以给MySQL打上补丁，这样就可以将查询时间精确到微秒级，具体可以看这儿。

mysqldumpslow的帮助选项如下

[root@CentOS_Test_Server ~]# mysqldumpslow --help
Usage: mysqldumpslow [ OPTS... ] [ LOGS... ]

Parse and summarize the MySQL slow query log. Options are

--verbose    verbose
--debug      debug
--help       write this text to standard output

-v           verbose
-d           debug
-s ORDER     what to sort by (t, at, l, al, r, ar etc), 'at' is default
-r           reverse the sort order (largest last instead of first)
-t NUM       just show the top n queries
-a           don't abstract all numbers to N and strings to 'S'
-n NUM       abstract numbers with at least n digits within names
-g PATTERN   grep: only consider stmts that include this string
-h HOSTNAME hostname of db server for *-slow.log filename (can be wildcard),
               default is '*', i.e. match all
-i NAME      name of server instance (if using mysql.server startup script)
-l           don't subtract lock time from total time

文件/tmp/mysqlslow.log的内容为
[root@CentOS_Test_Server tmp]# cat mysqlslow.log
/usr/local/webserver/mysql/libexec/mysqld, Version: 5.1.33-log (Source distribution). started with:
Tcp port: 3306 Unix socket: /tmp/mysql.sock
Time                 Id Command    Argument
/usr/local/webserver/mysql/libexec/mysqld, Version: 5.1.33-log (Source distribution). started with:
Tcp port: 3306 Unix socket: /tmp/mysql.sock
Time                 Id Command    Argument
# Time: 090908 15:36:05
# User@Host: root[root] @ [192.168.1.100]
# Query_time: 0.002321 Lock_time: 0.000629 Rows_sent: 8 Rows_examined: 8
SET timestamp=1252395365;
show variables like '%character%';
# User@Host: root[root] @ [192.168.1.100]
# Query_time: 0.000423 Lock_time: 0.000000 Rows_sent: 0 Rows_examined: 0
SET timestamp=1252395365;
Set character_set_connection=latin1;
# User@Host: root[root] @ [192.168.1.100]
# Query_time: 0.000294 Lock_time: 0.000000 Rows_sent: 0 Rows_examined: 0
SET timestamp=1252395365;
Set character_set_results=latin1;
# User@Host: root[root] @ [192.168.1.100]
# Query_time: 0.000202 Lock_time: 0.000000 Rows_sent: 0 Rows_examined: 0
SET timestamp=1252395365;
Set character_set_client=latin1;
# User@Host: root[root] @ [192.168.1.100]
# Query_time: 0.000265 Lock_time: 0.000000 Rows_sent: 0 Rows_examined: 0
SET timestamp=1252395365;
set sql_mode='';
# User@Host: root[root] @ [192.168.1.100]
# Query_time: 0.001412 Lock_time: 0.000180 Rows_sent: 5 Rows_examined: 5
SET timestamp=1252395365;
select * from mysql.user;

用mysqldumpslow命令分析文件/tmp/mysqlslow.log的内容
[root@CentOS_Test_Server tmp]# mysqldumpslow mysqlslow.log

Reading mysql slow query log from mysqlslow.log
Count: 1 Time=0.00s (0s) Lock=0.00s (0s) Rows=8.0 (8), root[root]@[192.168.1.100]
show variables like 'S'

Count: 1 Time=0.00s (0s) Lock=0.00s (0s) Rows=5.0 (5), root[root]@[192.168.1.100]
select * from mysql.user

Count: 1 Time=0.00s (0s) Lock=0.00s (0s) Rows=0.0 (0), root[root]@[192.168.1.100]
Set character_set_connection=latin1

Count: 1 Time=0.00s (0s) Lock=0.00s (0s) Rows=0.0 (0), root[root]@[192.168.1.100]
Set character_set_results=latin1

Count: 1 Time=0.00s (0s) Lock=0.00s (0s) Rows=0.0 (0), root[root]@[192.168.1.100]
set sql_mode='S'

Count: 1 Time=0.00s (0s) Lock=0.00s (0s) Rows=0.0 (0), root[root]@[192.168.1.100]
Set character_set_client=latin1

下面一段内容引用自这儿。

主要功能是, 统计不同慢sql的
出现次数(Count),
执行最长时间(Time),
累计总耗费时间(Time),
等待锁的时间(Lock),
发送给客户端的行总数(Rows),
扫描的行总数(Rows),

2.mysqlsla
[root@CentOS_Test_Server software]# wgethttp://hackmysql.com/scripts/mysqlsla-2.03.tar.gz
--18:44:18-- http://hackmysql.com/scripts/mysqlsla-2.03.tar.gz
正在解析主机 hackmysql.com... 64.13.232.157
Connecting to hackmysql.com|64.13.232.157|:80... 已连接。
已发出 HTTP 请求，正在等待回应... 200 OK
长度：33674 (33K) [application/x-tar]
Saving to: `mysqlsla-2.03.tar.gz'

100%[=====================================================================================>] 33,674      45.9K/s   in 0.7s

18:44:20 (45.9 KB/s) - `mysqlsla-2.03.tar.gz' saved [33674/33674]

[root@CentOS_Test_Server software]# tar zxvf mysqlsla-2.03.tar.gz
mysqlsla-2.03/
mysqlsla-2.03/Changes
mysqlsla-2.03/INSTALL
mysqlsla-2.03/README
mysqlsla-2.03/Makefile.PL
mysqlsla-2.03/bin/
mysqlsla-2.03/bin/mysqlsla
mysqlsla-2.03/META.yml
mysqlsla-2.03/lib/
mysqlsla-2.03/lib/mysqlsla.pm
mysqlsla-2.03/MANIFEST
[root@CentOS_Test_Server software]# cd mysqlsla-2.03
[root@CentOS_Test_Server mysqlsla-2.03]# ll
总计 68
drwxr-xr-x 2 1000 1000 4096 2008-11-11 bin
-rw-r--r-- 1 1000 1000 5630 2008-11-11 Changes
-rw-r--r-- 1 1000 1000 394 2008-07-13 INSTALL
drwxr-xr-x 2 1000 1000 4096 2008-11-11 lib
-rw-r--r-- 1 1000 1000 389 2008-07-13 Makefile.PL
-rw-r--r-- 1 1000 1000 152 2008-07-13 MANIFEST
-rw-r--r-- 1 1000 1000 303 2008-11-11 META.yml
-rw-r--r-- 1 1000 1000 1969 2008-11-11 README
[root@CentOS_Test_Server mysqlsla-2.03]# more INSTALL

Installing mysqlsla
===================

   perl Makefile.PL
   make
   make install

Depending on your system, the mysqlsla script will be copied to some common bin
directory (/usr/local/bin/ for example). Then you should be able to run it from
the command line like: mysqlsla -lt slow LOG

For quick help, man mysqlsla

For all documentation and guides, visit http://hackmysql.com/mysqlsla

[root@CentOS_Test_Server mysqlsla-2.03]# more Makefile.PL
use 5.008004;
use ExtUtils::MakeMaker;

WriteMakefile(
    NAME              => 'mysqlsla',
    VERSION_FROM      => 'lib/mysqlsla.pm',
    PREREQ_PM         => {},
    ($] >= 5.005 ?
      (ABSTRACT_FROM => 'lib/mysqlsla.pm', # retrieve abstract from module
       AUTHOR         => 'Daniel Nichter (http://hackmysql.com/contact)') : ()),
    EXE_FILES         => [ 'bin/mysqlsla' ],
);
[root@CentOS_Test_Server mysqlsla-2.03]# perl Makefile.PL
Checking if your kit is complete...
Looks good
Writing Makefile for mysqlsla
[root@CentOS_Test_Server mysqlsla-2.03]# ll
总计 96
drwxr-xr-x 2 1000 1000 4096 2008-11-11 bin
-rw-r--r-- 1 1000 1000 5630 2008-11-11 Changes
-rw-r--r-- 1 1000 1000   394 2008-07-13 INSTALL
drwxr-xr-x 2 1000 1000 4096 2008-11-11 lib
-rw-r--r-- 1 root root 22722 09-08 18:52 Makefile
-rw-r--r-- 1 1000 1000   389 2008-07-13 Makefile.PL
-rw-r--r-- 1 1000 1000   152 2008-07-13 MANIFEST
-rw-r--r-- 1 1000 1000   303 2008-11-11 META.yml
-rw-r--r-- 1 1000 1000 1969 2008-11-11 README
[root@CentOS_Test_Server mysqlsla-2.03]# make
cp lib/mysqlsla.pm blib/lib/mysqlsla.pm
cp bin/mysqlsla blib/script/mysqlsla
/usr/bin/perl "-MExtUtils::MY" -e "MY->fixin(shift)" blib/script/mysqlsla
Manifying blib/man3/mysqlsla.3pm
[root@CentOS_Test_Server mysqlsla-2.03]# make install
Installing /usr/lib/perl5/site_perl/5.8.8/mysqlsla.pm
Installing /usr/share/man/man3/mysqlsla.3pm
Installing /usr/bin/mysqlsla
Writing /usr/lib/perl5/site_perl/5.8.8/i386-linux-thread-multi/auto/mysqlsla/.packlist
Appending installation info to /usr/lib/perl5/5.8.8/i386-linux-thread-multi/perllocal.pod

默认将命令mysqlsla安装到/usr/bin目录
执行mysqlsla --help时提示没有安装DBI模块，mysqlsla是用perl写的

第一次通过CPAN安装perl模块时，需要进行相关的配置，大部分配置采用默认值，一路回车即可
#perl -MCPAN -e shell
然后在下面的各个询问中可以改已经配置好的选项
然后执行下面的命令保存
CPAN>reload index
CPAN>reload cpan
安装DBI模块
CPAN>install DBI

如果是第一次以后进入CPAN系统，想修改以前的配置，可以在CPAN命令行(即CPAN>)键入以下指令:
CPAN>o conf init
同时要执行下面的命令保存
CPAN>reload index
CPAN>reload cpan

man mysqlsla可以看到mysqlsla的相关选项
[root@CentOS_Test_Server software]# man mysqlsla
XXX
XXX WARNING: old character encoding and/or character set
XXX
mysqlsla(3)           User Contributed Perl Documentation          mysqlsla(3)

NAME
       mysqlsla - Parse, filter, analyze and sort MySQL slow, general and binary logs

SYNOPSIS
          # Basic operation: parse a MySQL slow or general log
           mysqlsla --log-type slow LOG
           mysqlsla --log-type general LOG

           # Parse output from mysqlbinlog
           # mysqlsla cannot directly parse binary logs
           mysqlbinlog LOG │ mysqlsla --log-type binary -

           # Parse a microslow patched slow log
           mysqlsla --log-type msl LOG

           # Replay a replay file
           mysqlsla --replay FILE

           # Parse a user-defined log specify its format
           mysqlsla --log-type udl --udl-format FILE

          # Let mysqlsla automatically determine the log type
           mysqlsla LOG

DESCRIPTION
       mysqlsla parses, filters, analyzes and sorts MySQL slow, general, binary and microslow patched slow logs.
       It also supports user-defined logs.

       This POD/man page is only a very brief outline of usage and command line options. For the full library of
       mysqlsla documentation visithttp://hackmysql.com/mysqlsla.

[root@CentOS_Test_Server ~]# mysqlsla /tmp/mysqlslow.log | more
Auto-detected logs as slow logs
Report for slow logs: /tmp/mysqlslow.log
6 queries total, 6 unique
Sorted by 't_sum'
Grand Totals: Time 0 s, Lock 0 s, Rows sent 13, Rows Examined 13

______________________________________________________________________ 001 ___
Count         : 1 (16.67%)
Time          : 2.321 ms total, 2.321 ms avg, 2.321 ms to 2.321 ms max (47.20%)
Lock Time (s) : 629 ?s total, 629 ?s avg, 629 ?s to 629 ?s max (77.75%)
Rows sent     : 8 avg, 8 to 8 max (61.54%)
Rows examined : 8 avg, 8 to 8 max (61.54%)
Database      :
Users         :
        root@ 192.168.1.100 : 100.00% (1) of query, 100.00% (6) of all users

Query abstract:
SET timestamp=N; SHOW variables LIKE 'S';

Query sample:
SET timestamp=1252395365;
show variables like '%character%';
......

以下内容引用自http://kenwu.me/compare-five-mysql-slowlog-analysis-tool
     整体来说, 功能非常强大. 数据报表,非常有利于分析慢查询的原因, 包括执行频率, 数据量, 查询消耗等.

格式说明如下:
总查询次数 (queries total), 去重后的sql数量 (unique)
输出报表的内容排序(sorted by)
最重大的慢sql统计信息, 包括平均执行时间, 等待锁时间, 结果行的总数, 扫描的行总数.

Count, sql的执行次数及占总的slow log数量的百分比.
Time, 执行时间, 包括总时间, 平均时间, 最小, 最大时间, 时间占到总慢sql时间的百分比.
95% of Time, 去除最快和最慢的sql, 覆盖率占95%的sql的执行时间.
Lock Time, 等待锁的时间.
95% of Lock , 95%的慢sql等待锁时间.
Rows sent, 结果行统计数量, 包括平均, 最小, 最大数量.
Rows examined, 扫描的行数量.
Database, 属于哪个数据库
Users, 哪个用户,IP, 占到所有用户执行的sql百分比

Query abstract, 抽象后的sql语句
Query sample, sql语句

除了以上的输出, 官方还提供了很多定制化参数, 是一款不可多得的好工具.