MySQL Optimization Learning(三)

杀神lwz

已于 2023-07-06 23:53:30 修改

阅读量204

点赞数

分类专栏：数据库文章标签： mysql 信息可视化数据库

于 2023-07-03 23:36:44 首次发布

本文链接：https://blog.csdn.net/weixin_42472027/article/details/131505713

版权

数据库专栏收录该内容

14 篇文章 2 订阅

订阅专栏

6.1、当使用索引列进行查询的时候尽量不要使用表达式，把计算放到业务层而不是数据库层

6.2、尽量使用主键查询，而不是其他索引，因此主键查询不会触发回表查询

6.3、使用前缀索引

6.4、使用索引扫描来排序

6.5、union all,in,or都能够使用索引，但是推荐使用in

7、索引监控

一、通过索引进行优化

数据结构

Data Structure Visualizations 数据可视化效果展示

Binary Search Tree

插入数据可视化效果展示

AVL Tree

Red/Black Tree

--MYISAM存储引擎数据和引用分开存储
DROP TABLE IF EXISTS `t_test`;
CREATE TABLE `t_test` (
  `id` int(11) NOT NULL,
 `test` varchar(9) DEFAULT NULL,
  PRIMARY KEY (`id`)
) ENGINE=MYISAM;

--InnoDB
DROP TABLE IF EXISTS `t_test2`;
CREATE TABLE `t_test2` (
  `id` int(11) NOT NULL,
 `test` varchar(9) DEFAULT NULL,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB;

--查看数据存储路径
show variables like '%datadir%';
mysql> show variables like '%datadir%';
+---------------+---------------------------------------------+
| Variable_name | Value                                       |
+---------------+---------------------------------------------+
| datadir       | C:\ProgramData\MySQL\MySQL Server 8.0\Data\ |
+---------------+---------------------------------------------+
1 row in set, 1 warning (0.00 sec)

--MYISAM存储引擎有如下2个文件
t_test.MYD:存储数据
t_test.MYI:存储索引

--InnoDB存储引擎有如下1个文件
t_test2.ibd:存储文件+索引

1、索引基本知识

索引的优点

1、大大减少了服务器需要扫描的数据量
2、帮助服务器避免排序和临时表
3、将随机io变成顺序io

索引的用处

1、快速查找匹配WHERE子句的行
2、从consideration中消除行,如果可以在多个索引之间进行选择，mysql通常会使用找到最少行的索引
3、如果表具有多列索引，则优化器可以使用索引的任何最左前缀来查找行
4、当有表连接的时候，从其他表检索行数据
5、查找特定索引列的min或max值
6、如果排序或分组时在可用索引的最左前缀上完成的，则对表进行排序和分组
7、在某些情况下，可以优化查询以检索值而无需查询数据行

索引的分类

主键索引
唯一索引
普通索引
全文索引
组合索引

技术名词

回表
覆盖索引
最左匹配
索引下推

索引采用的数据结构

哈希表
B+树

索引匹配方式

CREATE TABLE staffs (
	id INT PRIMARY KEY auto_increment,
	NAME VARCHAR ( 24 ) NOT NULL DEFAULT '' COMMENT '姓名',
	age INT NOT NULL DEFAULT 0 COMMENT '年龄',
	pos VARCHAR ( 20 ) NOT NULL DEFAULT '' COMMENT '职位',
	add_time TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '入职时间' 
) charset utf8 COMMENT '员工记录表';

ALTER TABLE staffs ADD INDEX idx_nap ( NAME, age, pos );

INSERT INTO staffs VALUES ('1', 'zhangsan', 30,'dev',now());
INSERT INTO staffs VALUES ('2', 'lisi', 36,'dev',now());
INSERT INTO staffs VALUES ('3', 'July', 23,'dev',now());
INSERT INTO staffs VALUES ('4', 'Mary', 30,'dev',now());
INSERT INTO staffs VALUES ('6', 'Maryz', 30,'dev',now());
INSERT INTO staffs VALUES ('7', 'July', 25,'dev',now());
INSERT INTO staffs VALUES ('8', 'July', 30,'dev',now());

mysql> show index from staffs;
+--------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+---------+------------+
| Table  | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment | Visible | Expression |
+--------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+---------+------------+
| staffs |          0 | PRIMARY  |            1 | id          | A         |           7 |     NULL |   NULL |      | BTREE      |         |               | YES     | NULL       |
| staffs |          1 | idx_nap  |            1 | NAME        | A         |           5 |     NULL |   NULL |      | BTREE      |         |               | YES     | NULL       |
| staffs |          1 | idx_nap  |            2 | age         | A         |           7 |     NULL |   NULL |      | BTREE      |         |               | YES     | NULL       |
| staffs |          1 | idx_nap  |            3 | pos         | A         |           7 |     NULL |   NULL |      | BTREE      |         |               | YES     | NULL       |
+--------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+---------+------------+
4 rows in set (0.02 sec)

全值匹配

--全值匹配指的是和索引中的所有列进行匹配
explain select * from staffs where name = 'July' and age = '23' and pos = 'dev';
mysql> explain select * from staffs where name = 'July' and age = '23' and pos = 'dev';
+----+-------------+--------+------------+------+---------------+---------+---------+-------------------+------+----------+-------+
| id | select_type | table  | partitions | type | possible_keys | key     | key_len | ref               | rows | filtered | Extra |
+----+-------------+--------+------------+------+---------------+---------+---------+-------------------+------+----------+-------+
|  1 | SIMPLE      | staffs | NULL       | ref  | idx_nap       | idx_nap | 140     | const,const,const |    1 |   100.00 | NULL  |
+----+-------------+--------+------------+------+---------------+---------+---------+-------------------+------+----------+-------+
1 row in set, 1 warning (0.00 sec)

匹配最左前缀

--只匹配前面的几列
explain select * from staffs where name = 'July' and age = '23';
explain select * from staffs where name = 'July';
mysql> explain select * from staffs where name = 'July' and age = '23';
+----+-------------+--------+------------+------+---------------+---------+---------+-------------+------+----------+-------+
| id | select_type | table  | partitions | type | possible_keys | key     | key_len | ref         | rows | filtered | Extra |
+----+-------------+--------+------------+------+---------------+---------+---------+-------------+------+----------+-------+
|  1 | SIMPLE      | staffs | NULL       | ref  | idx_nap       | idx_nap | 78      | const,const |    1 |   100.00 | NULL  |
+----+-------------+--------+------------+------+---------------+---------+---------+-------------+------+----------+-------+
1 row in set, 1 warning (0.00 sec)

mysql> explain select * from staffs where name = 'July';
+----+-------------+--------+------------+------+---------------+---------+---------+-------+------+----------+-------+
| id | select_type | table  | partitions | type | possible_keys | key     | key_len | ref   | rows | filtered | Extra |
+----+-------------+--------+------------+------+---------------+---------+---------+-------+------+----------+-------+
|  1 | SIMPLE      | staffs | NULL       | ref  | idx_nap       | idx_nap | 74      | const |    3 |   100.00 | NULL  |
+----+-------------+--------+------------+------+---------------+---------+---------+-------+------+----------+-------+
1 row in set, 1 warning (0.00 sec)

匹配列前缀

--可以匹配某一列的值的开头部分
explain select * from staffs where name like 'J%';
explain select * from staffs where name like '%y';

mysql> explain select * from staffs where name like 'J%';
+----+-------------+--------+------------+-------+---------------+---------+---------+------+------+----------+-----------------------+
| id | select_type | table  | partitions | type  | possible_keys | key     | key_len | ref  | rows | filtered | Extra                 |
+----+-------------+--------+------------+-------+---------------+---------+---------+------+------+----------+-----------------------+
|  1 | SIMPLE      | staffs | NULL       | range | idx_nap       | idx_nap | 74      | NULL |    3 |   100.00 | Using index condition |
+----+-------------+--------+------------+-------+---------------+---------+---------+------+------+----------+-----------------------+
1 row in set, 1 warning (0.00 sec)

mysql> explain select * from staffs where name like '%y';
+----+-------------+--------+------------+------+---------------+------+---------+------+------+----------+-------------+
| id | select_type | table  | partitions | type | possible_keys | key  | key_len | ref  | rows | filtered | Extra       |
+----+-------------+--------+------------+------+---------------+------+---------+------+------+----------+-------------+
|  1 | SIMPLE      | staffs | NULL       | ALL  | NULL          | NULL | NULL    | NULL |    7 |    14.29 | Using where |
+----+-------------+--------+------------+------+---------------+------+---------+------+------+----------+-------------+
1 row in set, 1 warning (0.00 sec)

匹配范围值

--可以查找某一个范围的数据
explain select * from staffs where name > 'Mary';
mysql> explain select * from staffs where name > 'Mary';
+----+-------------+--------+------------+-------+---------------+---------+---------+------+------+----------+-----------------------+
| id | select_type | table  | partitions | type  | possible_keys | key     | key_len | ref  | rows | filtered | Extra                 |
+----+-------------+--------+------------+-------+---------------+---------+---------+------+------+----------+-----------------------+
|  1 | SIMPLE      | staffs | NULL       | range | idx_nap       | idx_nap | 74      | NULL |    2 |   100.00 | Using index condition |
+----+-------------+--------+------------+-------+---------------+---------+---------+------+------+----------+-----------------------+
1 row in set, 1 warning (0.00 sec)

精确匹配某一列并范围匹配另外一列

--可以查询第一列的全部和第二列的部分
explain select * from staffs where name = 'July' and age > 25;
mysql> explain select * from staffs where name = 'July' and age > 25;
+----+-------------+--------+------------+-------+---------------+---------+---------+------+------+----------+-----------------------+
| id | select_type | table  | partitions | type  | possible_keys | key     | key_len | ref  | rows | filtered | Extra                 |
+----+-------------+--------+------------+-------+---------------+---------+---------+------+------+----------+-----------------------+
|  1 | SIMPLE      | staffs | NULL       | range | idx_nap       | idx_nap | 78      | NULL |    1 |   100.00 | Using index condition |
+----+-------------+--------+------------+-------+---------------+---------+---------+------+------+----------+-----------------------+
1 row in set, 1 warning (0.00 sec)

只访问索引的查询:也叫索引覆盖 Extra:Using index

--查询的时候只需要访问索引，不需要访问数据行，本质上就是覆盖索引
explain select name,age,pos from staffs where name = 'July' and age = 25 and pos = 'dev';
mysql> explain select name,age,pos from staffs where name = 'July' and age = 25 and pos = 'dev';
+----+-------------+--------+------------+------+---------------+---------+---------+-------------------+------+----------+-------------+
| id | select_type | table  | partitions | type | possible_keys | key     | key_len | ref               | rows | filtered | Extra       |
+----+-------------+--------+------------+------+---------------+---------+---------+-------------------+------+----------+-------------+
|  1 | SIMPLE      | staffs | NULL       | ref  | idx_nap       | idx_nap | 140     | const,const,const |    1 |   100.00 | Using index |
+----+-------------+--------+------------+------+---------------+---------+---------+-------------------+------+----------+-------------+
1 row in set, 1 warning (0.00 sec)

MySQL官网提供数据结构案例

下载sakila database，执行SQL语句

--导入DB结构
mysql> source D:\.m2\sakila-db\sakila-schema.sql
Query OK, 0 rows affected (0.01 sec)
.....

--导入数据
mysql> source D:\.m2\sakila-db\sakila-data.sql
Query OK, 0 rows affected (0.00 sec)

Database changed
Query OK, 0 rows affected (0.00 sec)
...

2、哈希索引

基于哈希表的实现，只有精确匹配索引所有列的查询才有效
在mysql中，只有memory的存储引擎显式支持哈希索引
哈希索引自身只需存储对应的hash值，所以索引的结构十分紧凑，这让哈希索引查找的速度非常快
哈希索引的限制
1、哈希索引只包含哈希值和行指针，而不存储字段值，索引不能使用索引中的值来避免读取行
2、哈希索引数据并不是按照索引值顺序存储的，所以无法进行排序
3、哈希索引不支持部分列匹配查找，哈希索引是使用索引列的全部内容来计算哈希值
4、哈希索引支持等值比较查询，也不支持任何范围查询
5、访问哈希索引的数据非常快，除非有很多哈希冲突，当出现哈希冲突的时候，存储引擎必须遍历链表中的所有行指针，逐行进行比较，直到找到所有符合条件的行
6、哈希冲突比较多的话，维护的代价也会很高

案例

当需要存储大量的URL，并且根据URL进行搜索查找，如果使用B+树，存储的内容就会很大
select id from url where url=""
也可以利用将url使用CRC32做哈希，可以使用以下查询方式：
select id fom url where url="" and url_crc=CRC32("")
此查询性能较高原因是使用体积很小的索引来完成查找

CRC（Cyclic Redundancy Check）校验实用程序库在数据存储和数据通讯领域，为了保证数据的正确，就不得不采用检错的手段。在诸多检错手段中，CRC是最著名的一种。CRC的全称是循环冗余校验。

CRC的本质
是模-2除法的余数，采用的除数不同，CRC的类型也就不一样。通常，CRC的除数用生成多项式来表示。最常用的CRC码的生成多项式如表1所示。最常用的CRC码及生成多项式名称生成多项式
CRC-12

CRC-16

CRC-CCITT

CRC-32

由于CRC在通讯和数据处理软件中经常采用，笔者在实际工作中对其算法进行了研究和比较，总结并编写了一个具有最高效率的CRC通用程序库。该程序采用查表法计算CRC，在速度上优于一般的直接模仿硬件的算法，可以应用于通讯和数据压缩程序。

3、组合索引

当包含多个列作为索引，需要注意的是正确的顺序依赖于该索引的查询，同时需要考虑如何更好的满足排序和分组的需要

案例

建立组合索引a,b,c，不同SQL语句使用索引情况

语句	索引是否发挥作用
where a=3	是，只使用了a
where a=3 and b=5	是，使用了a,b
where a=3 and b=5 and c=4	是，使用了a,b,c
where b=3 or c=4	否
where a=3 and c=4	是，使用了a
where a=3 and b>10 and c=7	是，使用了a,b
where a=3 and b like '%xx%' and c=7	是，使用了a

4、聚簇索引与非聚簇索引

4.1、聚簇索引

不是单独的索引类型，而是一种数据存储方式，指的是数据行跟相邻的键值紧凑的存储在一起

优点
   1、可以把相关数据保存在一起
   2、数据访问更快，因为索引和数据保存在同一个树中
   3、使用覆盖索引扫描的查询可以直接使用页节点中的主键值
缺点
   1、聚簇数据最大限度地提高了IO密集型应用的性能，如果数据全部在内存，那么聚簇索引就没有什么优势
   2、插入速度严重依赖于插入顺序，按照主键的顺序插入是最快的方式
   3、更新聚簇索引列的代价很高，因为会强制将每个被更新的行移动到新的位置
   4、基于聚簇索引的表在插入新行，或者主键被更新导致需要移动行的时候，可能面临页分裂的问题
   5、聚簇索引可能导致全表扫描变慢，尤其是行比较稀疏，或者由于页分裂导致数据存储不连续的时候
4.2、非聚簇索引
   数据文件跟索引文件分开存放

--MYISAM存储引擎数据和引用分开存储
DROP TABLE IF EXISTS `t_test`;
CREATE TABLE `t_test` (
  `id` int(11) NOT NULL,
 `test` varchar(9) DEFAULT NULL,
  PRIMARY KEY (`id`)
) ENGINE=MYISAM;

--InnoDB
DROP TABLE IF EXISTS `t_test2`;
CREATE TABLE `t_test2` (
  `id` int(11) NOT NULL,
 `test` varchar(9) DEFAULT NULL,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB;

--查看数据存储路径
show variables like '%datadir%';
mysql> show variables like '%datadir%';
+---------------+---------------------------------------------+
| Variable_name | Value                                       |
+---------------+---------------------------------------------+
| datadir       | C:\ProgramData\MySQL\MySQL Server 8.0\Data\ |
+---------------+---------------------------------------------+
1 row in set, 1 warning (0.00 sec)

--MYISAM存储引擎有如下2个文件
t_test.MYD:存储数据
t_test.MYI:存储索引

--InnoDB存储引擎有如下1个文件
t_test2.ibd:存储文件+索引

5、覆盖索引

基本介绍
   1、如果一个索引包含所有需要查询的字段的值，我们称之为覆盖索引
   2、不是所有类型的索引都可以称为覆盖索引，覆盖索引必须要存储索引列的值
   3、不同的存储实现覆盖索引的方式不同，不是所有的引擎都支持覆盖索引，memory不支持覆盖索引
优势
   1、索引条目通常远小于数据行大小，如果只需要读取索引，那么mysql就会极大的较少数据访问量
   2、因为索引是按照列值顺序存储的，所以对于IO密集型的范围查询会比随机从磁盘读取每一行数据的IO要少的多
   3、一些存储引擎如MYISAM在内存中只缓存索引，数据则依赖于操作系统来缓存，因此要访问数据需要一次系统调用，这可能会导致严重的性能问题
   4、由于INNODB的聚簇索引，覆盖索引对INNODB表特别有用

--1、当发起一个被索引覆盖的查询时，在explain的extra列可以看到using index的信息，此时就使用了覆盖索引
mysql> use sakila;
Database changed
mysql> explain select store_id,film_id from inventory\G
*************************** 1. row ***************************
           id: 1
  select_type: SIMPLE
        table: inventory
   partitions: NULL
         type: index
possible_keys: NULL
          key: idx_store_id_film_id
      key_len: 3
          ref: NULL
         rows: 4581
     filtered: 100.00
        Extra: Using index
1 row in set, 1 warning (0.01 sec)

--2、在大多数存储引擎中，覆盖索引只能覆盖那些只访问索引中部分列的查询。不过，可以进一步的进行优化，可以使用innodb的二级索引来覆盖查询。
--例如：actor使用innodb存储引擎，并在last_name字段又二级索引，虽然该索引的列不包括主键actor_id，但也能够用于对actor_id做覆盖查询
mysql> explain select actor_id,last_name from actor where last_name='HOPPER'\G
*************************** 1. row ***************************
           id: 1
  select_type: SIMPLE
        table: actor
   partitions: NULL
         type: ref
possible_keys: idx_actor_last_name
          key: idx_actor_last_name
      key_len: 182
          ref: const
         rows: 2
     filtered: 100.00
        Extra: Using index
1 row in set, 1 warning (0.01 sec)

6、优化小细节

6.1、当使用索引列进行查询的时候尽量不要使用表达式，把计算放到业务层而不是数据库层

select actor_id from actor where actor_id=4;
select actor_id from actor where actor_id+1=5;

mysql> explain select actor_id from actor where actor_id=4;
+----+-------------+-------+------------+-------+---------------+---------+---------+-------+------+----------+-------------+
| id | select_type | table | partitions | type  | possible_keys | key     | key_len | ref   | rows | filtered | Extra       |
+----+-------------+-------+------------+-------+---------------+---------+---------+-------+------+----------+-------------+
|  1 | SIMPLE      | actor | NULL       | const | PRIMARY       | PRIMARY | 2       | const |    1 |   100.00 | Using index |
+----+-------------+-------+------------+-------+---------------+---------+---------+-------+------+----------+-------------+
1 row in set, 1 warning (0.00 sec)

mysql> explain select actor_id from actor where actor_id+1=5;
+----+-------------+-------+------------+-------+---------------+---------------------+---------+------+------+----------+--------------------------+
| id | select_type | table | partitions | type  | possible_keys | key                 | key_len | ref  | rows | filtered | Extra                    |
+----+-------------+-------+------------+-------+---------------+---------------------+---------+------+------+----------+--------------------------+
|  1 | SIMPLE      | actor | NULL       | index | NULL          | idx_actor_last_name | 182     | NULL |  200 |   100.00 | Using where; Using index |
+----+-------------+-------+------------+-------+---------------+---------------------+---------+------+------+----------+--------------------------+
1 row in set, 1 warning (0.00 sec)

6.2、尽量使用主键查询，而不是其他索引，因此主键查询不会触发回表查询

6.3、使用前缀索引

有时候需要索引很长的字符串，这会让索引变的大且慢，通常情况下可以使用某个列开始的部分字符串，这样大大的节约索引空间，从而提高索引效率，但这会降低索引的选择性，索引的选择性是指不重复的索引值和数据表记录总数的比值，范围从1/#T到1之间。索引的选择性越高则查询效率越高，因为选择性更高的索引可以让mysql在查找的时候过滤掉更多的行。

一般情况下某个列前缀的选择性也是足够高的，足以满足查询的性能，但是对应BLOB,TEXT,VARCHAR类型的列，必须要使用前缀索引，因为mysql不允许索引这些列的完整长度，使用该方法的诀窍在于要选择足够长的前缀以保证较高的选择性，通过又不能太长。

案例演示：

--创建数据表
create table citydemo(city varchar(50) not null);
insert into citydemo(city) select city from city;

--重复执行5次下面的sql语句
insert into citydemo(city) select city from citydemo;

--更新城市表的名称
update citydemo set city=(select city from city order by rand() limit 1);

--查找最常见的城市列表，发现每个值都出现11-13次，
select count(*) as cnt,city from citydemo group by city order by cnt desc limit 10;
mysql> select count(*) as cnt,city from citydemo group by city order by cnt desc limit 10;
+-----+-----------------------+
| cnt | city                  |
+-----+-----------------------+
|  13 | Udine                 |
|  12 | Siliguri (Shiliguri)  |
|  12 | Sungai Petani         |
|  12 | Ogbomosho             |
|  11 | Yangor                |
|  11 | Jastrzebie-Zdrój     |
|  11 | Ueda                  |
|  11 | Monywa                |
|  11 | Allappuzha (Alleppey) |
|  11 | Garden Grove          |
+-----+-----------------------+
10 rows in set (0.01 sec)

--查找最频繁出现的城市前缀，先从3个前缀字母开始，发现比原来出现的次数更多，可以分别截取多个字符查看城市出现的次数
select count(*) as cnt,left(city,3) as pref from citydemo group by pref order by cnt desc limit 10;
select count(*) as cnt,left(city,7) as pref from citydemo group by pref order by cnt desc limit 10;
mysql> select count(*) as cnt,left(city,3) as pref from citydemo group by pref order by cnt desc limit 10;
+-----+------+
| cnt | pref |
+-----+------+
|  64 | San  |
|  27 | Sal  |
|  26 | Val  |
|  23 | Tan  |
|  22 | Cha  |
|  21 | Sou  |
|  21 | Man  |
|  21 | Bat  |
|  20 | al-  |
|  20 | Yan  |
+-----+------+
10 rows in set (0.01 sec)

mysql> select count(*) as cnt,left(city,7) as pref from citydemo group by pref order by cnt desc limit 10;
+-----+---------+
| cnt | pref    |
+-----+---------+
|  14 | Valle d |
|  13 | Udine   |
|  12 | Siligur |
|  12 | Sungai  |
|  12 | Ogbomos |
|  11 | Yangor  |
|  11 | San Fel |
|  11 | Jastrze |
|  11 | Ueda    |
|  11 | Monywa  |
+-----+---------+
10 rows in set (0.00 sec)
--此时前缀的选择性接近于完整列的选择性

--还可以通过另外一种方式来计算完整列的选择性，可以看到当前缀长度到达7之后，再增加前缀长度，选择性提升的幅度已经很小了
select count(distinct left(city,3))/count(*) as sel3,
count(distinct left(city,4))/count(*) as sel4,
count(distinct left(city,5))/count(*) as sel5,
count(distinct left(city,6))/count(*) as sel6,
count(distinct left(city,7))/count(*) as sel7,
count(distinct left(city,8))/count(*) as sel8 
from citydemo;

mysql> select count(distinct left(city,3))/count(*) as sel3,
    -> count(distinct left(city,4))/count(*) as sel4,
    -> count(distinct left(city,5))/count(*) as sel5,
    -> count(distinct left(city,6))/count(*) as sel6,
    -> count(distinct left(city,7))/count(*) as sel7,
    -> count(distinct left(city,8))/count(*) as sel8
    -> from citydemo;
+--------+--------+--------+--------+--------+--------+
| sel3   | sel4   | sel5   | sel6   | sel7   | sel8   |
+--------+--------+--------+--------+--------+--------+
| 0.1500 | 0.1857 | 0.1923 | 0.1950 | 0.1960 | 0.1960 |
+--------+--------+--------+--------+--------+--------+
1 row in set (0.01 sec)

--计算完成之后可以创建前缀索引
alter table citydemo add key(city(7));

mysql> show index from citydemo;
+----------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+---------+------------+
| Table    | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment | Visible | Expression |
+----------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+---------+------------+
| citydemo |          1 | city     |            1 | city        | A         |         588 |        7 |   NULL |      | BTREE      |         |               | YES     | NULL       |
+----------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+---------+------------+
1 row in set (0.01 sec)

Cardinality:基数   基数统计，某一单列，去重之后的唯一值就是基数  OLAP：在线数据分析；
mysql> select count(distinct city) from citydemo;
+----------------------+
| count(distinct city) |
+----------------------+
|                  591 |
+----------------------+
1 row in set (0.01 sec)

--注意：前缀索引是一种能使索引更小更快的有效方法，但是也包含缺点：mysql无法使用前缀索引做order by 和 group by。

原文链接：探索HyperLogLog算法（含Java实现） - 简书

HyperLogLog算法

HyperLogLog算法经常在数据库中被用来统计某一字段的Distinct Value（下文简称DV），比如Redis的HyperLogLog结构。

基数就是指一个集合中不同值的数目，比如[a,b,c,d]的基数就是4，[a,b,c,d,a]的基数还是4，因为a重复了一个，不算。基数也可以称之为Distinct Value，简称DV。HyperLogLog算法就是用来计算基数的。

6.4、使用索引扫描来排序

        mysql有两种方式可以生成有序的结果：通过排序操作或者按索引顺序扫描，如果explain出来的type列的值为index,则说明mysql使用了索引扫描来做排序
        扫描索引本身是很快的，因为只需要从一条索引记录移动到紧接着的下一条记录。但如果索引不能覆盖查询所需的全部列，那么就不得不每扫描一条索引记录就得回表查询一次对应的行，这基本都是随机IO，因此按索引顺序读取数据的速度通常要比顺序地全表扫描慢
        mysql可以使用同一个索引即满足排序，又用于查找行，如果可能的话，设计索引时应该尽可能地同时满足这两种任务。
        只有当索引的列顺序和order by子句的顺序完全一致，并且所有列的排序方式都一样时，mysql才能够使用索引来对结果进行排序，如果查询需要关联多张表，则只有当orderby子句引用的字段全部为第一张表时，才能使用索引做排序。order by子句和查找型查询的限制是一样的，需要满足索引的最左前缀的要求，否则，mysql都需要执行顺序操作，而无法利用索引排序

--sakila数据库中rental表在rental_date,inventory_id,customer_id上有rental_date的索引
--使用rental_date索引为下面的查询做排序
mysql> explain select rental_id,staff_id from rental where rental_date='2005-05-25' order by inventory_id,customer_id\G
*************************** 1. row ***************************
           id: 1
  select_type: SIMPLE
        table: rental
   partitions: NULL
         type: ref
possible_keys: rental_date
          key: rental_date
      key_len: 5
          ref: const
         rows: 1
     filtered: 100.00
        Extra: NULL
1 row in set, 1 warning (0.01 sec)

--order by子句不满足索引的最左前缀的要求，也可以用于查询排序，这是因为所以你的第一列被指定为一个常数

--该查询为索引的第一列提供了常量条件，而使用第二列进行排序，将两个列组合在一起，就形成了索引的最左前缀
mysql> explain select rental_id,staff_id from rental where rental_date='2005-05-25' order by inventory_id desc\G
*************************** 1. row ***************************
           id: 1
  select_type: SIMPLE
        table: rental
   partitions: NULL
         type: ref
possible_keys: rental_date
          key: rental_date
      key_len: 5
          ref: const
         rows: 1
     filtered: 100.00
        Extra: Backward index scan
1 row in set, 1 warning (0.00 sec)

--下面的查询不会利用索引
mysql> explain select rental_id,staff_id from rental where rental_date>'2005-05-25' order by rental_date,inventory_id\G
*************************** 1. row ***************************
           id: 1
  select_type: SIMPLE
        table: rental
   partitions: NULL
         type: ALL
possible_keys: rental_date
          key: NULL
      key_len: NULL
          ref: NULL
         rows: 16005
     filtered: 50.00
        Extra: Using where; Using filesort
1 row in set, 1 warning (0.01 sec)

--该查询使用了两中不同的排序方向，但是索引列都是正序排序的
mysql> explain select rental_id,staff_id from rental where rental_date>'2005-05-25' order by inventory_id desc,customer_id asc\G
*************************** 1. row ***************************
           id: 1
  select_type: SIMPLE
        table: rental
   partitions: NULL
         type: ALL
possible_keys: rental_date
          key: NULL
      key_len: NULL
          ref: NULL
         rows: 16005
     filtered: 50.00
        Extra: Using where; Using filesort
1 row in set, 1 warning (0.00 sec)
--该查询中引用了一个不在索引中的列
mysql> explain select rental_id,staff_id from rental where rental_date>'2005-05-25' order by inventory_id,staff_id\G
*************************** 1. row ***************************
           id: 1
  select_type: SIMPLE
        table: rental
   partitions: NULL
         type: ALL
possible_keys: rental_date
          key: NULL
      key_len: NULL
          ref: NULL
         rows: 16005
     filtered: 50.00
        Extra: Using where; Using filesort
1 row in set, 1 warning (0.00 sec)

6.5、union all,in,or都能够使用索引，但是推荐使用in

explain select * from actor where actor_id = 1 union all select * from actor where actor_id = 2;
mysql> explain select * from actor where actor_id = 1 union all select * from actor where actor_id = 2;
+----+-------------+-------+------------+-------+---------------+---------+---------+-------+------+----------+-------+
| id | select_type | table | partitions | type  | possible_keys | key     | key_len | ref   | rows | filtered | Extra |
+----+-------------+-------+------------+-------+---------------+---------+---------+-------+------+----------+-------+
|  1 | PRIMARY     | actor | NULL       | const | PRIMARY       | PRIMARY | 2       | const |    1 |   100.00 | NULL  |
|  2 | UNION       | actor | NULL       | const | PRIMARY       | PRIMARY | 2       | const |    1 |   100.00 | NULL  |
+----+-------------+-------+------------+-------+---------------+---------+---------+-------+------+----------+-------+
2 rows in set, 1 warning (0.00 sec)

explain select * from actor where actor_id in (1,2);
mysql> explain select * from actor where actor_id in (1,2);
+----+-------------+-------+------------+-------+---------------+---------+---------+------+------+----------+-------------+
| id | select_type | table | partitions | type  | possible_keys | key     | key_len | ref  | rows | filtered | Extra       |
+----+-------------+-------+------------+-------+---------------+---------+---------+------+------+----------+-------------+
|  1 | SIMPLE      | actor | NULL       | range | PRIMARY       | PRIMARY | 2       | NULL |    2 |   100.00 | Using where |
+----+-------------+-------+------------+-------+---------------+---------+---------+------+------+----------+-------------+
1 row in set, 1 warning (0.00 sec)

explain select * from actor where actor_id = 1 or actor_id =2;

mysql> explain select * from actor where actor_id = 1 or actor_id =2;
+----+-------------+-------+------------+-------+---------------+---------+---------+------+------+----------+-------------+
| id | select_type | table | partitions | type  | possible_keys | key     | key_len | ref  | rows | filtered | Extra       |
+----+-------------+-------+------------+-------+---------------+---------+---------+------+------+----------+-------------+
|  1 | SIMPLE      | actor | NULL       | range | PRIMARY       | PRIMARY | 2       | NULL |    2 |   100.00 | Using where |
+----+-------------+-------+------------+-------+---------------+---------+---------+------+------+----------+-------------+
1 row in set, 1 warning (0.00 sec)

6.6、范围列可以用到索引
范围条件是：<、>
范围列可以用到索引，但是范围列后面的列无法用到索引，索引最多用于一个范围列

6.7、强制类型转换会全表扫描

create table user(id int,name varchar(10),phone varchar(11));

alter table user add index idx_1(phone);

--不会触发索引
explain select * from user where phone=13800001234;
mysql> explain select * from user where phone=13800001234;
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-------------+
| id | select_type | table | partitions | type | possible_keys | key  | key_len | ref  | rows | filtered | Extra       |
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-------------+
|  1 | SIMPLE      | user  | NULL       | ALL  | idx_1         | NULL | NULL    | NULL |    1 |   100.00 | Using where |
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-------------+
1 row in set, 3 warnings (0.01 sec)

--触发索引
explain select * from user where phone='13800001234';
mysql> explain select * from user where phone='13800001234';
+----+-------------+-------+------------+------+---------------+-------+---------+-------+------+----------+-------+
| id | select_type | table | partitions | type | possible_keys | key   | key_len | ref   | rows | filtered | Extra |
+----+-------------+-------+------------+------+---------------+-------+---------+-------+------+----------+-------+
|  1 | SIMPLE      | user  | NULL       | ref  | idx_1         | idx_1 | 47      | const |    1 |   100.00 | NULL  |
+----+-------------+-------+------------+------+---------------+-------+---------+-------+------+----------+-------+
1 row in set, 1 warning (0.01 sec)

6.8、更新十分频繁，数据区分度不高的字段上不宜建立索引
1.更新会变更B+树，更新频繁的字段建议索引会大大降低数据库性能
2.类似于性别这类区分不大的属性，建立索引是没有意义的，不能有效的过滤数据，
3.一般区分度在80%以上的时候就可以建立索引，区分度可以使用 count(distinct(列名))/count(*) 来计算
6.9、创建索引的列，不允许为null，可能会得到不符合预期的结果
6.10、当需要进行表连接的时候，最好不要超过三张表，因为需要join的字段，数据类型必须一致
能使用limit的时候尽量使用limit

Nested-Loop Join Algorithms:嵌套循环联接算法

Index Nested-Loop Join：索引嵌套循环联接

Block Nested-Loop Joins:块嵌套循环联接 join buffer：把驱动表的列数据放到缓存join buffer中包括select的列，而不仅仅只缓存关联列，然后批量与匹配表进行匹配，将第一种多次比较合并为一次，降低了非驱动表的访问频率。在一个有N个JOIN关联的SQL当中会在执行的时候分配N-1个join buffer。

--查看join_buffer大小256K
mysql> show variables like '%join_buffer%';
+------------------+--------+
| Variable_name    | Value  |
+------------------+--------+
| join_buffer_size | 262144 |
+------------------+--------+
1 row in set, 1 warning (0.04 sec)

6.11、单表索引建议控制在5个以内
6.12、单索引字段数不允许超过5个（组合索引）
6.13、创建索引的时候应该避免以下错误概念
1.索引越多越好
2.过早优化，在不了解系统的情况下进行优化

7、索引监控

mysql> show status like 'Handler_read%';
+-----------------------+-------+
| Variable_name         | Value |
+-----------------------+-------+
| Handler_read_first    | 4     |
| Handler_read_key      | 9239  |
| Handler_read_last     | 0     |
| Handler_read_next     | 288   |
| Handler_read_prev     | 0     |
| Handler_read_rnd      | 30    |
| Handler_read_rnd_next | 13636 |
+-----------------------+-------+
7 rows in set (0.01 sec)

参数解释
	Handler_read_first：读取索引第一个条目的次数
	Handler_read_key：通过index获取数据的次数：越大越好，说明使用索引
	Handler_read_last：读取索引最后一个条目的次数
	Handler_read_next：通过索引读取下一条数据的次数
	Handler_read_prev：通过索引读取上一条数据的次数
	Handler_read_rnd：从固定位置读取数据的次数
	Handler_read_rnd_next：从数据节点读取下一条数据的次数：越大越好，说明使用索引

SET FOREIGN_KEY_CHECKS=0;
DROP TABLE IF EXISTS `itdragon_order_list`;
CREATE TABLE `itdragon_order_list` (
  `id` bigint(11) NOT NULL AUTO_INCREMENT COMMENT '主键id，默认自增长',
  `transaction_id` varchar(150) DEFAULT NULL COMMENT '交易号',
  `gross` double DEFAULT NULL COMMENT '毛收入(RMB)',
  `net` double DEFAULT NULL COMMENT '净收入(RMB)',
  `stock_id` int(11) DEFAULT NULL COMMENT '发货仓库',
  `order_status` int(11) DEFAULT NULL COMMENT '订单状态',
  `descript` varchar(255) DEFAULT NULL COMMENT '客服备注',
  `finance_descript` varchar(255) DEFAULT NULL COMMENT '财务备注',
  `create_type` varchar(100) DEFAULT NULL COMMENT '创建类型',
  `order_level` int(11) DEFAULT NULL COMMENT '订单级别',
  `input_user` varchar(20) DEFAULT NULL COMMENT '录入人',
  `input_date` varchar(20) DEFAULT NULL COMMENT '录入时间',
  PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=10003 DEFAULT CHARSET=utf8;

INSERT INTO itdragon_order_list VALUES ('10000', '81X97310V32236260E', '6.6', '6.13', '1', '10', 'ok', 'ok', 'auto', '1', 'itdragon', '2017-08-28 17:01:49');
INSERT INTO itdragon_order_list VALUES ('10001', '61525478BB371361Q', '18.88', '18.79', '1', '10', 'ok', 'ok', 'auto', '1', 'itdragon', '2017-08-18 17:01:50');
INSERT INTO itdragon_order_list VALUES ('10002', '5RT64180WE555861V', '20.18', '20.17', '1', '10', 'ok', 'ok', 'auto', '1', 'itdragon', '2017-09-08 17:01:49');

优化一：

select * from itdragon_order_list where transaction_id = "81X97310V32236260E";
--通过查看执行计划发现type=all,需要进行全表扫描
explain select * from itdragon_order_list where transaction_id = "81X97310V32236260E";
mysql> explain select * from itdragon_order_list where transaction_id = "81X97310V32236260E";
+----+-------------+---------------------+------------+------+---------------+------+---------+------+------+----------+-------------+
| id | select_type | table               | partitions | type | possible_keys | key  | key_len | ref  | rows | filtered | Extra       |
+----+-------------+---------------------+------------+------+---------------+------+---------+------+------+----------+-------------+
|  1 | SIMPLE      | itdragon_order_list | NULL       | ALL  | NULL          | NULL | NULL    | NULL |    3 |    33.33 | Using where |
+----+-------------+---------------------+------------+------+---------------+------+---------+------+------+----------+-------------+
1 row in set, 1 warning (0.00 sec)

--优化一、为transaction_id创建唯一索引
 create unique index idx_order_transaID on itdragon_order_list (transaction_id);
--当创建索引之后，唯一索引对应的type是const，通过索引一次就可以找到结果，普通索引对应的type是ref，表示非唯一性索引赛秒，找到值还要进行扫描，直到将索引文件扫描完为止，显而易见，const的性能要高于ref
 explain select * from itdragon_order_list where transaction_id = "81X97310V32236260E";
mysql> explain select * from itdragon_order_list where transaction_id = "81X97310V32236260E";
+----+-------------+---------------------+------------+-------+--------------------+--------------------+---------+-------+------+----------+-------+
| id | select_type | table               | partitions | type  | possible_keys      | key                | key_len | ref   | rows | filtered | Extra |
+----+-------------+---------------------+------------+-------+--------------------+--------------------+---------+-------+------+----------+-------+
|  1 | SIMPLE      | itdragon_order_list | NULL       | const | idx_order_transaID | idx_order_transaID | 453     | const |    1 |   100.00 | NULL  |
+----+-------------+---------------------+------------+-------+--------------------+--------------------+---------+-------+------+----------+-------+
1 row in set, 1 warning (0.00 sec)
 --优化二、使用覆盖索引，查询的结果变成 transaction_id,当extra出现using index,表示使用了覆盖索引
 explain select transaction_id from itdragon_order_list where transaction_id = "81X97310V32236260E";
mysql>  explain select transaction_id from itdragon_order_list where transaction_id = "81X97310V32236260E";
+----+-------------+---------------------+------------+-------+--------------------+--------------------+---------+-------+------+----------+-------------+
| id | select_type | table               | partitions | type  | possible_keys      | key                | key_len | ref   | rows | filtered | Extra       |
+----+-------------+---------------------+------------+-------+--------------------+--------------------+---------+-------+------+----------+-------------+
|  1 | SIMPLE      | itdragon_order_list | NULL       | const | idx_order_transaID | idx_order_transaID | 453     | const |    1 |   100.00 | Using index |
+----+-------------+---------------------+------------+-------+--------------------+--------------------+---------+-------+------+----------+-------------+
1 row in set, 1 warning (0.00 sec)

优化二：

--创建复合索引
create index idx_order_levelDate on itdragon_order_list (order_level,input_date);

--创建索引之后发现跟没有创建索引一样，都是全表扫描，都是文件排序
explain select * from itdragon_order_list order by order_level,input_date;
mysql> explain select * from itdragon_order_list order by order_level,input_date;
+----+-------------+---------------------+------------+------+---------------+------+---------+------+------+----------+----------------+
| id | select_type | table               | partitions | type | possible_keys | key  | key_len | ref  | rows | filtered | Extra          |
+----+-------------+---------------------+------------+------+---------------+------+---------+------+------+----------+----------------+
|  1 | SIMPLE      | itdragon_order_list | NULL       | ALL  | NULL          | NULL | NULL    | NULL |    3 |   100.00 | Using filesort |
+----+-------------+---------------------+------------+------+---------------+------+---------+------+------+----------+----------------+
1 row in set, 1 warning (0.00 sec)

--可以使用force index强制指定索引
explain select * from itdragon_order_list force index(idx_order_levelDate) order by order_level,input_date;
mysql> explain select * from itdragon_order_list force index(idx_order_levelDate) order by order_level,input_date;
+----+-------------+---------------------+------------+-------+---------------+---------------------+---------+------+------+----------+-------+
| id | select_type | table               | partitions | type  | possible_keys | key                 | key_len | ref  | rows | filtered | Extra |
+----+-------------+---------------------+------------+-------+---------------+---------------------+---------+------+------+----------+-------+
|  1 | SIMPLE      | itdragon_order_list | NULL       | index | NULL          | idx_order_levelDate | 68      | NULL |    3 |   100.00 | NULL  |
+----+-------------+---------------------+------------+-------+---------------+---------------------+---------+------+------+----------+-------+
1 row in set, 1 warning (0.00 sec)

--其实给订单排序意义不大，给订单级别添加索引意义也不大，因此可以先确定order_level的值，然后再给input_date排序
explain select * from itdragon_order_list where order_level=3 order by input_date;
mysql> explain select * from itdragon_order_list where order_level=3 order by input_date;
+----+-------------+---------------------+------------+------+---------------------+---------------------+---------+-------+------+----------+-------+
| id | select_type | table               | partitions | type | possible_keys       | key                 | key_len | ref   | rows | filtered | Extra |
+----+-------------+---------------------+------------+------+---------------------+---------------------+---------+-------+------+----------+-------+
|  1 | SIMPLE      | itdragon_order_list | NULL       | ref  | idx_order_levelDate | idx_order_levelDate | 5       | const |    1 |   100.00 | NULL  |
+----+-------------+---------------------+------------+------+---------------------+---------------------+---------+-------+------+----------+-------+
1 row in set, 1 warning (0.00 sec)

MySQL Optimization Learning(一)

MySQL Optimization Learning(二)

MySQL Optimization Learning(四)

不断学习才能不断提高！
生如蝼蚁，当立鸿鹄之志，命比纸薄，应有不屈之心。
乾坤未定，你我皆是黑马，若乾坤已定，谁敢说我不能逆转乾坤？
努力吧，机会永远是留给那些有准备的人，否则，机会来了，没有实力，只能眼睁睁地看着机会溜走。

杀神lwz

关注

0
点赞
踩
0

收藏

觉得还不错? 一键收藏
打赏
0
评论
MySQL Optimization Learning(三)

1、如果一个索引包含所有需要查询的字段的值，我们称之为覆盖索引2、不是所有类型的索引都可以称为覆盖索引，覆盖索引必须要存储索引列的值3、不同的存储实现覆盖索引的方式不同，不是所有的引擎都支持覆盖索引，memory不支持覆盖索引。
复制链接

扫一扫