exists是对外表作loop循环,每次loop循环再对内表进行查询,在项目中遇到类似的问题,数据量大时,not exists不管用并且执行SQL的时间很长,
在网上查了一些资料,所以记录下来:
原始SQL:
explain select relation.id,relation.invited_code,relation.uid,relation.create_time,relation.invited_uid from tran_user_invited_relation relation,
tran_user user where 1=1 and user.uid=relation.uid and user.register_time>='2017-01-01 19:40:09' and user.register_time<='2017-12-15 16:33:49' and
not EXISTS (select 1 from tran_user_reward reward where reward.sources='invite_auth' and reward.invited_relation_id=relation.id)
关于优化把以上SQL修改一下做几点思考:
对于原始语句,因为是NOT EXISTS判断,所以子查询中没有必要SELECT * FROM DUAL,只需SELECT ID甚至是SELECT 1都可以。
另外,鉴于MySQL子查询算法性能较差,考虑改写成对应的JOIN方式,因为这里是NOT EXIST判断,所以改写时有一定技巧,
即可使用左连接,然后过滤出未能连接的记录做一个条件筛选得到想要的数据。还有,原始语句最外层的查询也是SELECT 至于这个是否有必要应当依据具体业务,
尽量在满足业务要求的条件下取尽可能少的字段。最后,即时做了上述优化,若在连接条件上没有合适的索引SQL性能仍然会非常差(特别是在表的数据量巨大的时候),
所以考虑在连接的条件列reward.invited_relation_id和relation.id上分别创建索引。
优化后的SQL是:
SQL由原来的30S的运行时间减少到1-2左右秒内。
explain select invited.id,invited.invited_code,invited.uid,invited.create_time,invited.invited_uid from (
(select user.uid from tran_user user where 1=1 and user.rna_status=1 and user.register_time>='2017-01-01 19:40:09'
and user.register_time<='2017-12-15 16:33:49') as `user` left join
(select invited.id,invited.invited_code,invited.uid,invited.create_time,invited.invited_uid from
tran_user_invited_relation invited) as invited on user.uid=invited.uid left join
(select id as rewards_id ,invited_relation_id from tran_user_reward where 1=1 and sources='invite_auth' AND invited_relation_id !=0) reward ON
invited.id=reward.invited_relation_id ) where reward.rewards_id is null and invited.id is not null