类似group by的分组计数功能

最新推荐文章于 2024-07-10 16:11:12 发布
最新推荐文章于 2024-07-10 16:11:12 发布
阅读量2.9k 收藏
点赞数 1
分类专栏： sql优化 文章标签： oracle sql 优化 执行计划 全表扫描
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/gdmzlhj1/article/details/41204147
版权 

之前同事发过一个语句，实现的功能比较简单，类似group by的分组计数功能，因为where条件有like，又无法用group by来实现。
SELECT a.N0,b.N1,c.N2,d.N3,e.N4,f.N5,g.N6,h.N7,i.N8,j.N9 from 
 (select count(*) N0 from tbl_loginfo_20141110 where keyrecord like '0%' or keyrecord like 'GJ_0%') a,
 (select count(*) N1 from tbl_loginfo_20141110 where keyrecord like '1%' or keyrecord like 'GJ_1%') b,
 (select count(*) N2 from tbl_loginfo_20141110 where keyrecord like '2%' or keyrecord like 'GJ_2%') c,
 (select count(*) N3 from tbl_loginfo_20141110 where keyrecord like '3%' or keyrecord like 'GJ_3%') d,
 (select count(*) N4 from tbl_loginfo_20141110 where keyrecord like '4%' or keyrecord like 'GJ_4%') e,
 (select count(*) N5 from tbl_loginfo_20141110 where keyrecord like '5%' or keyrecord like 'GJ_5%') f,
 (select count(*) N6 from tbl_loginfo_20141110 where keyrecord like '6%' or keyrecord like 'GJ_6%') g,
 (select count(*) N7 from tbl_loginfo_20141110 where keyrecord like '7%' or keyrecord like 'GJ_7%') h,
 (select count(*) N8 from tbl_loginfo_20141110 where keyrecord like '8%' or keyrecord like 'GJ_8%') i,
 (select count(*) N9 from tbl_loginfo_20141110 where keyrecord like '9%' or keyrecord like 'GJ_9%') j;
 
为了了解语句的性能，我做了如下类似的测试： 
select * from v$version;   --Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - Production
drop table a;
create table a as select * from dba_objects where rownum<=50000;
begin
  for x in 1..6 loop
    insert into a select * from a;
  end loop;
  commit;
end;

select count(*) from a;   --3200000
select bytes/1024/1024 from user_segments where segment_name='A';  --357M

alter system flush shared_pool;
alter system flush buffer_cache;

SELECT a.N0,b.N1,c.N2,d.N3,e.N4,f.N5,g.N6,h.N7,i.N8,j.N9 from 
 (select count(*) N0 from a where object_name like 'A%' or object_name like 'V%') a,
 (select count(*) N1 from a where object_name like 'B%' or object_name like 'V%') b,
 (select count(*) N2 from a where object_name like 'C%' or object_name like 'V%') c,
 (select count(*) N3 from a where object_name like 'D%' or object_name like 'V%') d,
 (select count(*) N4 from a where object_name like 'E%' or object_name like 'V%') e,
 (select count(*) N5 from a where object_name like 'F%' or object_name like 'V%') f,
 (select count(*) N6 from a where object_name like 'G%' or object_name like 'V%') g,
 (select count(*) N7 from a where object_name like 'H%' or object_name like 'V%') h,
 (select count(*) N8 from a where object_name like 'I%' or object_name like 'V%') i,
 (select count(*) N9 from a where object_name like 'J%' or object_name like 'V%') j;
--58s
alter system flush shared_pool;
alter system flush buffer_cache;

--改写后
select 
sum(case when object_name like 'A%' or object_name like 'V%' then 1 else 0 end) N0,
sum(case when object_name like 'B%' or object_name like 'V%' then 1 else 0 end) N1, 
sum(case when object_name like 'C%' or object_name like 'V%' then 1 else 0 end) N2, 
sum(case when object_name like 'D%' or object_name like 'V%' then 1 else 0 end) N3, 
sum(case when object_name like 'E%' or object_name like 'V%' then 1 else 0 end) N4, 
sum(case when object_name like 'F%' or object_name like 'V%' then 1 else 0 end) N5, 
sum(case when object_name like 'G%' or object_name like 'V%' then 1 else 0 end) N6, 
sum(case when object_name like 'H%' or object_name like 'V%' then 1 else 0 end) N7, 
sum(case when object_name like 'I%' or object_name like 'V%' then 1 else 0 end) N8, 
sum(case when object_name like 'J%' or object_name like 'V%' then 1 else 0 end) N9
from a; 

--19s





--对比执行计划：
--前者执行计划：
SQL> explain plan for
  2  SELECT a.N0,b.N1,c.N2,d.N3,e.N4,f.N5,g.N6,h.N7,i.N8,j.N9 from
  3   (select count(*) N0 from a where object_name like 'A%' or object_name like 'V%') a,
  4   (select count(*) N1 from a where object_name like 'B%' or object_name like 'V%') b,
  5   (select count(*) N2 from a where object_name like 'C%' or object_name like 'V%') c,
  6   (select count(*) N3 from a where object_name like 'D%' or object_name like 'V%') d,
  7   (select count(*) N4 from a where object_name like 'E%' or object_name like 'V%') e,
  8   (select count(*) N5 from a where object_name like 'F%' or object_name like 'V%') f,
  9   (select count(*) N6 from a where object_name like 'G%' or object_name like 'V%') g,
 10   (select count(*) N7 from a where object_name like 'H%' or object_name like 'V%') h,
 11   (select count(*) N8 from a where object_name like 'I%' or object_name like 'V%') i,
 12   (select count(*) N9 from a where object_name like 'J%' or object_name like 'V%') j;

Explained.

Elapsed: 00:00:00.15
SQL> @getplan
'general,outline,starts'

Enter value for plan type:

PLAN_TABLE_OUTPUT
-----------------------------------------------------------------------------------------
Plan hash value: 2527411742

-------------------------------------------------------------------------------------
| Id  | Operation                    | Name | Rows  | Bytes | Cost (%CPU)| Time     |
-------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT             |      |     1 |   130 |   123K  (1)| 00:24:46 |
|   1 |  NESTED LOOPS                |      |     1 |   130 |   123K  (1)| 00:24:46 |
|   2 |   NESTED LOOPS               |      |     1 |   117 |   111K  (1)| 00:22:17 |
|   3 |    NESTED LOOPS              |      |     1 |   104 | 99032   (1)| 00:19:49 |
|   4 |     NESTED LOOPS             |      |     1 |    91 | 86653   (1)| 00:17:20 |
|   5 |      NESTED LOOPS            |      |     1 |    78 | 74274   (1)| 00:14:52 |
|   6 |       NESTED LOOPS           |      |     1 |    65 | 61895   (1)| 00:12:23 |
|   7 |        NESTED LOOPS          |      |     1 |    52 | 49516   (1)| 00:09:55 |
|   8 |         NESTED LOOPS         |      |     1 |    39 | 37137   (1)| 00:07:26 |
|   9 |          NESTED LOOPS        |      |     1 |    26 | 24758   (1)| 00:04:58 |
|  10 |           VIEW               |      |     1 |    13 | 12379   (1)| 00:02:29 |
|  11 |            SORT AGGREGATE    |      |     1 |    66 |            |          |
|* 12 |             TABLE ACCESS FULL| A    | 91587 |  5903K| 12379   (1)| 00:02:29 |
|  13 |           VIEW               |      |     1 |    13 | 12379   (1)| 00:02:29 |
|  14 |            SORT AGGREGATE    |      |     1 |    66 |            |          |
|* 15 |             TABLE ACCESS FULL| A    |   137K|  8831K| 12379   (1)| 00:02:29 |
|  16 |          VIEW                |      |     1 |    13 | 12379   (1)| 00:02:29 |
|  17 |           SORT AGGREGATE     |      |     1 |    66 |            |          |
|* 18 |            TABLE ACCESS FULL | A    | 85818 |  5531K| 12379   (1)| 00:02:29 |
|  19 |         VIEW                 |      |     1 |    13 | 12379   (1)| 00:02:29 |
|  20 |          SORT AGGREGATE      |      |     1 |    66 |            |          |
|* 21 |           TABLE ACCESS FULL  | A    |   111K|  7158K| 12379   (1)| 00:02:29 |
|  22 |        VIEW                  |      |     1 |    13 | 12379   (1)| 00:02:29 |
|  23 |         SORT AGGREGATE       |      |     1 |    66 |            |          |
|* 24 |          TABLE ACCESS FULL   | A    | 86539 |  5577K| 12379   (1)| 00:02:29 |
|  25 |       VIEW                   |      |     1 |    13 | 12379   (1)| 00:02:29 |
|  26 |        SORT AGGREGATE        |      |     1 |    66 |            |          |
|* 27 |         TABLE ACCESS FULL    | A    | 91587 |  5903K| 12379   (1)| 00:02:29 |
|  28 |      VIEW                    |      |     1 |    13 | 12379   (1)| 00:02:29 |
|  29 |       SORT AGGREGATE         |      |     1 |    66 |            |          |
|* 30 |        TABLE ACCESS FULL     | A    |   228K|    14M| 12379   (1)| 00:02:29 |
|  31 |     VIEW                     |      |     1 |    13 | 12379   (1)| 00:02:29 |
|  32 |      SORT AGGREGATE          |      |     1 |    66 |            |          |
|* 33 |       TABLE ACCESS FULL      | A    | 87981 |  5670K| 12379   (1)| 00:02:29 |
|  34 |    VIEW                      |      |     1 |    13 | 12379   (1)| 00:02:29 |
|  35 |     SORT AGGREGATE           |      |     1 |    66 |            |          |
|* 36 |      TABLE ACCESS FULL       | A    | 84376 |  5438K| 12379   (1)| 00:02:29 |
|  37 |   VIEW                       |      |     1 |    13 | 12379   (1)| 00:02:29 |
|  38 |    SORT AGGREGATE            |      |     1 |    66 |            |          |
|* 39 |     TABLE ACCESS FULL        | A    |   112K|  7251K| 12379   (1)| 00:02:29 |
-------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

  12 - filter("OBJECT_NAME" LIKE 'J%' OR "OBJECT_NAME" LIKE 'V%')
  15 - filter("OBJECT_NAME" LIKE 'I%' OR "OBJECT_NAME" LIKE 'V%')
  18 - filter("OBJECT_NAME" LIKE 'H%' OR "OBJECT_NAME" LIKE 'V%')
  21 - filter("OBJECT_NAME" LIKE 'G%' OR "OBJECT_NAME" LIKE 'V%')
  24 - filter("OBJECT_NAME" LIKE 'F%' OR "OBJECT_NAME" LIKE 'V%')
  27 - filter("OBJECT_NAME" LIKE 'E%' OR "OBJECT_NAME" LIKE 'V%')
  30 - filter("OBJECT_NAME" LIKE 'D%' OR "OBJECT_NAME" LIKE 'V%')
  33 - filter("OBJECT_NAME" LIKE 'C%' OR "OBJECT_NAME" LIKE 'V%')
  36 - filter("OBJECT_NAME" LIKE 'B%' OR "OBJECT_NAME" LIKE 'V%')
  39 - filter("OBJECT_NAME" LIKE 'A%' OR "OBJECT_NAME" LIKE 'V%')
  
  
  
  
  
--后者执行计划：  
SQL> explain plan for
  2  select
  3  sum(case when object_name like 'A%' or object_name like 'V%' then 1 else 0 end) N0,
  4  sum(case when object_name like 'B%' or object_name like 'V%' then 1 else 0 end) N1,
  5  sum(case when object_name like 'C%' or object_name like 'V%' then 1 else 0 end) N2,
  6  sum(case when object_name like 'D%' or object_name like 'V%' then 1 else 0 end) N3,
  7  sum(case when object_name like 'E%' or object_name like 'V%' then 1 else 0 end) N4,
  8  sum(case when object_name like 'F%' or object_name like 'V%' then 1 else 0 end) N5,
  9  sum(case when object_name like 'G%' or object_name like 'V%' then 1 else 0 end) N6,
 10  sum(case when object_name like 'H%' or object_name like 'V%' then 1 else 0 end) N7,
 11  sum(case when object_name like 'I%' or object_name like 'V%' then 1 else 0 end) N8,
 12  sum(case when object_name like 'J%' or object_name like 'V%' then 1 else 0 end) N9
 13  from a;

Explained.

Elapsed: 00:00:00.01
SQL> @getplan
'general,outline,starts'

Enter value for plan type:

PLAN_TABLE_OUTPUT
--------------------------------------------------------------------------------------------
Plan hash value: 3918351354

---------------------------------------------------------------------------
| Id  | Operation          | Name | Rows  | Bytes | Cost (%CPU)| Time     |
---------------------------------------------------------------------------
|   0 | SELECT STATEMENT   |      |     1 |    66 | 12349   (1)| 00:02:29 |
|   1 |  SORT AGGREGATE    |      |     1 |    66 |            |          |
|   2 |   TABLE ACCESS FULL| A    |  3097K|   194M| 12349   (1)| 00:02:29 |
---------------------------------------------------------------------------

Note
-----
   - dynamic sampling used for this statement (level=2)  


可以看出，前者10次全表扫描，后者1次全表扫描。从而时间上也大大降低了。由58s降低到19s。
优化这个sql主要还是思路的转换，难点在于怎样把10次全表扫描转化成1次全表扫描。
在OLAP中，可以加并行使sql速度更快。

gdmzlhj1
关注
专栏目录
分组统计group by
A_JOKER___的博客
10-14 680
使用group by子句对列进行分组 SELECT column1,column2,column3… FROM table GROUP BY column 使用having子句对分组后的结果进行过滤 SELECT column1，column2，column3… FROM table GROUP BY column HAVING 过滤条件 GROUP BY 用于对查询的结果分组统计，HAVING 子句用于限制分组显示结果 USE db_02; #创建部门表 CREATE TABLE dept( dept.
sas分组计数_SAS读书笔记：SQL
weixin_39730911的博客
11-21 3083
1.SQL语言概述SAS术语与SQL术语的区别SQl语言在SAS中通过PROC SQL来实现。PROC SQL可实现以下功能：制作报表与表；生成一些统计性数据；合并表；从其它表中抽取部分行、列；更新表的行、列；从其它数据管理系统（DBMS）中更新或抽取数据。2.使用SQL检索数据 2.1 SQL的基本结构PROC SQL; SELECT 表1.列1 , 表1.列2 ,``` FR...
用count求group by分组的个数
睶
09-08 1557
示例 select count(*) from (select count(*) num,sid from person group by sid )
group by 分组函数详解
最新发布
qq_43723636的博客
07-10 2173
如何分组数据，以便能汇总表内容的子集。这涉及两个新SELECT语句子句，分别是GROUP BY子句和HAVING子句。
文本相似性算法实现（二）-分组及分句热度统计
软件老王
03-03 1364
1. 场景描述 软件老王在上一节介绍到相似性热度统计的4个需求（文本相似性热度统计(python版)），本次介绍分组及分组分句热度统计（需求1和需求2）。 2. 解决方案 分组热度统计首先根据某列进行分组，然后再对这些句进行热度统计，主要是分组处理，分句仅仅是按照标点符号做了下拆分，在代码说明中可以替换下就可以了。 2.1 完整代码 完整代码，有需要的朋友可以直接拿走，不想看代码介绍的，可以直接拿...
ES6集合操作-分组、去重、排序
tonysh_zds的博客
03-12 1265
注意：Chrome浏览器会将Lodash库作为开发工具的一部分加载到控制台中，从而使你能够直接在控制台中使用Lodash的方法(比如_.groupBy等)。2.3使用第三方库lodash的.keyBy()和.mapValues()方法。4.2使用第三方库lodash的_. orderBy()方法。3.4使用第三方库lodash的_.groupBy()方法。1.3使用第三方库lodash的_.uniq()方法。2.1使用Array.map()方法。3.3使用Array.map()方法。3、对象数组进行分组。
Laravel 实现Eloquent模型分组查询并返回每个分组的数量 groupBy()
10-16
本文将详细介绍如何使用Eloquent模型实现分组查询并返回每个分组的数量，主要涉及`groupBy()`方法。 首先，`groupBy()`方法是SQL查询中的一个关键组成部分，它允许你根据一个或多个字段对查询结果进行分组。在...
Lucene group by ，分组实现
03-16
"Lucene group by" 指的就是在 Lucene 中实现基于特定字段的分组操作，类似于 SQL 中的 GROUP BY 子句。这使得用户能够按类别聚合文档，例如，根据作者、日期或其他分类标准来查看搜索结果。 在 Lucene 中，分组...
pandas获取groupby分组里最大值所在的行方法
09-20
通过`groupby`对指定列进行分组后，可以应用一个函数，该函数接收每个分组作为一个单独的DataFrame。在这个函数内部，可以使用条件筛选来定位到计数列（Count）中值等于该组最大值的行。 第二种方法涉及到了pandas...
java groupbykey_Java实现GroupBy/分组TopN功能
weixin_30676635的博客
02-24 867
介绍在Java 8 的Lambda(stream)之前，要在Java代码中实现类似SQL中的group by分组聚合功能，还是比较困难的。这之前Java对函数式编程支持不是很好，Scala则把函数式编程发挥到了机制，实现一个group by聚合对Scala来说就是几行代码的事情：val birds = List("Golden Eagle","Gyrfalcon", "American Robin...
es6 js group by
weixin_42351984的博客
04-28 985
text(){ const inventory = [ { name: 'asparagus', type: 'vegetables', quantity: 5 }, { name: 'bananas', type: 'fruit', quantity: 0 }, { name: 'goat', type: 'meat', quantity: 23 }, { name: 'cherries', type: 'f...
sql中分组，除了group by之外的用法
PowerKinging的专栏
11-14 1692
表(kzw): id       xianlu_id       point_name       x                 y                       h                   biaod_id   1               19                   草桥     314594.654     494252.073        ...
使用Stream流对集合中根据多个元素进行去重
正儿八经的胡说八道的博客
05-19 2214
最近工作中需要在有重复数据的数据表中查出数据集合并且不能重复，但由于数据量太大就不好用group by分组，这里用一次Stream的去重操作 oldSegments.stream().collect(Collectors.collectingAndThen( Collectors.toCollection(() -> new TreeSet<>( Comparator.comparing(o -> o.getDptTime
js对数据分组类似group by
qq_25970751的博客
07-02 2万+
前端处理后端的返回来的json, 对数据进行处理类似group by，看到大牛的写法，分享给大家自己也做一下记录。list = [ {"name": "John", "Average": 15, "High": 10, "DtmStamp": 1358226000000}, {"name": "Jane", "Average": 16, &q
字符串按相似度分组
Eric's Blog
09-06 1617
相关代码已经上传GitHub ： https://github.com/EricLi404/go-utils 0x00 概述 在做在做反垃圾账号业务的时候，检测发现有几十名用户都在某个IP上有过行为打点。 # 原始数据 ꧁❥????➦SAMI????MMS➣????❥꧂ ꧁❥????➦Ⓜ️AHI????MMS➣????❥꧂.. jgcch7b????V????O????I???? 2kc7fd7????V????O????I???? ꧁❥????➦VIDYA????MMS➣????❥꧂ RcR...
R语言-分组统计
热门推荐
hongweigg的专栏
10-26 6万+
分组统计 1、假定有一组成绩数据，要求根据性别进行分组统计： > score     ID   score1 score2 Gender 1  101 11.35321    0.9   male 2  102 12.58654    0.6   male 3  103 19.79380    0.9 female 4  104 12.01352    0.4 female 5 
js分组（group by）
风火一回，一生不毁
10-26 3万+
对数据进行分组，类似sql的groupBy。 原理：遍历数组，将符合条件的数据放在一起，最后返回一个分组后的二维数组。 //分组 export const groupBy = (list, fn) =&amp;gt; { const groups = {}; list.forEach(function (o) { const group = JSON.stringify...
groupby分组查询
10-18
groupby是一种常用的数据分组方法，它可以将数据按照指定的列进行分组，并对每个分组进行聚合操作。在SQL中，groupby通常与聚合函数（如sum、count、avg等）一起使用，用于统计每个分组的聚合结果。在Python中，...
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值