SQL提取大表的部分数据所在周/月的全部数据并根据周/月汇总,所遇到的性能问题解决

从一个大表(TEST_TAB,大于1亿行)中根据特定条件筛选出一部分数据,在筛选这部分数据所在的月/周的数据,再根据月/周进行汇总,筛选数据的字段和日期字段(天)均有索引。

系统中有日期定义表CALENDAR如下:

DATE_TIMEKEYWEEK_TIMEKEYMONTH_TIMEKEYYEAR_TIMEKEY
202310112023W412023M102023
202310122023W412023M102023
202310132023W412023M102023
202310142023W412023M102023
202310152023W422023M102023
202310162023W422023M102023
202310172023W422023M102023
202310182023W422023M102023
202310192023W422023M102023

SQL如下:

		SELECT 
			C.MONTH_TIMEKEY,
			A.COL1,
			A.COL2,
			A.COL3,
			SUM(A.QTY) QTY
			FROM TEST_TAB A
		JOIN (
			SELECT DISTINCT B.COL1,B.COL2,S2.DATE_TIMEKEY,S2.MONTH_TIMEKEY
				FROM TEST_TAB B
			JOIN CALENDAR S1 
				ON B.DATE_TIMEKEY = S1.DATE_TIMEKEY
			JOIN CALENDAR S2 
				ON S1.MONTH_TIMEKEY = S2.MONTH_TIMEKEY
			WHERE B.COL1 = 'AAA' AND B.COL2 = 'BBB'
			) C ON A.DATE_TIMEKEY = C.DATE_TIMEKEY 
			AND A.COL1 = C.COL1
			AND A.COL2 = C.COL2
		GROUP BY 
			C.MONTH_TIMEKEY,
			A.COL1,
			A.COL2,
			A.COL3

但这个SQL实际执行很慢,需要10min左右,查看了执行计划,A和B均走了相应索引。

测试1:如果不进行按月汇总,SQL只需1s就执行出来了

		SELECT 
			C.MONTH_TIMEKEY,
			A.COL1,
			A.COL2,
			A.COL3,
			A.QTY
			FROM TEST_TAB A
		JOIN (
			SELECT DISTINCT B.COL1,B.COL2,S2.DATE_TIMEKEY,S2.MONTH_TIMEKEY
				FROM TEST_TAB B
			JOIN CALENDAR S1 
				ON B.DATE_TIMEKEY = S1.DATE_TIMEKEY
			JOIN CALENDAR S2 
				ON S1.MONTH_TIMEKEY = S2.MONTH_TIMEKEY
			WHERE B.COL1 = 'AAA' AND B.COL2 = 'BBB'
			) C ON A.DATE_TIMEKEY = C.DATE_TIMEKEY 
			AND A.COL1 = C.COL1
			AND A.COL2 = C.COL2

测试2:MONTH_TIMEKEY不取C的字段,SQL同样1s执行出来

		SELECT 
			SUBSTR(A.DATE_TIMEKEY,1,4)||'M'||SUBSTR(A.DATE_TIMEKEY,5,2) MONTH_TIMEKEY,
			A.COL1,
			A.COL2,
			A.COL3,
			SUM(A.QTY) QTY
			FROM TEST_TAB A
		JOIN (
			SELECT DISTINCT B.COL1,B.COL2,S2.DATE_TIMEKEY,S2.MONTH_TIMEKEY
				FROM TEST_TAB B
			JOIN CALENDAR S1 
				ON B.DATE_TIMEKEY = S1.DATE_TIMEKEY
			JOIN CALENDAR S2 
				ON S1.MONTH_TIMEKEY = S2.MONTH_TIMEKEY
			WHERE B.COL1 = 'AAA' AND B.COL2 = 'BBB'
			) C ON A.DATE_TIMEKEY = C.DATE_TIMEKEY 
			AND A.COL1 = C.COL1
			AND A.COL2 = C.COL2
		GROUP BY 
			SUBSTR(A.DATE_TIMEKEY,1,4)||'M'||SUBSTR(A.DATE_TIMEKEY,5,2),
			A.COL1,
			A.COL2,
			A.COL3

这种方式可以解决按月汇总的问题,但由于周无法直接从日期字段中截取,因此无法解决按周汇总的问题。

测试3:参考测试1结果,使用临时表(使用materialize hint)固化后再按周汇总

WITH D AS (
		SELECT 
			C.WEEK_TIMEKEY,
			A.COL1,
			A.COL2,
			A.COL3,
			A.QTY
			FROM TEST_TAB A
		JOIN (
			SELECT DISTINCT B.COL1,B.COL2,S2.DATE_TIMEKEY,S2.WEEK_TIMEKEY
				FROM TEST_TAB B
			JOIN CALENDAR S1 
				ON B.DATE_TIMEKEY = S1.DATE_TIMEKEY
			JOIN CALENDAR S2 
				ON S1.WEEK_TIMEKEY = S2.WEEK_TIMEKEY
			WHERE B.COL1 = 'AAA' AND B.COL2 = 'BBB'
			) C ON A.DATE_TIMEKEY = C.DATE_TIMEKEY 
			AND A.COL1 = C.COL1
			AND A.COL2 = C.COL2
)
	SELECT /*+materialize*/
		D.WEEK_TIMEKEY,
		D.COL1,
		D.COL2,
		D.COL3,
		SUM(D.QTY) QTY
		FROM TEST_TAB D
	GROUP BY 
		D.WEEK_TIMEKEY,
		D.COL1,
		D.COL2,
		D.COL3

测试结果也是可以1s执行出来

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值