Trafodion更新统计信息原理分析

在前面的文章Trafodion 更新统计信息数据收集 中我们介绍了如何查看更新信息统计的详细步骤，今天正好有空对更新统计信息日志做了一些基本的分析，也算是大致了解了更新统计信息的内部原理了吧，以下内容对更新统计信息的步骤做了一些简化，希望没有不小心剔除掉重要的点。

1. Get Detail Update Statistics Log

sqlci
cqd ustat_log ‘ustat_2020.log’; --the log file will be generated under cdl
update statistics log on;
update statistics for table <table_name> on every column sample;
exit

2. Understanding Update Statistics Log

2.1. Set CQDs
CONTROL QUERY DEFAULT QUERY_CACHE ‘0’
CONTROL QUERY DEFAULT CACHE_HISTOGRAMS ‘OFF’
CONTROL QUERY DEFAULT USTAT_MODIFY_DEFAULT_UEC ‘0.05’
CONTROL QUERY DEFAULT OUTPUT_DATE_FORMAT ‘ANSI’
CONTROL QUERY DEFAULT HIST_MISSING_STATS_WARNING_LEVEL ‘0’
CONTROL QUERY DEFAULT USTAT_AUTOMATION_INTERVAL ‘0’
CONTROL QUERY DEFAULT MV_ALLOW_SELECT_SYSTEM_ADDED_COLUMNS ‘ON’
CONTROL QUERY DEFAULT HIST_ON_DEMAND_STATS_SIZE ‘0’
CONTROL QUERY DEFAULT ISOLATION_LEVEL ‘READ COMMITTED’
CONTROL QUERY DEFAULT ALLOW_DML_ON_NONAUDITED_TABLE ‘ON’
CONTROL QUERY DEFAULT MV_ALLOW_SELECT_SYSTEM_ADDED_COLUMNS ‘ON’
CONTROL QUERY DEFAULT ALLOW_NULLABLE_UNIQUE_KEY_CONSTRAINT ‘ON’
CONTROL QUERY DEFAULT CAT_ERROR_ON_NOTNULL_STOREBY ‘OFF’
CONTROL QUERY DEFAULT WMS_CHILD_QUERY_MONITORING ‘OFF’
CONTROL QUERY DEFAULT WMS_QUERY_MONITORING ‘OFF’
CONTROL QUERY DEFAULT TRAF_TINYINT_RETURN_VALUES ‘ON’
CONTROL QUERY DEFAULT TRAF_BOOLEAN_IO ‘ON’
CONTROL QUERY DEFAULT TRAF_LARGEINT_UNSIGNED_IO ‘ON’
CONTROL QUERY DEFAULT TRAF_ALLOW_RESERVED_COLNAMES ‘ON’
CONTROL QUERY DEFAULT TRAF_BLOB_AS_VARCHAR ‘OFF’
CONTROL QUERY DEFAULT TRAF_CLOB_AS_VARCHAR ‘OFF’

2.2. Check If Object Exists

call HSHbaseTableDef::objExists
naTbl_->objectUid() is 3872136939770385698

2.3. Creating histogram tables on demand

2.4. Get Table RowCount

SELECT COUNT(*) FROM TRAFODION.DAAS_GX.E_OT_CASE FOR READ UNCOMMITTED ACCESS
Output:
Total #rows= 78650, sample rows=10000, IntervalCount = 50*

2.5. Memory Estimates

Column S_EXT_DATATIME requires 80000 bytes of memory for internal sort.
Column S_EXT_FROMNODE requires 340000 bytes of memory for internal sort.
…

2.6. Create/populate sample table

CREATE TABLE TRAFODION.DAAS_GX.TRAF_SAMPLE_03872136939770385698_1533649412_993975 LIKE TRAFODION.DAAS_GX.E_OT_CASE WITHOUT LOB COLUMNS WITH PARTITIONS LIMIT COLUMN LENGTH TO 256
UPSERT USING LOAD INTO TRAFODION.DAAS_GX.TRAF_SAMPLE_03872136939770385698_1533649412_993975 SELECT SUBSTRING(“CASEID” FOR 64) AS “CASEID”, “REGNO”, “UNISCID”, SUBSTRING(“ENTNAME” FOR 64) AS “ENTNAME”, SUBSTRING(“PENDECNO” FOR 64) AS “PENDECNO”, SUBSTRING(“ILLEGACTTYPE” FOR 64) AS “ILLEGACTTYPE”, SUBSTRING(“PENTYPE” FOR 64) AS “PENTYPE”, SUBSTRING(“PENTYPE_CN” FOR 64) AS “PENTYPE_CN”, “PENAM”, “FORFAM”, SUBSTRING(“PENCONTENT” FOR 64) AS “PENCONTENT”, SUBSTRING(“JUDAUTH” FOR 64) AS “JUDAUTH”, “PENDECISSDATE”, SUBSTRING(“REMARK” FOR 64) AS “REMARK”, “PUBLICDATE”, SUBSTRING(“DATADEPT” FOR 64) AS “DATADEPT”, “S_EXT_FROMNODE”, “S_EXT_DATATIME” FROM TRAFODION.DAAS_GX.E_OT_CASE <<+ cardinality 7.865000e+04 >> SAMPLE RANDOM 12.714571 PERCENT FOR READ UNCOMMITTED ACCESS

2.7. Read/sort data before creating STATISTICS

2.8. Create statistics for internal sort

Estimating UEC for column group [S_EXT_DATATIME]
Estimating UEC for column group [DATADEPT]

2.9. Create Single-column stats: fetchBoundaries()

SELECT FMTVAL, SUMVAL FROM (SELECT “FORFAM”, TRIM(TRAILING FROM CAST(“FORFAM” AS VARCHAR(250) CHARACTER SET UCS2)), COUNT(*) FROM TRAFODION.DAAS_GX.TRAF_SAMPLE_03872136939770385698_1533649412_993975 <<+ cardinality 1.000000e+04 >> GROUP BY “FORFAM” FOR READ UNCOMMITTED ACCESS) T(“FORFAM”, FMTVAL, SUMVAL) ORDER BY “FORFAM”

2.10. Drop Sample Table

DROP TABLE TRAFODION.DAAS_GX.TRAF_SAMPLE_03872136939770385698_1533649412_993975

2.11. Flush out stats

SELECT HISTOGRAM_ID, COL_POSITION, COLUMN_NUMBER, COLCOUNT, cast(READ_TIME as char(19) character set iso88591), REASON FROM TRAFODION.DAAS_GX.SB_HISTOGRAMS WHERE TABLE_UID = 3872136939770385698 ORDER BY TABLE_UID, HISTOGRAM_ID, COL_POSITION FOR READ COMMITTED ACCESS

2.12. Write out new histograms/histogram intervals

2.13. Delete old stats rows

DELETE FROM TRAFODION.DAAS_GX.SB_HISTOGRAMS WHERE TABLE_UID = 3872136939770385698 AND HISTOGRAM_ID IN ( 2041743984, 2041743989, 2041743994, 2041743999, 2041744004, 2041744009, 2041744014, 2041744019, 2041744024, 2041744029, 2041744034, 2041744039, 2041744044, 2041744049, 2041744054, 2041744059, 2041744064, 2041744069)
DELETE FROM TRAFODION.DAAS_GX.SB_HISTOGRAM_INTERVALS WHERE TABLE_UID = 3872136939770385698 AND HISTOGRAM_ID IN ( 2041743984, 2041743989, 2041743994, 2041743999, 2041744004, 2041744009, 2041744014, 2041744019, 2041744024, 2041744029, 2041744034, 2041744039, 2041744044, 2041744049, 2041744054, 2041744059, 2041744064, 2041744069)