系统版本:aix6014

数据库版本:oracle11.2.0.2.0

 

故障情况,早晨公司清分系统执行异常缓慢,取了一份awr,发现由于一条频繁执行的sql原因,导致应用执行缓慢。查看该条sql的sqlplan情况

SQL> explain plan for
  2  select /* +index(BMS_CHK_DTL, INDEX2_BMS_CHK_DTL) */
  3   ACQ_DB_FEE, ACQ_CR_FEE, RSV1, RSV2, UMS_MCC, ORI_TRANS_DT
  4    from YSSETTZGS.BMS_CHK_DTL
  5   where RRN = :REF_NO
  6     and RUNTIME_ID = :RUNTIME_ID
  7     and MID = :MER_NO
  8     and TID = :TERM_NO
  9     and MSG_TYPE = :MSG_TYPE
 10     and FILE_SEQ = '1000'
 11     and PROC_CODE = :PROC_CODE
 12     and SER_CONCODE = :SVR_COND_CODE
 13     and TRANS_DT = :TRANS_DT;

Explained.

SQL> select * from table(DBMS_XPLAN.display); \
  2  
SQL> select * from table(DBMS_XPLAN.display); 

PLAN_TABLE_OUTPUT
------------------------------------------------------------------------------------------------------------------------------------------------------
Plan hash value: 142170735

-------------------------------------------------------------------------------------------------------------------------
| Id  | Operation                          | Name               | Rows  | Bytes | Cost (%CPU)| Time     | Pstart| Pstop |
-------------------------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT                   |                    |     1 |   169 |     0   (0)| 00:00:01 |       |       |
|*  1 |  TABLE ACCESS BY GLOBAL INDEX ROWID| BMS_CHK_DTL        |     1 |   169 |     0   (0)| 00:00:01 | ROWID | ROWID |
|*  2 |   INDEX RANGE SCAN                 | INDEX1_BMS_CHK_DTL |     1 |       |     0   (0)| 00:00:01 |       |       |
-------------------------------------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   1 - filter("FILE_SEQ"=1000 AND "RRN"=:REF_NO AND "MID"=:MER_NO AND "TID"=:TERM_NO AND "MSG_TYPE"=:MSG_TYPE
              AND "PROC_CODE"=:PROC_CODE AND "SER_CONCODE"=:SVR_COND_CODE AND "TRANS_DT"=:TRANS_DT)
   2 - access("RUNTIME_ID"=TO_NUMBER(:RUNTIME_ID))

 

发现sql走得索引是INDEX1_BMS_CHK_DTL ,并非hint指定的索引INDEX2_BMS_CHK_DTL

这两条索引的字段差别:

SQL> col column_name for a30
SQL> select INDEX_NAME,TABLE_NAME,COLUMN_NAME,COLUMN_POSITION from dba_ind_columns where INDEX_NAME in (select b.INDEX_NAME from dba_indexes b
  2  where b.owner=upper('&username') and b.TABLE_NAME=upper('&tablename')) order by INDEX_NAME,COLUMN_POSITION;
Enter value for username: yssettzgs
Enter value for tablename: BMS_CHK_DTL
old   2: where b.owner=upper('&username') and b.TABLE_NAME=upper('&tablename')) order by INDEX_NAME,COLUMN_POSITION
new   2: where b.owner=upper('yssettzgs') and b.TABLE_NAME=upper('BMS_CHK_DTL')) order by INDEX_NAME,COLUMN_POSITION

INDEX_NAME                     TABLE_NAME                     COLUMN_NAME                    COLUMN_POSITION
------------------------------ ------------------------------ ------------------------------ ---------------
BMS_CHK_DTL_IDX3               BMS_CHK_DTL                    INST_ID                                      1
INDEX1_BMS_CHK_DTL             BMS_CHK_DTL                    RUNTIME_ID                                   1
INDEX1_BMS_CHK_DTL             BMS_CHK_DTL                    PAN                                          2
INDEX2_BMS_CHK_DTL             BMS_CHK_DTL                    RRN                                          1
INDEX2_BMS_CHK_DTL             BMS_CHK_DTL                    TID                                          2
INDEX2_BMS_CHK_DTL             BMS_CHK_DTL                    MID                                          3
INDEX_BMS_CHK_DTL              BMS_CHK_DTL                    RUNTIME_ID                                   1
INDEX_BMS_CHK_DTL              BMS_CHK_DTL                    SYSTRACE                                     1
INDEX_BMS_CHK_DTL              BMS_CHK_DTL                    TID                                          2
INDEX_BMS_CHK_DTL              BMS_CHK_DTL                    TRANS_DT                                     2
INDEX_BMS_CHK_DTL              BMS_CHK_DTL                    MID                                          3
INDEX_BMS_CHK_DTL              BMS_CHK_DTL                    ACQ_ID                                       3
INDEX_BMS_CHK_DTL              BMS_CHK_DTL                    FWD_ID                                       4
INDEX_BMS_CHK_DTL              BMS_CHK_DTL                    RRN                                          4
INDEX_BMS_CHK_DTL              BMS_CHK_DTL                    CHK_FLG                                      5

 

查看那索引统计分析情况:

SQL> select INDEX_NAME,STATUS,TABLE_NAME,NUM_ROWS,LAST_ANALYZED,GLOBAL_STATS,VISIBILITY from dba_indexes where owner=upper('&owner') and table_name=upper('&tablename');
Enter value for owner: yssettzgs
Enter value for tablename: BMS_CHK_DTL
old   1: select INDEX_NAME,STATUS,TABLE_NAME,NUM_ROWS,LAST_ANALYZED,GLOBAL_STATS,VISIBILITY from dba_indexes where owner=upper('&owner') and table_name=upper('&tablename')
new   1: select INDEX_NAME,STATUS,TABLE_NAME,NUM_ROWS,LAST_ANALYZED,GLOBAL_STATS,VISIBILITY from dba_indexes where owner=upper('yssettzgs') and table_name=upper('BMS_CHK_DTL')

INDEX_NAME                     STATUS   TABLE_NAME                       NUM_ROWS LAST_ANALYZED       GLO VISIBILIT
------------------------------ -------- ------------------------------ ---------- ------------------- --- ---------
INDEX2_BMS_CHK_DTL             VALID    BMS_CHK_DTL                             0 2013-08-29:22:03:03 YES VISIBLE
INDEX1_BMS_CHK_DTL             VALID    BMS_CHK_DTL                             0 2013-08-29:22:03:03 YES VISIBLE
INDEX_BMS_CHK_DTL              VALID    BMS_CHK_DTL                             0 2013-08-29:22:03:03 YES VISIBLE
BMS_CHK_DTL_IDX3               N/A      BMS_CHK_DTL                             0 2013-08-29:22:03:03 YES VISIBLE

 

查看该表的统计分析情况

SQL> select OWNER,TABLE_NAME,NUM_ROWS,BLOCKS,EMPTY_BLOCKS,NUM_FREELIST_BLOCKS,LAST_ANALYZED from dba_tables where owner=upper('&username') and table_name=upper('&tablename');
Enter value for username: yssettzgs
Enter value for tablename: BMS_CHK_DTL
old   1: select OWNER,TABLE_NAME,NUM_ROWS,BLOCKS,EMPTY_BLOCKS,NUM_FREELIST_BLOCKS,LAST_ANALYZED from dba_tables where owner=upper('&username') and table_name=upper('&tablename')
new   1: select OWNER,TABLE_NAME,NUM_ROWS,BLOCKS,EMPTY_BLOCKS,NUM_FREELIST_BLOCKS,LAST_ANALYZED from dba_tables where owner=upper('yssettzgs') and table_name=upper('BMS_CHK_DTL')

OWNER                          TABLE_NAME                       NUM_ROWS     BLOCKS EMPTY_BLOCKS NUM_FREELIST_BLOCKS LAST_ANALYZED
------------------------------ ------------------------------ ---------- ---------- ------------ ------------------- -------------------
YSSETTZGS                      BMS_CHK_DTL                             0          0            0                   0 2013-08-29:22:03:01

 

查看该表当前数据总数

SQL> select count(*) from yssettzgs.BMS_CHK_DTL;

  COUNT(*)
----------
    232183

 

发现比较奇怪的是,20130829晚上22点左右,索引的统计分析自动更新, NUM_ROWS 都是0,说明那个时刻表的数据为0.

 

查看这两个索引第一个字段的筛选率

SQL> select count(*) from (select distinct RUNTIME_ID from YSSETTZGS.BMS_CHK_DTL);

  COUNT(*)
----------
         1

SQL> select count(*) from (select distinct RRN from YSSETTZGS.BMS_CHK_DTL);

  COUNT(*)
----------
    178663

 

可见INDEX2_BMS_CHK_DTL 的筛选率要远高于INDEX1_BMS_CHK_DTL ,但为什么不走INDEX2_BMS_CHK_DTL 呢?

 

继续查看该表的dml记录变化

TABLE_NAME                     PARTITION_NAME                 SUBPARTITION_NAME                 INSERTS    UPDATES    DELETES TRU DROP_SEGMENTS TIMESTAMP
------------------------------ ------------------------------ ------------------------------ ---------- ---------- ---------- --- ------------- -------------------
BMS_CHK_DTL                    BMS_CHK_DTL_48005840                                                   0          0          0 NO              0 2013-08-28:22:00:09
BMS_CHK_DTL                    BMS_CHK_DTL_48020000                                             1376507    1762186    1144331 YES             0 2013-08-30:05:30:30
BMS_CHK_DTL                    BMS_CHK_DTL_48021000                                                   0          0          0 NO              0 2013-08-28:22:00:09
BMS_CHK_DTL                    BMS_CHK_DTL_48021100                                                   0          0          0 NO              0 2013-08-28:22:00:09
BMS_CHK_DTL                    BMS_CHK_DTL_48021210                                                   0          0          0 NO              0 2013-08-28:22:00:09
BMS_CHK_DTL                    BMS_CHK_DTL_48021610                                                   0          0          0 NO              0 2013-08-28:22:00:09
BMS_CHK_DTL                    BMS_CHK_DTL_48021910                                                   0          0          0 NO              0 2013-08-28:22:00:09
BMS_CHK_DTL                    BMS_CHK_DTL_48022200                                                   0          0          0 NO              0 2013-08-28:22:00:09
BMS_CHK_DTL                    BMS_CHK_DTL_48022220                                                   0          0          0 NO              0 2013-08-28:22:00:09
BMS_CHK_DTL                    BMS_CHK_DTL_48022410                                                   0          0          0 NO              0 2013-08-28:22:00:09
BMS_CHK_DTL                    BMS_CHK_DTL_48022610                                                   0          0          0 NO              0 2013-08-28:22:00:09
BMS_CHK_DTL                    BMS_CHK_DTL_48022900                                                   0          0          0 NO              0 2013-08-28:22:00:09
BMS_CHK_DTL                    BMS_CHK_DTL_48023010                                                   0          0          0 NO              0 2013-08-28:22:00:09
BMS_CHK_DTL                    BMS_CHK_DTL_48023310                                                   0          0          0 NO              0 2013-08-28:22:00:09
BMS_CHK_DTL                    BMS_CHK_DTL_48023320                                                   0          0          0 NO              0 2013-08-28:22:00:09
BMS_CHK_DTL                    BMS_CHK_DTL_48023610                                                   0          0          0 NO              0 2013-08-28:22:00:09
BMS_CHK_DTL                    BMS_CHK_DTL_48023900                                                   0          0          0 NO              0 2013-08-28:22:00:09
BMS_CHK_DTL                    BMS_CHK_DTL_48023930                                                   0          0          0 NO              0 2013-08-28:22:00:09
BMS_CHK_DTL                    BMS_CHK_DTL_48024210                                                   0          0          0 NO              0 2013-08-28:22:00:09
BMS_CHK_DTL                    BMS_CHK_DTL_48024500                                                   0          0          0 NO              0 2013-08-28:22:00:09
BMS_CHK_DTL                    BMS_CHK_DTL_48024520                                                   0          0          0 NO              0 2013-08-28:22:00:09
BMS_CHK_DTL                    BMS_CHK_DTL_48024910                                                   0          0          0 NO              0 2013-08-28:22:00:09
BMS_CHK_DTL                    BMS_CHK_DTL_48025200                                                   0          0          0 NO              0 2013-08-28:22:00:09
BMS_CHK_DTL                    BMS_CHK_DTL_48025510                                                   0          0          0 NO              0 2013-08-28:22:00:09
BMS_CHK_DTL                    BMS_CHK_DTL_48025800                                                   0          0          0 NO              0 2013-08-28:22:00:09
BMS_CHK_DTL                    BMS_CHK_DTL_48025840                                                   0          0          0 NO              0 2013-08-28:22:00:09
BMS_CHK_DTL                    BMS_CHK_DTL_48026100                                                   0          0          0 NO              0 2013-08-28:22:00:09
BMS_CHK_DTL                    BMS_CHK_DTL_48026410                                                   0          0          0 NO              0 2013-08-28:22:00:09
BMS_CHK_DTL                    BMS_CHK_DTL_48026510                                                   0          0          0 NO              0 2013-08-28:22:00:09
BMS_CHK_DTL                    BMS_CHK_DTL_48026900                                                   0          0          0 NO              0 2013-08-28:22:00:09
BMS_CHK_DTL                    BMS_CHK_DTL_48027010                                                   0          0          0 NO              0 2013-08-28:22:00:09
BMS_CHK_DTL                    BMS_CHK_DTL_48027310                                                   0          0          0 NO              0 2013-08-28:22:00:09
BMS_CHK_DTL                    BMS_CHK_DTL_48027700                                                   0          0          0 NO              0 2013-08-28:22:00:09
BMS_CHK_DTL                    BMS_CHK_DTL_48027910                                                   0          0          0 NO              0 2013-08-28:22:00:09
BMS_CHK_DTL                    BMS_CHK_DTL_48028210                                                   0          0          0 NO              0 2013-08-28:22:00:09
BMS_CHK_DTL                    BMS_CHK_DTL_48028500                                                   0          0          0 NO              0 2013-08-28:22:00:09
BMS_CHK_DTL                    BMS_CHK_DTL_48028700                                                   0          0          0 NO              0 2013-08-28:22:00:09
BMS_CHK_DTL                    BMS_CHK_DTL_48028810                                                   0          0          0 NO              0 2013-08-28:22:00:09
BMS_CHK_DTL                    BMS_CHK_DTL_48031000                                                   0          0          0 NO              0 2013-08-28:22:00:09

 

可以发现,该表数据的变化,在20130830的凌晨五点左右,发生数据改变,但这时候当晚自动统计分析已经过了。因此统计分析数据相对当前情况是非常不准确。

可以通过立刻对该表和表上索引进行统计分析,可以纠正执行计划,恢复执行效率。

 

但为什么hint提示无效?

通过10053trace分析,看见结果:

****** finished trying bitmap/domain indexes ******
Best:: AccessPath: IndexRange
Index: INDEX1_BMS_CHK_DTL
       Cost: 0.00  Degree: 1  Resp: 0.00  Card: 0.00  Bytes: 0

 

导致HINT 失效的原因有如下2点: 

(1) 如果CBO 认为使用Hint 会导致错误的结果时,Hint将被忽略。

如索引中的记录因为空值而和表的记录不一致时,结果就是错误的,会忽略hint。

(2) 如果表中指定了别名,那么Hint中也必须使用别名,否则Hint也会忽略。

 

猜测可能是第一点,在执行sql时,rrn字段并未有值导致索引选择错误。

再研究分析后,再更新具体内容。。。先记录着。