之前遇到的一个EBS性能问题，更新至CSDN
Create Accounting - Cost Management_performance issue
SYMPTOMS
元旦过后，create accounting cost management执行时间过长，还时常报错，影响正常作业；
1.对比2018/12和2019/01的报表执行时间，确实能看到
Accounting Program和create accounting cost management比上个月的执行时间要长太多 (data from MTRDB)
通过具体对比每天的执行log，发现是从01/07日开始执行时间变长
由原来的平均15分钟左右，到超过1000分钟
ANALYZE
根据出现问题的时间点，首先想到是拉出AWR report，对比发生问题前后的AWR，发现在发生问题后的AWR有两个异常的SQL。
Sql1：
BEGIN xla_accounting_pkg.unit_processor_batch(:errbuf, :rc, :A0, :A1, :A2, :A3, :A4, :A5, :A6, :A7, :A8, :A9, :A10, :A11, :A12, :A13, :A14); END;
Sql2：
SELECT /*+ leading(xet) cardinality(xet,1) */
XET.ENTITY_ID,
XET.LEGAL_ENTITY_ID,
XET.ENTITY_CODE,
XET.TRANSACTION_NUMBER,
XET.EVENT_ID,
XET.EVENT_CLASS_CODE,
XET.EVENT_TYPE_CODE,
XET.EVENT_NUMBER,
XET.EVENT_DATE,
XET.TRANSACTION_DATE,
XET.REFERENCE_NUM_1,
XET.REFERENCE_NUM_2,
XET.REFERENCE_NUM_3,
XET.REFERENCE_NUM_4,
XET.REFERENCE_CHAR_1,
XET.REFERENCE_CHAR_2,
XET.REFERENCE_CHAR_3,
XET.REFERENCE_CHAR_4,
XET.REFERENCE_DATE_1,
XET.REFERENCE_DATE_2,
XET.REFERENCE_DATE_3,
XET.REFERENCE_DATE_4,
XET.EVENT_CREATED_BY,
XET.BUDGETARY_CONTROL_FLAG,
L2.LINE_NUMBER,
L2.CODE_COMBINATION_ID SOURCE_4,
L2.ACCOUNTING_LINE_TYPE_CODE SOURCE_5,
L2.DISTRIBUTION_IDENTIFIER SOURCE_6,
L2.ENTERED_AMOUNT SOURCE_8,
L2.CURRENCY_CODE SOURCE_9,
L2.CURRENCY_CONVERSION_DATE SOURCE_10,
L2.CURRENCY_CONVERSION_RATE SOURCE_11,
L2.CURRENCY_CONVERSION_TYPE SOURCE_12,
L2.ACCOUNTED_AMOUNT SOURCE_13
FROM XLA_EVENTS_GT XET, CST_XLA_INV_LINES_V L2
WHERE XET.EVENT_ID BETWEEN :B5 AND :B4
AND XET.EVENT_DATE BETWEEN :B3 and :B2其中sql1為sql2的top level sql，也就是sql2是由sql1 call起的
查看SQL2的执行计划，除了MTL_TRANSACTION_ACCOUNTS_N6的基数过大外，其他没有太大异常。当时也有检查该执行计划，查到MTL_TRANSACTION_ACCOUNTS_N6索引所在的基表是一个1.7亿数据的大表，但没有将该问题视为问题点，去对比测试区执行计划。
- 将此问题在MOS检索，查到相关Bug较多
Create Accounting - Cost Management Program Performance Problem (Doc ID 1380982.1)
This is explained in the following bug:
Bug 12898461 - CREATE ACCOUNTING - COST MANAGEMENT PROGRAM PERFORMANCE PROBLEM
select * from table(dbms_xplan.display_cursor('d4wkbj1knh6y8',null,'ADVANCED ALLSTATS LAST'));
select * from table(dbms_xplan.display_cursor('872zgwkyjdw09', null, 'peeked_binds'));发现生成sql执行计划的检索范围非常大
Peeked Binds (identified by position):
--------------------------------------
1 - (NUMBER): 82248273
2 - (NUMBER): 82260731
3 - (DATE): 06/01/2001 00:00:00
4 - (DATE): 01/02/2019 00:00:00
5 - (VARCHAR2(30), CSID=867): 'WIP_MTL'
Predicate Information (identified by operation id):考虑到01/05日有做过gather schema，怀疑是不是gather schema后第一个request把时间范围设置错误，但是查询request记录，发现用户数据只有当前时间点，也就是2001这个时间点是系统带出来的。
想到目前crp3可以正常运行，便对比crp3中该SQL的执行计划。并且联系华硕的Douglas，确认华硕那边该SQL的执行计划如下：
两者均是走的索引MTL_TRANSACTION_ACCOUNTS_N1，而非N6
考虑到时间紧急，于是在正式区直接将MTL_TRANSACTION_ACCOUNTS_N6设置为invisible（使优化器不可见），重新运行create accounting cost management，结果可以正常运行，执行计划也与测试区一致。
SOLUTION
-
cancel all request about create accounting cost management
-
check no sql_id （'872zgwkyjdw09','d4wkbj1knh6y8'）execute on db
-
mark index MTL_TRANSACTION_ACCOUNTS_N6 invisible
alter index inv.MTL_TRANSACTION_ACCOUNTS_N6 invisible;4.submit new request “create accounting cost management”
5. check explain plan is right
select * from table(dbms_xplan.display_cursor('d4wkbj1knh6y8',null,'ADVANCED ALLSTATS LAST'));
select * from table(dbms_xplan.display_cursor('872zgwkyjdw09', null, 'peeked_binds'));6.use sql profile bind explain plan with sql_id
SQL> @/home/ora920/jobs/coe_xfr_sql_profile.sql ---execute sql profrile sql
Parameter 1:
SQL_ID (required)
Enter value for 1: d4wkbj1knh6y8 ---enter the sql id
PLAN_HASH_VALUE AVG_ET_SECS
--------------- -----------
574263622 .032
2234976105 4032.48
Parameter 2:
PLAN_HASH_VALUE (required)
Enter value for 2: 574263622 ---select the short time hash
Values passed to coe_xfr_sql_profile:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
SQL_ID : "d4wkbj1knh6y8"
PLAN_HASH_VALUE: "574263622"
SQL>BEGIN
2 IF :sql_text IS NULL THEN
3 RAISE_APPLICATION_ERROR(-20100, 'SQL_TEXT for SQL_ID &&sql_id. was not found in memory (gv$sqltext_with_newlines) or AWR (dba_hist_sqltext).');
4 END IF;
5 END;
6 /
SQL>SET TERM OFF;
SQL>BEGIN
2 IF :other_xml IS NULL THEN
3 RAISE_APPLICATION_ERROR(-20101, 'PLAN for SQL_ID &&sql_id. and PHV &&plan_hash_value. was not found in memory (gv$sql_plan) or AWR (dba_hist_sql_plan).');
4 END IF;
5 END;
6 /
SQL>SET TERM OFF;
Execute coe_xfr_sql_profile_d4wkbj1knh6y8_574263622.sql
on TARGET system in order to create a custom SQL Profile
with plan 574263622 linked to adjusted sql_text.
COE_XFR_SQL_PROFILE completed.
SQL>
SQL>exit
ora920@erpdb(/home/ora920/jobs)$ ls -lrt
total 5872
-rw-rw---- 1 ora920 dba 18248 Jan 16 17:22 coe_xfr_sql_profile.sql
-rw-r--r-- 1 ora920 dba 6200 Jan 16 17:23 coe_xfr_sql_profile_d4wkbj1knh6y8_574263622.sql --- generate sql script
-rw-r--r-- 1 ora920 dba 19981 Jan 16 17:23 coe_xfr_sql_profile.log
ora920@erpdb(/home/ora920/jobs)$ sqlplus / as sysdba
SQL*Plus: Release 11.2.0.2.0 Production on Wed Jan 16 17:25:53 2019
Copyright (c) 1982, 2010, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
SQL> @/home/ora920/jobs/coe_xfr_sql_profile_d4wkbj1knh6y8_574263622.sql
---execute this sql script bind sql plan hash with sql_id
SQL> REM
SQL> REM $Header: 215187.1 coe_xfr_sql_profile_d4wkbj1knh6y8_574263622.sql 11.4.4.4 2019/01/16 carlos.sierra $
SQL> REM
SQL> REM Copyright (c) 2000-2012, Oracle Corporation. All rights reserved.
SQL> REM
SQL> REM AUTHOR
SQL> REM carlos.sierra@oracle.com
SQL> REM
SQL> REM SCRIPT
SQL> REM coe_xfr_sql_profile_d4wkbj1knh6y8_574263622.sql
SQL> REM
SQL> REM DESCRIPTION
SQL> REM This script is generated by coe_xfr_sql_profile.sql
SQL> REM It contains the SQL*Plus commands to create a custom
SQL> REM SQL Profile for SQL_ID d4wkbj1knh6y8 based on plan hash
SQL> REM value 574263622.
SQL> REM The custom SQL Profile to be created by this script
SQL> REM will affect plans for SQL commands with signature
SQL> REM matching the one for SQL Text below.
SQL> REM Review SQL Text and adjust accordingly.
SQL> REM
SQL> REM PARAMETERS
SQL> REM None.
SQL> REM
SQL> REM EXAMPLE
SQL> REM SQL> START coe_xfr_sql_profile_d4wkbj1knh6y8_574263622.sql;
SQL> REM
SQL> REM NOTES
SQL> REM 1. Should be run as SYSTEM or SYSDBA.
SQL> REM 2. User must have CREATE ANY SQL PROFILE privilege.
SQL> REM 3. SOURCE and TARGET systems can be the same or similar.
SQL> REM 4. To drop this custom SQL Profile after it has been created:
SQL> REM EXEC DBMS_SQLTUNE.DROP_SQL_PROFILE('coe_d4wkbj1knh6y8_574263622');
SQL> REM 5. Be aware that using DBMS_SQLTUNE requires a license
SQL> REM for the Oracle Tuning Pack.
SQL> REM 6. If you modified a SQL putting Hints in order to produce a desired
SQL> REM Plan, you can remove the artifical Hints from SQL Text pieces below.
SQL> REM By doing so you can create a custom SQL Profile for the original
SQL> REM SQL but with the Plan captured from the modified SQL (with Hints).
SQL> REM
SQL> WHENEVER SQLERROR EXIT SQL.SQLCODE;
SQL> REM
SQL> VAR signature NUMBER;
SQL> VAR signaturef NUMBER;
SQL> REM
SQL> DECLARE
2 sql_txt CLOB;
3 h SYS.SQLPROF_ATTR;
4 PROCEDURE wa (p_line IN VARCHAR2) IS
5 BEGIN
6 DBMS_LOB.WRITEAPPEND(sql_txt, LENGTH(p_line), p_line);
7 END wa;
8 BEGIN
9 DBMS_LOB.CREATETEMPORARY(sql_txt, TRUE);
10 DBMS_LOB.OPEN(sql_txt, DBMS_LOB.LOB_READWRITE);
11 -- SQL Text pieces below do not have to be of same length.
12 -- So if you edit SQL Text (i.e. removing temporary Hints),
13 -- there is no need to edit or re-align unmodified pieces.
14 wa(q'[SELECT /*+ leading(xet) cardinality(xet,1) */ XET.ENTITY_ID ,XET]');
15 wa(q'[.LEGAL_ENTITY_ID ,XET.ENTITY_CODE ,XET.TRANSACTION_NUMBER ,XET.E]');
16 wa(q'[VENT_ID ,XET.EVENT_CLASS_CODE ,XET.EVENT_TYPE_CODE ,XET.EVENT_NU]');
17 wa(q'[MBER ,XET.EVENT_DATE ,XET.TRANSACTION_DATE ,XET.REFERENCE_NUM_1 ]');
18 wa(q'[,XET.REFERENCE_NUM_2 ,XET.REFERENCE_NUM_3 ,XET.REFERENCE_NUM_4 ,]');
19 wa(q'[XET.REFERENCE_CHAR_1 ,XET.REFERENCE_CHAR_2 ,XET.REFERENCE_CHAR_3]');
20 wa(q'[ ,XET.REFERENCE_CHAR_4 ,XET.REFERENCE_DATE_1 ,XET.REFERENCE_DATE]');
21 wa(q'[_2 ,XET.REFERENCE_DATE_3 ,XET.REFERENCE_DATE_4 ,XET.EVENT_CREATE]');
22 wa(q'[D_BY ,XET.BUDGETARY_CONTROL_FLAG , L2.LINE_NUMBER , L2.CODE_COMB]');
23 wa(q'[INATION_ID SOURCE_4 , L2.ACCOUNTING_LINE_TYPE_CODE SOURCE_5 , L2]');
24 wa(q'[.DISTRIBUTION_IDENTIFIER SOURCE_6 , L2.ENTERED_AMOUNT SOURCE_8 ,]');
25 wa(q'[ L2.CURRENCY_CODE SOURCE_9 , L2.CURRENCY_CONVERSION_DATE SOURCE_]');
26 wa(q'[10 , L2.CURRENCY_CONVERSION_RATE SOURCE_11 , L2.CURRENCY_CONVERS]');
27 wa(q'[ION_TYPE SOURCE_12 , L2.ACCOUNTED_AMOUNT SOURCE_13 FROM XLA_EVEN]');
28 wa(q'[TS_GT XET , CST_XLA_INV_LINES_V L2 WHERE XET.EVENT_ID BETWEEN :B]');
29 wa(q'[5 AND :B4 AND XET.EVENT_DATE BETWEEN :B3 AND :B2 AND XET.EVENT_C]');
30 wa(q'[LASS_CODE = :B1 AND XET.EVENT_STATUS_CODE <> 'N' AND L2.EVENT_ID]');
31 wa(q'[ = XET.EVENT_ID ]');
32 DBMS_LOB.CLOSE(sql_txt);
33 h := SYS.SQLPROF_ATTR(
34 q'[BEGIN_OUTLINE_DATA]',
35 q'[IGNORE_OPTIM_EMBEDDED_HINTS]',
36 q'[OPTIMIZER_FEATURES_ENABLE('11.2.0.2')]',
37 q'[DB_VERSION('11.2.0.2')]',
38 q'[OPT_PARAM('_b_tree_bitmap_plans' 'false')]',
39 q'[OPT_PARAM('_fast_full_scan_enabled' 'false')]',
40 q'[ALL_ROWS]',
41 q'[OUTLINE_LEAF(@"SEL$D836F37B")]',
42 q'[ELIMINATE_JOIN(@"SEL$335DD26A" "TRFMMT"@"SEL$3")]',
43 q'[ELIMINATE_JOIN(@"SEL$335DD26A" "MP2"@"SEL$3")]',
44 q'[ELIMINATE_JOIN(@"SEL$335DD26A" "WSH"@"SEL$3")]',
45 q'[ELIMINATE_JOIN(@"SEL$335DD26A" "XGT"@"SEL$2")]',
46 q'[OUTLINE_LEAF(@"SEL$1")]',
47 q'[OUTLINE(@"SEL$335DD26A")]',
48 q'[MERGE(@"SEL$3")]',
49 q'[OUTLINE(@"SEL$2")]',
50 q'[OUTLINE(@"SEL$3")]',
51 q'[INDEX_RS_ASC(@"SEL$1" "XET"@"SEL$1" ("XLA_EVENTS_GT"."EVENT_ID"))]',
52 q'[NO_ACCESS(@"SEL$1" "L2"@"SEL$1")]',
53 q'[LEADING(@"SEL$1" "XET"@"SEL$1" "L2"@"SEL$1")]',
54 q'[USE_HASH(@"SEL$1" "L2"@"SEL$1")]',
55 q'[INDEX_RS_ASC(@"SEL$D836F37B" "XEG"@"SEL$3" ("XLA_EVENTS_GT"."EVENT_ID"))]',
56 q'[INDEX_RS_ASC(@"SEL$D836F37B" "GL"@"SEL$2" ("GL_LEDGERS"."LEDGER_ID"))]',
57 q'[INDEX_RS_ASC(@"SEL$D836F37B" "MMT"@"SEL$3" ("MTL_MATERIAL_TRANSACTIONS"."TRANSACTION_ID"))]',
58 q'[INDEX_RS_ASC(@"SEL$D836F37B" "MTA"@"SEL$2" ("MTL_TRANSACTION_ACCOUNTS"."TRANSACTION_ID"))]',
59 q'[INDEX_RS_ASC(@"SEL$D836F37B" "MMT"@"SEL$2" ("MTL_MATERIAL_TRANSACTIONS"."TRANSACTION_ID"))]',
60 q'[INDEX(@"SEL$D836F37B" "MP1"@"SEL$3" ("MTL_PARAMETERS"."ORGANIZATION_ID"))]',
61 q'[INDEX(@"SEL$D836F37B" "MP"@"SEL$2" ("MTL_PARAMETERS"."ORGANIZATION_ID"))]',
62 q'[LEADING(@"SEL$D836F37B" "XEG"@"SEL$3" "GL"@"SEL$2" "MMT"@"SEL$3" "MTA"@"SEL$2" "MMT"@"SEL$2" "MP1"@"SEL$3" "MP"@"SEL$2")]',
63 q'[USE_NL(@"SEL$D836F37B" "GL"@"SEL$2")]',
64 q'[USE_NL(@"SEL$D836F37B" "MMT"@"SEL$3")]',
65 q'[USE_NL(@"SEL$D836F37B" "MTA"@"SEL$2")]',
66 q'[USE_NL(@"SEL$D836F37B" "MMT"@"SEL$2")]',
67 q'[USE_NL(@"SEL$D836F37B" "MP1"@"SEL$3")]',
68 q'[USE_NL(@"SEL$D836F37B" "MP"@"SEL$2")]',
69 q'[END_OUTLINE_DATA]');
70 :signature := DBMS_SQLTUNE.SQLTEXT_TO_SIGNATURE(sql_txt);
71 :signaturef := DBMS_SQLTUNE.SQLTEXT_TO_SIGNATURE(sql_txt, TRUE);
72 DBMS_SQLTUNE.IMPORT_SQL_PROFILE (
73 sql_text => sql_txt,
74 profile => h,
75 name => 'coe_d4wkbj1knh6y8_574263622',
76 description => 'coe d4wkbj1knh6y8 574263622 '||:signature||' '||:signaturef||'',
77 category => 'DEFAULT',
78 validate => TRUE,
79 replace => TRUE,
80 force_match => FALSE /* TRUE:FORCE (match even when different literals in SQL). FALSE:EXACT (similar to CURSOR_SHARING) */ );
81 DBMS_LOB.FREETEMPORARY(sql_txt);
82 END;
83 /
PL/SQL procedure successfully completed.
SQL> WHENEVER SQLERROR CONTINUE
SQL> SET ECHO OFF;
SIGNATURE
---------------------
6475979124597627135
SIGNATUREF
---------------------
6475979124597627135
... manual custom SQL Profile has been created
COE_XFR_SQL_PROFILE_d4wkbj1knh6y8_574263622 completed
SQL>
SQL>
SQL> exit
2024
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
