Oracle ADDM 自动诊断监视工具介绍

最新推荐文章于 2023-10-16 11:41:05 发布

aaron8219

最新推荐文章于 2023-10-16 11:41:05 发布

阅读量1.2k

点赞数

分类专栏： Oracle Performance Tuning Oracle AWR 文章标签： ADDM AWR STA 诊断性能

Oracle Performance Tuning 同时被 2 个专栏收录

9 篇文章 0 订阅

订阅专栏

Oracle AWR

1 篇文章 0 订阅

订阅专栏

Oracle AWR 介绍(AWR -- Automatic Workload Repository)

http://blog.csdn.net/tianlesoftware/archive/2009/10/17/4682300.aspx

一. ADDM概述

ADDM(Automatic Database Diagnostic Monitor) 是植入Oracle数据库的一个自诊断引擎.ADDM 通过检查和分析AWR获取的数据来判断Oracle数据库中可能的问题.

在Oracle9i及之前，DBA们已经拥有了很多很好用的性能分析工具，比如，tkprof、sql_trace、statspack、set event 10046&10053等等。这些工具能够帮助DBA很快的定位性能问题。但这些工具都只给出一些统计数据，然后再由DBA们根据自己的经验进行优化。
Oracle10g中推出了新的优化诊断工具：数据库自动诊断监视工具（Automatic Database Diagnostic Monitor :ADDM）和SQL优化建议工具（SQL Tuning Advisor: STA）。这两个工具的结合使用，能使DBA节省大量优化时间，也大大减少了系统宕机的危险。简单点说，ADDM就是收集相关的统计数据到自动工作量知识库(Automatic Workload Repository :AWR)中，而STA则根据这些数据，给出优化建议。

例如，一个系统资源紧张，出现了明显的性能问题，由以往的办法，做个一个statspack快照，等30分钟，再做一次。查看报告，发现’ db file scattered read’事件在top 5 events里面。根据经验，这个事件一般可能是因为缺少索引、统计分析信息不够新、热表都放在一个数据文件上导致IO争用等原因引起的。根据这些经验，我们需要逐个来定位排除，比如查看语句的查询计划、查看user_tables的last_analysed子段，检查热块等等步骤来最后定位出原因，并给出优化建议。但是，有了STA以后，它就可以根据ADDM采集到的数据直接给出优化建议，甚至给出优化后的语句。

ADDM能发现定位的问题包括：
•操作系统内存页入页出问题
•由于Oracle负载和非Oracle负载导致的CPU瓶颈问题
•导致不同资源负载的Top SQL语句和对象——CPU消耗、IO带宽占用、潜在IO问题、RAC内部通讯繁忙
•按照PLSQL和JAVA执行时间排的Top SQL语句.
•过多地连接 (login/logoff).
•过多硬解析问题——由于shared pool过小、书写问题、绑定大小不适应、解析失败原因引起的。
•过多软解析问题
•索引查询过多导致资源争用.
•由于用户锁导致的过多的等待时间 (通过包dbms_lock加的锁)
•由于DML锁导致的过多等待时间(例如锁住表了)
•由于管道输出导致的过多等待时间(如通过包dbms_pipe.put进行管道输出)
•由于并发更新同一个记录导致的过多等待时间(行级锁等待)
•由于ITL不够导致的过多等待时间（大量的事务操作同一个数据块）
•系统中过多的commit和rollback(logfile sync事件).
•由于磁盘带宽太小和其他潜在问题（如由于logfile太小导致过多的checkpoint，MTTR设置问题，过多的undo操作等等）导致的IO性能问题I
•对于DBWR进程写数据块，磁盘IO吞吐量不足
•由于归档进程无法跟上redo日至产生的速度，导致系统变慢
•redo数据文件太小导致的问题
•由于扩展磁盘分配导致的争用
•由于移动一个对象的高水位导致的争用问题
•内存太小问题——SGA Target, PGA, Buffer Cache, Shared Pool
•在一个实例或者一个机群环境中存在频繁读写争用的热块
•在一个实例或者一个机群环境中存在频繁读写争用的热对象
•RAC环境中内部通讯问题
•LMS进程无法跟上导致锁请求阻塞
•在RAC环境中由于阻塞和争用导致的实例倾斜
•RMAN导致的IO和CPU问题
•Streams和AQ问题
•资源管理等待事件

ADDM提供了一个整体的优化方案.基于一段时间内的AWR snapshots(默认一小时一次)可以执行ADDM 分析,它可以帮我们诊断在这段期间内数据库可能存在的瓶颈.

ADDM分析的目标是减小吞吐量的度量值, 在这里我们将它称为"DB TIME". DB TIME是一个累积值(数据库服务器处理用户请求所花费的时间). 它包括了等待时间和CPU处理的时间(针对所有活跃的用户进程而言),可以通过查询下面两个视图来获得它的值: V$SESS_TIME_MODEL, V$SYS_TIME_MODEL.

AWR收集的数据时放到内存中（share pool），通过一个新的后台进程MMON定期写到磁盘中。所以10g的share pool要求比以前版本更大，一般推荐比以前大15-20%。

注意: ADDM不会将处理用户响应时间作为调优的目标, 你应该使用"TRACE"技术来监控它.

通过减小"DB TIME", 使用同样多的系统资源,数据库服务器可以处理更多的用户请求,也就是提高了吞吐量. 通过ADDM报告的问题是按照DB time排序的.

二. ADDM 分析的结果
ADDM 分析的结果以一些"Finding"的样式来表达. 每个"Finding"都属于以下三种类型之一:

1. 问题: 描述了导致数据库性能问题的根源;
2. 征兆: 包含了可能导致其他问题的信息
3. 信息: 报告其他没有问题的模块

三. 设置ADDM
缺省情况下,ADDM已经被自动启用,通过初始化参数文件中的STATISTICS_LEVEL来控制.
这个参数应该被设置成TYPICAL或者ALL(缺省值是TYPICAL).如果你将这个参数设置成basic,很多Oracle的特性将被屏蔽.
ALTER SESSION SET STATISTICS_LEVEL= TYPICAL;

ADDM 对于I/O性能的评估分析在部分程度上依赖于这个DBIO_EXPECTED. 这个参数的含义是读取一个数据块所花费的平均时间(以微秒为单位). Oracle使用的是缺省值(10毫秒), 对于现在流行的硬盘来说, 这是一个比较合适的值.如果你的硬盘比较陈旧,或者你有一个非常好的RAM DISK,请修改这个值.

为了决定DBIO_EXPECTED这个参数该怎样去正确地配置,需要完成下面的步骤:

1. 基于你的机器的硬件,估量一下读取单个数据库块所花费的平均时间.
注意:这个度量应该针对随机的I/O(包括寻道的时间).传统的值应该属于5000-20000微秒这个区间.

2. 为接下来的ADDM执行设置一个时间参数. 例如:如果估计的值是8000微秒,你应该以SYS的身份执行
下面的过程:

EXECUTE DBMS_ADVISOR.SET_DEFAULT_PARAMETER ('ADDM','DBIO_EXPECTED',8000);

四. 通过Oracle Enterprise Manager来访问ADDM:

五. 诊断与ADDM相关的问题:
为了诊断数据库性能问题, ADDM分析可以跨越任意两个snapshots,只要它们满足下面两个条件:
1. 两个快照在创建过程中没有错误并且没有被删除;
2. 两个快照期间数据库不能发生关闭和启动的事件
(同statspack).

最简单的运行ADDM分析的方法就是运行Enterprise Manager.
另外,也可以手工地执行 $ORACLE_HOME/rdbms/admin/addmrpt.sql以及dbms_advisor包.
这些脚本和包可以被任何用户执行,只要它们被赋予了ADVISOR的角色.

5.1 使用addmrpt.sql来运行
和statspack包中的spreport.sql非常相似

5.2 使用dbms_advisor包:
基本步骤:
1) 创建一个task: dbms_advisor.create_task ;

2) 设置相关的参数:
START_SNAPSHOT,END_SNAPSHOT
(通过DBMS_ADVISOR.SET_TASK_PARAMETER来完成)

3) 执行这个task: DBMS_ADVISOR.E

六. 与 ADDM相关的视图:
DBA_ADVISOR_TASKS
DBA_ADVISOR_LOG
DBA_ADVISOR_RECOMMENDATIONS
DBA_ADVISOR_FINDINGS

七．工作采集、诊断过程
Oracle10g提供了一个图形化的界面（通过OEM），使这个工具使用起来非常简单。下面这里介绍一下如何通过sqlplus使用这个工具。

第一步：创建测试用的表
SQL> CREATE TABLE bigtab AS SELECT rownum as "id", a.* FROM dba_objects a;
Table created.
SQL> create table smalltab as select rownum as "id", a.* FROM dba_tables a;
Table created.
SQL> ALTER TABLE bigtab MODIFY (empno NUMBER);
Table altered.
SQL> DECLARE
2       n NUMBER;
3    BEGIN
4       FOR n IN 1..100
5       LOOP
6           INSERT INTO bigtab SELECT rownum as "id", a.* FROM dba_objects a;
7           COMMIT;
8       END LOOP;
9   END;
/
PL/SQL procedure successfully completed.

第二步：采集一次工作量快照
SQL> begin
  2   dbms_workload_repository.create_snapshot('TYPICAL');
  3  end;
  4  /
PL/SQL procedure successfully completed.

第三步：进行一些高负荷操作
DECLARE
    v_var number;
BEGIN
    FOR n IN 1..6
    LOOP
        select count(*) into v_var from bigtab b, smalltab a;
    END LOOP;
END;
/
PL/SQL procedure successfully completed.

第四步：再次采集一次工作量快照
要注意的是：两次快照之间的间隔时间必须足够（一般推荐30分钟左右），否则得到的ADDM报告中就会提示：THERE WAS NOT ENOUGH DATABASE TIME FOR ADDM ANALYSIS.
SQL> begin
  2   dbms_workload_repository.create_snapshot('TYPICAL');
  3  end;
  4  /
PL/SQL procedure successfully completed.

第五步：创建一个优化诊断任务并执行

先获取到两次快照的ID:
SQL> select snap_id from
  2  (SELECT * FROM dba_hist_snapshot
  3  ORDER BY snap_id desc)
  4  where rownum <=2;
SNAP_ID
--------
      66
      65

然后创建优化任务，并执行。

DECLARE
    task_name VARCHAR2(30) := 'DEMO_ADDM01';
    task_desc VARCHAR2(30) := 'ADDM Feature Test';
    task_id NUMBER;
BEGIN
    dbms_advisor.create_task('ADDM', task_id, task_name, task_desc, null);
    dbms_advisor.set_task_parameter(task_name, 'START_SNAPSHOT', 65);
    dbms_advisor.set_task_parameter(task_name, 'END_SNAPSHOT', 66);
    dbms_advisor.set_task_parameter(task_name, 'INSTANCE', 1);
    dbms_advisor.set_task_parameter(task_name, 'DB_ID', 1712582900);
    dbms_advisor.execute_task(task_name);
END;
/
PL/SQL procedure successfully completed.

DBID 查看sql

SQL> select dbid from v$database;

DBID

----------

1712582900

其中，set_task_parameter是用来设置任务参数的。START_SNAPSHOT是起始快照ID，END_SNAPSHOT是结束快照ID，INSTANCE是实例号，对于单实例，一般是1，在RAC环境下，可以通过查询视图v$instance得到，DB_ID是数据库的唯一识别号，可以通过查询v$database查到。

第六步：查看优化建议结果
通知函数dbms_advisor.get_task_report可以得到优化建议结果。

SQL> SET LONG 1000000 PAGESIZE 0 LONGCHUNKSIZE 1000
SQL> COLUMN get_clob FORMAT a80
SQL> SELECT dbms_advisor.get_task_report('DEMO_ADDM01', 'TEXT', 'ALL') FROM DUAL;

DBMS_ADVISOR.GET_TASK_REPORT('
--------------------------------------------------------------------------------
          DETAILED ADDM REPORT FOR TASK 'DEMO_ADDM01' WITH ID 243
          -------------------------------------------------------

              Analysis Period: 23-NOV-2005 from 15:02:27 to 16:06:42
         Database ID/Instance: 1712582900/1
      Database/Instance Names: EDGAR/edgar
                    Host Name: HUANGED
             Database Version: 10.2.0.1.0
               Snapshot Range: from 65 to 66
                Database Time: 1463 seconds
        Average Database Load: .4 active sessions

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

FINDING 1: 100% impact (1463 seconds)
-------------------------------------
Significant virtual memory paging was detected on the host operating system.

   RECOMMENDATION 1: Host Configuration, 100% benefit (1463 seconds)
      ACTION: Host operating system was experiencing significant paging but no
         particular root cause could be detected. Investigate processes that
         do not belong to this instance running on the host that are consuming
         significant amount of virtual memory. Also consider adding more
         physical memory to the host.

FINDING 2: 100% impact (1463 seconds)
-------------------------------------
SQL statements consuming significant database time were found.

   RECOMMENDATION 1: SQL Tuning, 68% benefit (998 seconds)
      ACTION: Tune the PL/SQL block with SQL_ID "064wqx7c5b81z". Refer to the
         "Tuning PL/SQL Applications" chapter of Oracle's "PL/SQL User's Guide
         and Reference"
         RELEVANT OBJECT: SQL statement with SQL_ID 064wqx7c5b81z
         DECLARE
         v_var number;
         BEGIN
         FOR n IN 1..10000
         LOOP
         select count(*) into v_var from bigtab b, smalltab a;
         END LOOP;
         END;

   RECOMMENDATION 2: SQL Tuning, 67% benefit (986 seconds)
      ACTION: Run SQL Tuning Advisor on the SQL statement with SQL_ID
         "fvqfghq71cqns".
         RELEVANT OBJECT: SQL statement with SQL_ID fvqfghq71cqns and
         PLAN_HASH 3281046854
         SELECT COUNT(*) FROM BIGTAB B, SMALLTAB A
      RATIONALE: SQL statement with SQL_ID "fvqfghq71cqns" was executed 6
         times and had an average elapsed time of 166 seconds.

FINDING 3: 69% impact (1002 seconds)
------------------------------------
Time spent on the CPU by the instance was responsible for a substantial part
of database time.

   RECOMMENDATION 1: SQL Tuning, 67% benefit (986 seconds)
      ACTION: Run SQL Tuning Advisor on the SQL statement with SQL_ID
         "fvqfghq71cqns".
         RELEVANT OBJECT: SQL statement with SQL_ID fvqfghq71cqns and
         PLAN_HASH 3281046854
         SELECT COUNT(*) FROM BIGTAB B, SMALLTAB A
      RATIONALE: SQL statement with SQL_ID "fvqfghq71cqns" was executed 6
         times and had an average elapsed time of 166 seconds.
      RATIONALE: Average CPU used per execution was 162 seconds.

   RECOMMENDATION 2: SQL Tuning, 2.1% benefit (30 seconds)
      ACTION: Tune the PL/SQL block with SQL_ID "2b064ybzkwf1y". Refer to the
         "Tuning PL/SQL Applications" chapter of Oracle's "PL/SQL User's Guide
         and Reference"
         RELEVANT OBJECT: SQL statement with SQL_ID 2b064ybzkwf1y
         BEGIN EMD_NOTIFICATION.QUEUE_READY(:1, :2, :3); END;
      RATIONALE: SQL statement with SQL_ID "2b064ybzkwf1y" was executed 125
         times and had an average elapsed time of 0.26 seconds.
      RATIONALE: Average CPU used per execution was 0.24 seconds.

FINDING 4: 2.2% impact (33 seconds)
-----------------------------------
PL/SQL execution consumed significant database time.

   RECOMMENDATION 1: SQL Tuning, 2.2% benefit (33 seconds)
      ACTION: Tune the PL/SQL block with SQL_ID "2b064ybzkwf1y". Refer to the
         "Tuning PL/SQL Applications" chapter of Oracle's "PL/SQL User's Guide
         and Reference"
         RELEVANT OBJECT: SQL statement with SQL_ID 2b064ybzkwf1y
         BEGIN EMD_NOTIFICATION.QUEUE_READY(:1, :2, :3); END;
      RATIONALE: SQL statement with SQL_ID "2b064ybzkwf1y" was executed 125
         times and had an average elapsed time of 0.26 seconds.
      RATIONALE: Average time spent in PL/SQL execution was 0.26 seconds.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

          ADDITIONAL INFORMATION
          ----------------------

Wait class "Application" was not consuming significant database time.
Wait class "Commit" was not consuming significant database time.
Wait class "Concurrency" was not consuming significant database time.
Wait class "Configuration" was not consuming significant database time.
Wait class "Network" was not consuming significant database time.
Wait class "User I/O" was not consuming significant database time.
Session connect and disconnect calls were not consuming significant database
time.
Hard parsing of SQL statements was not consuming significant database time.

The analysis of I/O performance is based on the default assumption that the
average read time for one database block is 10000 micro-seconds.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

          TERMINOLOGY
          -----------

DATABASE TIME: This is the ADDM's measurement of throughput. From the user's
   point of view: this is the total amount of time spent by users waiting for
   a response from the database after issuing a call (not including
   networking). From the database instance point of view: this is the total
   time spent by forground processes waiting for a database resource (e.g.,
   read I/O), running on the CPU and waiting for a free CPU (run-queue). The
   target of ADDM analysis is to reduce this metric as much as possible,
   thereby reducing the instance's response time.

AVERAGE DATABASE LOAD: At any given time we can count how many users (also
   called 'Active Sessions') are waiting for an answer from the instance. This
   is the ADDM's measurement for instance load. The 'Average Database Load' is
   the average of the the load measurement taken over the entire analysis
   period. We get this number by dividing the 'Database Time' by the analysis
   period. For example, if the analysis period is 30 minutes and the 'Database
   Time' is 90 minutes, we have an average of 3 users waiting for a response.

IMPACT: Each finding has an 'Impact' associated with it. The impact is the
   portion of the 'Database Time' the finding deals with. If we assume that
   the problem described by the finding is completely solved, then the
   'Database Time' will be reduced by the amount of the 'Impact'.

BENEFIT: Each recommendation has a 'benefit' associated with it. The ADDM
   analysis estimates that the 'Database Time' can be reduced by the 'benefit'
   amount if all the actions of the recommendation are performed.

说明：其中第五步到第六步可以直接执行$ORACLE_HOME/rdbms/admin/addmrpt.sql来得到，这个脚本的执行过程和statspack脚本执行过程类似：

SQL> @addmrpt

Current Instance
~~~~~~~~~~~~~~~~

   DB Id    DB Name      Inst Num Instance
----------- ------------ -------- ------------
1712582900 EDGAR               1 edgar

Instances in this Workload Repository schema
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

   DB Id     Inst Num DB Name      Instance     Host
------------ -------- ------------ ------------ ------------
* 1712582900        1 EDGAR        edgar        HUANGED

Using 1712582900 for database Id
Using          1 for instance number

Specify the number of days of snapshots to choose from
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Entering the number of days (n) will result in the most recent
(n) days of snapshots being listed.  Pressing <return> without
specifying a number lists all completed snapshots.

Listing the last 3 days of Completed Snapshots

                                                        Snap
Instance     DB Name        Snap Id    Snap Started    Level
------------ ------------ --------- ------------------ -----
edgar        EDGAR                7 22 Nov 2005 00:00      1
... ...
                                 64 23 Nov 2005 15:02      1
                                 65 23 Nov 2005 16:00      1
                                 66 23 Nov 2005 16:06      1

Specify the Begin and End Snapshot Ids
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Enter value for begin_snap: 65
Begin Snapshot Id specified: 66

Enter value for end_snap: 66
End   Snapshot Id specified: 66

Specify the Report Name
~~~~~~~~~~~~~~~~~~~~~~~
The default report file name is addmrpt_1_65_66.txt.  To use this name,
press <return> to continue, otherwise enter an alternative.

Enter value for report_name:

Using the report name addmrpt_1_65_66.txt

Running the ADDM analysis on the specified pair of snapshots ...

Generating the ADDM report for this analysis ...

... ...

此外，如果是RAC环境下，可以执行$ORACLE_HOME/rdbms/admin/addmrpti.sql，这脚本的执行，会多出要求输入DB ID和instance ID的要求。

八．诊断结果分析

我们从上面的建议结果看到了，ADDM Report的结果与Statspack Report的结果大不相同。Statspack Report的结果给出的都是统计数据、各种事件，然后由DBA根据这些数据给出优化建议，而ADDM Report的结果包含就已经是给出的优化建议了

第一部分：
      Analysis Period: 23-NOV-2005 from 15:02:27 to 16:06:42
      Database ID/Instance: 1712582900/1
      Database/Instance Names: EDGAR/edgar
      Host Name: HUANGED
      Database Version: 10.2.0.1.0
      Snapshot Range: from 65 to 66
      Database Time: 1463 seconds
      Average Database Load: .4 active sessions

   这一部分包括一些基础信息，分析时间段、DB和instance ID&名字、主机名字、Oracle版本、快照范围、数据库消耗时间、多少个活动会话。

第二部分：
下面就是ADDM发现的问题，并给出的相应建议。在我们这个例子中总共发现4个问题，下面一一解释一下。

第一个问题：
FINDING 1: 100% impact (1463 seconds)
-------------------------------------
Significant virtual memory paging was detected on the host operating system.

   RECOMMENDATION 1: Host Configuration, 100% benefit (1463 seconds)
    ACTION: Host operating system was experiencing significant paging but no
         particular root cause could be detected. Investigate processes that
         do not belong to this instance running on the host that are consuming
         significant amount of virtual memory. Also consider adding more
         physical memory to the host.
    先看第一行：100% impact (1463 seconds)，这是这个问题所持续的实践及其对系统的影响，它的时间是1463秒，和分析期间的数据库消耗时间（在第一部分中）是一样（1463秒），所以对系统的影响是1463/1463*100=100%的。
    再看第二行：Significant virtual memory paging was detected on the host operating system.，这是ADDM发现的这个问题的具体描述：在操作系统中发现有显著的虚拟内存页入页出的问题。
    然后看ADDM给出的建议及其作用：Host Configuration, 100% benefit (1463 seconds)——更改主机配置，100%有效。
    最后是具体该如何操作：略——在主机的操作系统上发现了明显的页入页出，但是没有发现明显导致内存频繁换如换出的根本原因。需要仔细检查那些消耗大量虚拟内存的进程（除Oracle实例外）。此外，还可以考虑增大主机的物理内存。说明一下：我的这个实例是跑在我自己的PC机上，Oracle运行的同时有大量的其他消耗内存的程序（word等）在运行，所以肯定有大量的内存交换存在。

再看第二个问题：
FINDING 2: 100% impact (1463 seconds)
-------------------------------------
SQL statements consuming significant database time were found.

   RECOMMENDATION 1: SQL Tuning, 68% benefit (998 seconds)
      ACTION: Tune the PL/SQL block with SQL_ID "064wqx7c5b81z". Refer to the
         "Tuning PL/SQL Applications" chapter of Oracle's "PL/SQL User's Guide
         and Reference"
         RELEVANT OBJECT: SQL statement with SQL_ID 064wqx7c5b81z
         DECLARE
         v_var number;
         BEGIN
         FOR n IN 1..10000
         LOOP
         select count(*) into v_var from bigtab b, smalltab a;
         END LOOP;
         END;
    ADDM发现有SQL语句在消耗大量数据库时间，它的影响是100%的。给出的建议是优化SQL，能取得68%的效果。
    具体操作是优化ADDM找到的PL/SQL块，它的SQL_ID是"064wqx7c5b81z"（可以通过select sql_text from v$sql where sql_id=’064wqx7c5b81z’;查到）。至于如何优化SQL语句，可以参考Oracle文档PL/SQL User's Guide and Reference中的Tuning PL/SQL Applications章节。下面的内容便是我们用来插入数据的测试语句。

下面是ADDM发现的其他问题语句：
FINDING 3: 69% impact (1002 seconds)
------------------------------------
Time spent on the CPU by the instance was responsible for a substantial part
of database time.

   RECOMMENDATION 1: SQL Tuning, 67% benefit (986 seconds)
      ACTION: Run SQL Tuning Advisor on the SQL statement with SQL_ID
         "fvqfghq71cqns".
         RELEVANT OBJECT: SQL statement with SQL_ID fvqfghq71cqns and
         PLAN_HASH 3281046854
         SELECT COUNT(*) FROM BIGTAB B, SMALLTAB A
      RATIONALE: SQL statement with SQL_ID "fvqfghq71cqns" was executed 6
         times and had an average elapsed time of 166 seconds.
      RATIONALE: Average CPU used per execution was 162 seconds.

   RECOMMENDATION 2: SQL Tuning, 2.1% benefit (30 seconds)
      ACTION: Tune the PL/SQL block with SQL_ID "2b064ybzkwf1y". Refer to the
         "Tuning PL/SQL Applications" chapter of Oracle's "PL/SQL User's Guide
         and Reference"
         RELEVANT OBJECT: SQL statement with SQL_ID 2b064ybzkwf1y
         BEGIN EMD_NOTIFICATION.QUEUE_READY(:1, :2, :3); END;
      RATIONALE: SQL statement with SQL_ID "2b064ybzkwf1y" was executed 125
         times and had an average elapsed time of 0.26 seconds.
      RATIONALE: Average CPU used per execution was 0.24 seconds.
    这个问题的描述是，实例消耗的CPU事件占据了大量的数据库运行时间。由于发现了两条问题语句，所以这里有两个建议。
    第一个建议就是优化我们的测试语句。并且说明了这个问题的根本原因：这条语句总共执行过6次，平均每次消耗了166秒。平均这个问题消耗的CPU时间是162秒。
    第二个建议实际上是针对一个系统过程，这个过程是用来读取队列信息的，消耗的资源比较小，我们这里就不需要关心了。

再看最后一个问题：
FINDING 4: 2.2% impact (33 seconds)
-----------------------------------
PL/SQL execution consumed significant database time.

   RECOMMENDATION 1: SQL Tuning, 2.2% benefit (33 seconds)
      ACTION: Tune the PL/SQL block with SQL_ID "2b064ybzkwf1y". Refer to the
         "Tuning PL/SQL Applications" chapter of Oracle's "PL/SQL User's Guide
         and Reference"
         RELEVANT OBJECT: SQL statement with SQL_ID 2b064ybzkwf1y
         BEGIN EMD_NOTIFICATION.QUEUE_READY(:1, :2, :3); END;
      RATIONALE: SQL statement with SQL_ID "2b064ybzkwf1y" was executed 125
         times and had an average elapsed time of 0.26 seconds.
      RATIONALE: Average time spent in PL/SQL execution was 0.26 seconds.
     从内容上看，这个问题就是上一个问题中的第二个建议。但是，它导致的结果是不一样的。看这个问题的描述：PL/SQL的执行次数消耗了大量的数据库时间。它的根本原因是因为执行次数太多（125次）。可见ADDM的问题检查相当全面。

第三部分：
        这一部分的内容是关于此次优化建议的一些附加信息：
          ADDITIONAL INFORMATION
          ----------------------

Wait class "Application" was not consuming significant database time.
Wait class "Commit" was not consuming significant database time.
Wait class "Concurrency" was not consuming significant database time.
Wait class "Configuration" was not consuming significant database time.
Wait class "Network" was not consuming significant database time.
Wait class "User I/O" was not consuming significant database time.
Session connect and disconnect calls were not consuming significant database
time.
Hard parsing of SQL statements was not consuming significant database time.

The analysis of I/O performance is based on the default assumption that the
average read time for one database block is 10000 micro-seconds.

这是关于这次优化诊断对各类事件（在Oracle10g，新增了很多新的事件，主要是将原先一些较含糊的事件细化了，同时将所有事件进行了归类。可以查看视图V$SYSTEM_WAIT_CLASS）的一些总结：Application、Commit、Concurrency、Configuration、Network、User I/O类等待事件没有显著消耗数据库时间；会话连接、断连请求没有消耗大量数据库时间；对SQL语句的硬解析没有消耗大量数据库时间；对IO性能的分析是基于默认假设每次读一个数据块的时间是10000微秒的。

第四部分：
         这部分是对诊断报告中用到的术语的解释：
          TERMINOLOGY
          -----------

DATABASE TIME: This is the ADDM's measurement of throughput. From the user's
   point of view: this is the total amount of time spent by users waiting for
   a response from the database after issuing a call (not including
   networking). From the database instance point of view: this is the total
   time spent by forground processes waiting for a database resource (e.g.,
   read I/O), running on the CPU and waiting for a free CPU (run-queue). The
   target of ADDM analysis is to reduce this metric as much as possible,
   thereby reducing the instance's response time.

AVERAGE DATABASE LOAD: At any given time we can count how many users (also
   called 'Active Sessions') are waiting for an answer from the instance. This
   is the ADDM's measurement for instance load. The 'Average Database Load' is
   the average of the the load measurement taken over the entire analysis
   period. We get this number by dividing the 'Database Time' by the analysis
   period. For example, if the analysis period is 30 minutes and the 'Database
   Time' is 90 minutes, we have an average of 3 users waiting for a response.

IMPACT: Each finding has an 'Impact' associated with it. The impact is the
   portion of the 'Database Time' the finding deals with. If we assume that
   the problem described by the finding is completely solved, then the
   'Database Time' will be reduced by the amount of the 'Impact'.

BENEFIT: Each recommendation has a 'benefit' associated with it. The ADDM
   analysis estimates that the 'Database Time' can be reduced by the 'benefit'
   amount if all the actions of the recommendation are performed.

DATABASE TIME：是ADDM的度量数据。从用户角度看：这是从向数据库请求开始，消耗在用户等待响应上的全部时间（不包括网络响应时间）；从数据库实例角度看：前台进程消耗在等待一种数据库资源（例如，IO读）、CPU运行和等待CPU释放（队列等待）的总共时间。 ADDM分析的目标就尽量降低这个数字，也就是减少实例响应时间
     AVERAGE DATABASE LOAD：所有能统计到的有多少用户（也称为“活动会话”）等待实例响应。这是实例负荷的度量指标。平均数据库负荷是由整个分析计算出来的平均负荷。通过“Database Time”除以分析周期时间得到。例如，分析周期时30分钟，而数据库运行消耗时间是90分钟，那就说明平均有3个用户在等待响应。
     IMPACT：每一个找到的问题都有“影响”这一项。“影响”是数据库消耗时间用于处理这个问题的时间不分。假定我们所找到的这个问题完全解决，那么数据库消耗时间就会相应减少“影响”时间。
     BENEFIT：每一个找到的问题都“受益”这一项。如果所有建议操作得到实施，ADDM分析估计数据库消耗时间能减少“受益”的全部时间。

找到了有问题的SQL后我们就可以用 Oracle SQL Tuning Advisor 工具来优化该SQL，关于STA的使用，请参考Blog：

如何用 SQL Tuning Advisor (STA) 优化SQL语句

http://blog.csdn.net/tianlesoftware/archive/2010/05/28/5630888.aspx