EM 12C报数据库报磁盘read errors的错误:
EM12C-Warning:+ASM_fd02-cluster:Disk-DATA_FD02.DATA_FD02_CD_08_FD02CEL06-has-279-Read-errors.:Nov-25,-2015-11:47:04-AM-CST(2015-11-25-11:46)
oracle官方分析认为是触发bug导致:
当cell节点出现内存溢出,ORA-27626: Exadata error: 215 (Cell failed to allocate memory)错误就会出在数据库的alert日志中产生。 这个错误发生在大量的smart scan操作需要分配更多的内存用于smart scan。这不是说cell节点需增加更多的内存用于smart scan,而是说在这一点上的smart scan需要内存比较高。 这是一个与I/O相关的错误。所以在trace文件中可以看到很多类似的以下信息 =======Dumping Fail Data blocks======= Fail Block 0: RDBA: -1328289280, Diskname: o/172.16.247.110/DATA_FD02_CD_08_fd02cel06, FOB: 0x437774f78, Disknum: 0, AUNUM: 277141 所以在相同的时间里,在EM12c中可以看到以下的read-error相关的信息。 EM12C-Warning:+ASM_fd02-cluster:Disk-DATA_FD02.DATA_FD02_CD_08_FD02CEL06-has-279-Read-errors.:Nov-25,-2015-11:47:04-AM-CST(2015-11-25-11:46) 这个错误可以打patch 18169273来解决,这样ORA-27603和ORA-27626就不会再报出来,EM12c中报出的错误也不会报出来了。 |
=== ODM Cause Determination ===
The issue is caused by Bug 18169273 - FLOOD OF ALERT WHEN CELL GOES TO PASS THROUGH MODE DUE TO NO CELL MEMORY
=== ODM Cause Justification ===
On 11.2.0.3 Exadata, ORA-27626: Exadata error: 215 (Cell failed to allocate memory) generated in db alert log. when the cell runs out of memory, for that scan it returns error 215 (OSS_ERRCODE_NOCELLMEM) to RDBMS. Once received the IO result, we print that in alert.log. Those errors mean that some smart scans didn't actually go through smart scan but did get serviced through Block IO. This happens when there is a surge in smart scan sessions which need more memory to be allocated for the scan. It is not just number of smart scan sessions being more but number of smart scan sessions which need more memory are high at that point. ORA-27603: Cell storage I/O error, I/O failed on disk o/172.16.247.111//box/predicate at offset 0 for data length 0 ORA-27626: Exadata error: 215 (Cell failed to allocate memory) Above version, symptom matches Bug 18169273. |
估计和sga比较小也是有关系的
下面是oracle的分析思路
下面是oracle的分析思路
1、出现这个问题可能需要分析的日志
+ 所有数据库节点ASM和所有RDBMS实例的alert log + GI/RDBMS的opatch输出 - ‘opatch lsinventory -detail' + Exadata image版本 - imageinfo + 所有cell节点sundiag输出 + ASM中的以下输出: select name, read_errs from v$asm_disk; |
2、结合em的报错信息,查看alert日志时发现db节点1 alert日志有报错
FileName = alert_reportdb1.log Wed Nov 25 11:28:33 2015 Errors in file /u01/app/oracle/diag/rdbms/reportdb/reportdb1/trace/reportdb1_ora_90598.trc: ORA-27603: Cell storage I/O error, I/O failed on disk o/172.16.247.111//box/predicate at offset 0 for data length 0 ORA-27626: Exadata error: 215 (Cell failed to allocate memory) ... Wed Nov 25 11:43:49 2015 Errors in file /u01/app/oracle/diag/rdbms/reportdb/reportdb1/trace/reportdb1_ora_95186.trc: ORA-27603: Cell storage I/O error, I/O failed on disk o/172.16.247.110//box/predicate at offset 0 for data length 0 ORA-27626: Exadata error: 215 (Cell failed to allocate memory) ... Wed Nov 25 11:43:50 2015 Errors in file /u01/app/oracle/diag/rdbms/reportdb/reportdb1/trace/reportdb1_ora_94153.trc: ORA-27603: Cell storage I/O error, I/O failed on disk o/172.16.247.110//box/predicate at offset 0 for data length 0 ORA-27626: Exadata error: 215 (Cell failed to allocate memory) ... Wed Nov 25 12:30:21 2015 Errors in file /u01/app/oracle/diag/rdbms/reportdb/reportdb1/trace/reportdb1_ora_127407.trc: ORA-27603: Cell storage I/O error, I/O failed on disk o/172.16.247.110//box/predicate at offset 0 for data length 0 ORA-27626: Exadata error: 215 (Cell failed to allocate memory) ... Wed Nov 25 12:52:51 2015 Errors in file /u01/app/oracle/diag/rdbms/reportdb/reportdb1/trace/reportdb1_ora_124095.trc: ORA-27603: Cell storage I/O error, I/O failed on disk o/172.16.247.107//box/predicate at offset 0 for data length 0 ORA-27626: Exadata error: 215 (Cell failed to allocate memory) ... Wed Nov 25 12:52:52 2015 Errors in file /u01/app/oracle/diag/rdbms/reportdb/reportdb1/trace/reportdb1_ora_71945.trc: ORA-27603: Cell storage I/O error, I/O failed on disk o/172.16.247.107//box/predicate at offset 0 for data length 0 ORA-27626: Exadata error: 215 (Cell failed to allocate memory) |
3、分析trace文件
FileName = reportdb1_ora_90598.trc Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP, Data Mining and Real Application Testing options ORACLE_HOME = /u01/app/oracle/product/11.2.0.3/dbhome_1 System name: Linux Node name: fd02db01.deppon.com Release: 2.6.39-400.128.17.el5uek Version: #1 SMP Tue May 27 13:20:24 PDT 2014 Machine: x86_64 Instance name: reportdb1 Redo thread mounted by this instance: 1 Oracle process number: 359 Unix process pid: 90598, image: oracle@fd02db01.deppon.com *** 2015-11-25 09:56:43.270 *** SESSION ID:(4162.57941) 2015-11-25 09:56:43.270 *** CLIENT ID:(192.168.4.221) 2015-11-25 09:56:43.270 *** SERVICE NAME:(report) 2015-11-25 09:56:43.270 *** MODULE NAME:(JDBC Thin Client) 2015-11-25 09:56:43.270 *** ACTION NAME:() 2015-11-25 09:56:43.270 =======Dumping Fail Data blocks======= Fail Block 0: RDBA: -1328289280, Diskname: o/172.16.247.110/DATA_FD02_CD_08_fd02cel06, FOB: 0x437774f78, Disknum: 0, AUNUM: 277141 Fail Block 1: RDBA: -1328288896, Diskname: o/172.16.247.110/DATA_FD02_CD_08_fd02cel06, FOB: 0x437774f78, Disknum: 0, AUNUM: 277141 Fail Block 2: RDBA: -1328252800, Diskname: o/172.16.247.110/DATA_FD02_CD_03_fd02cel06, FOB: 0x437774f78, Disknum: 2, AUNUM: 119987 Fail Block 3: RDBA: -1328252800, Diskname: o/172.16.247.110/DATA_FD02_CD_03_fd02cel06, FOB: 0x437774f78, Disknum: 2, AUNUM: 119987 Fail Block 4: RDBA: -1328252672, Diskname: o/172.16.247.110/DATA_FD02_CD_03_fd02cel06, FOB: 0x437774f78, Disknum: 2, AUNUM: 119987 Fail Block 5: RDBA: 1103997444, Diskname: o/172.16.247.110/DATA_FD02_CD_03_fd02cel06, FOB: 0x856eb2c8, Disknum: 2, AUNUM: 407353 Fail Block 6: RDBA: 1103997444, Diskname: o/172.16.247.110/DATA_FD02_CD_03_fd02cel06, FOB: 0x856eb2c8, Disknum: 2, AUNUM: 407353 Fail Block 7: RDBA: -1328289280, Diskname: o/172.16.247.110/DATA_FD02_CD_08_fd02cel06, FOB: 0x437774f78, Disknum: 0, AUNUM: 277141 Fail Block 8: RDBA: -1328289152, Diskname: o/172.16.247.110/DATA_FD02_CD_08_fd02cel06, FOB: 0x437774f78, Disknum: 0, AUNUM: 277141 Fail Block 9: RDBA: -1329264622, Diskname: o/172.16.247.106/DATA_FD02_CD_05_fd02cel02, FOB: 0x437774f78, Disknum: 0, AUNUM: 276729 Fail Block 10: RDBA: -1628508668, Diskname: o/172.16.247.106/DATA_FD02_CD_05_fd02cel02, FOB: 0x437782538, Disknum: 0, AUNUM: 417675 Fail Block 11: RDBA: -1628508544, Diskname: o/172.16.247.106/DATA_FD02_CD_05_fd02cel02, FOB: 0x437782538, Disknum: 0, AUNUM: 417675 Fail Block 12: RDBA: -1329264256, Diskname: o/172.16.247.106/DATA_FD02_CD_05_fd02cel02, FOB: 0x437774f78, Disknum: 0, AUNUM: 276729 Fail Block 13: RDBA: -1329264622, Diskname: o/172.16.247.106/DATA_FD02_CD_05_fd02cel02, FOB: 0x437774f78, Disknum: 0, AUNUM: 276729 Fail Block 14: RDBA: -1329264622, Diskname: o/172.16.247.106/DATA_FD02_CD_05_fd02cel02, FOB: 0x437774f78, Disknum: 0, AUNUM: 276729 Fail Block 15: RDBA: -1624484864, Diskname: o/172.16.247.111/DATA_FD02_CD_09_fd02cel07, FOB: 0x437776f58, Disknum: 2, AUNUM: 418098 ====End of Dumping Fail Data blocks=== =======Dumping Fail Data blocks======= Fail Block 0: RDBA: -1328289280, Diskname: o/172.16.247.110/DATA_FD02_CD_08_fd02cel06, FOB: 0x437774f78, Disknum: 0, AUNUM: 277141 Fail Block 1: RDBA: -1328288896, Diskname: o/172.16.247.110/DATA_FD02_CD_08_fd02cel06, FOB: 0x437774f78, Disknum: 0, AUNUM: 277141 Fail Block 2: RDBA: -1328252800, Diskname: o/172.16.247.110/DATA_FD02_CD_03_fd02cel06, FOB: 0x437774f78, Disknum: 2, AUNUM: 119987 Fail Block 3: RDBA: -1328252800, Diskname: o/172.16.247.110/DATA_FD02_CD_03_fd02cel06, FOB: 0x437774f78, Disknum: 2, AUNUM: 119987 Fail Block 4: RDBA: -1328252672, Diskname: o/172.16.247.110/DATA_FD02_CD_03_fd02cel06, FOB: 0x437774f78, Disknum: 2, AUNUM: 119987 Fail Block 5: RDBA: 1103997444, Diskname: o/172.16.247.110/DATA_FD02_CD_03_fd02cel06, FOB: 0x856eb2c8, Disknum: 2, AUNUM: 407353 Fail Block 6: RDBA: 1103997444, Diskname: o/172.16.247.110/DATA_FD02_CD_03_fd02cel06, FOB: 0x856eb2c8, Disknum: 2, AUNUM: 407353 Fail Block 7: RDBA: -1328289280, Diskname: o/172.16.247.110/DATA_FD02_CD_08_fd02cel06, FOB: 0x437774f78, Disknum: 0, AUNUM: 277141 Fail Block 8: RDBA: -1328289152, Diskname: o/172.16.247.110/DATA_FD02_CD_08_fd02cel06, FOB: 0x437774f78, Disknum: 0, AUNUM: 277141 Fail Block 9: RDBA: -1329264622, Diskname: o/172.16.247.106/DATA_FD02_CD_05_fd02cel02, FOB: 0x437774f78, Disknum: 0, AUNUM: 276729 Fail Block 10: RDBA: -1628508668, Diskname: o/172.16.247.106/DATA_FD02_CD_05_fd02cel02, FOB: 0x437782538, Disknum: 0, AUNUM: 417675 Fail Block 11: RDBA: -1628508544, Diskname: o/172.16.247.106/DATA_FD02_CD_05_fd02cel02, FOB: 0x437782538, Disknum: 0, AUNUM: 417675 Fail Block 12: RDBA: -1329264256, Diskname: o/172.16.247.106/DATA_FD02_CD_05_fd02cel02, FOB: 0x437774f78, Disknum: 0, AUNUM: 276729 Fail Block 13: RDBA: -1329264622, Diskname: o/172.16.247.106/DATA_FD02_CD_05_fd02cel02, FOB: 0x437774f78, Disknum: 0, AUNUM: 276729 Fail Block 14: RDBA: -1329264622, Diskname: o/172.16.247.106/DATA_FD02_CD_05_fd02cel02, FOB: 0x437774f78, Disknum: 0, AUNUM: 276729 Fail Block 15: RDBA: -1624484864, Diskname: o/172.16.247.111/DATA_FD02_CD_09_fd02cel07, FOB: 0x437776f58, Disknum: 2, AUNUM: 418098 ====End of Dumping Fail Data blocks=== *** 2015-11-25 09:56:46.861 =======Dumping Fail Data blocks======= Fail Block 0: RDBA: -1328252800, Diskname: o/172.16.247.110/DATA_FD02_CD_03_fd02cel06, FOB: 0x437774f78, Disknum: 2, AUNUM: 119987 Fail Block 1: RDBA: -1328252800, Diskname: o/172.16.247.110/DATA_FD02_CD_03_fd02cel06, FOB: 0x437774f78, Disknum: 2, AUNUM: 119987 Fail Block 2: RDBA: -1328252672, Diskname: o/172.16.247.110/DATA_FD02_CD_03_fd02cel06, FOB: 0x437774f78, Disknum: 2, AUNUM: 119987 Fail Block 3: RDBA: 1103997444, Diskname: o/172.16.247.110/DATA_FD02_CD_03_fd02cel06, FOB: 0x856eb2c8, Disknum: 2, AUNUM: 407353 Fail Block 4: RDBA: 1103997444, Diskname: o/172.16.247.110/DATA_FD02_CD_03_fd02cel06, FOB: 0x856eb2c8, Disknum: 2, AUNUM: 407353 Fail Block 5: RDBA: -1328289280, Diskname: o/172.16.247.110/DATA_FD02_CD_08_fd02cel06, FOB: 0x437774f78, Disknum: 0, AUNUM: 277141 Fail Block 6: RDBA: -1329264622, Diskname: o/172.16.247.106/DATA_FD02_CD_05_fd02cel02, FOB: 0x437774f78, Disknum: 0, AUNUM: 276729 Fail Block 7: RDBA: -1628508668, Diskname: o/172.16.247.106/DATA_FD02_CD_05_fd02cel02, FOB: 0x437782538, Disknum: 0, AUNUM: 417675 Fail Block 8: RDBA: -1628508544, Diskname: o/172.16.247.106/DATA_FD02_CD_05_fd02cel02, FOB: 0x437782538, Disknum: 0, AUNUM: 417675 Fail Block 9: RDBA: -1329264256, Diskname: o/172.16.247.106/DATA_FD02_CD_05_fd02cel02, FOB: 0x437774f78, Disknum: 0, AUNUM: 276729 Fail Block 10: RDBA: -1329264622, Diskname: o/172.16.247.106/DATA_FD02_CD_05_fd02cel02, FOB: 0x437774f78, Disknum: 0, AUNUM: 276729 Fail Block 11: RDBA: -1329264622, Diskname: o/172.16.247.106/DATA_FD02_CD_05_fd02cel02, FOB: 0x437774f78, Disknum: 0, AUNUM: 276729 Fail Block 12: RDBA: -1624484864, Diskname: o/172.16.247.111/DATA_FD02_CD_09_fd02cel07, FOB: 0x437776f58, Disknum: 2, AUNUM: 418098 ====End of Dumping Fail Data blocks=== *** 2015-11-25 09:56:47.636 =======Dumping Fail Data blocks======= Fail Block 0: RDBA: -1328252800, Diskname: o/172.16.247.110/DATA_FD02_CD_03_fd02cel06, FOB: 0x437774f78, Disknum: 2, AUNUM: 119987 Fail Block 1: RDBA: 1103997444, Diskname: o/172.16.247.110/DATA_FD02_CD_03_fd02cel06, FOB: 0x856eb2c8, Disknum: 2, AUNUM: 407353 Fail Block 2: RDBA: -1328289280, Diskname: o/172.16.247.110/DATA_FD02_CD_08_fd02cel06, FOB: 0x437774f78, Disknum: 0, AUNUM: 277141 Fail Block 3: RDBA: -1329264622, Diskname: o/172.16.247.106/DATA_FD02_CD_05_fd02cel02, FOB: 0x437774f78, Disknum: 0, AUNUM: 276729 Fail Block 4: RDBA: -1628508668, Diskname: o/172.16.247.106/DATA_FD02_CD_05_fd02cel02, FOB: 0x437782538, Disknum: 0, AUNUM: 417675 Fail Block 5: RDBA: -1628508544, Diskname: o/172.16.247.106/DATA_FD02_CD_05_fd02cel02, FOB: 0x437782538, Disknum: 0, AUNUM: 417675 Fail Block 6: RDBA: -1329264256, Diskname: o/172.16.247.106/DATA_FD02_CD_05_fd02cel02, FOB: 0x437774f78, Disknum: 0, AUNUM: 276729 Fail Block 7: RDBA: -1329264622, Diskname: o/172.16.247.106/DATA_FD02_CD_05_fd02cel02, FOB: 0x437774f78, Disknum: 0, AUNUM: 276729 Fail Block 8: RDBA: -1329264622, Diskname: o/172.16.247.106/DATA_FD02_CD_05_fd02cel02, FOB: 0x437774f78, Disknum: 0, AUNUM: 276729 Fail Block 9: RDBA: -1624484864, Diskname: o/172.16.247.111/DATA_FD02_CD_09_fd02cel07, FOB: 0x437776f58, Disknum: 2, AUNUM: 418098 ====End of Dumping Fail Data blocks=== =======Dumping Fail Data blocks======= Fail Block 0: RDBA: -1328252800, Diskname: o/172.16.247.110/DATA_FD02_CD_03_fd02cel06, FOB: 0x437774f78, Disknum: 2, AUNUM: 119987 Fail Block 1: RDBA: 1103997444, Diskname: o/172.16.247.110/DATA_FD02_CD_03_fd02cel06, FOB: 0x856eb2c8, Disknum: 2, AUNUM: 407353 Fail Block 2: RDBA: -1328289280, Diskname: o/172.16.247.110/DATA_FD02_CD_08_fd02cel06, FOB: 0x437774f78, Disknum: 0, AUNUM: 277141 Fail Block 3: RDBA: -1329264622, Diskname: o/172.16.247.106/DATA_FD02_CD_05_fd02cel02, FOB: 0x437774f78, Disknum: 0, AUNUM: 276729 Fail Block 4: RDBA: -1628508668, Diskname: o/172.16.247.106/DATA_FD02_CD_05_fd02cel02, FOB: 0x437782538, Disknum: 0, AUNUM: 417675 Fail Block 5: RDBA: -1628508544, Diskname: o/172.16.247.106/DATA_FD02_CD_05_fd02cel02, FOB: 0x437782538, Disknum: 0, AUNUM: 417675 Fail Block 6: RDBA: -1329264622, Diskname: o/172.16.247.106/DATA_FD02_CD_05_fd02cel02, FOB: 0x437774f78, Disknum: 0, AUNUM: 276729 Fail Block 7: RDBA: -1329264622, Diskname: o/172.16.247.106/DATA_FD02_CD_05_fd02cel02, FOB: 0x437774f78, Disknum: 0, AUNUM: 276729 Fail Block 8: RDBA: -1624484864, Diskname: o/172.16.247.111/DATA_FD02_CD_09_fd02cel07, FOB: 0x437776f58, Disknum: 2, AUNUM: 418098 ====End of Dumping Fail Data blocks=== *** 2015-11-25 09:56:48.971 Start of KCFIS reconnect diagnostics ---KCFIS is responsible for ExaData-optimized I/O. It is related to smart scan. Appstate: 0x7f7e9e0c82d8 Cell: 172.16.247.110/, Device: /box/predicate1600118 Printing reconnect diagnostics: app_type 1 tsn 16 objd 1674375 objn 1674375 Dumping sqltext for sqlid: 5dmxfk16wnkav kxsDumpSqlTextFromSQLID SQL= select count(*) recNum from (select rownum 序号,m.* from ((SELECT FBASE_NAME, FBASE_CODE, FORG_NAME, FORG_CODE, FBIG_NAME, FBIG_CODE, ... +++++++++++++++++ Dumping sqltext in chunks of 2048 bytes ++++++++ select count(*) recNum from (select rownum 序号,m.* from ((SELECT FBASE_NAME, FBASE_CODE, FORG_NAME, FORG_CODE, FBIG_NAME, FBIG_CODE, FSMALL_NAME, FSMALL_CODE, FDEPT_NAME, ... ... TO_CHAR(T.FTOTAL_AMOUNT, 'fm999999990.00') FTOTAL_AMOUNT FROM t_etl_papers t LEFT JOIN crm.t_cust_custbasedata CUS ON t.fnumber = cus.fcustnumber AND cus.fcuststatus <> '2' WHERE 1 = 1 and t.FDEPT_CODE IN (SELECT T.FDEPTCODE CODE FROM DPRP.V_BUSINESS_ORGANIZATION T WHERE T.FDEPTCODE = 'W011302040502' OR T.FSMALLAREACODE = 'W011302040502' OR T.FBIGAREACODE = 'W011302040502' OR T.FORGCODE = 'W011302040502' OR T.FBASECODE ='W011302040502' OR t.fbosscode='W011302040502' OR T.fex_boss_code='W011302040502' union all select w.FDEPTCODE from DPRP.v_operation_organization w WHERE w.FDEPTCODE = 'W011302040502' OR w.FSMALLAREACODE = 'W011302040502' OR w.FBIGAREACODE = 'W011302040502' OR w.FORGCODE = 'W011302040502' OR w.FBASECODE = 'W011302040502' OR w.fbosscode = 'W011302040502' OR w.fex_boss_code='W011302040502' ) and t.FDATE = to_date('2015-11-24','yyyy-mm-dd') ORDER BY t.FDEPT_CODE,T.FNUMBER,T.FCHARGE_TYPE,T.FPAYMENT_TYPE,T.FBILL_TYPE) m ) ) +++++++++++++ End of SQl dump +++++++++++++ Dumping session sid: 4162 ser: 57941 audsid: 231250857 user: 52/DPRP flags: (0x100045) USR/- flags_idl: (0x1) BSY/-/-/-/-/- flags2: (0x40009) -/-/INC pid: 359 O/S info: user: grid, term: UNKNOWN, ospid: 90598 image: oracle@fd02db01.deppon.com client details: O/S info: user: root, term: unknown, ospid: 1234 machine: fspbaobiao-221 program: JDBC Thin Client client info: 192.168.4.221 application name: JDBC Thin Client, hash value=2546894660 current SQL: select count(*) recNum from (select rownum 序号,m.* from ((SELECT FBASE_NAME, FBASE_CODE, FORG_NAME, FORG_CODE, FBIG_NAME, FBIG_CODE, FSMALL_NAME, FSMALL_CODE, FDEPT_NAME, ... *** 2015-11-25 09:56:49.621 =======Dumping Fail Data blocks======= Fail Block 0: RDBA: -1328252800, Diskname: o/172.16.247.110/DATA_FD02_CD_03_fd02cel06, FOB: 0x437774f78, Disknum: 2, AUNUM: 119987 Fail Block 1: RDBA: 1103997444, Diskname: o/172.16.247.110/DATA_FD02_CD_03_fd02cel06, FOB: 0x856eb2c8, Disknum: 2, AUNUM: 407353 Fail Block 2: RDBA: -1328289280, Diskname: o/172.16.247.110/DATA_FD02_CD_08_fd02cel06, FOB: 0x437774f78, Disknum: 0, AUNUM: 277141 Fail Block 3: RDBA: -1329264622, Diskname: o/172.16.247.106/DATA_FD02_CD_05_fd02cel02, FOB: 0x437774f78, Disknum: 0, AUNUM: 276729 Fail Block 4: RDBA: -1329264622, Diskname: o/172.16.247.106/DATA_FD02_CD_05_fd02cel02, FOB: 0x437774f78, Disknum: 0, AUNUM: 276729 Fail Block 5: RDBA: -1624484864, Diskname: o/172.16.247.111/DATA_FD02_CD_09_fd02cel07, FOB: 0x437776f58, Disknum: 2, AUNUM: 418098 ====End of Dumping Fail Data blocks=== kcfis: Exadata error: Device 'o/172.16.247.111//box/predicate' error code 215 (Cell failed to allocate memory) *** 2015-11-25 11:28:33.613 ORA-27603: Cell storage I/O error, I/O failed on disk o/172.16.247.111//box/predicate at offset 0 for data length 0 ORA-27626: Exadata error: 215 (Cell failed to allocate memory) |
3、然后通过相关报错查看文档确定是bug
-- Search -- Note: This is INTERNAL ONLY research. No action should be taken by the customer on this information. This is research only, and may NOT be applicable to your specific situation. Searched on: Cell failed to allocate memory Bug 20310116 - ORA-27603 AND ORA-27626 IN ALERT LOG Bug 18169273 - FLOOD OF ALERT WHEN CELL GOES TO PASS THROUGH MODE DUE TO NO CELL MEMORY |
4、采用打one-off-patch的方式来避免这个告警,或者忽略
=== ODM Proposed Solution === Apply one-off patch 18169273 on 11.2.0.3.28 Exadata Database. === ODM Proposed Solution Justif === Bug 18169273 is fixed in 12.1.0.2 and above. There is a one-off patch on 11.2.0.3.28 Exadata Database. These messages should not be considered buggy but only informative. === ODM Solution / Action Plan === This happens when there is a surge in smart scan sessions which need more memory to be allocated for the scan. It is not just number of smart scan sessions being more but number of smart scan sessions which need more memory are high at that point. To resolve the issue, please apply the patch 18169273. Download and review the readme and pre-requisites for Patch 18169273. Download link: https://updates.oracle.com/download/10098816.html Select a Release: Oracle 11.2.0.3.28 Exadata Database |
来自 “ ITPUB博客 ” ,链接:http://blog.itpub.net/29953799/viewspace-1973100/,如需转载,请注明出处,否则将追究法律责任。
转载于:http://blog.itpub.net/29953799/viewspace-1973100/