最近处理的一个故障

异常现象特别多，主要有如下：

1、onstat -g ckp 部分检查点点时间明显变长

2、sysadmin中的一个任务导致了长事务

3、重启数据库之后有一个create index的会话，但是看不到是什么样的DDL语句；

4、数据库频繁的抛出af文件，里面大致如下，基本上都指向某个相对复杂的SQL语句：

14:03:16  Context ptr = 0x0000000001cc36c0
14:03:16  Siginfo ptr = 0x0000000001cc37f0
14:03:16  Reg  0(R8)      0x0000000190f49220 Reg  1(R9)      0x00000002047d6720
14:03:16  Reg  2(R10)     0x0000000000000000 Reg  3(R11)     0x0000000000000000
14:03:16  Reg  4(R12)     0x0000000000000000 Reg  5(R13)     0x0000000000000000
14:03:16  Reg  6(R14)     0x0000000000000000 Reg  7(R15)     0x00000001bb7bdf00
14:03:16  Reg  8(RDI)     0x0000000000000000 Reg  9(RSI)     0x00000001bb7bdf00
14:03:16  Reg 10(RBP)     0x0000000000000000 Reg 11(RBX)     0x0000000000000000
14:03:16  Reg 12(RDX)     0x00000001e23a53a8 Reg 13(RAX)     0x0000000000000000
14:03:16  Reg 14(RCX)     0x00000001bb7bdff0 Reg 15(RSP)     0x00000001bb7bde90
14:03:16  Reg 16(RIP)     0x0000000001580ccb Reg 17(EFL)     0x0000000000010246
14:03:16  Reg 18(CSGSFS)  0x0000000000000033 Reg 19(ERR)     0x0000000000000004
14:03:16  Reg 20(TRAPNO)  0x000000000000000e Reg 21(OLDMASK) 0x0000000000000000
14:03:16  Reg 22(CR2)     0x0000000000000008
14:03:16  SHM Globals and Master Pool/Master Block Adresses:

14:03:16  shmcb =           0x0000000045f89a08
14:03:16  rhead =           0x0000000045ff5800
14:03:16  pool list =       0x0000000045f89ae0
14:03:16  block pool list = 0x0000000045fef2e8
14:03:16  TRANSP =          0x0000000102342ae8
14:03:16  PARTP =           0x000000015d1a22f0
14:03:16  PARTNP =          0x000000015d1a2400
14:03:16  OPENP =           0x00000001460d9028
14:03:16  FILEP =           0x0000000166903e10
14:03:16  Raw hex dump of stack located in /opt/informix/tmp/af.12b8c5a3.rawstk
14:03:16  Stack for thread: 3792 sqlexec

 base: 0x00000001bb3c2000
  len:   4198400
   pc: 0x0000000001392326
  tos: 0x0000000001cc2e10
state: running
   vp: 12

0x0000000001392326 (oninit) afstack
0x000000000139956c (oninit) mt_ex_throw_sig
0x0000000001338503 (oninit) afsig_handler
0x000000375100f710 (Linux) <signal frame>
0x0000000001580ccb (oninit) new_eval
0x00000000007f9bf8 (oninit) sort_open
0x00000000007d7292 (oninit) prepselect
0x00000000008507cc (oninit) subqprep
0x00000000007d32b9 (oninit) scan_next
0x00000000007d9d2e (oninit) getrow
0x00000000007dabb1 (oninit) doopen
0x000000000084f16b (oninit) doopen_allany
0x00000000008509f7 (oninit) exsubqm
0x000000000085101e (oninit) subqcmp
0x0000000001581636 (oninit) ev_anyall
0x0000000001580cce (oninit) new_eval
0x00000000007d11b7 (oninit) gettupl
0x00000000007d3563 (oninit) scan_next
0x0000000000b1e0c9 (oninit) next_row
0x0000000000b1ea21 (oninit) get_first_row_from_producer
0x0000000000b210c6 (oninit) hash_process_all_groups
0x0000000000b22a66 (oninit) group_open
0x0000000000b2265b (oninit) group_open
0x00000000007feb05 (oninit) merge_open
0x00000000007feb05 (oninit) merge_open
0x00000000007c322a (oninit) filltemp
0x00000000007cad76 (oninit) scan_open
0x00000000007b9652 (oninit) materialize_viewtmp
0x00000000007b9596 (oninit) materialize_viewtmp
0x00000000007b9527 (oninit) materialize_viewtmp
0x00000000007b95ae (oninit) materialize_viewtmp
0x00000000007b9596 (oninit) materialize_viewtmp
0x00000000007b9596 (oninit) materialize_viewtmp
0x00000000007d70f1 (oninit) prepselect
0x00000000009e25ea (oninit) open_cursor
0x00000000009e5926 (oninit) sql_open
0x00000000009e5ac6 (oninit) sq_open
0x0000000000aaae6b (oninit) sqmain
0x0000000001476169 (oninit) spawn_thread
0x000000000138f470 (oninit) th_init_initgls
0x0000000001358cd7 (oninit) startup

 ucontext: 0x0000000001cc36c0
  siginfo: 0x0000000001cc37f0
           si_signo: 11  si_code: 1   si_errno: 0 si_pid: 8 si_uid: 0 
           si_addr: 0x0000000000000008 si_value: 0x0000000000000000  si_fd: 0

5、抛出了的一个af文件有几十个索引异常；

 

处理过程：

1、关闭schedluer

2、重建索引

3、重启数据库

4、oncheck -cc检查catalog table

5、发现数据库重启之后会有一个重建索引的过程，这个太讨厌了！！！