oracle 服务器硬盘满了,【案例】Oracle服务器diag进程占据了12g的磁盘空间分析解决办法...

【案例】Oracle服务器diag进程占据了12g的磁盘空间分析解决办法

时间:2016-11-13 20:10   来源:Oracle研究中心   作者:网络   点击:

次

天萃荷净

Oracle研究中心案例分析：巡检时发现某套rac的awr居然没有生成，而且手工创建快照也没反应。发现某个节点的/oracle 一共20g，oracle本身才8G左右，另外的12g不知道被什么东西占据了，ls -ltr居然无法看见，经主机工程师检查，发现是diag进程占据了12g的磁盘空间。

在操作之前，我做了一个systemstate dump，对该dump我简单的分析了一下，发现了如下有用信息：

System global information:

processes: base c0000000ad649960, size 500, cleanup c0000000ad68c020

allocation: free sessions c0000000ad78d620, free calls 0000000000000000

control alloc errors: 0 (process), 0 (session), 0 (call)

PMON latch cleanup depth: 0

seconds since PMON's last scan for dead processes: 19  ###### 有19个死进程

###### 我们首先来搜索diag进程：######

PROCESS 3:

—————————————-

SO: c0000000ad68c810, type: 2, owner: 0000000000000000, flag: INIT/-/-/0x00

(process) Oracle pid=3, calls cur/top: c0000000adab8358/c0000000adab8358, flag: (2) SYSTEM

int error: 0, call error: 0, sess error: 0, txn error 0

(post info) last post received: 0 0 24

last post received-location: ksasnd

last process to post me: c0000000ad6956f0 235 0

last post sent: 0 0 48

last post sent-location: ksoreq_reply

last process posted by me: c0000000ad782910 1 0

(latch info) wait_event=0 bits=0

Process Group: DEFAULT, pseudo proc: c0000000ad784f20

O/S info: user: oracle, term: UNKNOWN, ospid: 27628

OSD pid info: Unix process pid: 27628, image: oracle@uamdb2 (DIAG)

###### 然后搜索c0000000ad68c810 字符串，发现如下内容：######

SO: c0000000ab2bac00, type: 19, owner: c0000000ad68c810, flag: INIT/-/-/0x00

GES MSG BUFFERS: st=emp chunk=0x0000000000000000 hdr=0x0000000000000000 lnk=0x0000000000000000 flags=0x0 inc=0

outq=0 sndq=0 opid=0 prmb=0x0

mbg[i]=(0 0) mbg[b]=(0 0) mbg[r]=(0 0)

fmq[i]=(0 0) fmq[b]=(0 0) fmq[r]=(0 0)

mop[s]=0 mop[q]=0 pendq=0 zmbq=0

nonksxp_recvs=0

————process 0xc0000000ab2bac00——————–

proc version      : 0

Local node        : 1

pid               : 27628

lkp_node          : 1

svr_mode          : 0

proc state        : KJP_UNFREEZE

Last drm hb acked : 0

Total accesses    : 5

Imm.  accesses    : 0

Locks on ASTQ     : 0

Locks Pending AST : 0

Granted locks     : 0

AST_Q:

PENDING_Q:

GRANTED_Q:

—————————————-

SO: c0000000ada72528, type: 4, owner: c0000000ad68c810, flag: INIT/-/-/0x00

(session) sid: 554 trans: 0000000000000000, creator: c0000000ad68c810, flag: (51) USR/- BSY/-/-/-/-/-

DID: 0000-0000-00000000, short-term DID: 0000-0000-00000000

txn branch: 0000000000000000

oct: 0, prv: 0, sql: 0000000000000000, psql: 0000000000000000, user: 0/SYS

service name: SYS$BACKGROUND

waiting for 'DIAG idle wait' blocking sess=0x0000000000000000 seq=331 wait_time=0 seconds since wait started=62193

component=1, where=1, wait time(millisec)=c8

Dumping Session Wait History

for 'DIAG idle wait' count=1 wait_time=195274

component=1, where=1, wait time(millisec)=c8

for 'DIAG idle wait' count=1 wait_time=195286

component=1, where=1, wait time(millisec)=c8

for 'DIAG idle wait' count=1 wait_time=214760

component=1, where=1, wait time(millisec)=c8

for 'DIAG idle wait' count=1 wait_time=195267

component=1, where=1, wait time(millisec)=c8

for 'DIAG idle wait' count=1 wait_time=195282

component=1, wherhttp://www.oracleplus.nete=1, wait time(millisec)=c8

for 'DIAG idle wait' count=1 wait_time=195282

component=1, where=1, wait time(millisec)=c8

for 'DIAG idle wait' count=1 wait_time=195281

component=1, where=1, wait time(millisec)=c8

for 'DIAG idle wait' count=1 wait_time=195278

component=1, where=1, wait time(millisec)=c8

for 'DIAG idle wait' count=1 wait_time=195280

component=1, where=1, wait time(millisec)=c8

for 'DIAG idle wait' count=1 wait_time=195270

component=1, where=1, wait time(millisec)=c8

temporary object counter: 0

—————————————-

UOL used : 0 locks(used=0, free=0)

KGX Atomic Operation Log c0000000acb7a5a0

Mutex 0000000000000000(0, 0) idn 0 oper NONE

Library Cache uid 554 efd 0 whr 0 slp 0

KGX Atomic Operation Log c0000000acb7a5e8

Mutex 0000000000000000(0, 0) idn 0 oper NONE

Library Cache uid 554 efd 0 whr 0 slp 0

KGX Atomic Operation Log c0000000acb7a630

Mutex 0000000000000000(0, 0) idn 0 oper NONE

Library Cache uid 554 efd 0 whr 0 slp 0

—————————————-

SO: c0000000a1a7fa18, type: 41, owner: c000