最近linux工作站闹“机瘟”，故障处理过程-CSDN博客

一台服务器重启后smtp报警，dmesg查看有以下信息，
EXT3-fs error (device sda6) in start_transaction: Journal has aborted
EXT3-fs error (device sda6) in start_transaction: Journal has aborted
EXT3-fs error (device sda6) in start_transaction: Journal has aborted
EXT3-fs error (device sda6) in start_transaction: Journal has aborted
EXT3-fs error (device sda6) in start_transaction: Journal has aborted
EXT3-fs error (device sda6) in start_transaction: Journal has aborted
EXT3-fs error (device sda6) in start_transaction: Journal has aborted
EXT3-fs error (device sda6) in start_transaction: Journal has aborted
EXT3-fs error (device sda6) in start_transaction: Journal has aborted
EXT3-fs error (device sda6) in start_transaction: Journal has aborted
EXT3-fs error (device sda6) in start_transaction: Journal has aborted
EXT3-fs error (device sda6) in start_transaction: Journal has aborted
EXT3-fs error (device sda6) in start_transaction: Journal has aborted
EXT3-fs error (device sda6) in start_transaction: Journal has aborted
EXT3-fs error (device sda6) in start_transaction: Journal has aborted
EXT3-fs error (device sda6) in start_transaction: Journal has aborted
EXT3-fs error (device sda6) in start_transaction: Journal has aborted
EXT3-fs error (device sda6) in start_transaction: Journal has aborted
EXT3-fs error (device sda6) in start_transaction: Journal has aborted
EXT3-fs error (device sda6) in start_transaction: Journal has aborted
EXT3-fs error (device sda6) in start_transaction: Journal has aborted

明显是sda6分区文件系统损坏了，退出所有调用/dev/sda6分区的进程。可以通过以下命令查看：
fuser -vm /dev/sda6
也可通过 fuser -kvm /dev/sda6直接强制关闭调用此分区的进程

再umount /dev/sda6把分区卸载下来，要不强制修复会有问题。

开始修复：
[root@localhost ~]#fsck -C -y /dev/sda6
fsck 1.35 (28-Feb-2004)
e2fsck 1.35 (28-Feb-2004)
/mail: recovering journal
/mail contains a file system with errors, check forced.
Pass 1: Checking inodes, blocks, and sizes
Inodes that were part of a corrupted orphan linked list found. Fix? yes

Inode 11567146 was part of the orphaned inode list. FIXED.
Deleted inode 88002036 has zero dtime. Fix? yes

Inode 142868618, i_blocks is 336, should be 280. Fix? yes

Duplicate blocks found... invoking duplicate block passes.
Pass 1B: Rescan for duplicate/bad blocks

Duplicate/bad block(s) in inode 140690903: 300506198 300531891 301339371 301339372 301339373 301339374 301339375 301339376 301339377 301339378 301339388 301339389 301339955 301339981 301340019 301340020 301340021 301340083 301340084 301340085 301340086 301340087 301340088 301340089 301340090 301340091 301340092 301340093 301340094 301340095 301340096 301340097 301340098 301340099 301340100

Duplicate/bad block(s) in inode 142868618: 300506198 300531891 301339371 301339372 301339373 301339374 301339375 301339376 301339377 301339378 301339388 301339389 301339955 301339981 301340019 301340020 301340021 301340083 301340084 301340085 301340086 301340087 301340088 301340089 301340090 301340091 301340092 301340093 301340094 301340095 301340096 301340097 301340098 301340099 301340100
Pass 1C: Scan directories for inodes with dup blocks.
Pass 1D: Reconciling duplicate blocks
(There are 2 inodes containing duplicate/bad blocks.)

File /l/libotic.com.cn/l/lwg/cur/1341374309.M886694P10938V0000000000000806I000000000862C5D7_0.mail.mil6.c4e.com,S=135350:2,S (inode #140690903, mod time Wed Jul 4 11:58:30 2012)
has 35 duplicate block(s), shared with 1 file(s):
/l/lnb-ch.com/j/jian_pan/.wbaillist_inbox (inode #142868618, mod time Wed Jul 4 11:25:19 2012)
Clone duplicate/bad blocks? yes

File /l/ln-chi.com/j/jian_pan/.weailit_inbox (inode #142868618, mod time Wed Jul 4 11:25:19 2012)
has 35 duplicate block(s), shared with 1 file(s):
/l/liastic.co.c/l/lwg/cur/1341374309.M886694P10938V0000000000000806I000000000862C5D7_0.ml.ml63.c4e.com,S=135350:2,S (inode #140690903, mod time Wed Jul 4 11:58:30 2012)
Duplicated blocks already reassigned or cloned.

Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Unattached inode 153240106
Connect to /lost+found? yes

Inode 153240106 ref count is 2, should be 1. Fix? yes

Inode 153240109 ref count is 1, should be 2. Fix? yes

Unattached inode 153240112
Connect to /lost+found? yes

Inode 153240112 ref count is 2, should be 1. Fix? yes

Inode 153240116 ref count is 1, should be 2. Fix? yes

Inode 153240117 ref count is 1, should be 2. Fix? yes

Unattached inode 153240118
Connect to /lost+found? yes

Inode 153240118 ref count is 2, should be 1. Fix? yes

Inode 153240121 ref count is 1, should be 2. Fix? yes

Inode 153240123 ref count is 1, should be 2. Fix? yes

Pass 5: Checking group summary information
Block bitmap differences: -(23153596--23153599) -(51560568--51560573) +(51560580--51560581) +(51560584--51560588) -(51560589--51560591) -(51560593--51560601) -(51560604--51560606) +(51560608--51560619) +(51560648--51560659) -(51560660--51560667) -51560670 -51560724 -(51560726--51560739) +(51560740--51560747) +(51560752--51560757) -176108310 -286972966 -(301339956--301339980) -301340028 -(301340030--301340032) -(301340034--301340037) -(301340070--301340072) -390707346 -390707348 +(390707349--390707350)
Fix? yes

Free blocks count wrong for group #3 (13089, counted=13054).
Fix? yes

Free blocks count wrong for group #706 (9481, counted=9485).
Fix? yes

Free blocks count wrong for group #5374 (14575, counted=14576).
Fix? yes

Free blocks count wrong for group #8720 (0, counted=1).
Fix? yes

Free blocks count wrong for group #8757 (1, counted=2).
Fix? yes

Free blocks count wrong for group #8760 (0, counted=1).
Fix? yes

Free blocks count wrong for group #8774 (0, counted=2).
Fix? yes

Free blocks count wrong for group #9134 (0, counted=1).
Fix? yes

Free blocks count wrong for group #9196 (17050, counted=17086).
Fix? yes

Free blocks count wrong (174274933, counted=174274945).
Fix? yes

Inode bitmap differences: -11567146 -88002036
Fix? yes

Free inodes count wrong for group #706 (16314, counted=16315).
Fix? yes