Unable to mount or check XFS filesystem

环境

• Red Hat Enterprise Linux 5.7 or later
• Red Hat Enterprise Linux 6
• Red Hat Enterprise Linux 7
• XFS file system

问题

• In Red Hat Enterprise Linux 6, an xfs filesystem hung and running xfs_check gave the following error:

Raw

# xfs_check /dev/sdm1
ERROR: The filesystem has valuable metadata changes in a log which needs to
be replayed. Mount the filesystem to replay the log, and unmount it before
re-running xfs_check. If you are unable to mount the filesystem, then use
the xfs_repair -L option to destroy the log and attempt a repair.
Note that destroying the log may cause corruption -- please attempt a mount
of the filesystem before doing this.

NOTE: Older xfsprogs packages contained the xfs_check tool. This tool is very slow and does not scale well for large filesystems. As such, it has been deprecated in favor of xfs_repair -n.

• In the above scenario, the filesystem would also not mount:

Raw

# mount /dev/sdm1 /new/
mount: Structure needs cleaning
# mount -f /dev/sdm1 /new/
/dev/sdm1 100G 49G 46G 52% /new

• The last resort was running xfs_repair -L which was able to cause the filesystem to mount and work.

Raw

kernel: XFS internal error XFS_WANT_CORRUPTED_GOTO at line 1572 of file fs/xfs/xfs_alloc.c.  Caller 0xffffffff88871969
kernel: 
kernel: Call Trace:
kernel:  [<ffffffff8886fe2f>] :xfs:xfs_free_ag_extent+0x19e/0x67e
kernel:  [<ffffffff88871969>] :xfs:xfs_free_extent+0xa9/0xc9
kernel:  [<ffffffff8887a755>] :xfs:xfs_bmap_finish+0xee/0x15f
kernel:  [<ffffffff88899927>] :xfs:xfs_itruncate_finish+0x185/0x2af
kernel:  [<ffffffff888b1001>] :xfs:xfs_inactive+0x1de/0x40f
kernel:  [<ffffffff888bb8c1>] :xfs:xfs_fs_clear_inode+0xa4/0xeb
kernel:  [<ffffffff80022eee>] clear_inode+0xd2/0x123
kernel:  [<ffffffff8002f3ec>] generic_delete_inode+0xde/0x143
kernel:  [<ffffffff8003c716>] do_unlinkat+0xd5/0x141
kernel:  [<ffffffff8005d229>] tracesys+0x71/0xe0
kernel:  [<ffffffff8005d28d>] tracesys+0xd5/0xe0
kernel: 
kernel: xfs_force_shutdown(sdm1,0x8) called from line 4269 of file fs/xfs/xfs_bmap.c.  Return address = 0xffffffff8887a78a
kernel: Filesystem "sdm1": Corruption of in-memory data detected.  Shutting down filesystem: sdm1
kernel: Please umount the filesystem, and rectify the problem(s)
kernel: nfsd: non-standard errno: 5

决议

• The remedy is to xfs_repair -L to discard the log and repair the filesystem, which also reconstructs the free space trees on-disk.
• Then, remount the filesystem.

根源

• Corruption has occurred in a free space btree (some unknown time ago), but this has only been detected now when you tried to delete a file and free an extent.

• Also check your storage integrity (though no scsi errors seems to be present in the logs) - this can be caused by all sorts of conditions outside of XFS itself (storage errors, or memory corruption, etc).

诊断步骤

• Examine syslog for the signature messages, as per Issue section: XFS_WANT_CORRUPTED_GOTO at line 1572