背景
一年前由于本地备份空间不够,加入了一个存储上面的磁盘(LUN),这样就造成了备份用的这个VG是本地磁盘和存储磁盘混用了,最近通过监控发现存储其中一个控制器存在故障,初步诊断为光纤模块存在问题,这个时候如果另外一个控制器出现故障就会导致整个存储不能用,防止单点故障需要停存储进行更换控制器,在维护过程中我们只把存储停了,但是AIX主机并没有关机,当存储启来后我们重启了操作系统
AIX操作系统版本:
# oslevel -s
6100-06-01-1043
存储类型
IBM DS5200
问题描述和诊断过程
1.当重启操作系统后发现/backup挂载不上,手动挂载报错如下:
bash-3.00# mount /backup
Replaying log for /dev/backuplv.
Fatal: I/O error
mount: 0506-324 Cannot mount /dev/backuplv on /backup: The media is not formatted or the format is not correct.
0506-342 The superblock on /dev/backuplv is dirty. Run a full fsck to fix.
bash-3.00#
2.根据上面提示手动执行fsck后问题还是一样,查看对应用的LV,再根据对应的LV查看对应的磁盘,发现其中一磁盘,也就是存储上面的一个磁盘状态为missing
bash-3.00# more /etc/filesystems
/backup:
dev = /dev/backuplv
vfs = jfs2
log = /dev/loglv00
mount = true
options = rw
account = false
bash-3.00# lsvg -l backupvg
backupvg:
LV NAME TYPE LPs PPs PVs LV STATE MOUNT POINT
backuplv jfs2 3600 3600 3 open/syncd /backup
loglv00 jfs2log 1 1 1 open/syncd N/A
bash-3.00# lspv|grep backupvg
hdisk2 00f6886135525b61 backupvg active
hdisk3 00f6886135525c3b backupvg active
hdisk20 00f6886132cfde40 backupvg missing
bash-3.00# lspv hdisk20
PHYSICAL VOLUME: hdisk20 VOLUME GROUP: backupvg
PV IDENTIFIER: 00f6886132cfde40 VG IDENTIFIER 00f6886100004c000000012f35525ca0
PV STATE: missing
STALE PARTITIONS: 0 ALLOCATABLE: yes
PP SIZE: 256 megabyte(s) LOGICAL VOLUMES: 0
TOTAL PPs: 1459 (373504 megabytes) VG DESCRIPTORS: 1
FREE PPs: 1459 (373504 megabytes) HOT SPARE: no
USED PPs: 0 (0 megabytes) MAX REQUEST: 256 kilobytes
FREE DISTRIBUTION: 292..292..291..292..292
USED DISTRIBUTION: 00..00..00..00..00
MIRROR POOL: None
bash-3.00#
3.试图使用smit chpv把Physical volume STATE从no active改为active,执行过程中没有报错,但是执行成功后,状态还是没有变,说明这个方法行不通
上面执行成功了,状态和原来一样,说明这个方法无效
bash-3.00# lspv hdisk20
PHYSICAL VOLUME: hdisk20 VOLUME GROUP: backupvg
PV IDENTIFIER: 00f6886132cfde40 VG IDENTIFIER 00f6886100004c000000012f35525ca0
PV STATE: missing
STALE PARTITIONS: 0 ALLOCATABLE: yes
PP SIZE: 256 megabyte(s) LOGICAL VOLUMES: 0
TOTAL PPs: 1459 (373504 megabytes) VG DESCRIPTORS: 1
FREE PPs: 1459 (373504 megabytes) HOT SPARE: no
USED PPs: 0 (0 megabytes) MAX REQUEST: 256 kilobytes
FREE DISTRIBUTION: 292..292..291..292..292
USED DISTRIBUTION: 00..00..00..00..00
MIRROR POOL: None
bash-3.00#
解决方案
因为这个目录都是备份用的,里面数据不是很重要,所以打算重建,我想一定有不需要破坏原数据的就能把这个磁盘改为active的方法,先记录一下,等有空再找一下
重建步骤如下:
先使用smit rmfs把backup文件和LV删除再把这个这个磁盘从backupvg中卸载,最后把这个磁盘添加到backupvg中去,以下为操作日志
bash-3.00# smit reducevg
Remove a Physical Volume from a Volume Group
Type or select values in entry fields.
Press Enter AFTER making all desired changes.
[Entry Fields]
* VOLUME GROUP name backupvg
* PHYSICAL VOLUME names [hdisk20] +
FORCE deallocation of all partitions on no +
this physical volume?
F1=Help F2=Re+------------------------------------------------------+ F4=List
Esc+5=Reset Esc+6| Running command... | Esc+8=Image
Esc+9=Shell Esc+0+------------------------------------------------------+
COMMAND STATUS
Command: running stdout: no stderr: no
Before command completion, additional instructions may appear below.
Reduce a Volume Group
Move cursor to desired item and press Enter.
Remove a Physical Volume from a Volume Group
Remove a Volume Group
F1=Help F2=Refresh F3=Cancel Esc+8=Image
Esc+9=Shell Esc+0=Exit Enter=Do
bash-3.00# lspv
hdisk0 00f68861b9e00f54 rootvg active
hdisk1 00f688611026530f rootvg active
hdisk2 00f6886135525b61 backupvg active
hdisk3 00f6886135525c3b backupvg active
hdisk20 00f6886132cfde40 None
bash-3.00# smit extendvg
Add a Physical Volume to a Volume Group
Type or select values in entry fields.
Press Enter AFTER making all desired changes.
[Entry Fields]
Force the creation of a volume group? no +
* VOLUME GROUP name [backupvg] +
* PHYSICAL VOLUME names [hdisk20] +
COMMAND STATUS
Command: running stdout: yes stderr: no
Before command completion, additional instructions may appear below.
0516-1398 /usr/sbin/extendvg: The physical volume hdisk20, appears to belong to
another volume group. Use the force option to add this physical volume
to a volume group.
0516-792 /usr/sbin/extendvg: Unable to extend volume group.
Add a Physical Volume to a Volume Group
Type or select values in entry fields.
Press Enter AFTER making all desired changes.
[Entry Fields]
Force the creation of a volume group? yes
* VOLUME GROUP name [backupvg]
* PHYSICAL VOLUME names [hdisk20]
bash-3.00# lspv hdisk20
PHYSICAL VOLUME: hdisk20 VOLUME GROUP: backupvg
PV IDENTIFIER: 00f6886132cfde40 VG IDENTIFIER 00f6886100004c000000012f35525ca0
PV STATE: active
STALE PARTITIONS: 0 ALLOCATABLE: yes
PP SIZE: 256 megabyte(s) LOGICAL VOLUMES: 0
TOTAL PPs: 1459 (373504 megabytes) VG DESCRIPTORS: 1
FREE PPs: 1459 (373504 megabytes) HOT SPARE: no
USED PPs: 0 (0 megabytes) MAX REQUEST: 256 kilobytes
FREE DISTRIBUTION: 292..292..291..292..292
USED DISTRIBUTION: 00..00..00..00..00
MIRROR POOL: None
bash-3.00#
总结
在维护存储时,如果有服务器挂载了这些磁盘,务必停掉服务器或者umount相关的文件系统再进行操作
来自 “ ITPUB博客 ” ,链接:http://blog.itpub.net/21605631/viewspace-755840/,如需转载,请注明出处,否则将追究法律责任。
转载于:http://blog.itpub.net/21605631/viewspace-755840/