ceph踩坑指南
环境
机器:centos 7.5 ceph Luminous版本 源:阿里云
ceph-deploy new node
问题:
Traceback (most recent call last):
File "/usr/bin/ceph-deploy", line 18, in <module>
from ceph_deploy.cli import main
File "/usr/lib/python2.7/site-packages/ceph_deploy/cli.py", line 1, in <module>
import pkg_resources
ImportError: No module named pkg_resources
解决办法:
python 版本问题 ,注意检查系统的python版本,此处ceph-deploy实则为通过执行python脚本来实现安装,
更改/usr/bin/ceph-deploy文件中 更改python2.6的部分为2.7(此处应当与系统python版本相同) 更改第一行文件为 #!/usr/bin/python2.7 重新执行ceph-deploy new即可
ceph-deploy disk zap node12:sdb
问题:
[ceph_deploy][ERROR ] RuntimeError: zap command needs both HOSTNAME and DISK but got "None node12:sdb"
解决办法:
# 注意中间是空格,一个是名称,一个是设备
ceph-deploy disk zap node12 /dev/sdb
ceph-deploy disk list node12
问题:
[ceph_deploy][ERROR ] Traceback (most recent call last):
[ceph_deploy][ERROR ] File "/usr/lib/python2.7/site-packages/ceph_deploy/util/decorators.py", line 69, in newfunc
[ceph_deploy][ERROR ] return f(*a, **kw)
[ceph_deploy][ERROR ] File "/usr/lib/python2.7/site-packages/ceph_deploy/cli.py", line 164, in _main
[ceph_deploy][ERROR ] return args.func(args)
[ceph_deploy][ERROR ] File "/usr/lib/python2.7/site-packages/ceph_deploy/osd.py", line 434, in disk
[ceph_deploy][ERROR ] disk_list(args, cfg)
[ceph_deploy][ERROR ] File "/usr/lib/python2.7/site-packages/ceph_deploy/osd.py", line 376, in disk_list
[ceph_deploy][ERROR ] distro.conn.logger(line)
[ceph_deploy][ERROR ] TypeError: 'Logger' object is not callable
[ceph_deploy][ERROR ]
解决办法:
待解决 这个是为了查看当前所有空闲盘用的,当拟清楚所有盘的情况时可以不执行。
[root@node12 my-cluster]# ceph-deploy disk zap node12 /dev/sdb
问题:
[ceph_deploy][ERROR ] Traceback (most recent call last):
[ceph_deploy][ERROR ] File "/usr/lib/python2.7/site-packages/ceph_deploy/util/decorators.py", line 69, in newfunc
[ceph_deploy][ERROR ] return f(*a, **kw)
[ceph_deploy][ERROR ] File "/usr/lib/python2.7/site-packages/ceph_deploy/cli.py", line 164, in _main
[ceph_deploy][ERROR ] return args.func(args)
[ceph_deploy][ERROR ] File "/usr/lib/python2.7/site-packages/ceph_deploy/osd.py", line 438, in disk
[ceph_deploy][ERROR ] disk_zap(args)
[ceph_deploy][ERROR ] File "/usr/lib/python2.7/site-packages/ceph_deploy/osd.py", line 336, in disk_zap
[ceph_deploy][ERROR ] if args.debug:
[ceph_deploy][ERROR ] AttributeError: 'Namespace' object has no attribute 'debug'
[ceph_deploy][ERROR ]
解决:
vim /usr/lib/python2.7/site-packages/ceph_deploy/osd.py
# 修改第336行为
#if args.debug:
if False:
ceph-deploy --overwrite-conf osd create node231:vdb
问题:
[ceph_deploy][ERROR ] NeedDiskError: Must supply disk/path argument: node12:sdb
解决:
在ceph luminous中创建bluestore的过程为指定data,block-db,block-wal
例如执行
# 数据日志存在不同的盘符
ceph-deploy osd create node1 --data /dev/sde --block-db /dev/sdf1 --block-wal /dev/sdf2
# 省略db 与wal的说明,只指定data则为 (创建于同一个盘)
ceph-deploy osd create node1 --data /dev/sdb
ceph -s
问题:
health: HEALTH_WARN
no active mgr
解决:
Ceph Manager Daemon,简称ceph-mgr。 该组件的主要作用是分担和扩展monitor的部分功能,减轻monitor的负担,让更好地管理ceph存储系统ceph
ceph在 luminous中新加入了mgr功能模块,手动安装mgr即可
ceph-deploy mgr create node01 node02 node03
crush rule 常用命令记录
ceph osd getcrushmap -o /tmp/crush
crushtool -d /tmp/crush -o /tmp/crush.txt
crushtool -c /tmp/crush.txt -o /tmp/crush.bin
ceph osd setcrushmap -i /tmp/crush.bin