问题
我遇到这个问题是因为在之前安装了ceph-0.94.6的环境中重新安装ceph-10.2.5,只是卸载了ceph的包,没有对ceph的一些配置文件进行删除。
[root@node-1 tmp]# ceph-deploy new node-1
[ceph_deploy.conf][DEBUG ] found configuration file at: /root/.cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (1.5.37): /usr/bin/ceph-deploy new node-1
[ceph_deploy.cli][INFO ] ceph-deploy options:
[ceph_deploy.cli][INFO ] username : None
[ceph_deploy.cli][INFO ] func : <function new at 0x152d398>
[ceph_deploy.cli][INFO ] verbose : False
[ceph_deploy.cli][INFO ] overwrite_conf : False
[ceph_deploy.cli][INFO ] quiet : False
[ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0x159a368>
[ceph_deploy.cli][INFO ] cluster : ceph
[ceph_deploy.cli][INFO ] ssh_copykey : True
[ceph_deploy.cli][INFO ] mon : ['node-1']
[ceph_deploy.cli][INFO ] public_network : None
[ceph_deploy.cli][INFO ] ceph_conf : None
[ceph_deploy.cli][INFO ] cluster_network : None
[ceph_deploy.cli][INFO ] default_release : False
[ceph_deploy.cli][INFO ] fsid : None
[ceph_deploy.new][DEBUG ] Creating new cluster named ceph
[ceph_deploy.new][INFO ] making sure passwordless SSH succeeds
[node-1][DEBUG ] connected to host: node-1
[node-1][DEBUG ] detect platform information from remote host
[node-1][DEBUG ] detect machine type
[node-1][DEBUG ] find the location of an executable
[node-1][INFO ] Running command: /usr/sbin/ip link show
[node-1][INFO ] Running command: /usr/sbin/ip addr show
[node-1][DEBUG ] IP addresses found: [u'192.168.122.182']
[ceph_deploy.new][DEBUG ] Resolving host node-1
[ceph_deploy.new][DEBUG ] Monitor node-1 at 192.168.122.182
[ceph_deploy.new][DEBUG ] Monitor initial members are ['node-1']
[ceph_deploy.new][DEBUG ] Monitor addrs are ['192.168.122.182']
[ceph_deploy.new][DEBUG ] Creating a random mon key...
[ceph_deploy.new][DEBUG ] Writing monitor keyring to ceph.mon.keyring...
[ceph_deploy.new][DEBUG ] Writing initial config to ceph.conf...
[root@node-1 tmp]# ceph-deploy --overwrite-conf mon create-initial
[ceph_deploy.conf][DEBUG ] found configuration file at: /root/.cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (1.5.37): /usr/bin/ceph-deploy --overwrite-conf mon create-initial
[ceph_deploy.cli][INFO ] ceph-deploy options:
[ceph_deploy.cli][INFO ] username : None
[ceph_deploy.cli][INFO ] verbose : False
[ceph_deploy.cli][INFO ] overwrite_conf : True
[ceph_deploy.cli][INFO ] subcommand : create-initial
[ceph_deploy.cli][INFO ] quiet : False
[ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0x1a66b00>
[ceph_deploy.cli][INFO ] cluster : ceph
[ceph_deploy.cli][INFO ] func : <function mon at 0x1a60230>
[ceph_deploy.cli][INFO ] ceph_conf : None
[ceph_deploy.cli][INFO ] default_release : False
[ceph_deploy.cli][INFO ] keyrings : None
[ceph_deploy.mon][DEBUG ] Deploying mon, cluster ceph hosts node-1
[ceph_deploy.mon][DEBUG ] detecting platform for host node-1 ...
[node-1][DEBUG ] connected to host: node-1
[node-1][DEBUG ] detect platform information from remote host
[node-1][DEBUG ] detect machine type
[node-1][DEBUG ] find the location of an executable
[ceph_deploy.mon][INFO ] distro info: CentOS Linux 7.2.1511 Core
[node-1][DEBUG ] determining if provided host has same hostname in remote
[node-1][DEBUG ] get remote short hostname
[node-1][DEBUG ] deploying mon to node-1
[node-1][DEBUG ] get remote short hostname
[node-1][DEBUG ] remote hostname: node-1
[node-1][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf
[node-1][DEBUG ] create the mon path if it does not exist
[node-1][DEBUG ] checking for done path: /var/lib/ceph/mon/ceph-node-1/done
[node-1][DEBUG ] create a done file to avoid re-doing the mon deployment
[node-1][DEBUG ] create the init path if it does not exist
[node-1][INFO ] Running command: systemctl enable ceph.target
[node-1][INFO ] Running command: systemctl enable ceph-mon@node-1
[node-1][INFO ] Running command: systemctl start ceph-mon@node-1
[node-1][INFO ] Running command: ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.node-1.asok mon_status
[node-1][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory
[node-1][WARNIN] monitor: mon.node-1, might not be running yet
[node-1][INFO ] Running command: ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.node-1.asok mon_status
[node-1][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory
[node-1][WARNIN] monitor node-1 does not exist in monmap
[node-1][WARNIN] neither `public_addr` nor `public_network` keys are defined for monitors
[node-1][WARNIN] monitors may not be able to form quorum
[ceph_deploy.mon][INFO ] processing monitor mon.node-1
[node-1][DEBUG ] connected to host: node-1
[node-1][DEBUG ] detect platform information from remote host
[node-1][DEBUG ] detect machine type
[node-1][DEBUG ] find the location of an executable
[node-1][INFO ] Running command: ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.node-1.asok mon_status
[node-1][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory
[ceph_deploy.mon][WARNIN] mon.node-1 monitor is not yet in quorum, tries left: 5
[ceph_deploy.mon][WARNIN] waiting 5 seconds before retrying
[ceph_deploy][ERROR ] KeyboardInterrupt
解决方法
检查防火墙有没有关闭
iptables -F
getenforce
setenforce 0
删除之前版本ceph残留的文件
rm -rf /etc/ceph/*
rm -rf /var/lib/ceph/*/*
rm -rf /var/log/ceph/*
rm -rf /var/run/ceph/*
[root@node-1 tmp]# ceph-deploy new node-1
[ceph_deploy.conf][DEBUG ] found configuration file at: /root/.cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (1.5.37): /usr/bin/ceph-deploy new node-1
[ceph_deploy.cli][INFO ] ceph-deploy options:
[ceph_deploy.cli][INFO ] username : None
[ceph_deploy.cli][INFO ] func : <function new at 0xe5b398>
[ceph_deploy.cli][INFO ] verbose : False
[ceph_deploy.cli][INFO ] overwrite_conf : False
[ceph_deploy.cli][INFO ] quiet : False
[ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0xec8368>
[ceph_deploy.cli][INFO ] cluster : ceph
[ceph_deploy.cli][INFO ] ssh_copykey : True
[ceph_deploy.cli][INFO ] mon : ['node-1']
[ceph_deploy.cli][INFO ] public_network : None
[ceph_deploy.cli][INFO ] ceph_conf : None
[ceph_deploy.cli][INFO ] cluster_network : None
[ceph_deploy.cli][INFO ] default_release : False
[ceph_deploy.cli][INFO ] fsid : None
[ceph_deploy.new][DEBUG ] Creating new cluster named ceph
[ceph_deploy.new][INFO ] making sure passwordless SSH succeeds
[node-1][DEBUG ] connected to host: node-1
[node-1][DEBUG ] detect platform information from remote host
[node-1][DEBUG ] detect machine type
[node-1][DEBUG ] find the location of an executable
[node-1][INFO ] Running command: /usr/sbin/ip link show
[node-1][INFO ] Running command: /usr/sbin/ip addr show
[node-1][DEBUG ] IP addresses found: [u'192.168.122.182']
[ceph_deploy.new][DEBUG ] Resolving host node-1
[ceph_deploy.new][DEBUG ] Monitor node-1 at 192.168.122.182
[ceph_deploy.new][DEBUG ] Monitor initial members are ['node-1']
[ceph_deploy.new][DEBUG ] Monitor addrs are ['192.168.122.182']
[ceph_deploy.new][DEBUG ] Creating a random mon key...
[ceph_deploy.new][DEBUG ] Writing monitor keyring to ceph.mon.keyring...
[ceph_deploy.new][DEBUG ] Writing initial config to ceph.conf...
[root@node-1 tmp]# ceph-deploy mon create-initial
[ceph_deploy.conf][DEBUG ] found configuration file at: /root/.cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (1.5.37): /usr/bin/ceph-deploy mon create-initial
[ceph_deploy.cli][INFO ] ceph-deploy options:
[ceph_deploy.cli][INFO ] username : None
[ceph_deploy.cli][INFO ] verbose : False
[ceph_deploy.cli][INFO ] overwrite_conf : False
[ceph_deploy.cli][INFO ] subcommand : create-initial
[ceph_deploy.cli][INFO ] quiet : False
[ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0x2778b00>
[ceph_deploy.cli][INFO ] cluster : ceph
[ceph_deploy.cli][INFO ] func : <function mon at 0x2772230>
[ceph_deploy.cli][INFO ] ceph_conf : None
[ceph_deploy.cli][INFO ] default_release : False
[ceph_deploy.cli][INFO ] keyrings : None
[ceph_deploy.mon][DEBUG ] Deploying mon, cluster ceph hosts node-1
[ceph_deploy.mon][DEBUG ] detecting platform for host node-1 ...
[node-1][DEBUG ] connected to host: node-1
[node-1][DEBUG ] detect platform information from remote host
[node-1][DEBUG ] detect machine type
[node-1][DEBUG ] find the location of an executable
[ceph_deploy.mon][INFO ] distro info: CentOS Linux 7.2.1511 Core
[node-1][DEBUG ] determining if provided host has same hostname in remote
[node-1][DEBUG ] get remote short hostname
[node-1][DEBUG ] deploying mon to node-1
[node-1][DEBUG ] get remote short hostname
[node-1][DEBUG ] remote hostname: node-1
[node-1][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf
[node-1][DEBUG ] create the mon path if it does not exist
[node-1][DEBUG ] checking for done path: /var/lib/ceph/mon/ceph-node-1/done
[node-1][DEBUG ] done path does not exist: /var/lib/ceph/mon/ceph-node-1/done
[node-1][INFO ] creating keyring file: /var/lib/ceph/tmp/ceph-node-1.mon.keyring
[node-1][DEBUG ] create the monitor keyring file
[node-1][INFO ] Running command: ceph-mon --cluster ceph --mkfs -i node-1 --keyring /var/lib/ceph/tmp/ceph-node-1.mon.keyring --setuser 167 --setgroup 167
[node-1][DEBUG ] ceph-mon: mon.noname-a 192.168.122.182:6789/0 is local, renaming to mon.node-1
[node-1][DEBUG ] ceph-mon: set fsid to 7c4acee4-333a-4ee8-ac2f-6dbd05d29f10
[node-1][DEBUG ] ceph-mon: created monfs at /var/lib/ceph/mon/ceph-node-1 for mon.node-1
[node-1][INFO ] unlinking keyring file /var/lib/ceph/tmp/ceph-node-1.mon.keyring
[node-1][DEBUG ] create a done file to avoid re-doing the mon deployment
[node-1][DEBUG ] create the init path if it does not exist
[node-1][INFO ] Running command: systemctl enable ceph.target
[node-1][INFO ] Running command: systemctl enable ceph-mon@node-1
[node-1][INFO ] Running command: systemctl start ceph-mon@node-1
[node-1][INFO ] Running command: ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.node-1.asok mon_status
[node-1][DEBUG ] ********************************************************************************
[node-1][DEBUG ] status for monitor: mon.node-1
[node-1][DEBUG ] {
[node-1][DEBUG ] "election_epoch": 3,
[node-1][DEBUG ] "extra_probe_peers": [],
[node-1][DEBUG ] "monmap": {
[node-1][DEBUG ] "created": "2018-01-02 08:17:13.035137",
[node-1][DEBUG ] "epoch": 1,
[node-1][DEBUG ] "fsid": "7c4acee4-333a-4ee8-ac2f-6dbd05d29f10",
[node-1][DEBUG ] "modified": "2018-01-02 08:17:13.035137",
[node-1][DEBUG ] "mons": [
[node-1][DEBUG ] {
[node-1][DEBUG ] "addr": "192.168.122.182:6789/0",
[node-1][DEBUG ] "name": "node-1",
[node-1][DEBUG ] "rank": 0
[node-1][DEBUG ] }
[node-1][DEBUG ] ]
[node-1][DEBUG ] },
[node-1][DEBUG ] "name": "node-1",
[node-1][DEBUG ] "outside_quorum": [],
[node-1][DEBUG ] "quorum": [
[node-1][DEBUG ] 0
[node-1][DEBUG ] ],
[node-1][DEBUG ] "rank": 0,
[node-1][DEBUG ] "state": "leader",
[node-1][DEBUG ] "sync_provider": []
[node-1][DEBUG ] }
[node-1][DEBUG ] ********************************************************************************
[node-1][INFO ] monitor: mon.node-1 is running
[node-1][INFO ] Running command: ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.node-1.asok mon_status
[ceph_deploy.mon][INFO ] processing monitor mon.node-1
[node-1][DEBUG ] connected to host: node-1
[node-1][DEBUG ] detect platform information from remote host
[node-1][DEBUG ] detect machine type
[node-1][DEBUG ] find the location of an executable
[node-1][INFO ] Running command: ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.node-1.asok mon_status
[ceph_deploy.mon][INFO ] mon.node-1 monitor has reached quorum!
[ceph_deploy.mon][INFO ] all initial monitors are running and have formed quorum
[ceph_deploy.mon][INFO ] Running gatherkeys...
[ceph_deploy.gatherkeys][INFO ] Storing keys in temp directory /tmp/tmp2TpoFD
[node-1][DEBUG ] connected to host: node-1
[node-1][DEBUG ] detect platform information from remote host
[node-1][DEBUG ] detect machine type
[node-1][DEBUG ] get remote short hostname
[node-1][DEBUG ] fetch remote file
[node-1][INFO ] Running command: /usr/bin/ceph --connect-timeout=25 --cluster=ceph --admin-daemon=/var/run/ceph/ceph-mon.node-1.asok mon_status
[node-1][INFO ] Running command: /usr/bin/ceph --connect-timeout=25 --cluster=ceph --name mon. --keyring=/var/lib/ceph/mon/ceph-node-1/keyring auth get client.admin
[node-1][INFO ] Running command: /usr/bin/ceph --connect-timeout=25 --cluster=ceph --name mon. --keyring=/var/lib/ceph/mon/ceph-node-1/keyring auth get client.bootstrap-mds
[node-1][INFO ] Running command: /usr/bin/ceph --connect-timeout=25 --cluster=ceph --name mon. --keyring=/var/lib/ceph/mon/ceph-node-1/keyring auth get client.bootstrap-osd
[node-1][INFO ] Running command: /usr/bin/ceph --connect-timeout=25 --cluster=ceph --name mon. --keyring=/var/lib/ceph/mon/ceph-node-1/keyring auth get client.bootstrap-rgw
[ceph_deploy.gatherkeys][INFO ] Storing ceph.client.admin.keyring
[ceph_deploy.gatherkeys][INFO ] Storing ceph.bootstrap-mds.keyring
[ceph_deploy.gatherkeys][INFO ] keyring 'ceph.mon.keyring' already exists
[ceph_deploy.gatherkeys][INFO ] Storing ceph.bootstrap-osd.keyring
[ceph_deploy.gatherkeys][INFO ] Storing ceph.bootstrap-rgw.keyring
[ceph_deploy.gatherkeys][INFO ] Destroy temp directory /tmp/tmp2TpoFD