cinder对接两套ceph存储

生产环境中，可能会遇见一套openstack对接多套ceph的情况，具体步骤如下：
1：首先在ceph的mon节点，为第二套ceph添加用户认证，如下;

ceph auth get-or-create client.volume mon 'allow *'  osd 'allow class-read object_prefix rbd_children, allow * pool=volumes, allow * pool=vms, allow * pool=images'
[client.volume]          ####以下为返回的key值
 key = AQBJ/1ZfEsgHKxAAw/In1l/oIYghkl7+SyHipg==

2：修改cinder-volume文件，添加如下配置：

[DEFAULT]
enabled_backends = ceph1,ceph2
[ceph2]
volume_driver = cinder.volume.drivers.rbd.RBDDriver
volume_backend_name = ceph2
rbd_pool = volumes2
rbd_ceph_conf = /etc/ceph/ceph2.conf
rbd_flatten_volume_from_snapshot = false
rbd_max_clone_depth = 5
rados_connect_timeout = -1
glance_api_version = 2
rbd_user = volume
rbd_secret_uuid =  22003ebb-0f32-400e-9584-fa90b6efd874   ####此uuid为手动生成并在libvirt里面注册

注释：拷贝ceph2.conf到cinder_volume目录下。
[root@ceph1 cinder-volume]# ll
total 24
-rw-r----- 1 root root  669 Sep  8 16:34 ceph2.conf
-rw------- 1 root root   67 Sep  8 09:46 ceph.client.cinder.keyring
-rw------- 1 root root   67 Sep  8 16:34 ceph.client.volume.keyring
-rw-rw---- 1 root root  669 Sep  8 09:46 ceph.conf
-rw-rw---- 1 root root 2768 Sep  8 09:47 cinder.conf
-rw-rw---- 1 root root 1215 Sep  8 09:46 config.json
[root@ceph1 cinder-volume]# 

3：在计算节点执行生成uuid，并在libvirt注册

(nova-libvirt)[root@compute1 /]# uuidgen
bb7873a4-2d06-4742-8b82-e8956d5ae333
定义secret.xml文件
cat > secret.xml <<EOF
<secret ephemeral='no' private='no'>
<uuid>bb7873a4-2d06-4742-8b82-e8956d5ae333</uuid>  ###替换实际的uuid
<usage type='ceph'>
<name>client.volume secret</name>  ##替换在ceph中创建的用户
</usage>
</secret>
EOF
virsh secret-define --file secret.xml   ###注册secret__uuid
查看注册进的volume用户
(nova-libvirt)[root@compute1 /]# virsh secret-list
 UUID                                  Usage
--------------------------------------------------------------------------------
 22003ebb-0f32-400e-9584-fa90b6efd874  ceph client.volume secret ##
 96c020b5-bc43-4513-8085-a547487bf5fe  ceph client.cinder secret
 e5d162eb-03ae-4ba9-89e6-cfa6cc3b0c64  ceph client.nova secret
设置secret的value值
virsh secret-set-value  --secret  22003ebb-0f32-400e-9584-fa90b6efd874 --base64 AQBJ/1ZfEsgHKxAAw/In1l/oIYghkl7+SyHipg== 
注释：替换实际的uuid和key值

4：生成钥匙环key文件

ceph auth get client.volume -o /etc/ceph/ceph.client.volume.keyring
并拷贝到cinder-volume节点的cinder目录下

5：重启cinder_volume服务

docker restart cinder_volume
创建ceph的volume_type
cinder type-create ceph2
cinder type-key ceph2 set volume_backend_name=ceph2