ceph-ansible安装ceph

1. 准备机器， 1台做ansible, 另外3台安装ceph

http://docs.ceph.com/ceph-ansible/master/#demos

2.  在安装了ansible的机器上, vi /etc/hosts. 加三台机器的hostname

10.0.221.100 ceph1
10.0.221.101 ceph2
10.0.221.102 ceph3

3. 添加秘钥

ssh-keygen -t rsa

ssh-copy-id root@ceph1

ssh-copy-id root@ceph2

ssh-copy-id root@ceph3

4. 安装ceph-ansible

git clone https://github.com/ceph/ceph-ansible.git

pip install -r requirements.txt

5. 配置文件设置

vi nodes.yml

[mons]
10.0.221.100
10.0.221.101
10.0.221.102

[osds]
10.0.221.100
10.0.221.101
10.0.221.102

[mgrs]
10.0.221.100
10.0.221.101
10.0.221.102
==========================================

cp group_vars/all.yml.sample group_vars/all.yml

vi group_vars/all.yml

cluster: ceph                                #集群名
ceph_origin: repository                          #使用distro，则不会使用公网源
ceph_repository: community                       #被操作节点使用本地的repo文件
ceph_stable_release: luminous                #安装版本
public_network: "10.0.221.0/24"
cluster_network: "10.0.221.0/24"
mon_host: 10.0.221.100,10.0.221.101,10.0.221.102
monitor_interface: ens160
devices:                                     #OSD磁盘，有几块写几块
  - '/dev/sdb'
osd_scenario: collocated
 

========================================

cp site.yml.sample site.yml

6. ceph 安装

ansible-playbook site.yml -i nodes.yml

7. ceph -s 检查

如果osd未up,

chown ceph:ceph /dev/sdb1

chown ceph:ceph /dev/sdb2

systemctl restart ceph-osd@2.service

如果restart有问题，
systemctl reset-failed ceph-osd@2.service 再 restart

8 检查

ceph osd tree

 ceph -s
ceph osd pool create kube 128 128
ceph osd pool ls
ceph osd pool ls detail

netstat -anp | grep 6789

 rbd list kube
 rbd create kube/vol1 --size 1000

9.  ceph dashboard, 在ceph mon的机器上

  ceph mgr module enable dashboard
  netstat -antpl | grep ceph-mgr|grep LISTE
  access http://10.0.221.100:7000

10. enable ceph-rest-api，在ceph mon的机器上

  ceph-rest-api -n client.admin &
  netstat -anp | grep 5000 

https://ceph.com/category/ceph-rest-api/

https://blog.csdn.net/hit1944/article/details/46551753

可以通过5000 访问restapi. 

 11. 一个基于ceph-rest-api的dashboard

docker run -d --net=host tobegit3hub/ceph-web

access http://10.0.221.100:8080/

12. openattic存储管理，安装openattic
 sudo add-apt-repository "deb http://apt.openattic.org/ $(lsb_release -cs) main"
  sudo apt-get install openattic
  sudo apt-get install openattic-gui
  oaconfig install
  apt-get install openattic-module-drbd
  mkdir /etc/ceph
  cd /etc/ceph 

  copy /etc/ceph/ceph.conf and /etc/ceph/ceph.client.admin.keyring to this dir from ceph mon node
  chgrp openattic /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring
  chmod g+r /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring
  apt-get install openattic-module-ceph
  apt-get install openattic-filter
   vi /etc/dbconfig-common/openattic.conf 
   vi ceph.conf 
   oaconfig install

  access http://10.0.221.103/openattic

reference:

https://docs.openattic.org/en/latest/install_guides/accessing_the_webui.html

https://blog.csdn.net/qq_23348071/article/details/76222955

13, 安装inkscope, ceph dashboard, 需要安装到ceph mon node上

  git clone https://github.com/inkscope/inkscope-ansible.git
  cd inkscope-ansible/
  ansible-galaxy install -r requirements.yaml

vi host

[inkscope]
10.0.221.100

vi inkscope.yaml 
vi vars/playbook.yaml 

vi vars/my_inkscope.yaml 

ansible-playbook inkscope.yaml -i host

access http://10.0.221.100/inkscopeViz

refer to https://github.com/inkscope/inkscope-ansible

             https://github.com/inkscope/inkscope

14. k8s 接ceph:

https://www.cnblogs.com/chimeiwangliang/p/8386353.html

15. k8s的rook(ceph)

https://blog.csdn.net/liukuan73/article/details/84664922

https://blog.csdn.net/hxpjava1/article/details/80175238