Ceph基础之搭建ceph集群

• 初始环境配置
  配置四台虚拟机，设置好主机名，关防火墙，关selinux，设置/etc/hosts，做ssh互信，时间同步等。

• 配置源
  在所有ceph节点上准备下面的源。
  1、下载wget工具

  yum install -y wget
  
  2、清除yum缓存

  yum clean all
  
  3、删除国外yum源

  rm -rf /etc/yum.repos.d/*.repo
  
  4、下载阿里centos源
  wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo
  
  5、下载阿里epel源

  wget -O /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-7.repo
  
  6、修改源

  sed -i ‘/aliyuncs/d’ /etc/yum.repos.d/CentOS-Base.repo
  sed -i ‘/aliyuncs/d’ /etc/yum.repos.d/epel.repo
  sed -i ‘s/$releasever/7/g’ /etc/yum.repos.d/CentOS-Base.repo
  
  7、设置ceph源

  vi /etc/yum.repos.d/ceph.repo
  [ceph]
  name=ceph
  baseurl=http://mirrors.163.com/ceph/rpm-jewel/el7/x86_64/
  gpgcheck=0
  [ceph-noarch]
  name=cephnoarch
  baseurl=http://mirrors.163.com/ceph/rpm-jewel/el7/noarch/
  gpgcheck=0

8、更新源

yum update -y

• 操作系统配置
  1:无密码访问配置
  在deploy节点(ceph-node1)上执行。配置deploy节点和其他ceph节点之间的无密码访问。

  sudo su -
  ssh-keygen
  

  ssh-copy-id root@ceph-node2
  
  ssh-copy-id root@cloud
  2:启用Ceph monitor OSD端口，在所有ceph节点上执行
  firewall-cmd --zone=public --add-port=6789/tcp --permanent
  firewall-cmd --zone=public --add-port=6800-7100/tcp --permanent
  firewall-cmd --reload
  firewall-cmd --zone=public --list-all

  3:禁用Selinux，在所有ceph节点上执行

setenforce 0

4:安装ntp，在所有ceph节点上执行
yum install ntp ntpdate -y
systemctl restart ntpdate.service
systemctl restart ntpd.service
systemctl enable ntpd.service ntpdate.service

• 部署ceph集群
  1:安装ceph-deploy

yum install ceph-deploy -y

2:用Ceph-deploy创建Ceph集群

mkdir /etc/ceph
cd /etc/ceph

ceph-deploy new ceph-node1

生成一个新的ceph集群，集群包括ceph配置文件以及monitor的密钥环。

3:安装ceph二进制软件包

ceph-deploy install --no-adjust-repos ceph-node1 ceph-node2

4:修改ceph 配置文件

[global]
fsid = 7bac6963-0e1d-4cea-9e2e-f02bbae96ba7
mon_initial_members = ceph-node1
mon_host = 192.168.1.101
auth_cluster_required = cephx
auth_service_required = cephx
auth_client_required = cephx
public network = 192.168.1.0/24

5:在ceph-node1上创建第一个ceph monitor

ceph-deploy mon create-initial

6:在ceph-node1上创建OSD
ceph-deploy disk list ceph-node1
(列出disk）

ceph-deploy disk zap ceph-node1:sdb ceph-node1:sdc ceph-node1:sdd

ceph-deploy osd create ceph-node1:sdb ceph-node1:sdc ceph-node1:sdd

7:用Ceph-deploy在ceph-node2上创建monitor
ceph-deploy mon create ceph-node2

ceph –s

ceph mon stat

8:在ceph-node2上创建OSD
ceph-deploy disk zap ceph-node2:sdb ceph-node2:sdc ceph-node2:sdd
ceph-deploy osd create ceph-node2:sdb ceph-node2:sdc ceph-node2:sdd
9:调整rbd存储池的pg_num和pgp_num
ceph osd pool set rbd pg_num 256
ceph osd pool set rbd pgp_num 256