Centos7搭建ceph分布式集群

Ceph架构

Ceph核心组件及概念介绍

1. Monitor：保存，同步OSD元数据 （可以是OSD中的一员充当Monitor）
2. OSD ：OSD全称Object Storage Device，也就是负责响应客户端请求返回具体数据的进程。一个Ceph集群一般都有很多个OSD。
3. MDS ：MDS全称Ceph Metadata Server，是CephFS服务依赖的元数据服务。
4. Object：Ceph最底层的存储单元是Object对象，每个Object包含元数据和原始数据。

ceph环境

主机名	IP	OS
ceph1	192.168.16.10	Centos7
ceph2	192.168.16.20	Centos7
ceph3	192.168.16.30	Centos7

添加本地解析

vim /etc/hosts
192.168.16.10 ceph1
192.168.16.11 ceph2
192.168.16.12 ceph3

关闭防火墙和selinux

systemctl stop firewalld && systemctl disable firewalld && setenforce 0
vim /etc/selinux/config
SELINUX = disabled

配置ssh互信

ssh-keygen
ssh-copy-id -i ~/.ssh/id_rsa.pub root@ceph1
ssh-copy-id -i ~/.ssh/id_rsa.pub root@ceph2
ssh-copy-id -i ~/.ssh/id_rsa.pub root@ceph3

配置yum源

cd /etc/yum.repos.d/
mkdir backup
mv C* backup

//安装wget命令，方便下载新的yum源。
yum install wget -y

//用wget命令下载新的yum源。
wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo
wget -O /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-7.repo

配置ceph源

//配置ceph源
vim /etc/yum.repos.d/ceph.repo
[ceph]
name=Ceph packages for
baseurl=https://mirrors.aliyun.com/ceph/rpm-15.2.0/el7/$basearch
enabled=1
gpgcheck=1
type=rpm-md
gpgkey=https://mirrors.aliyun.com/ceph/keys/release.asc
priority=1

[ceph-noarch]
name=Ceph noarch packages 
baseurl=https://mirrors.aliyun.com/ceph/rpm-15.2.0/el7/noarch/
enabled=1
gpgcheck=1
type=rpm-md
gpgkey=https://mirrors.aliyun.com/ceph/keys/release.asc
priority=1

[ceph-source]
name=Ceph source packages 
baseurl=https://mirrors.aliyun.com/ceph/rpm-15.2.0/el7/SRPMS/
enabled=1
gpgcheck=1
type=rpm-md
gpgkey=https://mirrors.aliyun.com/ceph/keys/release.asc
priority=1

配置ntp时间同步

yum install -y ntp
systemctl start ntpd
systemctl enable ntpd

安装ceph

//ceph01主节点
yum install ceph-deploy ceph python-setuptools -y


//ceph02、ceph03节点
yum install ceph python-setuptools -y
mkdir /etc/ceph

管理节点创建mon并初始化，收集秘钥

cd /etc/ceph
ceph-deploy new ceph01 ceph02 ceph03 	'//创建mon'
ceph-deploy mon create-initial	'//初始化，收集秘钥'

将配置文件和admin秘钥下发到节点并给秘钥增加权限

[root@ceph1 ceph]# ceph-deploy admin ceph1 ceph2 ceph3
[root@ceph1 ceph]# chmod +r ceph.client.admin.keyring 
[root@ceph2 ceph]# chmod +r ceph.client.admin.keyring 
[root@ceph3 ceph]# chmod +r ceph.client.admin.keyring 

创建osd

ceph-deploy osd create --data /dev/sdb ceph1
ceph-deploy osd create --data /dev/sdb ceph2
ceph-deploy osd create --data /dev/sdb ceph3
ceph-deploy osd create --data /dev/sdc ceph1
ceph-deploy osd create --data /dev/sdc ceph2
ceph-deploy osd create --data /dev/sdc ceph3
ceph-deploy osd create --data /dev/sdd ceph1
ceph-deploy osd create --data /dev/sdd ceph2
ceph-deploy osd create --data /dev/sdd ceph3

查看集群状态