搭建和扩容ceph集群完整过程

最新推荐文章于 2024-10-08 09:30:10 发布

okc_champion

最新推荐文章于 2024-10-08 09:30:10 发布

阅读量694

点赞数 1

分类专栏：存储文章标签：分布式

本文链接：https://blog.csdn.net/okc_champion/article/details/124865041

版权

存储专栏收录该内容

1 篇文章 0 订阅

订阅专栏

环境准备
1. 准备三台centos7虚拟机。
2. 准备两块硬盘，一块系统盘，另一块作为ceph使用大小100G。
3. 配置IP地址
4. 主机 ip
  ceph01 172.20.20.166
  ceph02 172.20.20.167
  ceph03 172.20.20.168

主机	ip
ceph01	172.20.20.166
ceph02	172.20.20.167
ceph03	172.20.20.168

搭建集群前的配置

以下操作每个节点都要做

每个节点的修改主机名，和hosts文件

[root@ceph02 ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
172.20.20.166  ceph01
172.20.20.167  ceph02
172.20.20.168  ceph03
[root@ceph02 ~]#

关闭防火墙和核心防护

root@ceph01 ~]# systemctl stop firewalld
[root@ceph01 ~]# systemctl disable firewalld
Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
[root@ceph01 ~]# setenforce 0
[root@ceph01 ~]# vi /etc/selinux/config

三个节点创建免密登录

ssh-keygen
ssh-copy-id root@ceph01
ssh-copy-id root@ceph02
ssh-copy-id root@ceph03

配置YUM源

//安装wget命令，方便下载新的yum源。
yum install wget -y

cd /etc/yum.repos.d/
mkdir backup
mv C* backup

//用wget命令下载新的yum源。
wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo
wget -O /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-7.repo

//配置ceph源
vi /etc/yum.repos.d/ceph.repo
[ceph]
name=Ceph packages for
baseurl=https://mirrors.aliyun.com/ceph/rpm-mimic/el7/$basearch
enabled=1
gpgcheck=1
type=rpm-md
gpgkey=https://mirrors.aliyun.com/ceph/keys/release.asc
priority=1

[ceph-noarch]
name=Ceph noarch packages 
baseurl=https://mirrors.aliyun.com/ceph/rpm-mimic/el7/noarch/
enabled=1
gpgcheck=1
type=rpm-md
gpgkey=https://mirrors.aliyun.com/ceph/keys/release.asc
priority=1

[ceph-source]
name=Ceph source packages 
baseurl=https://mirrors.aliyun.com/ceph/rpm-mimic/el7/SRPMS/
enabled=1
gpgcheck=1
type=rpm-md
gpgkey=https://mirrors.aliyun.com/ceph/keys/release.asc
priority=1

安装ceph集群
1. 安装好相应的工具
  1. ```
  //ceph01节点
  yum install ceph-deploy ceph python-setuptools -y
  
  //ceph02节点
  yum install ceph python-setupto ols -y
  
  //ceph03节点
  yum install ceph python-setupto ols -y
  
  //每个节点都创建目录
  mkdir /etc/ceph 
```
2. 管理节点创建mon
  1. [root@ceph01 ceph]# ceph-deploy new ceph01 ceph02
3. 管理节点初始化，收集秘钥
  1. [root@ceph01 ceph]#ceph-deploy --overwrite-conf mon create-initial
4. 查看集群状态
  1. [root@ceph01 ceph]#ceph -s
5. 创建osd
  1. [root@ceph01 ceph]# ceph-deploy osd create --data /dev/sdb ceph01
  2. [root@ceph01 ceph]# ceph-deploy osd create --data /dev/sdb ceph02
  3. [root@ceph01 ceph]# ceph osd tree
  4. [root@ceph01 ceph]# ceph osd stat
6. 将配置文件和admin秘钥下发到节点
  1. [root@ceph01 ceph]# ceph-deploy admin ceph01 ceph02
7. 给秘钥增加权限
  1. [root@ceph01 ceph]# chmod +r ceph.client.admin.keyring
添加一个节点成为3节点的ceph集群
1. 添加ceph03节点的osd
  1. [root@ceph01 ceph]# ceph-deploy osd create --data /dev/sdb ceph03
2. vi ceph.conf
  1. [root@ceph01 ceph]# cat ceph.conf
    [global]
    fsid = 3ce96247-8fdc-405f-87e6-8bc5ac56a238
    mon_initial_members = ceph01,ceph02
    mon_host = 172.20.20.166,172.20.20.167,172.20.20.168
    auth_cluster_required = cephx
    auth_service_required = cephx
    auth_client_required = cephx
    public network = 172.20.20.0/23
    [root@ceph01 ceph]#
3. 添加ceph03节点的mon
  1. [root@ceph01 ceph]# ceph-deploy --overwrite-conf mon add ceph03
4. 重新下发秘钥和配置文件
  1. [root@ceph01 ceph]# ceph-deploy --overwrite-conf config push ceph01 ceph02 ceph03
5. 三台主机重启mon服务
  1. [root@ceph01 ceph]# systemctl list-unit-files |grep mon
  2. [root@ceph01 ceph]# systemctl restart ceph-mon.target
6. 三台主机添加mgr服务
  1. [root@ceph01 ceph]# ceph-deploy mgr create ceph01 ceph02 ceph03
7. 查看ceph集群状态和版本
  1. [root@ceph01 ceph]# ceph -s
  3. [root@ceph01 ceph]# ceph -v