一 摘要
本文基于三台物理机部署ceph ,物理机都安装了centos7.6.1810,三台物理机品牌不一样,硬盘配置不一样。
二 环境信息及部署规划
(一)操作系统版本
[root@localhost ~]# cat /etc/centos-release
CentOS Linux release 7.6.1810 (Core)
[root@localhost ~]#
(二)ceph 版本
(三)物理机信息
2.3.1 机器配置
品牌 | 物理机配置 |
---|---|
Huashan Technologies R4900 G2 | Intel® Xeon® CPU E5-2650 v4 @ 2.20GHz*2;内存:128G;硬盘:5块硬盘大小未标明,总大小4.4T |
Sugon I620-G20 | Intel® Xeon® CPU E5-2650 v4 @ 2.20GHz*2;内存:224G;硬盘:7块硬盘大小未标明,总大小4.4T+558.4G |
浪潮 SA5212M5 | Intel Silver 4210 2.2G * 2 ;内存:总256G 32G*8;硬盘:240G SSD * 2 ;8T/SATA/7.2K * 6;网卡:网卡 X710 10G * 1 |
2.3.2机器raid 方式
系统盘用raid1,数据盘若raid 卡支持单盘raid0,就做单盘raid0,不支持 就是单盘volumn,不要把几张数据盘放在一起做raid0.
(四)部署规划
| 主机名 | IP |磁盘 |角色|
| ---- | ---- | ---- | ---- | ---- |—|
| cephtest001.ceph.kxdigit.com | 10.3.176.10 | 系统盘:/dev/sda 数据盘:/dev/sdb /dev/sdc /dev/sdd |ceph-deploy,monitor,mgr,mds,osd|
| cephtest002.ceph.kxdigit.com | 10.3.176.16 | 系统盘:/dev/sda 数据盘:/dev/sdb /dev/sdc /dev/sdd /dev/sde /dev/sdf |monitor,mgr,mds,osd|
| cephtest003.ceph.kxdigit.com | 10.3.176.44 | 系统盘:/dev/sda 数据盘:/dev/sdb /dev/sdc /dev/sdd /dev/sde /dev/sdf /dev/sdg |monitor,mgr,mds,osd|
四 部署
(一)基础安装
4.1.1 配置dns(若内网无dns系统,请跳过该步骤)
CENTOS7 修改dns
4.1.1.1首先新增/etc/resolv.conf
[dev@10-3-170-32 centos]$ cat resolv.conf
# Generated by NetworkManager
nameserver 10.3.157.201
[dev@10-3-170-32 centos]$
4.1.1.2其次是修改/etc/NetworkManager/NetworkManager.conf
[dev@10-3-170-32 NetworkManager]$ cat NetworkManager.conf
# Configuration file for NetworkManager.
#
# See "man 5 NetworkManager.conf" for details.
#
# The directory /etc/NetworkManager/conf.d/ can contain additional configuration
# snippets. Those snippets override the settings from this main file.
#
# The files within conf.d/ directory are read in asciibetical order.
#
# If two files define the same key, the one that is read afterwards will overwrite
# the previous one.
[main]
plugins=ifcfg-rh
dns=none
[logging]
#level=DEBUG
#domains=ALL
[dev@10-3-170-32 NetworkManager]$
4.1.1.3 最后是重启NETWORKD
systemctl restart NetworkManager
4.1.1.4
我这边都封装到ansible 脚本里
- hosts: 00base
tasks:
- name: register var
command: date +%Y-%m-%d_%H-%M-%S
register: datetime
- name: display vars
debug: msg="{
{datetime.stdout}}"
- name: check /etc/resolv.conf
shell: ls /etc/resolv.conf
register: resolvmsg
ignore_errors: true
- name: file not exist
when: resolvmsg is failed #调试文件不存在
debug: msg="resolv.conf is not found..."
- name: backup old dns file
when: resolvmsg is success
command: cp /etc/resolv.conf /etc/resolv.conf.bak."{
{datetime.stdout}}"
- name: backup old
command: cp /etc/NetworkManager/NetworkManager.conf /etc/NetworkManager/NetworkManager.conf.bak."{
{datetime.stdout}}"
- name: copy intranet resolv.conf to server
copy:
src: "{
{ item.src }}"
dest: "{
{ item.dest }}"
with_items:
- { src: "/home/dev/ansible/files/centos/resolv.conf", dest: "/etc/resolv.conf" }
- name: copy intranet networkmanager.conf to server
copy:
src: "{
{ item.src }}"
dest: "{
{ item.dest }}"
with_items:
- { src: "/home/dev/ansible/files/centos7/NetworkManager.conf", dest: "/etc/NetworkManager/NetworkManager.conf" }
when: ansible_distribution == "CentOS" and ansible_distribution_major_version == "7"
- name: restart NetworkManager
command: systemctl restart NetworkManager
when: ansible_distribution == "CentOS" and ansible_distribution_major_version == "7"
- name: copy intranet networkmanager.conf to server
copy:
src: "{
{ item.src }}"
dest: "{
{ item.dest }}"
with_items:
- { src: "/home/dev/ansible/files/centos8/NetworkManager.conf", dest: "/etc/NetworkManager/NetworkManager.conf" }
when: ansible_distribution == "CentOS" and ansible_distribution_major_version == "8"
- name: restart NetworkManager
command: systemctl restart NetworkManager
when: ansible_distribution == "CentOS" and ansible_distribution_major_version == "8"
- name: test ansible own variable
debug: msg="OS ansible_distribution {
{ ansible_distribution }} major version {
{ ansible_distribution_major_version }}"
4.1.2 配置yum
配置内网yum,方便后面安装依赖。
4.1.2.1 内网yum 搭建
搭建内网yum 此处略。后续专门出专门文章。
4.1.2.2配置yum
修改该目录下
/etc/yum.repos.d 文件即可
4.1.2.3 ansible 脚本
我这边也对yum 源更新编写了脚本
- hosts: 00base
tasks:
- name: register var
command: date +%Y-%m-%d_%H-%M-%S
register: datetime
- name: display vars
debug: msg="{
{datetime.stdout}}"
- name: backup old yum file
command: tar -zcvf /etc/yum.repos.d."{
{datetime.stdout}}".tgz /etc/yum.repos.d
- name: list the files of /etc/yum.repos.d
shell: ls
args:
chdir: /etc/yum.repos.d
register: files_list
- name: remove files
file:
path: /etc/yum.repos.d/{
{ item }}
state: absent
with_items:
- "{
{ files_list.stdout_lines }}"
when: ansible_distribution == "CentOS" and ansible_distribution_major_version == "7"
- name: remove files
file:
path: /etc/yum.repos.d/{
{ item }}
state: absent
with_items:
- "{
{ files_list.stdout_lines }}"
when: ansible_distribution == "CentOS" and ansible_distribution_major_version == "8"
- name: copy intranet yum to 731611 version server
copy:
src: "{
{ item.src }}"
dest: "{
{ item.dest }}"
with_items:
- { src: "/home/dev/ansible/files/yumconfig/CentOS-73.repo", dest: "/etc/yum.repos.d/CentOS-73.repo" }
when: ansible_distribution == "CentOS" and ansible_distribution_version == "7.3"
- name: copy intranet yum to 771908 version server
copy:
src: "{
{ item.src }}"
dest: "{
{ item.dest }}"
with_items:
- { src: "/home/dev/ansible/files/yumconfig/CentOS-77.repo", dest: "/etc/yum.repos.d/CentOS-77.repo" }
when: ansible_distribution == "CentOS" and ansible_distribution_version == "7.7"
- name: copy intranet yum to 771908 version server
copy:
src: "{
{ item.src }}"
dest: "{
{ item.dest }}"
with_items:
- { src: "/home/dev/ansible/files/yumconfig/CentOS-76.repo", dest: "/etc/yum.repos.d/CentOS-76.repo" }
when: ansible_distribution == "CentOS" and ansible_distribution_version == "7.6"
- name: copy intranet yum to 7 version server
copy:
src: "{
{ item.src }}"
dest: "{
{ item.dest }}"
with_items:
- { src: "/home/dev/ansible/files/yumconfig/openstack.repo", dest: "/etc/yum.repos.d/openstack.repo" }
- { src: "/home/dev/ansible/files/yumconfig/