[PROJECT] DEPLOY CLUSTER

最新推荐文章于 2023-09-06 00:02:07 发布
最新推荐文章于 2023-09-06 00:02:07 发布
阅读量423 收藏
点赞数
分类专栏： PROJECT 文章标签： PCC
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/IDreamsComeTrue/article/details/89162687
版权

Kubernetes Cluster DEPLOY [UAT]

1 Environment Preparation

1.1 All Nodes Setup Root Login

vim /etc/ssh/sshd_config

---------------------------
PermitRootLogin yes

systemctl restart sshd.service

1.2 All Nodes Disk Mount


fdisk -l

pvs
pvcreate /dev/vdb
pvs

vgs
vgextend vg00 /dev/vdb
vgs

lvs
lvcreate -l 50%free -n lv_data vg00
lvs
mkfs.xfs /dev/vg00/lv_data

mkdir /data


mkdir /data
mount /dev/vg00/lv_data /data



cat >> /etc/fstab << EOF
/dev/vg00/lv_data      /data                    xfs     defaults        0 0
EOF

mount -a
reboot

df -h




pvcreate /dev/vdb
vgextend vg00 /dev/vdb

lvcreate -l 50%free -n lv_data vg00
lvcreate -l 100%free -n lv_pcc vg00

mkfs.xfs /dev/vg00/lv_data
mkfs.xfs /dev/vg00/lv_pcc
mkdir /data
mount /dev/vg00/lv_data /data
mkdir /pcc
mount /dev/vg00/lv_pcc /pcc

cat >> /etc/fstab << EOF
/dev/vg00/lv_data      /data                    xfs     defaults        0 0
/dev/vg00/lv_pcc       /pcc                     xfs     defaults        0 0
EOF
mount -a
reboot



pvcreate /dev/vdb
vgcreate vg00 /dev/vdb
lvcreate -l 100%free -n LVdata vg00
mkfs.xfs /dev/vg00/LVdata
mkdir /data
echo "/dev/vg00/LVdata  /data  xfs  defaults        0 0" >> /etc/fstab
mount -a
把data，换成要挂载的目录名字，然后把这些命令粘进去就行了


--------------------------------------------------------------------
pvcreate /dev/vdb
vgextend vg00 /dev/vdb
lvcreate -l 50%Free -n lv_data vg00
lvcreate -L 50G -n lv_data vg00
mkfs -t xfs /dev/vg00/lv_data
mkfs -t xfs /dev/vg00/lv_docker

#创建目录并挂载
mkdir /data
mkdir /var/lib/docker
mount /dev/vg00/lv_data /data
mount /dev/vg00/lv_docker /var/lib/docker

#开机自动挂载
cat >> /etc/fstab/ << EOF
/dev/vg00/lv_data      /data                    xfs     defaults        0 0
/dev/vg00/lv_docker    /data                    xfs     defaults        0 0
EOF

1.3 All Nodes SSH Login

# check ssh-keygen
ls  /root/.ssh/ -l

#################
# create ssh-keygen
# 一路回车
#################
ssh-keygen


# master and worker node: host-ip
ssh-copy-id xxx.xxx.xxx.xxx(master and worker node host-ip)

# deploy-node: host-ip
ssh-copy-id xxx.xxx.xxx.xxx(deploy-node host-ip)

# loging into the machine
ssh 'xxx.xxx.xxx.xxx'

# exit machine
exit

2. Deploy-Node Ready

2.1 Copy the Required File to Deploy Node

# Deploy-Node
cd /pcc
mkdir breeze
cd breeze
rz
  • 拷贝离线包到Deploy-Node
docker-rpm.tar.gz    		#包含docker、docker-compose离线rpm包，解压后为文件夹./rpms/
deploy-image.tar           	#包含部署程序docker镜像
docker-compose.yml  		#拉起部署程序的yml文件

2.2 Install docker & docker-compose

  • Decompression docker yum rpm bkg
cd /pcc/breeze

# Unzip to get ./rpms/
tar -zxvf docker-rpm.tar.gz
  • 配置本地yum源
cd /etc/yum.repo.d/
ll

cat >> /etc/yum.repos.d/docker-local.repo << EOF
[docker]
name=docker-rpm
baseurl=file:///pcc/breeze/rpms/
gpgcheck=0
enabled=1
EOF

##########################
# 注意事项：baseurl 配置路径为解压的路径
  • install docker & docker-compose
# yum ready
yum clean all && yum makecache

# local-install docker & docker-compose
yum install -y docker docker-compose

# set start
systemctl start docker && systemctl enable docker
# 加载docker镜像到本地
docker load -i deploy-image.tar

# 查看镜像
docker images
#
docker-compose up -d 
# 查看容器
docker ps -a

3 Deploy K8S Cluster

3.1 创建集群

3.2 添加主机

3.3 添加组件

3.4 Deploy K8S Cluster

  • Verify cluster status
# Master-Node
kubectl get nodes

# Master-node: 查看启动的容器
docker ps -a

3.5 Check Node status

# Deploy-Node Reset
kubeadm reset

# View deploy-main logs
docker logs -f deploy-main

# Master Node: View nodes
kubectl get cs

# Master Node: View nodes
kubectl get nodes

# Master Node: View pods
kubectl -n kube-system get pods -o wide

# View Plugin: Master-Node
cd /etc/kubernetes/manifests
ll
kube-apiserver.yaml
kube-controller-manager.yaml
kube-scheduler.yaml

# restart docker
systemctl restart docker

# View daemon.json
cat /etc/docker/daemon.json

# View docker config
cd /etc/sysconfig/
ll | grep docker

# View Docker
docker ps 
docker ps -a
# View deploy-main logs
docker logs -f deploy-main

4 Add Delpoy-Node to Cluster

最后通过breeze部署程序将Deploy节点加入到K8S Cluster，问题解决如下

4.1 hostname

  • Modify hostname
# Deploy-Node
hostnamectl set-hostname xxx

# 断开重连
exit

# 查看主机名
hostname
  • Modify hosts
# 查看 /etc/hosts 下的配置
# Deploy-Node Modify hosts 参考其他节点配置hosts
cat /etc/hosts

4.2 configure yum source

  • configure yum source
# deploy-node
cd /etc/yum.repo.d/
ls

mkdir /pcc/ori
mv docker-local.repo /pcc/ori/

# master-node or other worker-node except deploy-node
cd  /etc/yum.repos.d/
ls
scp wise2c.repo deploy-node-ip:/etc/yum.repos.d/

# 执行下面命令，如果没错，说明yum源配置没有问题
yum makecache

4.3 Only Add delpoy-node to cluster

4.4 [Error] & [solution]

[Error] NotReady

  • [Error] 完成后查看node状态为NotReady
kubectl get nodes
  • [solution]
    新增的node节点docker为手动安装，缺失配置文件，从任意节点scp配置文件到新的节点即可
# /etc/docker/daemon.json
cd /etc/docker/
ls
cat daemon.json

#  /etc/cni file floder
cd /etc/cni/net.d/
ls
cat /etc/cni/net.d/flannel.conflist

# worker-node or master-node
cd /etc/docker/
scp daemon.json xxx.xxx.xxx.xxx:/etc/docker/

cd /etc
scp -r cni/ xxx.xxx.xxx.xxx:/etc/

scp daemon.json 100.70.88.97:/etc/docker
scp -r cni/ 100.70.88.97:/etc

[Error] docker配置冲突问题

  • [solution] 清空以下4个配置文件内容
[root@nodecti2 ~]# cd /etc/sysconfig/
[root@nodecti2 sysconfig]# ll | grep docker
cd /etc/sysconfig/
ll | grep docker
-rw-r--r--  1 root root    1 Mar 25 20:05 docker
-rw-r--r--  1 root root    0 Mar 25 20:06 
-rw-r--r--  1 root root    1 Mar 25 20:05 docker-storage
-rw-r--r--  1 root root    0 Mar 25 20:06 docker-storage-setup

cat docker
cat docker-network
cat docker-storage
cat docker-storage-setup

cp docker /pcc/ori
cp docker-netword /pcc/ori
cp docker-storage /pcc/ori
cp docker-storage-setup /pcc/ori

echo > docker
echo > docker-network
echo > docker-storage
echo > docker-storage-setup

示例 echo > docker-network

  • 最后重启docker服务，查看你node状态是否为Ready
# deploy-node restart docker
systemctl restart docker

# master node or other node except deploy-node
kubectl get nodes

4.5 Verify cluster status

4.5.1 Check Node status

# Deploy-Node Reset
kubeadm reset

# View deploy-main logs
docker logs -f deploy-main

# Master Node: View nodes
kubectl get cs

# Master Node: View nodes
kubectl get nodes

# Master Node: View pods
kubectl -n kube-system get pods -o wide

# View Plugin: Master-Node
cd /etc/kubernetes/manifests
ll
kube-apiserver.yaml
kube-controller-manager.yaml
kube-scheduler.yaml

# restart docker
systemctl restart docker

# View daemon.json
cat /etc/docker/daemon.json

# View docker config
cd /etc/sysconfig/
ll | grep docker

# View Docker
docker ps 
docker ps -a
# View deploy-main logs
docker logs -f deploy-main

4.5.2 Master

# 
cd /var/tmp/wise2c/kubernetes/
cat kubeadm.conf


# View Plugin: Master-Node
cd /etc/kubernetes/manifests
ls
kube-apiserver.yaml
kube-controller-manager.yaml
kube-scheduler.yaml

# check node info
kubectl describe node node-name

# check pod logs
kubectl -n kube-system logs -f pod-name


# check pod status: Running
kubectl get pod -n kube-system -o wide
NAME	READY	STATUS	RESTARTS	AGE	IP	NODE	NOMINATED-NODE

# Plugin: Running
coredns(one of the master-nodes)
kube-apiserver-master
kube-controller-manager-master
kube-scheduler-master
kube-flannel(master & worker)
kube-proxy(master & worker)
kubernetes-dashboard(one of the master-nodes)

# delete node
kubectl delete node-name
  • View cluster status
# 查看集群状态：
kubectl get cs     #正常状态为healthy
# 查看节点状态：
kubectl get nodes  #正常状态为ready
# 查看集群系统组件pod状态：
kubectl -n kube-system get pod   #pod状态应该全部为running

4.5.3 Access Harbor 仓库

浏览器输入registry节点IP
默认用户名密码admin/Harbor12345

Deploy APP

Common troubleshooting

# deploy-node 部署日志查看
docker logs -f deploy-main

# 登录容器内部
docker exec -it deploy-main sh

Wise2c



vim /etc/


# delete /app and extend root 
umount /app/
lvremove vg00 app 

lvextend -L +40G /dev/vg00/root
xfs_growfs /dev/vg00/root



# extend var 
lvextend -L +180G /dev/vg00/var
xfs_growfs /dev/vg00/var



mkdir /data
lvcreate -l 100%Free -n lv_data vg00

mkfs.xfs /dev/vg00/lv_data
mount /dev/vg00/lv_data /data


# 开机挂载df 

# 删除

vim /etc/fstab
/dev/mapper/vg00-app    /app                    xfs     defaults        0 0


# 添加

cat >> /etc/fstab << EOF
/dev/vg00/lv_data      /data                    xfs     defaults        0 0
EOF


mount -a 
reboot



cat >> /etc/yum.repos.d/docker-ce-local.repo << EOF
[docker]
name=docker-local-repo
baseurl=file:///breeze/rpms/
gpgcheck=0
enabled=1
EOF


#构建本地安装源
yum clean all && yum makecache

#安装docker和docker-compose
yum install -y docker docker-compose

#启动docker服务并设为开机启动
systemctl start docker && systemctl enable docker


docker load -i docker-image.tar
docker images

docker-compose up -d
docker ps


ssh-keygen   一路回车
ls  /root/.ssh/ -l


 # 登陆容器内部： docker exec -it deploy-main sh  ，  exit 退出容器
 
 
 
 添加最后一个node节点
 
 breeze 仅勾选kubernetes组件    勾选just add new worker nodes, do notreinstall this cluster
 不选择master节点，仅选择最后一个node节点，最后点击开始安装
 
 
 完成后查看node状态为notready
 kubectl get nodes
 
 解决步骤
 1.新增的node节点手动安装docker,缺失两项配置文件
  从任意node节点scp配置到新节点即可
 /etc/docker/daemon.json
 /etc/cni
 

 scp daemon.json 100.67.33.10:/etc/docker
 scp -r cni/ 100.67.33.10:/etc
 
 
 2.另外需要解决docker配置冲突问题
清空以下4个配置文件内容
[root@nodecti2 ~]# cd /etc/sysconfig/
[root@nodecti2 sysconfig]# ll | grep docker
-rw-r--r--  1 root root    1 Mar 25 20:05 docker
-rw-r--r--  1 root root    0 Mar 25 20:06 
-rw-r--r--  1 root root    1 Mar 25 20:05 docker-storage
-rw-r--r--  1 root root    0 Mar 25 20:06 docker-storage-setup
示例 echo > docker-network

最后重启docker服务，查看你node状态恢复为Running

A. SIP-PROXY安装

1. localrpm ready

  • bzip2-1.0.6-13.el7.x86_64.rpm
  • docker-rpm.tar.gz

docker local install

cat >> /etc/yum.repos.d/docker-ce-local.repo << EOF
[docker]
name=docker-local-repo
baseurl=file:///breeze/rpms/
gpgcheck=0
enabled=1
EOF

# 刷新缓存
yum clean all && yum makecache

#安装docker和docker-compose
yum install -y docker docker-compose

#启动docker服务并设为开机启动
systemctl start docker && systemctl enable docker

docker load -i docker-image.tar
docker images


docker load -i fsdeb.tar
  • 安装记录
    1  [2019-03-28 17:29:19] vim /etc/ssh/sshd_config 
    2  [2019-03-28 17:30:06] cp /etc/ssh/sshd_config /etc/ssh/sshd_config.ori
    3  [2019-03-28 17:30:08] vim /etc/ssh/sshd_config 
    4  [2019-03-28 17:31:04] systemctl restart sshd
    5  [2019-03-28 17:35:06] cd /etc/ssh
    6  [2019-03-28 17:35:07] ls
    7  [2019-03-28 17:35:14] cp sshd_config sshd_config.ori
    8  [2019-03-28 17:35:18] cp sshd_config sshd_config.ori2
    9  [2019-03-28 17:35:21] vi sshd_config
   10  [2019-03-28 17:35:30] ifconfig
   11  [2019-03-28 17:35:39] systemctl restart sshd
   12  [2019-03-29 14:07:24] ls
   13  [2019-03-29 14:07:29] cd /app
   14  [2019-03-29 14:07:30] ls
   15  [2019-03-29 14:07:54] cat >> /etc/yum.repos.d/docker-ce-local.repo << EOF
[docker]
name=docker-local-repo
baseurl=file:///breeze/rpms/
gpgcheck=0
enabled=1
EOF

   16  [2019-03-29 14:08:17] vim /etc/yum.repos.d/docker-ce-local.repo 
   17  [2019-03-29 14:09:32] ls
   18  [2019-03-29 14:09:34] cd localrpm/
   19  [2019-03-29 14:09:35] ls
   20  [2019-03-29 14:09:51] tar -zxvf docker-rpm.tar.gz 
   21  [2019-03-29 14:10:02] ls
   22  [2019-03-29 14:10:14] vim /etc/yum.repos.d/docker-ce-local.repo 
   23  [2019-03-29 14:10:28] yum clean all && yum makecache
   24  [2019-03-29 14:11:00] yum install -y docker docker-compose
   25  [2019-03-29 14:11:59] systemctl start docker && systemctl enable docker
   26  [2019-03-29 14:12:11] ls
   27  [2019-03-29 14:12:34] yum localinstall bzip2-1.0.6-13.el7.x86_64.rpm 
   28  [2019-03-29 14:34:24] cd ../SIPPROXY/
   29  [2019-03-29 14:34:25] ls
   30  [2019-03-29 14:34:37] tar -zxvf rainny.tar.gz 
   31  [2019-03-29 14:34:44] ls
   32  [2019-03-29 14:34:55] cd app/
   33  [2019-03-29 14:34:57] ls
   34  [2019-03-29 14:35:05] mv rainny ../
   35  [2019-03-29 14:35:06] ls
   36  [2019-03-29 14:35:10] cd ..
   37  [2019-03-29 14:35:11] ls
   38  [2019-03-29 14:35:21] rm -rf app
   39  [2019-03-29 14:35:23] ls
   40  [2019-03-29 14:35:31] cd rainny/
   41  [2019-03-29 14:35:33] ls
   42  [2019-03-29 14:46:22] vim FreeswitchWatchdog 
   43  [2019-03-29 14:47:27] chmod +x FreeswitchWatchdog 
   44  [2019-03-29 14:47:31] ll
   45  [2019-03-29 14:47:46] chmod +x docker.sh 
   46  [2019-03-29 14:47:49] ll
   47  [2019-03-29 14:48:08] ./docker.sh 
   48  [2019-03-29 14:48:25] vim docker.sh 
   49  [2019-03-29 15:40:01] ps -ef |grep docker.sh 
   50  [2019-03-29 15:40:06] ps -ef |grep docke
   51  [2019-03-29 15:40:20] ps -ef |grep freeswitch
   52  [2019-03-29 15:43:30] docker images
   53  [2019-03-29 15:43:51] ls
   54  [2019-03-29 15:43:58] cd /app
   55  [2019-03-29 15:43:59] ls
   56  [2019-03-29 15:44:04] cd SIPPROXY/
   57  [2019-03-29 15:44:05] ls
   58  [2019-03-29 15:44:14] cd rainny/
   59  [2019-03-29 15:44:15] ls
   60  [2019-03-29 15:47:02] cd ..
   61  [2019-03-29 15:47:04] ls
   62  [2019-03-29 15:47:07] cd ..
   63  [2019-03-29 15:47:09] ls
   64  [2019-03-29 15:47:13] cd ..
   65  [2019-03-29 15:47:15] ls
   66  [2019-03-29 15:56:48] history

常用的命令

PATH：/var/rainny/ippbx

运维

常用的命令

  • 抓包分析
tcpdump -np -s 0 -i eth0 -w /app/demp.pcap udp

tcpdump -np -s 0 -i eth0   udp

  • K8S调试命令

  • Master节点

  • Path：/root/

kubectl delete -f pcc.yml;
kubectl apply -f pcc.yml ; sleep 6;
kubectl get pod -n pcc -o wide;

kubectl exec -it ippbx-1 -n pcc sh
功能命令备注
保存操作history >> /root/nodeippbx2
查看日志kubectl logs -f ippbx-0
删除podkubectl delete -f pcc.yml;
创建podkubectl apply -f pcc.yml ; sleep 6;
查看podkubectl get pod -n pcc -o wide;
进入podkubectl exec -it ippbx-1 -n pcc sh
  • 搜索文件
  252  [2019-03-27 11:43:11] find -name "*.yml"
  253  [2019-03-27 11:44:24] find -name "*.yaml"
kubectl delete -f pcc.yml ; kubectl apply -f pcc.yml ; sleep 6 ;kubectl get pod -n pcc -o wide


./fs_cli -H 127.0.0.1 -p wzw -P 4521
docker load -i fsdeb.tar

docker images

mv  freeswitch freeswitch.old

kubectl delete -f pcc.yml ; kubectl apply -f pcc.yml ; sleep 6 ;kubectl get pod -n pcc -o wide

ippbx cti 应用节点部署

1. 文件准备

  • 拷贝文件 zs招商copy.tar.gz 到跳板机，
  • 从跳板机上拷贝到每个机器的/root下，
  • 解压得到 keepalived,freeswtich,a生产环境部署三个目录

zs招商copy.tar.gz

2 准备本地镜像文件

准备应用节点镜像文件

  • load freeswtich压缩文件到docker 镜像，并上传到harbor仓库：

(a) 在deploy节点，即100.67.33.10上操作：

  • 准备工作
[root@nodecti2 ~]# cd /root
[root@nodecti2 ~]# tar -xzvf zs*.tar.gz  
[root@nodecti2 ~]# ls
bin  slogs  zs????copy.tar.gz  zs招商copy
[root@nodecti2 ~]# cd zs招商copy/
[root@nodecti2 zs招商copy]# ls
a生产环境部署  freeswitch  keepalived
[root@nodecti2 zs招商copy]# cd freeswitch/
[root@nodecti2 freeswitch]# ls
freeswitch.tar.bz2  fsdeb.tar  rainny.tar.bz2
  • load freeswtich压缩文件到docker 镜像
[root@nodecti2 freeswitch]# docker load -i fsdeb.tar 
f33a13616df9: Loading layer [==================================================>] 82.96 MB/82.96 MB
c8f952ba8693: Loading layer [==================================================>] 1.024 kB/1.024 kB
5ca3e1235786: Loading layer [==================================================>] 1.024 kB/1.024 kB
8cda936b4d69: Loading layer [==================================================>]  5.12 kB/5.12 kB
5329b221f182: Loading layer [==================================================>] 93.21 MB/93.21 MB
9c9167e39e1a: Loading layer [==================================================>] 359.9 MB/359.9 MB
9aa6a792bc60: Loading layer [==================================================>] 443.9 MB/443.9 MB

[root@nodecti2 freeswitch]# docker images
REPOSITORY                             TAG                 IMAGE ID            CREATED             SIZE
wise2c/playbook                        v1.11.8             e52b775e14b3        2 weeks ago         1.14 GB
wise2c/yum-repo                        v1.11.8             6ff733f70993        3 weeks ago         758 MB
100.67.33.9/library/kube-proxy-amd64   v1.11.8             2ed65dca1a98        3 weeks ago         98.1 MB
fs                                     deb1                9c3b419a16e0        4 weeks ago         960 MB
100.67.33.9/library/flannel            v0.11.0-amd64       ff281650a721        8 weeks ago         52.6 MB
wise2c/pagoda                          v1.1                d4f2b4cabdec        2 months ago        483 MB
wise2c/deploy-ui                       v1.3                2f159b37bf13        2 months ago        40.1 MB
100.67.33.9/library/pause              3.1                 da86e6ba6ca1        15 months ago       742 kB
[root@nodecti2 freeswitch]#
  • 补充： docker images 查看镜像
删除镜像docker rmi 仓库:TAG
删除镜像docker rmi ${IMAGE ID}
docker load -i 镜像经过压缩的tar文件
docker tag 旧镜像repo:tag 新镜像repo:tag
docker push 镜像repo:tag （执行前，需要docker login docker-register-server）
docker login docker-register-server
  • 补充： docker images 查看镜像
    docker rmi 仓库:TAG 用于删除镜像
    docker rmi ${IMAGE ID} 用于删除镜像
    docker load -i 镜像经过压缩的tar文件
    docker tag 旧镜像repo:tag 新镜像repo:tag
    docker push 镜像repo:tag （执行前，需要docker login docker-register-server）

b) 上传镜像到Harbor仓库

  • 浏览器打开100.67.33.9 harbor仓库（登陆置灰，也能点击登陆），打开library 项目，可以查看到一些镜像，此时不需要在界面上做任何操作。

  • 在deploy节点上执行：

[root@nodecti2 freeswitch]# docker images
REPOSITORY                             TAG                 IMAGE ID            CREATED             SIZE
wise2c/playbook                        v1.11.8             e52b775e14b3        2 weeks ago         1.14 GB
wise2c/yum-repo                        v1.11.8             6ff733f70993        3 weeks ago         758 MB
100.67.33.9/library/kube-proxy-amd64   v1.11.8             2ed65dca1a98        3 weeks ago         98.1 MB
fs                                     deb1                9c3b419a16e0        4 weeks ago         960 MB
100.67.33.9/library/flannel            v0.11.0-amd64       ff281650a721        8 weeks ago         52.6 MB
wise2c/pagoda                          v1.1                d4f2b4cabdec        2 months ago        483 MB
wise2c/deploy-ui                       v1.3                2f159b37bf13        2 months ago        40.1 MB
100.67.33.9/library/pause              3.1                 da86e6ba6ca1        15 months ago       742 kB
  • 重新打标签
[root@nodecti2 freeswitch]# docker  tag fs:deb1 100.67.33.9/library/fs:deb1

[root@nodecti2 freeswitch]# 
[root@nodecti2 freeswitch]# docker images
REPOSITORY                             TAG                 IMAGE ID            CREATED             SIZE
wise2c/playbook                        v1.11.8             e52b775e14b3        2 weeks ago         1.14 GB
wise2c/yum-repo                        v1.11.8             6ff733f70993        3 weeks ago         758 MB
100.67.33.9/library/kube-proxy-amd64   v1.11.8             2ed65dca1a98        3 weeks ago         98.1 MB
100.67.33.9/library/fs                 deb1                9c3b419a16e0        4 weeks ago         960 MB
fs                                     deb1                9c3b419a16e0        4 weeks ago         960 MB
100.67.33.9/library/flannel            v0.11.0-amd64       ff281650a721        8 weeks ago         52.6 MB
wise2c/pagoda                          v1.1                d4f2b4cabdec        2 months ago        483 MB
wise2c/deploy-ui                       v1.3                2f159b37bf13        2 months ago        40.1 MB
100.67.33.9/library/pause              3.1                 da86e6ba6ca1        15 months ago       742 kB

[root@nodecti2 freeswitch]# docker rmi fs:deb1
Untagged: fs:deb1

[root@nodecti2 freeswitch]# docker images     
REPOSITORY                             TAG                 IMAGE ID            CREATED             SIZE
wise2c/playbook                        v1.11.8             e52b775e14b3        2 weeks ago         1.14 GB
wise2c/yum-repo                        v1.11.8             6ff733f70993        3 weeks ago         758 MB
100.67.33.9/library/kube-proxy-amd64   v1.11.8             2ed65dca1a98        3 weeks ago         98.1 MB
100.67.33.9/library/fs                 deb1                9c3b419a16e0        4 weeks ago         960 MB
100.67.33.9/library/flannel            v0.11.0-amd64       ff281650a721        8 weeks ago         52.6 MB
wise2c/pagoda                          v1.1                d4f2b4cabdec        2 months ago        483 MB
wise2c/deploy-ui                       v1.3                2f159b37bf13        2 months ago        40.1 MB
100.67.33.9/library/pause              3.1                 da86e6ba6ca1        15 months ago       742 kB

[root@nodecti2 freeswitch]# cat /etc/docker/daemon.json 
{
  "exec-opt": [
    "native.cgroupdriver=systemd"
  ],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m",
    "max-file": "5"
  },
  "insecure-registries": [
      "100.67.33.9"
  ],
  "storage-driver": "overlay2"
}
  • 远程登陆Docker
[root@nodecti2 freeswitch]# docker login 100.67.33.9
Username: admin
Password:           #此处输入Harbor12345
Login Succeeded
  • push image 到Harbor
[root@nodecti2 freeswitch]# docker push 100.67.33.9/library/fs:deb1
The push refers to a repository [100.67.33.9/library/fs]
8c7f9e41cf2f: Pushed 
0e8f26b8afa9: Pushed 
683e839e85ce: Pushed 
1999ce8fd68d: Pushed 
5f70bf18a086: Pushed 
16ada34affd4: Pushed 
deb1: digest: sha256:37a3b7114091c8ea7dfd2f6b16a6c708469443e6da6edf96a2b0201c226b0ed2 size: 1787
  • 此时浏览器刷新harbor,可查看到推送的镜像

3 四个node-worker节点安装和配置keepalived

a 本地安装keepalived

此处在 100.67.33.7 上执行，其它三个节点类似：

[root@nodeippbx1 ~]# cd zs招商copy/
[root@nodeippbx1 zs招商copy]# ls
a生产环境部署  freeswitch  keepalived
[root@nodeippbx1 zs招商copy]# cd keepalived/
[root@nodeippbx1 keepalived]# ls
keepalived-1.3.5-8.el7_6.x86_64.rpm  lm_sensors-libs-3.4.0-6.20160601gitf9185e5.el7.x86_64.rpm  net-snmp-agent-libs-5.7.2-37.el7.x86_64.rpm  net-snmp-libs-5.7.2-37.el7.x86_64.rpm
  • 本地安装keepalived
    与上面路径一致
[root@nodeippbx1 keepalived]# yum localinstall *.rpm -y
...
Installed:
  keepalived.x86_64 0:1.3.5-8.el7_6                         net-snmp-agent-libs.x86_64 1:5.7.2-37.el7                         net-snmp-libs.x86_64 1:5.7.2-37.el7                        

Updated:
  lm_sensors-libs.x86_64 0:3.4.0-6.20160601gitf9185e5.el7                                                                                                                                 

Complete!
[root@nodeippbx1 keepalived]# 
systemctl enable keepalived.service;systemctl start keepalived.service

b 配置ippbx

注意keepavlied.conf 里面，主从的virtual_router_id 必须一致，最好不要大于255，而 router_id 必须不一样，主从的router_id，state，priority 不能一样，具体如下。

  • 100.67.33.7
 # Backup
cp /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf.ori



vim /etc/keepalived/keepalived.conf
  ###### begin
  [root@nodeippbx1 keepalived]# cat /etc/keepalived/keepalived.conf
! Configuration File for keepalived

global_defs {
   router_id ippbx_master
}

vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 188
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
       100.67.33.16/23 dev eth0 label eth0:1
    }
}
###### end
  • 100.67.33.8
###### begin
[root@nodeippbx2 keepalived]# cat /etc/keepalived/keepalived.conf
! Configuration File for keepalived

global_defs {
   router_id ippbx_backup
}

vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 188
    priority 20
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
       100.67.33.16/23 dev eth0 label eth0:1
    }
}
 ###### end
  • 补充
 两台主机分别：  systemctl  enable keepalived     && systemctl start keepalived   && ps -ef | grep keepalived
 分别查看ip:   ifconfig，会发现在主节点上有：
   eth0:1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 100.67.33.16  netmask 255.255.254.0  broadcast 0.0.0.0
        ether fa:16:3e:dd:97:61  txqueuelen 1000  (Ethernet)
 在主节点关闭keepalived,再分别查看ip，会发现从节点出现,主节点没有此vip了：
   eth0:1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 100.67.33.16  netmask 255.255.254.0  broadcast 0.0.0.0
        ether fa:16:3e:06:a4:f6  txqueuelen 1000  (Ethernet)

c 配置两台CTI

  • 100.67.33.9
##################begin
[root@nodecti1 keepalived]# cat keepalived.conf
! Configuration File for keepalived

global_defs {
   router_id cti_master
}

vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 189
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 2222
    }
    virtual_ipaddress {
       100.67.33.17/23 dev eth0 label eth0:1
    }
}
##################end
  • 100.67.33.10
 ##################begin
 [root@nodecti2 keepalived]# cat /etc/keepalived/keepalived.conf
global_defs {
   router_id cti_backup
}

vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 189
    priority 20
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 2222
    }
    virtual_ipaddress {
       100.67.33.17/23 dev eth0 label eth0:1
    }
}
##################end

d 测试Keepalived

两台节点分别执行：

[root@nodecti1 keepalived]# systemctl enable keepalived
[root@nodecti1 keepalived]# systemctl start keepalived 
[root@nodecti1 keepalived]# ps -ef | grep keep
root      1780     1  0 Jan29 ttyS0    00:00:00 /sbin/agetty --keep-baud 115200 38400 9600 ttyS0 vt220
root     16941     1  0 19:20 ?        00:00:00 /usr/sbin/keepalived -D
root     16942 16941  0 19:20 ?        00:00:00 /usr/sbin/keepalived -D
root     16943 16941  0 19:20 ?        00:00:00 /usr/sbin/keepalived -D
root     18826  4506  0 19:22 pts/1    00:00:00 grep --color=auto keep

[root@nodecti1 keepalived]# ifconfig | grep eth0   ## 在主节点能查到两个ip，从节点只有一个，当主节点停止keepalived服务后，从节点多了vip
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
eth0:1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
[root@nodecti1 keepalived]# ifconfig | grep 67
        inet 100.67.33.9  netmask 255.255.254.0  broadcast 100.67.33.255
        inet 100.67.33.17  netmask 255.255.254.0  broadcast 0.0.0.0
        RX packets 723588  bytes 667124730 (636.2 MiB)
        RX packets 459  bytes 86709 (84.6 KiB)
vethe922676: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        TX packets 750673  bytes 669843972 (638.8 MiB)
        inet6 fe80::867:1dff:fe29:55d0  prefixlen 64  scopeid 0x20<link>
        ether 0a:67:1d:29:55:d0  txqueuelen 0  (Ethernet)

4 配置fs app 的yaml文件

a 本地安装bzip2 包并解压文件

  • rpm包准备
    需要从外拷贝bzip2-1.0.6-13.el7.x86_64.rpm 到deploy机器，并 yum localinstall ./bzip2-1.0.6-13.el7.x86_64.rpm

  • 本地安装安装bzip2

yum localinstall ./bzip2-1.0.6-13.el7.x86_64.rpm
  • 解压rainny.tar.bz2与freeswitch.tar.bz2
[root@nodecti2 freeswitch]# yum localinstall ./bzip2-1.0.6-13.el7.x86_64.rpm
[root@nodecti2 freeswitch]# pwd
/root/zs招商copy/freeswitch
[root@nodecti2 freeswitch]# ls
freeswitch.tar.bz2  fsdeb.tar  rainny.tar.bz2    
[root@nodecti2 freeswitch]# tar -xjvf rainny.tar.bz2  
[root@nodecti2 freeswitch]# tar -xjvf freeswitch.tar.bz2
[root@nodecti2 freeswitch]# ls
freeswitch  freeswitch.tar.bz2  fsdeb.tar  rainny  rainny.tar.bz2

b 准备映射到pod内的host目录

  • 为四个node_worker节点打标签
    运行节点：在master上执行：
[root@master1 ~]# kubectl get nodes -o wide
NAME         STATUS    ROLES     AGE       VERSION
master1      Ready     master    1d        v1.11.8
master2      Ready     master    1d        v1.11.8
master3      Ready     master    1d        v1.11.8
nodecti1     Ready     <none>    1d        v1.11.8
nodecti2     Ready     <none>    1d        v1.11.8
nodeippbx1   Ready     <none>    1d        v1.11.8
nodeippbx2   Ready     <none>    1d        v1.11.8
[root@master1 ~]# kubectl label node nodeippbx1 pcc/ippbx=true
[root@master1 ~]# kubectl label node nodeippbx2 pcc/ippbx=true
[root@master1 ~]# kubectl label node nodecti1 pcc/cti=true
[root@master1 ~]# kubectl label node nodecti2 pcc/cti=true
  • [?]配置免密传输
免密登陆
  • 准备映射到pod内的host目录：
    将上面解压的rainny目录，拷贝到四个工作节点的 /data (单独逻辑卷)目录下：
 scp -r rainny 100.67.33.7:/data
 scp -r rainny 100.67.33.8:/data
 scp -r rainny 100.67.33.9:/data
 scp -r rainny 100.67.33.10:/data

补充脚本

IPPBX
 cp /root/zs招商copy/freeswitch/config/rc.local-pbbxc rc.local

c 修改配置文件 pcc.yaml

  • Path: /data
  • command

Keepalived config

code

运维

常用的命令

  • 搜索文件
  252  [2019-03-27 11:43:11] find -name "*.yml"
  253  [2019-03-27 11:44:24] find -name "*.yaml"
ActionCommandRemarks
kubectl logs -f pod/ippbx-0 -n pcc
kubectl describe pod/ippbx-0 -n pcc
kubectl get nodes,pod -n pcc
restartsystemctl start keepalived
check processps -ef | grep keepalived
check ipifconfig | grep eth0
check ipip -a

Reference

[1] tar.bz2文件解压命令
[2] Linux运行shell脚本提示No such file or directory错误的解决办法

Coderove
关注
  • 0
    点赞
  • 踩
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
专栏目录
CC00398.CloudKubernetes——|KuberNetes&CI/CD.V36|——|Jenkins.v02|生产UAT流水线设计.v02|
yanqi_vip
04-01 115
一、在kubernetes下创建nodejs-uat资源 ### --- 创建一个nodejs-uat的namespace [root@k8s-master01 ~]# kubectl create namespace nodejs-uat namespace/nodejs-uat created 二、在 nodejs-uat命名空间下创建registrykey.Secret...
Kubernetes In cloud (一、deploy cluster)
测试0901-1
03-16 293
一、写在前面： 云提供了kubernetes的Paas服务，但是很多同学对kubernetes的使用不是很清楚，最根本的原因就是出发点不同。cloud是要把技术门槛降低，通过可视化的配置降低学习成本；而企业要的是稳定性，以及故障恢复的时效性，以及故障复盘；这就造成了一系列的问题：support视角：1.cloud的support 不知道客户做了什么 操作...
Kubernetes Pod概述
技术积累-LZ
08-24 783
Kubernetes Pod概述 本文主要介绍Pod，了Kubernetes对象模型中可部署的最小对象。 了解Pod Pod是Kubernetes创建或部署的最小/最简单的基本单位，一个Pod代表集群上正在运行的一个进程。 一个Pod封装一个应用容器（也可以有多个容器），存储资源、一个独立的网络IP以及管理控制容器运行方式的策略选项。Pod代表部署的一个单位：Kubernetes中单个...
K8S命令记录
IDreamsComeTrue的博客
03-26 345
K8S 命令记录K8SReference K8S 180 [2019-02-20 17:04:14] docker load -i fsdeb.tar 181 [2019-02-20 17:06:55] docker images 182 [2019-02-20 17:07:44] docker login 100.69.216.165 183 [2019-02-20 17...
flink on yarn 提交任务报错 —— Could not deploy Yarn job cluster
一纸春秋的博客
01-18 3946
这里到项目路径里面查看一下打包好的jar包 打包之后的jar包如果是这两个，然后提交jar包的时候提交的flink-1.0-SNAPSHOT-jar-with-dependencies.jar 这个jar包，那么就会报上面的错误，这里应该是jar包里面的依赖出现了问题 我的解决措施是，直接重新创建了一个项目 然后是项目的整体pom.xml文件 <?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.a.
kubernetesl yaml deploy rancher server
爱死亡机器人
09-06 750
Rancher是一个开源的企业级全栈化容器部署及管理平台。已有超过 1900 万次下载，4000+ 生产环境的应用。简单的说，就是一个可以让你通过 web 界面管理 docker 容器的平台。定位上和 K8s 比较接近，都是通过 web 界面赋予完全的 docker 服务编排功能。特色：平台部署方便。管理 docker 的平台本身也基于 docker 部署。只要你有 docker ，一句命令就完成平台的部署了。平台扩展方便。
PROJECＴ
weixin_46796555的博客
04-24 615
外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-vNgDoYQv-1682339328434)(/root/下载/NSD2302笔记资料（整理）/第二阶段/nsd2210/imgs/image-20220111164543156.png)][外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-l55OaZTK-1682339328431)(…/imgs/image-20220110145139164.png)]注意：以上页面为后台管理页面。
SPARK spark-submit提交deploy-mode client改为cluster模式
80后大叔爱学习
08-04 1万+
报错内容：ERROR deploy.ClientEndpoint: Exception from cluster was: java.io.FileNotFoundException: /usr/log_data/java/spark-project-0.0.1-SNAPSHOT.jar (No such file or directory) 查找原因是： spark-submit 提交任务时...
使用ceph-deploy 部署集群
discsthnew的博客
03-26 3546
Quick Ceph Deploy 集群内有两个节点（tom-1, tom-2）,在 tom-1 中通过 ceph-deploy 来部署安装整个集群。均为 centos7.1 系统。 PREFLIGHT CHECKLIST 1. Add ceph repositories 官方的镜像源较慢，这里使用阿里提供的yum源 [root@tom-1 yum.repos.d]# cat c...
tensorflow on spark yarn model deploy on CDH5.12 cluster
mtj66的博客，交流WX:SpringBreeze1104
01-18 1412
TensorFlow是谷歌提供的开源深度学习框架 TensorFlowOnSpark: 是雅虎提供的集成到Spark上的深度学习框架 鉴于我们使用的是Spark核心计算框架，现在搭建TensorFlow ON Hadoop Yarn开发环境 架构 https://www.jianshu.com/p/62b4ebb5a2f4 http://yahoohadoop.tumblr.com/
使用ceph-deploy 2.0.0 部署ceph 12.2.5集群
热门推荐
for_tech的博客
05-10 1万+
1. 环境 1.1 硬件 4台 Linux虚拟机： server0, server1, server2, server3 每台有两块磁盘 ： /dev/vdb, /dev/vdc 每台有两块网卡 ：eth0, ens9 1.2 软件 linux版本： CentOS 7.2.1511 内核版本 ： 3.10.0-327.el7.x86_64 ceph版本： 12.2.5...
Apache HTTP Server通过mod_cluster模块与Tomcat连接
yakoo5的专栏
12-11 3326
使用mod_cluster的好处 1、httpd workers 可动态配置； 2、Server端负载均衡因子计算； 3、更细粒度的web-app生命周期控制 4、支持多种协议（HTTP、HTTPS、AJP）
在centos7下安装fate cluster.
weixin_51143734的博客
05-22 694
将主机10.10.44.127作为host，10.10.44.128作为guest，进行fate cluster的部署 1.环境配置 1.1修改主机名（host guest同时执行） 首先将10.10.44.127及10.10.44.128进行更改主机名操作： 将主机10.10.44.127更名为：vm_0_1_centos,将主机10.10.44.128更名为：vm_0_2_centos 在192.168.0.1 root用户下执行： hostnamectl set-hostname VM_
apimanager-wso2-OKD-deploy
m0_51713482的博客
08-16 864
apim-wso2-OKD-deploy（APIM部署到OKD）
Matlab界面GUI设计的车牌定位[Matlab界面GUI设计].zip
最新发布
09-10
linux常用命令大全
【独家首发】基于matlab沙猫群算法SCSO-GMDH风电数据回归预测【含Matlab源码 7530期】.zip
09-10
CSDN海神之光上传的全部代码均可运行，亲测可用，直接替换数据即可，适合小白； 1、代码压缩包内容 主函数：Main.m； 调用函数：其他m文件；无需运行 运行结果效果图； 2、代码运行版本 Matlab 2019b；若运行有误，根据提示修改；若不会，可私信博主； 3、运行操作步骤 步骤一：将所有文件放到Matlab的当前文件夹中； 步骤二：双击打开除Main.m的其他m文件； 步骤三：点击运行，等程序运行完得到结果； 4、仿真咨询 如需其他服务，可私信博主或扫描博主博客文章底部QQ名片； 4.1 CSDN博客或资源的完整代码提供 4.2 期刊或参考文献复现 4.3 Matlab程序定制 4.4 科研合作 智能优化算法优化GMDH回归预测系列程序定制或科研合作方向： 4.4.1 遗传算法GA/蚁群算法ACO优化GMDH回归预测 4.4.2 粒子群算法PSO/蛙跳算法SFLA优化GMDH回归预测 4.4.3 灰狼算法GWO/狼群算法WPA优化GMDH回归预测 4.4.4 鲸鱼算法WOA/麻雀算法SSA优化GMDH回归预测 4.4.5 萤火虫算法FA/差分算法DE优化GMDH回归预测 4.4.6 其他优化算法优化GMDH回归预测
【2024首发原创】蜣螂优化算法DBO-TCN-LSTM-Multihead-Attention负荷预测Matlab实现.rar
09-10
1.版本：matlab2014/2019a/2024a 2.附赠案例数据可直接运行matlab程序。 3.代码特点：参数化编程、参数可方便更改、代码编程思路清晰、注释明细。 4.适用对象：计算机，电子信息工程、数学等专业的大学生课程设计、期末大作业和毕业设计。 替换数据可以直接使用，注释清楚，适合新手
deploy-mode cluster
06-07
deploy-mode是Spark应用程序在Cluster模式下运行的一种配置方式。deploy-mode有两种模式：client和cluster。 在client模式下，Spark Driver程序运行在提交应用程序的客户端上，而Executor程序运行在集群的工作节点...
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值