环境
ubuntu18.04
docker 安装
1、卸载docker(没安装过docker就跳过)
sudo apt-get remove docker \
docker-engine \
docker.io
2、添加下载资源
sudo add-apt-repository \
"deb [arch=amd64] https://mirrors.ustc.edu.cn/docker-ce/linux/ubuntu \
$(lsb_release -cs) \
stable"
3、添加软件源的 GPG 密钥
curl -fsSL https://mirrors.ustc.edu.cn/docker-ce/linux/ubuntu/gpg | sudo apt-key add -
4、安装docker
wget -qO- https://get.docker.com/ | sh
5、设置开机自启动,并且现在启动docker
sudo systemctl enable docker
sudo systemctl start docker
6、将当前用户加入 docker 组
sudo usermod -aG docker $USER
7、国内image下载源
vi /etc/docker/daemon.json
daemon.json内容如下
{
"registry-mirrors": [
"https://registry.docker-cn.com"
]
}
8、重启docker
sudo systemctl daemon-reload
sudo systemctl restart docker
docker consul 安装
1、consul镜像下载
sudo docker pull consul
2、查看镜像
consul@ubuntu:~$ sudo docker image ls
REPOSITORY TAG IMAGE ID CREATED SIZE
microsoft/dotnet 2.1-sdk 540aa875e6c2 2 weeks ago 1.73GB
microsoft/dotnet 2.1-aspnetcore-runtime db366d73508b 2 weeks ago 253MB
consul latest 2d7f56f4c166 7 weeks ago 104MB
docker consul 集群搭建
1、搭建目标:2个数据中心,数据中心A是由5个server,6个client组成的集群,
数据中心B是由3个server,4个client组成的集群。
2、双数据中心集群草图:
3、server服务端节点配置
路径为:/opt/config/consul/consul_server_config
sudo vim basic_config_a1.json
basic_config_a1.json的内容如下:
{
"datacenter": "dc1",
"log_level": "INFO",
"node_name": "s_a1",
"server": true,
"bootstrap_expect": 3,
"bind_addr": "0.0.0.0",
"client_addr": "0.0.0.0",
"ui": true,
"ports": {
"dns": 8600,
"http": 8500,
"https": -1,
"server": 8300,
"serf_lan": 8301,
"serf_wan": 8302
},
"rejoin_after_leave": true,
"retry_join": [
"172.18.0.2",
"172.18.0.3",
"172.18.0.4",
"172.18.0.5",
"172.18.0.6"
],
"retry_interval": "30s",
"reconnect_timeout": "72h"
}
要点分析:datacenter为数据中心名称设置,
node_name为consul节点名称,也就是后面启动的容器的名称
bootstrap_expect为consul启动集群进行leader选举的期望服务数量,当达到此数值时才会进行选举
retry_join为consul的数据中心的服务节点地址配置,此参数是为选举leader准备的(自己yy的)
其他参数请度娘。
此配置文件是数据中心1的第一个服务节点的配置内容,剩余数据中心1的4个服务节点:s_a2,s_a3,s_a4,s_a5的配置均只用改
node_name的名称即可。
4、客户端节点配置:
路径为:/opt/config/consul/consul_client_config
sudo vim basic_config_a1.json
basic_config_a1.json的内容如下:
{
"datacenter": "dc1",
"log_level": "INFO",
"node_name": "ca1",
"server": false,
"bind_addr": "0.0.0.0",
"client_addr": "0.0.0.0",
"ui": false,
"ports": {
"dns": 8600,
"http": 8500,
"https": -1,
"server": 8300,
"serf_lan": 8301,
"serf_wan": 8302
},
"rejoin_after_leave": true,
"retry_join": [
"172.18.0.2",
"172.18.0.3",
"172.18.0.4",
"172.18.0.5",
"172.18.0.6"
],
"retry_interval": "30s",
"reconnect_timeout": "72h"
}
同样的
此配置文件是数据中心1的第一个客户端节点的配置内容,剩余数据中心1的4个服务节点:ca2,ca3,ca4,ca5,ca6的配置均只用改
node_name的名称即可。
5、数据中心 2的配置类似数据中心1,就不贴了。
6、编写shell脚本,方便部署,保存为 consulrestart.sh
sudo docker rm consul_cb2 -f
sudo docker rm consul_cb3 -f
sudo docker rm consul_cb4 -f
echo "start restart all servers and clients"
sudo docker run -d --name consul_sa1 --net staticnet --ip 172.18.0.2 -v /opt/config/consul/consul_server_config/basic_config_a1.json:/consul/config/basic_config_a1.json consul agent -config-dir /consul/config
sudo docker run -d --name consul_sa2 --net staticnet --ip 172.18.0.3 -v /opt/config/consul/consul_server_config/basic_config_a2.json:/consul/config/basic_config_a2.json consul agent -config-dir /consul/config
sudo docker run -d --name consul_sa3 --net staticnet --ip 172.18.0.4 -v /opt/config/consul/consul_server_config/basic_config_a3.json:/consul/config/basic_config_a3.json consul agent -config-dir /consul/config
sudo docker run -d --name consul_sa4 --net staticnet --ip 172.18.0.5 -v /opt/config/consul/consul_server_config/basic_config_a4.json:/consul/config/basic_config_a4.json consul agent -config-dir /consul/config
sudo docker run -d --name consul_sa5 --net staticnet --ip 172.18.0.6 -v /opt/config/consul/consul_server_config/basic_config_a5.json:/consul/config/basic_config_a5.json consul agent -config-dir /consul/config
sudo docker run -d --name consul_ca1 --net staticnet --ip 172.18.0.10 -v /opt/config/consul/consul_client_config/basic_config_a1.json:/consul/config/basic_config_a1.json consul agent -config-dir /consul/config
sudo docker run -d --name consul_ca2 --net staticnet --ip 172.18.0.11 -v /opt/config/consul/consul_client_config/basic_config_a2.json:/consul/config/basic_config_a2.json consul agent -config-dir /consul/config
sudo docker run -d --name consul_ca3 --net staticnet --ip 172.18.0.12 -v /opt/config/consul/consul_client_config/basic_config_a3.json:/consul/config/basic_config_a3.json consul agent -config-dir /consul/config
sudo docker run -d --name consul_ca4 --net staticnet --ip 172.18.0.13 -v /opt/config/consul/consul_client_config/basic_config_a4.json:/consul/config/basic_config_a4.json consul agent -config-dir /consul/config
sudo docker run -d --name consul_ca5 --net staticnet --ip 172.18.0.14 -v /opt/config/consul/consul_client_config/basic_config_a5.json:/consul/config/basic_config_a5.json consul agent -config-dir /consul/config
sudo docker run -d --name consul_ca6 --net staticnet --ip 172.18.0.15 -v /opt/config/consul/consul_client_config/basic_config_a6.json:/consul/config/basic_config_a6.json consul agent -config-dir /consul/config
sudo docker run -d --name consul_sb1 --net staticnet --ip 172.18.0.21 -v /opt/config/consul/consul_server_config/basic_config_b1.json:/consul/config/basic_config_b1.json consul agent -config-dir /consul/config
sudo docker run -d --name consul_sb2 --net staticnet --ip 172.18.0.22 -v /opt/config/consul/consul_server_config/basic_config_b2.json:/consul/config/basic_config_b2.json consul agent -config-dir /consul/config
sudo docker run -d --name consul_sb3 --net staticnet --ip 172.18.0.23 -v /opt/config/consul/consul_server_config/basic_config_b3.json:/consul/config/basic_config_b3.json consul agent -config-dir /consul/config
sudo docker run -d --name consul_cb1 --net staticnet --ip 172.18.0.24 -v /opt/config/consul/consul_client_config/basic_config_b1.json:/consul/config/basic_config_b1.json consul agent -config-dir /consul/config
sudo docker run -d --name consul_cb2 --net staticnet --ip 172.18.0.25 -v /opt/config/consul/consul_client_config/basic_config_b2.json:/consul/config/basic_config_b2.json consul agent -config-dir /consul/config
sudo docker run -d --name consul_cb3 --net staticnet --ip 172.18.0.26 -v /opt/config/consul/consul_client_config/basic_config_b3.json:/consul/config/basic_config_b3.json consul agent -config-dir /consul/config
sudo docker run -d --name consul_cb4 --net staticnet --ip 172.18.0.27 -v /opt/config/consul/consul_client_config/basic_config_b4.json:/consul/config/basic_config_b4.json consul agent -config-dir /consul/config
echo "start connect dt1,dt2"
sudo docker exec consul_sb1 consul join -wan 172.18.0.2
sleep 3s
sudo docker exec consul_sb1 consul join -wan 172.18.0.2
sleep 3s
sudo docker exec consul_sb1 consul join -wan 172.18.0.2
sleep 3s
sudo docker exec consul_sb1 consul join -wan 172.18.0.2
echo "completed!"
echo "visit websit:http://172.18.0.2:8500/ui/dc1/services"
sleep 10s
7、更改consulrestart.sh属性
chmod 777 consulrestart.sh
8、执行shell脚本
./consulrestart.sh
输出如下:
staticnet: 172.18.0.0/24
start remove all servers and clients
consul_sa1
consul_sa2
consul_sa3
consul_sa4
consul_sa5
consul_ca1
consul_ca2
consul_ca3
consul_ca4
consul_ca5
consul_ca6
consul_sb1
consul_sb2
consul_sb3
consul_cb1
consul_cb2
consul_cb3
consul_cb4
start restart all servers and clients
562fae5b739d9f804bb9673a5469cfe6640ff2d3974cf0ee3c5cd45fb40d8e60
32f4ca17b40402fe5644c78156ba33d62124d925d139c2dfed89107802cee972
14a257ccc48dcb7849cfbd6355c8443594994aecde252f8c598c5d7595472e92
3444c6f93cf0fddf98cc2c13612cfd278b45aa502732d47e6b1622a4ddda976f
52c5e01112d0e01774d87f9f2704e02473334bb6f5c926d1a0925ae10ff56f5c
9694260f7a898e3773b7a0281b0f4f9f4307994335a43b803537dce8d8ee1a26
bd24cf404fe1b76a2b2e575c2760c07b14f68681e14c8ecb57d95fe20a32d96c
cea2f47886c8f44adb7fdc8834b201a41a7dd5f026ca7b4b57bd2f7a027921a9
1ae3f8845583cae5d150a5ef0e40b591949c2228ff586a7829d1480a678268a3
a306604c77e13dc2a61cab4d700d2bb6eb9ec766489dd999af61a9a7192fc7c5
0682edeb31c43220ff045cb611bd41c5df74c90d8645fc3436c643c6f68195a0
d36b1e92682d84ee3bb2da71ba9a7a8610e5bcb4e963aae83001f52af25808be
ae0cfb5413c297ed6a6b181973119e1b2ac195231392d1ed3334cb9fd89e993b
5708c85b8ff6d490d12a8b0f48d7237bed96d4577f0f2ebe30b8e6e3df80f637
6abbbc608c66c486fcb0b1a36b51df9cf378324fc8ea89b68d28ba0ec344bf91
61e10a66ea4cd7d527a6c2a0747e105469e362b0c061219874e57ab9ed0dcb15
72356b64c20db2d1e81df6cace2b93a88b0449e3f09bf7a48b7fa34d67534e93
9a33d93f032f990a900bc808b5e31a9debd4b95275f9cf7557c00e0ec87b4c7a
start connect dt1,dt2
Successfully joined cluster by contacting 1 nodes.
Successfully joined cluster by contacting 1 nodes.
Successfully joined cluster by contacting 1 nodes.
Successfully joined cluster by contacting 1 nodes.
completed!
visit websit:http://172.18.0.2:8500/ui/dc1/services
9、点击链接:http://172.18.0.2:8500/ui/dc/services
10、进入容器consul_sb1的控制台
sudo docker exec -it consul_sb1 /bin/sh
11、查看当前容器的consul下的集群的状态
consul operator raft list-peers
输出:
consul@ubuntu:/opt/config/consul$ sudo docker exec -it consul_sb1 /bin/sh
/ # consul operator raft list-peers
Node ID Address State Voter RaftProtocol
s_b1 d15dfe7f-ceeb-670a-6f59-c84b6aeba1e4 172.18.0.21:8300 leader true 3
s_b2 3c68b227-253c-929a-35b4-e55fe9231ff6 172.18.0.22:8300 follower true 3
s_b3 bc01975d-6c5f-09c4-0d78-5b997d668dba 172.18.0.23:8300 follower true 3
/ #
12、查看当前容器的consul中所有LAN和WAN的server节点
/ # consul members -wan
Node Address Status Type Build Protocol DC Segment
s_a1.dc1 172.18.0.2:8302 alive server 1.3.0 2 dc1 <all>
s_a2.dc1 172.18.0.3:8302 alive server 1.3.0 2 dc1 <all>
s_a3.dc1 172.18.0.4:8302 alive server 1.3.0 2 dc1 <all>
s_a4.dc1 172.18.0.5:8302 alive server 1.3.0 2 dc1 <all>
s_a5.dc1 172.18.0.6:8302 alive server 1.3.0 2 dc1 <all>
s_b1.dc2 172.18.0.21:8302 alive server 1.3.0 2 dc2 <all>
s_b2.dc2 172.18.0.22:8302 alive server 1.3.0 2 dc2 <all>
s_b3.dc2 172.18.0.23:8302 alive server 1.3.0 2 dc2 <all>
/ #
ubuntu下目录结构:
consul@ubuntu:/opt/config/consul$ ls
consul_client_config consulrestart.sh consul_server_config
consul@ubuntu:/opt/config/consul$
consul@ubuntu:/opt/config/consul/consul_client_config$ ll
total 48
drwxr-xr-x 2 root root 4096 Dec 6 00:51 ./
drwxr-xr-x 4 root root 4096 Dec 6 02:54 ../
-rw-r--r-- 1 root root 549 Dec 5 22:16 basic_config_a1.json
-rw-r--r-- 1 root root 549 Dec 5 22:17 basic_config_a2.json
-rw-r--r-- 1 root root 549 Dec 5 22:17 basic_config_a3.json
-rw-r--r-- 1 root root 549 Dec 5 22:18 basic_config_a4.json
-rw-r--r-- 1 root root 549 Dec 5 22:18 basic_config_a5.json
-rw-r--r-- 1 root root 549 Dec 5 22:18 basic_config_a6.json
-rw-r--r-- 1 root root 522 Dec 6 00:50 basic_config_b1.json
-rw-r--r-- 1 root root 522 Dec 6 00:50 basic_config_b2.json
-rw-r--r-- 1 root root 522 Dec 6 00:50 basic_config_b3.json
-rw-r--r-- 1 root root 522 Dec 6 00:51 basic_config_b4.json
consul@ubuntu:/opt/config/consul/consul_client_config$
consul@ubuntu:/opt/config/consul/consul_server_config$ ll
total 40
drwxr-xr-x 2 root root 4096 Dec 6 00:55 ./
drwxr-xr-x 4 root root 4096 Dec 6 02:54 ../
-rw-r--r-- 1 root root 679 Dec 5 22:00 basic_config_a1.json
-rw-r--r-- 1 root root 679 Dec 5 22:01 basic_config_a2.json
-rw-r--r-- 1 root root 679 Dec 5 22:02 basic_config_a3.json
-rw-r--r-- 1 root root 679 Dec 5 22:03 basic_config_a4.json
-rw-r--r-- 1 root root 679 Dec 5 22:04 basic_config_a5.json
-rw-r--r-- 1 root root 644 Dec 6 00:54 basic_config_b1.json
-rw-r--r-- 1 root root 644 Dec 6 00:54 basic_config_b2.json
-rw-r--r-- 1 root root 644 Dec 6 00:55 basic_config_b3.json
consul@ubuntu:/opt/config/consul/consul_server_config$
脚本及配置文件:
https://github.com/jiahengaa/dockerconsulconfig/tree/master/consul