Docker-compose搭建(单机/多机)RabbitMQ集群

一、(单台服务器)docker-compose搭建RabbitMQ集群

这里我们提到的是用一台Centos7服务器上用docker搭建的RabbitMQ集群,只能用作测试和练手用。

1.环境准备

服务器IP容器分配网段容器hostname容器的IP
172.17.202.82172.24.0.0/16rabbit1172.24.0.5
rabbit2172.24.0.6
rabbit3172.24.0.7

2.Centos7关闭防火墙

启动一个服务:systemctl start firewalld.service
关闭一个服务:systemctl stop firewalld.service
重启一个服务:systemctl restart firewalld.service
显示一个服务的状态:systemctl status firewalld.service
在开机时启用一个服务:systemctl enable firewalld.service
在开机时禁用一个服务:systemctl disable firewalld.service

选择合适的命令,关闭防火墙

3.安装Docker,Docker-compose

4.单机版docker-compose文件

说明:docker-compose.yml文件中创建了network桥接网卡,和三个RabbitMQ的容器

version: '3.6'

networks:
    rabbit:
        driver: bridge
        ipam:
          config:
            - subnet: 172.24.0.0/16

services:
    rabbit1:
        image: rabbitmq:management-alpine
        hostname: rabbit1
        ports:
            - "25672:25672"
            - "5672:5672"
            - "15672:15672"
            - "4369:4369"
        networks:
            rabbit:
                ipv4_address: 172.24.0.5
        environment:
            - RABBITMQ_NODENAME=rabbit1
            - RABBITMQ_ERLANG_COOKIE=CURIOAPPLICATION
            - RABBITMQ_DEFAULT_USER=admin
            - RABBITMQ_DEFAULT_PASS=admin
        volumes:
            - "/var/lib/rabbitmq/.erlang.cookie:/root/.erlang.cookie"    

    rabbit2:
        image: rabbitmq:management-alpine
        hostname: rabbit2
        environment:
            - RABBITMQ_NODENAME=rabbit2
            - RABBITMQ_ERLANG_COOKIE=CURIOAPPLICATION
        ports:
            - "5673:5672"
            - "15673:15672"
        networks:
            rabbit:
                ipv4_address: 172.24.0.6
        volumes:
            - "/var/lib/rabbitmq/.erlang.cookie:/root/.erlang.cookie"

    rabbit3:
        image: rabbitmq:management-alpine
        hostname: rabbit3
        environment:
            - RABBITMQ_NODENAME=rabbit3
            - RABBITMQ_ERLANG_COOKIE=CURIOAPPLICATION
        ports:
            - "5674:5672"
            - "15674:15672"
        networks:
            rabbit:
                ipv4_address: 172.24.0.7
        volumes:
            - "/var/lib/rabbitmq/.erlang.cookie:/root/.erlang.cookie"

5.创建并启动RabbitMQ容器

执行命令:

[root@localhost home]# docker-compose –up -d

6.组成集群

说明: rabbit1@rabbit1 第一个RabbitMQ的节点名,第二个是容器主机名
如果想用ram存储就用第一个shell文件, 用硬盘存储就用第二个shell文件

ram集群服务器shell文件
rabbitmqctl stop_app
rabbitmqctl reset
rabbitmqctl join_cluster rabbit1@rabbit1 --ram
rabbitmqctl start_app

disk集群服务器shell文件
rabbitmqctl stop_app
rabbitmqctl reset
rabbitmqctl join_cluster rabbit1@rabbit1
rabbitmqctl start_app

Rabbit2加入rabbit1节点:

[root@linuxpei sqa]# docker exec -it 675f /bin/bash
bash-4.4# hostname
rabbit2
bash-4.4# 
bash-4.4# rabbitmqctl stop_app
Stopping rabbit application on node rabbit2@rabbit2 ...
bash-4.4# 
bash-4.4# rabbitmqctl reset
Resetting node rabbit2@rabbit2 ...
bash-4.4# 
bash-4.4# rabbitmqctl join_cluster rabbit1@rabbit1 --ram
Clustering node rabbit2@rabbit2 with rabbit1@rabbit1
bash-4.4# 
bash-4.4# rabbitmqctl start_app
Starting node rabbit2@rabbit2 ...
 completed with 3 plugins.
bash-4.4# exit
exit
[root@linuxpei sqa]#

Rabbit3加入rabbit1节点:

[root@linuxpei sqa]# docker exec -it f634 /bin/bash
bash-4.4# hostname
rabbit3
bash-4.4# 
bash-4.4# rabbitmqctl stop_app
Stopping rabbit application on node rabbit3@rabbit3 ...
bash-4.4# 
bash-4.4# rabbitmqctl reset
Resetting node rabbit3@rabbit3 ...
bash-4.4# 
bash-4.4# rabbitmqctl join_cluster rabbit1@rabbit1
Clustering node rabbit3@rabbit3 with rabbit1@rabbit1
bash-4.4# 
bash-4.4# rabbitmqctl start_app
Starting node rabbit3@rabbit3 ...
 completed with 3 plugins.
bash-4.4# 
bash-4.4# exit
exit
[root@linuxpei sqa]# 

7.查看结果

在rabbit1容器中查看集群状态,此时可以看到集群中有三个节点在运行

[root@linuxpei sqa]# docker exec -it 7793d /bin/bash
bash-4.4# 
bash-4.4# hostname
rabbit1
bash-4.4# rabbitmqctl cluster_status
Cluster status of node rabbit1@rabbit1 ...
[{nodes,[{disc,[rabbit1@rabbit1,rabbit3@rabbit3]},{ram,[rabbit2@rabbit2]}]},
 {running_nodes,[rabbit3@rabbit3,rabbit2@rabbit2,rabbit1@rabbit1]},
 {cluster_name,<<"rabbit1@rabbit1">>},
 {partitions,[]},
 {alarms,[{rabbit3@rabbit3,[]},{rabbit2@rabbit2,[]},{rabbit1@rabbit1,[]}]}]
bash-4.4# 
bash-4.4#

登陆网页:http://172.17.202.82:15672 登陆用户名密码:admin/admin 查看结果:

8.RabbitMQ集群设置镜像队列策略

在任意一个节点(如:rabbit1)上执行:

[root@linuxpei sqa]# docker exec -it 7793d /bin/bash
bash-4.4# 
bash-4.4# hostname
rabbit1
bash-4.4# 
bash-4.4# rabbitmqctl set_policy ha-all "^" '{"ha-mode":"all"}'
Setting policy "ha-all" for pattern "^" to "{"ha-mode":"all"}" with priority "0" for vhost "/" ...
bash-4.4# 
bash-4.4#

将所有队列设置为镜像队列,即队列会被复制到各个节点,各个节点状态保持一直。
我们可以在rabbit2上查看策略

[root@linuxpei sqa]# docker exec -it 675 /bin/bash
bash-4.4# 
bash-4.4# hostname
rabbit2
bash-4.4# rabbitmqctl list_policies
Listing policies for vhost "/" ...
vhost	name	pattern	apply-to	definition	priority
/	ha-all	^	all	{"ha-mode":"all"}	0
bash-4.4#

二、(多台服务器)docker-compose搭建RabbitMQ集群

上面我们使用Docker搭建出了一个RabbitMQ单机集群,这种模式只能用来测试玩玩,无法再生产环境中使用。在生产环境中,需要把3个节点的集群分布到各个主机上面去。这个时候docker-compose就需要做调整,环境部署如下:

1.环境准备

服务器IP容器分配网段容器hostname容器的IP
172.17.202.82172.24.0.0/16rabbit1172.24.0.5
172.17.202.144172.21.0.0/16rabbit4172.21.0.4
172.17.202.155172.22.0.0/16rabbit5172.22.0.2

2.Centos7关闭防火墙

启动一个服务:systemctl start firewalld.service
关闭一个服务:systemctl stop firewalld.service
重启一个服务:systemctl restart firewalld.service
显示一个服务的状态:systemctl status firewalld.service
在开机时启用一个服务:systemctl enable firewalld.service
在开机时禁用一个服务:systemctl disable firewalld.service

3.安装Docker,Docker-compose

4.多机版docker-compose文件

在服务器:172.17.202.82上创建rabbit1(IP:172.24.0.5)容器

version: '3.6'

networks:
    rabbit:
        driver: bridge
        ipam:
          config:
            - subnet: 172.24.0.0/16

services:
    rabbit1:
        image: rabbitmq:management-alpine
        hostname: rabbit1
        ports:
            - "25672:25672"
            - "5672:5672"
            - "15672:15672"
            - "4369:4369"
        networks:
            rabbit:
                ipv4_address: 172.24.0.5
        environment:
            - RABBITMQ_NODENAME=rabbit1
            - RABBITMQ_ERLANG_COOKIE=CURIOAPPLICATION
            - RABBITMQ_DEFAULT_USER=admin
            - RABBITMQ_DEFAULT_PASS=admin
        volumes:
            - "/var/lib/rabbitmq/.erlang.cookie:/root/.erlang.cookie"    

在服务器:172.17.202.144上创建rabbit4(IP:172.21.0.4)容器

version: '3.6'

networks:
    rabbit:
        driver: bridge
        ipam:
          config:
            - subnet: 172.21.0.0/16

services:
    
    rabbit4:
        image: rabbitmq:management-alpine
        hostname: rabbit4
        environment:
            - RABBITMQ_NODENAME=rabbit4
            - RABBITMQ_ERLANG_COOKIE=CURIOAPPLICATION
        ports:
            - "5672:5672"
            - "15672:15672"
            - "25672:25672"
            - "4369:4369"
        networks:
            rabbit:
                ipv4_address: 172.21.0.4 
        extra_hosts:
            - "rabbit1:172.24.0.5"
        volumes:
            - "/var/lib/rabbitmq/.erlang.cookie:/root/.erlang.cookie"


在服务器:172.17.202.155上创建rabbit5(IP:172.22.0.2)容器

version: '3.6'

networks:
    rabbit:
        driver: bridge
        ipam:
          config:
            - subnet: 172.22.0.0/16

services:
    
    rabbit5:
        image: rabbitmq:management-alpine
        hostname: rabbit5
        environment:
            - RABBITMQ_NODENAME=rabbit5
            - RABBITMQ_ERLANG_COOKIE=CURIOAPPLICATION
        ports:
            - "5672:5672"
            - "15672:15672"
            - "25672:25672"
            - "4369:4369"
        networks:
            rabbit:
                ipv4_address: 172.22.0.2 
        extra_hosts:
            - "rabbit1:172.24.0.5"
        volumes:
            - "/var/lib/rabbitmq/.erlang.cookie:/root/.erlang.cookie"


5.创建并启动RabbitMQ容器

执行命令:

[root@localhost home]# docker-compose  –up  -d

6.两个主机的容器互访测试

要将两个主机上容器组成一个集群,必须要确保这两个容器能够互访。
在(172.17.202.144)服务器的容器内做以下测试:

144的容器ip	操作	目标服务器

172.21.0.4	ping	172.17.202.144 (ping本容器的主机)
172.21.0.4	ping	172.17.202.82  (ping82主机)
172.21.0.4	ping	172.21.0.4     (ping本容器) 
172.21.0.4	ping	172.24.0.5 (ping82主机的容器->不通)

可以发现,ping 82主机的容器 ping不通。 跨主机容器互访失败

[root@whsqadt02 ~]# docker exec -it c3f8 /bin/bash
bash-4.4# hostname
rabbit4
bash-4.4# 
bash-4.4# ping 172.17.202.144
PING 172.17.202.144 (172.17.202.144): 56 data bytes
64 bytes from 172.17.202.144: seq=0 ttl=64 time=0.257 ms
64 bytes from 172.17.202.144: seq=1 ttl=64 time=0.124 ms
bash-4.4#
bash-4.4# ping 172.17.202.82
PING 172.17.202.82 (172.17.202.82): 56 data bytes
64 bytes from 172.17.202.82: seq=0 ttl=63 time=0.922 ms
64 bytes from 172.17.202.82: seq=1 ttl=63 time=0.868 ms
bash-4.4#
bash-4.4# ping 172.21.0.4
PING 172.21.0.4 (172.21.0.4): 56 data bytes
64 bytes from 172.21.0.4: seq=0 ttl=64 time=0.145 ms
64 bytes from 172.21.0.4: seq=1 ttl=64 time=0.115 ms
bash-4.4#
bash-4.4# ping 172.24.0.5
PING 172.21.0.6 (172.24.0.5): 56 data bytes

没有响应

7.配置路由规则

方法一、添加路由方式

在144主机上执行以下命令行,添加路由规则:

route add -net 172.24.0.0/16 gw 172.17.202.82
或
ip route add 172.24.0.0/16 via 172.17.202.82

可以看到,以上命令是加的82主机和82容器的相关配置

反之,对82服务器添加路由规则:

route add -net 172.21.0.0/16 gw 172.17.202.144
或
ip route add 172.21.0.0/16 via 172.17.202.144

可以看到,以上命令是加的144主机和144容器的相关配置

删除路由方式:

route del -net 172.24.0.2 netmask 255.255.255.254 gw 172.17.202.82

注:172.24.0.0/16填写16还是24 是根据容器的网段网关决定,如下:

[root@linuxpei ~]# docker exec -it 779 /bin/bash
bash-4.4# 
bash-4.4# hostname
rabbit1
bash-4.4# 
bash-4.4# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
72: eth0@if73: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue state UP 
    link/ether 02:42:ac:18:00:05 brd ff:ff:ff:ff:ff:ff
    inet 172.24.0.5/16 brd 172.24.255.255 scope global eth0
       valid_lft forever preferred_lft forever
bash-4.4#

可以执行以下命令查看添加的路由规则:

[root@whsqadt02 ~]# route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
172.17.202.0    0.0.0.0         255.255.254.0   U     100    0        0 ens33
172.18.0.0      0.0.0.0         255.255.0.0     U     0      0        0 br-
172.24.0.0      linuxpei        255.255.0.0     UG    0      0        0 ens33
192.168.192.0   0.0.0.0         255.255.255.0   U     0      0        0 docker0
[root@whsqadt02 ~]#

再登陆到144主机的容器中,对82主机的容器进行ping操作:

[root@whsqadt02 ~]# docker exec -it c3f8 /bin/bash
bash-4.4# 
bash-4.4# ping 172.24.0.5
PING 172.24.0.5 (172.24.0.5): 56 data bytes
64 bytes from 172.24.0.5: seq=0 ttl=62 time=1.124 ms
64 bytes from 172.24.0.5: seq=1 ttl=62 time=1.128 ms
bash-4.4#

反之,再登陆82主机的容器做同样ping操作,查看互访结果。
同理:在对82和155服务器做以上操作
注:以上方式添加的路由规则,在重启网络后,路由规则会消失

方法二、添加永久路由

1、在144主机上编辑 /etc/sysconfig/static-routes : (没有static-routes的话就手动建立一个这样的文件)
例如:

any net 172.24.0.0/16 gw 172.17.202.82
any net 172.21.0.0/16 gw 172.17.202.144
或
any net 172.24.0.0 netmask 255.255.0.0 gw 172.17.202.82
any net 172.21.0.0 netmask 255.255.0.0 gw 172.17.202.144

2、在144主机上编辑/etc/sysconfig/network: (写入以下内容)

 # Add non interface-specific static-routes. 
  if [ -f /etc/sysconfig/static-routes ]; then 
     grep "^any" /etc/sysconfig/static-routes | while read ignore args ; do 
         /sbin/route add -$args 
      done 
  fi

做好以上操作后,重启144服务器,然后登陆容器内,进行144->82,144->155容器间互信测试。

同理:在对82和155服务器做以上操作

8.组成集群

说明: rabbit1@rabbit1 第一个RabbitMQ的节点名,第二个是容器主机名
如果想用ram存储就用第一个shell文件, 用硬盘存储就用第二个shell文件

ram集群服务器shell文件
rabbitmqctl stop_app
rabbitmqctl reset
rabbitmqctl join_cluster rabbit1@rabbit1 --ram
rabbitmqctl start_app

disk集群服务器shell文件
rabbitmqctl stop_app
rabbitmqctl reset
rabbitmqctl join_cluster rabbit1@rabbit1
rabbitmqctl start_app

Rabbit4加入rabbit1节点:

[root@whsqadt02 sqa]# docker exec -it 675f /bin/bash
bash-4.4# hostname
rabbit4
bash-4.4# 
bash-4.4# rabbitmqctl stop_app
Stopping rabbit application on node rabbit2@rabbit2 ...
bash-4.4# 
bash-4.4# rabbitmqctl reset
Resetting node rabbit4@rabbit4 ...
bash-4.4# 
bash-4.4# rabbitmqctl join_cluster rabbit1@rabbit1 --ram
Clustering node rabbit4@rabbit4 with rabbit1@rabbit1
bash-4.4# 
bash-4.4# rabbitmqctl start_app
Starting node rabbit4@rabbit4 ...
 completed with 3 plugins.
bash-4.4# exit
exit
[root@whsqadt02 sqa]#

rabbit5加入rabbit1节点:

[root@whsqadt03 sqa]# docker exec -it f634 /bin/bash
bash-4.4# hostname
rabbit5
bash-4.4# 
bash-4.4# rabbitmqctl stop_app
Stopping rabbit application on node rabbit5@rabbit5 ...
bash-4.4# 
bash-4.4# rabbitmqctl reset
Resetting node rabbit5@rabbit5 ...
bash-4.4# 
bash-4.4# rabbitmqctl join_cluster rabbit1@rabbit1
Clustering node rabbit5@rabbit5 with rabbit1@rabbit1
bash-4.4# 
bash-4.4# rabbitmqctl start_app
Starting node rabbit5@rabbit5 ...
 completed with 3 plugins.
bash-4.4# 
bash-4.4# exit
exit
[root@whsqadt03 sqa]# 

9.查看结果

在rabbit1容器中查看集群状态,此时可以看到集群中有三个节点在运行

[root@linuxpei sqa]# docker exec -it 7793d /bin/bash
bash-4.4# 
bash-4.4# hostname
rabbit1
bash-4.4# rabbitmqctl cluster_status
Cluster status of node rabbit1@rabbit1 ...
[{nodes,[{disc,[rabbit1@rabbit1,rabbit5@rabbit5]},{ram,[rabbit4@rabbit4]}]},
 {running_nodes,[rabbit4@rabbit4,rabbit5@rabbit5,rabbit1@rabbit1]},
 {cluster_name,<<"rabbit1@rabbit1">>},
 {partitions,[]},
 {alarms,[{rabbit4@rabbit4,[]},{rabbit5@rabbit5,[]},{rabbit1@rabbit1,[]}]}]
bash-4.4# 
bash-4.4#

登陆网页:http://172.17.202.82:15672 登陆用户名密码:admin/admin 查看结果:

10.RabbitMQ集群设置镜像队列策略

在任意一个节点(如:rabbit1)上执行:

[root@linuxpei sqa]# docker exec -it 7793d /bin/bash
bash-4.4# 
bash-4.4# hostname
rabbit1
bash-4.4# 
bash-4.4# rabbitmqctl set_policy ha-all "^" '{"ha-mode":"all"}'
Setting policy "ha-all" for pattern "^" to "{"ha-mode":"all"}" with priority "0" for vhost "/" ...
bash-4.4# 
bash-4.4#

将所有队列设置为镜像队列,即队列会被复制到各个节点,各个节点状态保持一直。
我们可以在rabbit4上查看策略

[root@whsqadt03 sqa]# docker exec -it 675 /bin/bash
bash-4.4# 
bash-4.4# hostname
rabbit4
bash-4.4# rabbitmqctl list_policies
Listing policies for vhost "/" ...
vhost	name	pattern	apply-to	definition	priority
/	ha-all	^	all	{"ha-mode":"all"}	0
bash-4.4#

@ 参考其他的博客文档,并结合自己的整理,完成部署教程,共勉。

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值