【Linux】RocketMQ-Cluster集群搭建实战

1.RocketMQ-Cluster集群搭建

1.1.说明

RocketMQ是一款分布式消息中间件,其各方面的性能都比目前已有的消息队列要好。RocketMQ默认采用长轮询的拉模式,单机支持千万级别的消息堆积,可以非常好的应用在海量消息系统中。
集群的模式有四种,单master模式、多master模式、多master多slave异步复制模式、多master多slave同步复制模式,推荐生产环境使用多master多slave同步复制模式,本文也使用此模式进行搭建集群。该模式下每个Master配置一个Slave,有多对Master-Slave,采用同步双写方式,即只有主备都写成功,才向应用返回成功,数据与服务都无单点故障,Master宕机情况下,消息无延迟,服务可用性与数据可用性都非常高。

1.2.环境配置

1.2.1.准备

1.下载rocketmq-all-4.2.0-bin-release.zip,下载地址:

http://rocketmq.apache.org/dowloading/releases/。

解压后并将文件移动到指定位置/usr/local/。

unzip rocketmq-all-4.2.0-bin-release.zip -d rocketmq-all-4.2.0-bin-release
mv rocketmq-all-4.2.0-bin-release /usr/local/

可使用安装命令:

 yum install java-1.8.0-openjdk*

2.关闭防火墙

本次搭建使用虚拟机,为了防止出现防火墙问题,直接关闭了集群中节点服务器的防火墙,如果是在生产环境不允许关闭防火墙,那么就开放集群中用到的nameserver和broker端口号即可。

firewall-cmd --zone=public --add-port=(端口号)/tcp  --permanent & firewall-cmd --reload(开启指定端口)
或
systemctl stop firewalld.service (禁用防火墙)
systemctl disable firewalld.service(停止并禁用开机启动)

3.配置环境变量
配置JAVA_HOME(本机jdk安装后的地址为/usr/lib/jvm):

echo 'export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-0.uelc20.x86_64' >> /etc/profile
source /etc/profile
echo $JAVA_HOME 

配置rocketmq的环境变量:ROCKETMQ_HOME,值为rocketmq解压安装文件夹的路径,然后将ROCKETMQ_HOME/bin添加到PATH中去。

vim /etc/profile
在profile 文件末尾添加
set rocketmq
ROCKETMQ_HOME=/usr/local/rocketmq-all-4.2.0-bin-release
PATH=$PATH:$ROCKETMQ_HOME/bin
export ROCKETMQ_HOME PATH
#保存退出并生效
:wq
source /etc/profile

1.3.搭建集群

1.3.1.修改vim /etc/hosts

nameserver
10.100.20.91  rocketmq-nameserver1
10.100.20.92  rocketmq-nameserver2
broker
10.100.20.91  rocketmq-master1
10.100.20.91  rocketmq-slave2
10.100.20.92  rocketmq-master2
10.100.20.92  rocketmq-slave1

1.3.2.重启网卡

systemctl restart NetworkManager

1.3.3.创建消息存储路径

10.100.20.91服务器
mkdir -p /usr/local/rocketmq/store/broker-a
mkdir -p /usr/local/rocketmq/store/broker-a/commitlog
mkdir -p /usr/local/rocketmq/store/broker-a/consumequeue
mkdir -p /usr/local/rocketmq/store/broker-a/index
mkdir -p /usr/local/rocketmq/store/broker-b-s
mkdir -p /usr/local/rocketmq/store/broker-b-s/commitlog
mkdir -p /usr/local/rocketmq/store/broker-b-s/consumequeue
mkdir -p /usr/local/rocketmq/store/broker-b-s/index
10.100.20.92服务器
mkdir -p /usr/local/rocketmq/store/broker-b
mkdir -p /usr/local/rocketmq/store/broker-b/commitlog
mkdir -p /usr/local/rocketmq/store/broker-b/consumequeue
mkdir -p /usr/local/rocketmq/store/broker-b/index
mkdir -p /usr/local/rocketmq/store/broker-a-s
mkdir -p /usr/local/rocketmq/store/broker-a-s/commitlog
mkdir -p /usr/local/rocketmq/store/broker-a-s/consumequeue
mkdir -p /usr/local/rocketmq/store/broker-a-s/index

1.3.4.修改broker配置文件

10.100.20.91服务器 master
vim /usr/local/rocketmq-all-4.2.0-bin-release/conf/2m-2s-sync/broker-a.properties
修改配置如下:
#暴露的外网IP
brokerIP1=1号服务器公网IP
brokerIP2=1号服务器公网IP
#所属集群名字
brokerClusterName=rocketmq-cluster
#broker名字,注意此处不同的配置文件填写的不一样
brokerName=broker-a
#0 表示 Master,>0 表示 Slave
brokerId=0
#nameServer地址,分号分割
namesrvAddr=rocketmq-nameserver1:9876;rocketmq-nameserver2:9876
#在发送消息时,自动创建服务器不存在的topic,默认创建的队列数
defaultTopicQueueNums=4
#是否允许 Broker 自动创建Topic,建议线下开启,线上关闭
autoCreateTopicEnable=true
#是否允许 Broker 自动创建订阅组,建议线下开启,线上关闭
autoCreateSubscriptionGroup=true
#Broker 对外服务的监听端口
listenPort=10911
#删除文件时间点,默认凌晨 4点
deleteWhen=04
#文件保留时间,默认 48 小时
fileReservedTime=120
#commitLog每个文件的大小默认1G
mapedFileSizeCommitLog=1073741824
#ConsumeQueue每个文件默认存30W条,根据业务情况调整
mapedFileSizeConsumeQueue=300000
#destroyMapedFileIntervalForcibly=120000
#redeleteHangedFileInterval=120000
#检测物理文件磁盘空间
diskMaxUsedSpaceRatio=88
#存储路径
storePathRootDir=/usr/local/rocketmq/store/broker-a
#commitLog 存储路径
storePathCommitLog=/usr/local/rocketmq/store/broker-a/commitlog
#消费队列存储路径存储路径
storePathConsumeQueue=/usr/local/rocketmq/store/broker-a/consumequeue
#消息索引存储路径
storePathIndex=/usr/local/rocketmq/store/broker-a/index
#checkpoint 文件存储路径
storeCheckpoint=/usr/local/rocketmq/store/broker-a/checkpoint
#abort 文件存储路径
abortFile=/usr/local/rocketmq/store/broker-a/abort
#限制的消息大小
maxMessageSize=65536
#flushCommitLogLeastPages=4
#flushConsumeQueueLeastPages=2
#flushCommitLogThoroughInterval=10000
#flushConsumeQueueThoroughInterval=60000
#Broker 的角色
#- ASYNC_MASTER 异步复制Master
#- SYNC_MASTER 同步双写Master
#- SLAVE
brokerRole=SYNC_MASTER
#刷盘方式
#- ASYNC_FLUSH 异步刷盘
#- SYNC_FLUSH 同步刷盘
flushDiskType=SYNC_FLUSH
#checkTransactionMessageEnable=false
#发消息线程池数量
#sendMessageThreadPoolNums=128
#拉消息线程池数量
#pullMessageThreadPoolNums=128
10.100.20.91服务器 slave

vim /usr/local/rocketmq-all-4.2.0-bin-release/conf/2m-2s-sync/broker-b-s.properties
修改配置如下:
#所属集群名字
brokerClusterName=rocketmq-cluster
brokerName=broker-b
brokerId=1
namesrvAddr=rocketmq-nameserver1:9876;rocketmq-nameserver2:9876
defaultTopicQueueNums=4
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
listenPort=11011
deleteWhen=04
fileReservedTime=120
mapedFileSizeCommitLog=1073741824
mapedFileSizeConsumeQueue=300000
#destroyMapedFileIntervalForcibly=120000
#redeleteHangedFileInterval=120000
diskMaxUsedSpaceRatio=88
storePathRootDir=/usr/local/rocketmq/store/broker-b-s
storePathCommitLog=/usr/local/rocketmq/store/broker-b-s/commitlog
storePathConsumeQueue=/usr/local/rocketmq/store/broker-b-s/consumequeue
storePathIndex=/usr/local/rocketmq/store/broker-b-s/index
storeCheckpoint=/usr/local/rocketmq/store/broker-b-s/checkpoint
abortFile=/usr/local/rocketmq/store/broker-b-s/abort
maxMessageSize=65536
#flushCommitLogLeastPages=4
#flushConsumeQueueLeastPages=2
#flushCommitLogThoroughInterval=10000
#flushConsumeQueueThoroughInterval=60000
brokerRole=SLAVE
flushDiskType=ASYNC_FLUSH
#checkTransactionMessageEnable=false
#sendMessageThreadPoolNums=128
#pullMessageThreadPoolNums=128
10.100.20.92服务器 master
vim /usr/local/rocketmq-all-4.2.0-bin-release/conf/2m-2s-sync/broker-b.properties
修改配置如下:
brokerIP1=2号服务器公网IP
brokerIP2=2号服务器公网IP
brokerClusterName=rocketmq-cluster
brokerName=broker-b
brokerId=0
namesrvAddr=rocketmq-nameserver1:9876;rocketmq-nameserver2:9876
defaultTopicQueueNums=4
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
listenPort=10911
deleteWhen=04
fileReservedTime=120
mapedFileSizeCommitLog=1073741824
mapedFileSizeConsumeQueue=300000
#destroyMapedFileIntervalForcibly=120000
#redeleteHangedFileInterval=120000
diskMaxUsedSpaceRatio=88
storePathRootDir=/usr/local/rocketmq/store/broker-b
storePathCommitLog=/usr/local/rocketmq/store/broker-b/commitlog
storePathConsumeQueue=/usr/local/rocketmq/store/broker-b/consumequeue
storePathIndex=/usr/local/rocketmq/store/broker-b/index
storeCheckpoint=/usr/local/rocketmq/store/broker-b/checkpoint
abortFile=/usr/local/rocketmq/store/broker-b/abort
maxMessageSize=65536
#flushCommitLogLeastPages=4
#flushConsumeQueueLeastPages=2
#flushCommitLogThoroughInterval=10000
#flushConsumeQueueThoroughInterval=60000
brokerRole=SYNC_MASTER
flushDiskType=SYNC_FLUSH
#checkTransactionMessageEnable=false
#sendMessageThreadPoolNums=128
#pullMessageThreadPoolNums=128
10.100.20.92服务器 slave
vim /usr/local/rocketmq-all-4.2.0-bin-release/conf/2m-2s-sync/broker-a-s.properties
修改配置如下:
brokerClusterName=rocketmq-cluster
brokerName=broker-a
brokerId=1
namesrvAddr=rocketmq-nameserver1:9876;rocketmq-nameserver2:9876
defaultTopicQueueNums=4
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
listenPort=11011
deleteWhen=04
fileReservedTime=120
mapedFileSizeCommitLog=1073741824
mapedFileSizeConsumeQueue=300000
#destroyMapedFileIntervalForcibly=120000
#redeleteHangedFileInterval=120000
diskMaxUsedSpaceRatio=88
storePathRootDir=/usr/local/rocketmq/store/broker-a-s
storePathCommitLog=/usr/local/rocketmq/store/broker-a-s/commitlog
storePathConsumeQueue=/usr/local/rocketmq/store/broker-a-s/consumequeue
storePathIndex=/usr/local/rocketmq/store/broker-a-s/index
storeCheckpoint=/usr/local/rocketmq/store/broker-a-s/checkpoint
abortFile=/usr/local/rocketmq/store/broker-a-s/abort
maxMessageSize=65536
#flushCommitLogLeastPages=4
#flushConsumeQueueLeastPages=2
#flushCommitLogThoroughInterval=10000
#flushConsumeQueueThoroughInterval=60000
brokerRole=SLAVE
flushDiskType=ASYNC_FLUSH
#checkTransactionMessageEnable=false
#sendMessageThreadPoolNums=128
#pullMessageThreadPoolNums=128

1.3.5.修改启动脚本文件

vim /usr/local/rocketmq-all-4.2.0-bin-release/bin/runbroker.sh
JAVA_OPT="${JAVA_OPT} -server -Xms256m -Xmx256m -Xmn128m"

在这里插入图片描述

vim /usr/local/rocketmq-all-4.2.0-bin-release/bin/runserver.sh
JAVA_OPT="${JAVA_OPT} -server -Xms256m -Xmx256m -Xmn128m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m"

在这里插入图片描述

1.3.6.服务启动

1.启动NameServe集群(分别在服务器启动)

cd /usr/local/rocketmq-all-4.2.0-bin-release/bin
nohup sh mqnamesrv &

2.启动Broker集群
10.100.90.91服务器master1:

cd /usr/local/rocketmq-all-4.2.0-bin-release/bin
nohup sh mqbroker -c /usr/local/rocketmq-all-4.2.0-bin-release/conf/2m-2s-sync/broker-a.properties &

10.100.90.91服务器slave1:

cd /usr/local/rocketmq-all-4.2.0-bin-release/bin
nohup sh mqbroker -c /usr/local/rocketmq-all-4.2.0-bin-release/conf/2m-2s-sync/broker-b-s.properties &

10.100.90.92服务器master2:

cd /usr/local/rocketmq-all-4.2.0-bin-release/bin/
nohup sh mqbroker -c /usr/local/rocketmq-all-4.2.0-bin-release/conf/2m-2s-sync/broker-b.properties &

10.100.90.92服务器slave2:

cd /usr/local/rocketmq-all-4.2.0-bin-release/bin/
nohup sh mqbroker -c /usr/local/rocketmq-all-4.2.0-bin-release/conf/2m-2s-sync/broker-a-s.properties &

1.3.7.查看进程状态

启动后通过JPS查看启动进程
查看nameServer日志
tail -500f ~/logs/rocketmqlogs/namesrv.log
查看broker日志
tail -500f ~/logs/rocketmqlogs/broker.log
在集群中任意一个服务器上,执行以下命令查看该集群上所启动的各实例信息(namesrvAddr只写该节点的nameserver地址即可):
mqadmin clusterList --namesrvAddr=10.100.20.91:9876

在这里插入图片描述

1.3.8.Rocketmq-console控制台连接集群

RocketMQ-Console是RocketMQ项目的扩展插件,是一个图形化管理控制台,提供Broker集群状态查看,Topic管理,Producer、Consumer状态展示,消息查询等常用功能,这个功能在安装好RocketMQ后需要额外单独安装、运行。
1.rocketmq-console下载

进入rocketmq-externals项目的GitHub地址,如下图,可看到RocketMQ项目的诸多扩展项目,其中就包含我们需要下载的rocketmq-console。
在这里插入图片描述

克隆项目到本地
git clone https://github.com/apache/rocketmq-externals.git
#比较慢的话,可以用国内的gitee
git clone https://gitee.com/mirrors/RocketMQ-Externals.git
cd rocketmq-externals/rocketmq-console //切换到目录

2.修改rocketmq-console中配置namesrv集群地址

vim src/main/resources/application.properties
内容改为
rocketmq.config.namesrvAddr=10.100.20.91:9876;10.100.20.92:9876

在这里插入图片描述

3.打包(跳过测试打包)

mvn clean package -Dmaven.test.skip=true
若没有mvn则:
yum install maven

在这里插入图片描述

4.启动rocketmq-console:(其中一台服务器上运行都可以)

cd target/
java -jar rocketmq-console-ng-2.0.0.jar

在这里插入图片描述

5.访问工作台(访问地址10.100.20.91:8080/#/)
在这里插入图片描述

  • 0
    点赞
  • 1
    收藏
    觉得还不错? 一键收藏
  • 打赏
    打赏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

小白鸽i

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付:¥1
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值