RocketMQ集群部署

新公司用的消息中间件为RocketMQ,由于我之前用的都是ActiveMQ、RabbitMQ和Kafka,所以就在这整理下RocketMQ的部署。
一、整体规划

192.168.0.121NameServer1 / broker-01-master / broker-02-slave
192.168.0.122NameServer2 / broker-02-master / broker-01-slave

二、准备工作
https://www.oracle.com/technetwork/java/javase/archive-139210.html 下载jdk,这边选择的是jdk-8u144-linux-x64.tar.gz

http://rocketmq.apache.org/dowloading/releases/ 下载RocketMQ,这边选择的版本是rocketmq-all-4.3.1-bin-release.zip

官网给出的环境要求是:
1.JDK版本1.8+ (RocketMQ是JAVA写的)
2.Maven3.2.x以上 (用来打包console管理页面)

三、部署JAVA环境

tar -zxvf jdk-8u144-linux-x64.tar.gz -C /usr/local/
ln -s /usr/local/jdk1.8.0_144 /usr/local/jdk
sed -i.ori '$a export JAVA_HOME=/usr/local/jdk\nexport PATH=$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$PATH\nexport CLASSPATH=.:$JAVA_HOME/lib:$JAVA_HOME/jre/lib:$JAVA_HOME/lib/tools.jar' /etc/profile
source /etc/profile

#出现下面结果证明部署成功
[root@localhost opt]# java -version
java version "1.8.0_144"
Java(TM) SE Runtime Environment (build 1.8.0_144-b01)
Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)

四、安装RocketMQ
1.解压重命名

unzip rocketmq-all-4.3.1-bin-release.zip -C /usr/local/
mv /usr/local/rocketmq-all-4.3.1-bin-release /usr/local/apache-rocketmq-4.3.1

2.配置模式介绍
rocketmq默认给出了三种建议配置模式:
1)2m-2s-async(主从异步),----本文采用这种
2)2m-2s-sync(主从同步)
3)2m-noslave(仅master)

3.修改第一台服务器(192.168.0.121)上的配置文件
我们在这做个约定,就是把要用到的配置文件都放到/usr/local/apache-rocketmq-4.3.1/conf目录下。

cd /usr/local/apache-rocketmq-4.3.1/conf/2m-2s-async/
cp broker-a.properties ../broker-01-master.properties
cp broker-b.properties ../broker-02-slave.properties
cd ..

3.1修改 broker-01-master.properties
vim /usr/local/apache-rocketmq-4.3.1/conf/broker-01-master.properties

#所属集群名字
brokerClusterName=rocketmq-cluster
#存在broker主从时,在broker主节点上配置了brokerIP2的话,broker从节点会连接主节点配置的brokerIP2来同步
#默认不配置brokerIP1和brokerIP2时,都会根据当前网卡选择一个IP使用,当你的机器有多块网卡时,很有可能会有问题。比如,我遇到的问题是我机器上有两个IP,一个公网IP,一个私网IP,结果默认选择的走公网IP,这是不正确的,我期望的是所有业务内部通信都走内网。
brokerIP1=192.168.0.121
brokerIP2=192.168.0.121
#broker名字,名字可重复,为了管理,每个master起一个名字,他的slave同他
brokerName=broker-01
#0 表示 Master,>0 表示 Slave
brokerId=0
#nameServer地址,分号分割
namesrvAddr=192.168.0.121:9876;192.168.0.122:9876
#在发送消息时,自动创建服务器不存在的topic,默认创建的队列数
defaultTopicQueueNums=4
#是否允许 Broker 自动创建Topic,建议线下开启,线上关闭
autoCreateTopicEnable=true
#是否允许 Broker 自动创建订阅组,建议线下开启,线上关闭
autoCreateSubscriptionGroup=true
#Broker 对外服务的监听端口,
listenPort=10911
#删除文件时间点,默认凌晨 4点
deleteWhen=04
#文件保留时间,默认 48 小时
fileReservedTime=120
#commitLog每个文件的大小,默认为1G
mapedFileSizeCommitLog=1073741824
#ConsumeQueue每个文件默认存30W条,根据业务情况调整
mapedFileSizeConsumeQueue=300000
#destroyMapedFileIntervalForcibly=120000
#redeleteHangedFileInterval=120000
#检测物理文件磁盘空间
diskMaxUsedSpaceRatio=88
#存储路径
storePathRootDir=/usr/local/apache-rocketmq-4.3.1/store-01-master
#commitLog存储路径
storePathCommitLog=/usr/local/apache-rocketmq-4.3.1/store-01-master/commitlog
#消费队列存储路径
storePathConsumeQueue=/usr/local/apache-rocketmq-4.3.1/store-01-master/consumequeue
#消息索引存储路径
storePathIndex=/usr/local/apache-rocketmq-4.3.1/store-01-master/index
#checkpoint 文件存储路径
storeCheckpoint=/usr/local/apache-rocketmq-4.3.1/store-01-master/checkpoint
#abort文件存储路径
abortFile=/usr/local/apache-rocketmq-4.3.1/store-01-master/abort
#限制的消息大小
maxMessageSize=65536
#flushCommitLogLeastPages=4
#flushConsumeQueueLeastPages=2
#flushCommitLogThoroughInterval=10000
#flushConsumeQueueThoroughInterval=60000
#Broker 的角色
#- ASYNC_MASTER 异步复制Master
#- SYNC_MASTER 同步双写Master
#- SLAVE
brokerRole=ASYNC_MASTER
#刷盘方式
#- ASYNC_FLUSH 异步刷盘
#- SYNC_FLUSH 同步刷盘
flushDiskType=ASYNC_FLUSH
#checkTransactionMessageEnable=false
#发消息线程池数量
#sendMessageThreadPoolNums=128
#拉消息线程池数量
#pullMessageThreadPoolNums=128

3.2修改 broker-02-slave.properties
vim /usr/local/apache-rocketmq-4.3.1/conf/broker-02-slave.properties

#所属集群名称
brokerClusterName=rocketmq-cluster
#brokerIP1为当前broker监听的IP 
brokerIP1=192.168.0.121
#broker名字,名字可重复,为了管理,每个master起一个名字,他的slave同他
brokerName=broker-02
#0 表示 Master,>0 表示 Slave
brokerId=1
#nameServer地址,分号分割
namesrvAddr=192.168.0.121:9876;192.168.0.122:9876
#在发送消息时,自动创建服务器不存在的topic,默认创建的队列数
defaultTopicQueueNums=4
#是否允许 Broker 自动创建Topic,建议线下开启,线上关闭
autoCreateTopicEnable=true
#是否允许 Broker 自动创建订阅组,建议线下开启,线上关闭
autoCreateSubscriptionGroup=true
#Broker 对外服务的监听端口,
listenPort=10921
#删除文件时间点,默认凌晨4点
deleteWhen=04
#文件保留时间,默认48小时
fileReservedTime=120
#commitLog每个文件的大小,默认1G
mapedFileSizeCommitLog=1073741824
#ConsumeQueue每个文件默认存30W条,根据业务情况调整
mapedFileSizeConsumeQueue=300000
#destroyMapedFileIntervalForcibly=120000
#redeleteHangedFileInterval=120000
#检测物理文件磁盘空间
diskMaxUsedSpaceRatio=88
#存储路径
storePathRootDir=/usr/local/apache-rocketmq-4.3.1/store-02-slave
#commitLog 存储路径
storePathCommitLog=/usr/local/apache-rocketmq-4.3.1/store-02-slave/commitlog
#消费队列存储路径存储路径
storePathConsumeQueue=/usr/local/apache-rocketmq-4.3.1/store-02-slave/consumequeue
#消息索引存储路径
storePathIndex=/usr/local/apache-rocketmq-4.3.1/store-02-slave/index
#checkpoint 文件存储路径
storeCheckpoint=/usr/local/apache-rocketmq-4.3.1/store-02-slave/checkpoint
#abort 文件存储路径
abortFile=/usr/local/apache-rocketmq-4.3.1/store-02-slave/abort
#限制的消息大小
maxMessageSize=65536
#flushCommitLogLeastPages=4
#flushConsumeQueueLeastPages=2
#flushCommitLogThoroughInterval=10000
#flushConsumeQueueThoroughInterval=60000
#Broker 的角色
#- ASYNC_MASTER 异步复制Master
#- SYNC_MASTER 同步双写Master
#- SLAVE
brokerRole=SLAVE
#刷盘方式
#- ASYNC_FLUSH 异步刷盘
#- SYNC_FLUSH 同步刷盘
flushDiskType=ASYNC_FLUSH
#checkTransactionMessageEnable=false
#发消息线程池数量
#sendMessageThreadPoolNums=128
#拉消息线程池数量
#pullMessageThreadPoolNums=128

3.3修改NameServer的启动文件
由于默认的NameServer的启动文件的JVM内存给的比较大,由于这里是测试环境,所以这边需要改小点。如果是线上的话,就根据实际情况来。
vim /usr/local/apache-rocketmq-4.3.1/bin/runserver.sh
在这里插入图片描述
3.4修改broker的启动文件
和上面一样,broker的启动文件的JVM参数也需要调整下。
vim /usr/local/apache-rocketmq-4.3.1/bin/runbroker.sh
在这里插入图片描述
4.拷贝RocketMQ到第二台服务器(192.168.0.122)
将192.168.0.121上的/usr/local/apache-rocketmq-4.3.1拷贝到192.168.0.122上

scp -r /usr/local/apache-rocketmq-4.3.1 root@192.168.0.122:/usr/local/

5.修改第二台服务器(192.168.0.122)上的配置文件

cd /usr/local/apache-rocketmq-4.3.1/conf
mv broker-01-master.properties broker-02-master.properties
mv broker-02-slave.properties broker-01-slave.properties

vim broker-02-master.properties

brokerClusterName=rocketmq-cluster
brokerIP1=192.168.0.122
brokerIP2=192.168.0.122
brokerName=broker-02
brokerId=0
namesrvAddr=192.168.0.121:9876;192.168.0.122:9876
defaultTopicQueueNums=4
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
listenPort=10911
deleteWhen=04
fileReservedTime=120
mapedFileSizeCommitLog=1073741824
mapedFileSizeConsumeQueue=300000
diskMaxUsedSpaceRatio=88
storePathRootDir=/usr/local/apache-rocketmq-4.3.1/store-02-master
storePathCommitLog=/usr/local/apache-rocketmq-4.3.1/store-02-master/commitlog
storePathConsumeQueue=/usr/local/apache-rocketmq-4.3.1/store-02-master/consumequeue
storePathIndex=/usr/local/apache-rocketmq-4.3.1/store-02-master/index
storeCheckpoint=/usr/local/apache-rocketmq-4.3.1/store-02-master/checkpoint
abortFile=/usr/local/apache-rocketmq-4.3.1/store-02-master/abort
maxMessageSize=65536
brokerRole=ASYNC_MASTER
flushDiskType=ASYNC_FLUSH

vim broker-01-slave.properties

brokerClusterName=rocketmq-cluster
brokerIP1=192.168.0.122
brokerName=broker-01
brokerId=1
namesrvAddr=192.168.0.121:9876;192.168.0.122:9876
defaultTopicQueueNums=4
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
listenPort=10921
deleteWhen=04
fileReservedTime=120
mapedFileSizeCommitLog=1073741824
mapedFileSizeConsumeQueue=300000
diskMaxUsedSpaceRatio=88
storePathRootDir=/usr/local/apache-rocketmq-4.3.1/store-01-slave
storePathCommitLog=/usr/local/apache-rocketmq-4.3.1/store-01-slave/commitlog
storePathConsumeQueue=/usr/local/apache-rocketmq-4.3.1/store-01-slave/consumequeue
storePathIndex=/usr/local/apache-rocketmq-4.3.1/store-01-slave/index
storeCheckpoint=/usr/local/apache-rocketmq-4.3.1/store-01-slave/checkpoint
abortFile=/usr/local/apache-rocketmq-4.3.1/store-01-slave/abort
maxMessageSize=65536
brokerRole=SLAVE
flushDiskType=ASYNC_FLUSH

6.创建日志和存储文件等的存放目录

#节点1上执行
mkdir -p /usr/local/apache-rocketmq-4.3.1/logs /usr/local/apache-rocketmq-4.3.1/store-01-master /usr/local/apache-rocketmq-4.3.1/store-01-master/commitlog /usr/local/apache-rocketmq-4.3.1/store-01-master/consumequeue /usr/local/apache-rocketmq-4.3.1/store-01-master/index /usr/local/apache-rocketmq-4.3.1/store-02-slave /usr/local/apache-rocketmq-4.3.1/store-02-slave/commitlog /usr/local/apache-rocketmq-4.3.1/store-02-slave/consumequeue /usr/local/apache-rocketmq-4.3.1/store-02-slave/index
#节点2上执行
mkdir -p /usr/local/apache-rocketmq-4.3.1/logs /usr/local/apache-rocketmq-4.3.1/store-01-slave /usr/local/apache-rocketmq-4.3.1/store-01-slave/commitlog /usr/local/apache-rocketmq-4.3.1/store-01-slave/consumequeue /usr/local/apache-rocketmq-4.3.1/store-01-slave/index /usr/local/apache-rocketmq-4.3.1/store-02-master /usr/local/apache-rocketmq-4.3.1/store-02-master/commitlog /usr/local/apache-rocketmq-4.3.1/store-02-master/consumequeue /usr/local/apache-rocketmq-4.3.1/store-02-master/index

五、启动
1.启动NameServer

#两台上都执行
nohup sh /usr/local/apache-rocketmq-4.3.1/bin/mqnamesrv > /usr/local/apache-rocketmq-4.3.1/logs/mqnamesrv.log 2>&1 &

2.启动broker

#在节点1上启动broker-01-master
nohup sh /usr/local/apache-rocketmq-4.3.1/bin/mqbroker -c /usr/local/apache-rocketmq-4.3.1/conf/broker-01-master.properties > /usr/local/apache-rocketmq-4.3.1/logs/broker-01-master.log 2>&1 &

#在节点2上启动broker-01-slave
nohup sh /usr/local/apache-rocketmq-4.3.1/bin/mqbroker -c /usr/local/apache-rocketmq-4.3.1/conf/broker-01-slave.properties > /usr/local/apache-rocketmq-4.3.1/logs/broker-01-slave.log 2>&1 &

#在节点2上启动broker-02-maste
nohup sh /usr/local/apache-rocketmq-4.3.1/bin/mqbroker -c /usr/local/apache-rocketmq-4.3.1/conf/broker-02-master.properties > /usr/local/apache-rocketmq-4.3.1/logs/broker-02-master.log 2>&1 &

#在节点1上启动broker-02-slave
nohup sh /usr/local/apache-rocketmq-4.3.1/bin/mqbroker -c /usr/local/apache-rocketmq-4.3.1/conf/broker-02-slave.properties > /usr/local/apache-rocketmq-4.3.1/logs/broker-02-slave.log 2>&1 &

全部启动完成之后可以通过jps命令查看进程
在这里插入图片描述
3.查看集群的启动状况
可以通过下面命令来查看集群的启动状况

/usr/local/apache-rocketmq-4.3.1/bin/mqadmin clusterList -n 192.168.0.121:9876

在这里插入图片描述
六、安装web管理界面
rocketmq除了命令行的管理方式之外,还有web界面的管理方式。
apache提供一个开源的扩展项目: https://github.com/apache/rocketmq-externals 里面包含一个子项目rocketmq-console。
我们要使用这个web管理界面,要做的就是克隆代码,修改配置文件,编译打包,启动服务4个步骤。
1.克隆代码到电脑上(我电脑上有maven环境,不想在服务器上重新安装了。。)
2.修改配置文件
配置文件的路径为:rocketmq-externals\rocketmq-console\src\main\resources\application.properties
我们在"rocketmq.config.namesrvAddr="后面加上自己的NameServer服务器的地址即可。

rocketmq.config.namesrvAddr=192.168.0.121:9876;192.168.0.122:9876

3.打包

mvn clean package -Dmaven.test.skip=true

在这里插入图片描述
4.将jar包上传到服务器并启动

java -jar rocketmq-console-ng-1.0.0.jar > /usr/local/apache-rocketmq-4.3.1/logs/mq-console.log 2>&1 &

5.页面管理
在这里插入图片描述
参考文章:
https://blog.csdn.net/weixin_40533111/article/details/84451219

  • 0
    点赞
  • 7
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值