Rocketmq异步集群部署

1、部署说明

CentOS 7.7
RocketMQ: 4.6.0
MASETER:

• 10.10.0.221 A
• 10.10.0.222 B
  SLAVE :
• 10.10.0.221 B
• 10.10.0.222 A

2、原理：

1） nameServer
Name Server是一个几乎无状态节点，可集群部署，节点之间无任何信息同步。
2) Broker
分为master和slave，1的master可以对应多个slave ，但是1个slave只能对应1个master
，通过brokerId来区分。
每个Broker与NameServer集群中的所有节点建立长连接，定时(每隔30s)注册Topic信息到所有Name Server。Name Server定时(每隔10s)扫描所有存活broker的连接，如果Name Server超过2分钟没有收到心跳，则Name Server断开与Broker的连接。

3、准备工作

以下操作在rocketmq01和rocketmq02都需要操作

[root@rocketmq01 ~]# cat /etc/hostname
[root@rocketmq01 ~]# systemctl stop firewalld
[root@rocketmq01 ~]# systemctl disable firewalld
[root@rocketmq01 ~]# sed -i 's/enforcing/disabled/' /etc/selinux/config
[root@rocketmq01 ~]# setenforce 0
[root@rocketmq01 ~]# yum install ntp wget -y
[root@rocketmq01 ~]# ntpdate ntp.api.bz
[root@rocketmq01 ~]# cat /etc/hosts
10.10.0.221	rocketmq01
10.10.0.222	rocketmq02
[root@rocketmq01 ~]# wget http://mirrors.tuna.tsinghua.edu.cn/apache/rocketmq/4.6.0/rocketmq-all-4.6.0-bin-release.zip
[root@rocketmq01 ~]# unzip rocketmq-all-4.6.0-bin-release.zip
[root@rocketmq01 ~]# mv rocketmq-all-4.6.0-bin-release /usr/local/rocketmq
[root@rocketmq01 ~]# chown rocketmq.rocketmq -R /usr/local/rocketmq
[root@rocketmq01 ~]# mkdir /data/rocketmq/{store,logs} -p
[root@rocketmq01 ~]# chown rocketmq.rocketmq -R /data/rocketmq

以下只在RocketMQ01上执行

[root@rocketmq01 ~]# ssh-keygen -t rsa
[root@rocketmq01 ~]# ssh-copy-id rocketmq02 
[root@rocketmq01 ~]# useradd rocketmq
[root@rocketmq01 ~]# passwd rocketmq 
Changing password for user rocketmq.
New password:rootroot
[root@rocketmq01 ~]# ssh-copy-id rocketmq@rocketmq01

4、部署Java环境

以下操作在rocketmq01和rocketmq02都需要操作

[root@rocketmq01 ~]# tar xf jdk-8u231-linux-x64.tar.gz
[root@rocketmq01 ~]# mkdir /usr/java 
[root@rocketmq01 ~]# mv jdk1.8.0_231 /usr/java
添加环境变量
[root@rocketmq01 ~]# vim /etc/profile
//在文件的末尾添加Java 环境变量
JAVA_HOME=/usr/java/jdk1.8.0_231 
JRE_HOME=/usr/java/jdk1.8.0_231/jre
CLASS_PATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar:$JRE_HOME/lib
PATH=$PATH:$JAVA_HOME/bin:$JRE_HOME/bin
export JAVA_HOME JRE_HOME CLASS_PATH PATH
export PATH=/usr/local/spark-2.4.3:$PATH
//保存退出
[root@rocketmq01 ~]# source /etc/profile

5、修改配置文件（）

在Rocketmq01上修改：

[root@rocketmq01 ~]# cd /usr/local/rocketmq/conf/2m-2s-async/
[root@rocketmq01 2m-2s-async]# ls
broker-a.properties  broker-b-s.properties

在Rocketmq02上修改：

[root@rocketmq02 ~]# cd /usr/local/rocketmq/conf/2m-2s-async/
[root@rocketmq02 2m-2s-async]# ls
broker-a-s.properties  broker-b.properties

broker-a.properties：

[root@rocketmq01 2m-2s-async]# cat broker-a.properties 
//添加下面的内容
brokerClusterName=test-rocketmq-cluster
brokerName=rocketmq01
brokerId=0
listenPort=10911
namesrvAddr=rocketmq01:9876;rocketmq02:9876
deleteWhen=04
defaultTopicQueueNums=20
mapedFileSizeCommitLog=1073741824
mapedFileSizeConsumeQueue=3000000
diskMaxUsedSpaceRatio=75
fileReservedTime=48
brokerRole=SYNC_MASTER
flushDiskType=ASYNC_FLUSH   #开启异步刷盘，同步刷盘TPS过低，较难满足业务发展需求
storePathRootDir=/data/rocketmq/store/a-data-m
storePathCommitLog=/data/rocketmq/store/a-commitlog-m
storePathConsumeQueue=/data/rocketmq/store/a-consumequeue-m
storePathIndex=/data/rocketmq/store/a-index-m
storeCheckpoint=/data/rocketmq/store/a-checkpoint-m
abortFile=/data/rocketmq/store/a-abort-m
maxMessageSize=512
sendMessageThreadPoolNums=128
useReentrantLockWhenPutMessage=true
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
transientStorePoolEnable=true #开启堆外内存设置
warmMapedFileEnable=true  #开启文件预热
transferMsgByHeap=false    #关闭堆内存据传输

备注：broker-b-properies与broker–properies的配置是大体相同的，只有路径不同
broker-b-s-properies:

brokerClusterName=test-rocketmq-cluster
brokerName=rocketmq02
listenPort=10921
namesrvAddr=rocketmq01:9876;rocketmq02:9876
brokerId=1
deleteWhen=04
fileReservedTime=72
defaultTopicQueueNums=20
mapedFileSizeCommitLog=1073741824
mapedFileSizeConsumeQueue=3000000
diskMaxUsedSpaceRatio=75
brokerRole=SLAVE
storePathRootDir=/data/rocketmq/store/b-data-s
storePathCommitLog=/data/rocketmq/store/b-commitlog-s
storePathConsumeQueue=/data/rocketmq/store/b-consumequeue-s
storePathIndex=/data/rocketmq/store/b-index-s
storeCheckpoint=/data/rocketmq/store/b-checkpoint-s
abortFile=/data/rocketmq/store/b-abort-s
maxMessageSize=512
sendMessageThreadPoolNums=128
useReentrantLockWhenPutMessage=true
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
slaveReadEnable=true #开启slave读取权限
flushDiskType=SYNC_FLUSH
transientStorePoolEnable=true
warmMapedFileEnable=true
transferMsgByHeap=false

6、配置RocketMQ的Java虚拟机参数

修改JAVA虚拟机参数

[root@rocketmq01 bin]# vim runserver.sh
JAVA_OPT="${JAVA_OPT} -server -Xms256m -Xmx1g -Xmn256m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m
[root@rocketmq01 bin]# vim runbroker.sh
JAVA_OPT="${JAVA_OPT} -server -Xms256m -Xmx1g -Xmn256g"
[root@rocketmq01 bin]# vim tools.sh
JAVA_OPT="${JAVA_OPT} -server -Xms1g -Xmx1g -Xmn256m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=128m"
JAVA_OPT="${JAVA_OPT} -Djava.ext.dirs=${BASE_DIR}/lib:${JAVA_HOME}/jre/lib/ext:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.232.b09-0.el7_7.x86_64/jre/lib/ext"

7、编写启动脚本

[root@rocketmq01 src]# ls
master.sh  nameserverd.sh  slave.sh

Rocketmq01:

[root@rocketmq01 src]# cat master.sh 
#!/bin/bash
nohup sh /usr/local/rocketmq/bin/mqbroker -c /usr/local/rocketmq/conf/2m-2s-async/broker-a.properties > /data/rocketmq/store/broker-a-m.log 2>&1 &
[root@rocketmq01 src]# cat nameserverd.sh 
#!/bin/bash
source /etc/profile
nohup sh /usr/local/rocketmq/bin/mqnamesrv > /data/rocketmq/store/mqnamesrv.log 2>&1 &
[root@rocketmq01 src]# cat slave.sh 
#!/bin/bash
nohup sh /usr/local/rocketmq/bin/mqbroker -c /usr/local/rocketmq/conf/2m-2s-async/broker-b-s.properties > /data/rocketmq/store/broker-b-s.log 2>&1 &

备注： 启动脚本有多不同，注意配置文件路径
Rocketmq02:

#!/bin/bash
nohup sh /usr/local/rocketmq/bin/mqbroker -c /usr/local/rocketmq/conf/2m-2s-async/broker-b.properties > /data/rocketmq/store/broker-b-m.log 2>&1 &
[root@rocketmq02 src]# cat nameserverd.sh 
#!/bin/bash
source /etc/profile
nohup sh /usr/local/rocketmq/bin/mqnamesrv > /data/rocketmq/store/mqnamesrv.log 2>&1 &
[root@rocketmq02 src]# cat slave.sh 
#!/bin/bash
nohup sh /usr/local/rocketmq/bin/mqbroker -c /usr/local/rocketmq/conf/2m-2s-async/broker-a-s.properties > /data/rocketmq/store/broker-a-s.log 2>&1 &

总的启动脚本：

[root@rocketmq01 init.d]# ls
start.sh functions  rocketmq-cluster
[root@rocketmq01 init.d]# cat rocketmq-cluster 
#!/bin/bash
IP=10.10.0.222 #rocketmq02的Ip地址
SCRIPT_PATH=/usr/local/rocketmq/src

start(){ 
sh $SCRIPT_PATH/nameserverd.sh
ssh $IP "sh $SCRIPT_PATH/nameserverd.sh"
sleep 2
sh $SCRIPT_PATH/master.sh
ssh $IP "sh $SCRIPT_PATH/master.sh"
sleep 2

sh $SCRIPT_PATH/slave.sh
ssh $IP "sh $SCRIPT_PATH/slave.sh"
}

stop(){
mqshutdown  namesrv
ssh $IP "/usr/local/rocketmq/bin/mqshutdown namesrv"
mqshutdown broker
ssh $IP "/usr/local/rocketmq/bin/mqshutdown broker"
}
restart(){
$stop
$start
}

case $1 in 
"start") start
;;
"stop") stop
;;
"restart") restart
;;
"*")
echo "please input ./rocketmq_start 'start |stop'"
;;
esac
[root@rocketmq01 init.d]# chown rocketmq.rocketmq /etc/init.d/rocketmq-cluster

8、脚本启动和停止

启动rocketmq集群（可以用rocketmq用户启动）

[root@rocketmq01 init.d]# /etc/init.d/rocketmq-cluster start

关闭rocketmq集群（可以用rocketmq用户关闭）

[root@rocketmq01 init.d]# /etc/init.d/rocketmq-cluster stop

9、查看集群状态

//查看mqadmin的操作明令
[root@rocketmq01 ~]# mqadmin 
[root@rocketmq01 ~]# mqadmin 命令 help
[root@rocketmq01 ~]# mqadmin  clusterList -n 10.10.0.221:9876
RocketMQLog:WARN No appenders could be found for logger (io.netty.util.internal.PlatformDependent0).
RocketMQLog:WARN Please initialize the logger system properly.
#Cluster Name     #Broker Name            #BID  #Addr                  #Version                #InTPS(LOAD)       #OutTPS(LOAD) #PCWait(ms) #Hour #SPACE
test-rocketmq-cluster  rocketmq01              0     10.10.0.221:10911      V4_6_0                   0.00(0,0ms)         0.00(0,0ms)          0 654.69 0.0438
test-rocketmq-cluster  rocketmq01              1     10.10.0.222:10921      V4_6_0                   0.00(0,0ms)         0.00(0,0ms)          0 654.69 0.0345
test-rocketmq-cluster  rocketmq02              0     10.10.0.222:10911      V4_6_0                   0.00(0,0ms)         0.00(0,0ms)          0 654.68 0.0345
test-rocketmq-cluster  rocketmq02              1     10.10.0.221:10921      V4_6_0                   0.00(0,0ms)         0.00(0,0ms)          0 654.68 0.0438
查看broker状态

[root@rocketmq01 ~]# mqadmin brokerStatus -b 10.10.0.221:1091

10、查看和创建、删除topic

[root@rocketmq01 ~]# mqadmin topicList -n 10.10.0.221:9876
test-rocketmq-cluster_REPLY_TOPIC
RMQ_SYS_TRANS_HALF_TOPIC
rocketmq02
rocketmq01
%RETRY%please_rename_unique_group_name_4
BenchmarkTest
OFFSET_MOVED_EVENT
%RETRY%rmq_group_devops_stresstest
TBW102
test-rocketmq-cluster
SELF_TEST_TOPIC
TopicTest
%RETRY%TOOLS_CONSUMER
please_rename_unique_group_name_4
//查看单个topic状态
[root@rocketmq01 ~]# mqadmin topicStatus -n 10.10.0.221:9876 -t TopicTest
[root@rocketmq01 ~]# mqadmin topicRoute -n 10.10.0.221:9876 -t TopicTest
[root@rocketmq01 ~]# mqadmin updateTopic -n 10.10.0.221:9876 -c test-rocketmq-cluster -t zto-example
create topic to 10.10.0.222:10911 success.
create topic to 10.10.0.221:10911 success.
[root@rocketmq01 ~]# mqadmin deleteTopic -n 10.10.0.221:9876 -c test-rocketmq-cluster -t zto-example
delete topic [zto-example] from cluster [test-rocketmq-cluster] success.
delete topic [zto-example] from NameServer success.
//打印消息
[root@rocketmq01 ~]# mqadmin printMsg -n 10.10.0.221:9876 -t TopicTest