文章目录
下载二进制包
下载链接:RocketMQ官网
注:二进制包是编译好的,拿来可以直接用,如果不需要定制化,改动源码,可选择二进制包
机器准备
准备三台机器,每台机器需要安装好jdk,1.8以上版本;
集群使用交叉部署:三台机器,每台各自装一个nameserver,三个broker:一个master,两个其他master的slave;
目录结构:
机器1:namesever-1、master-a、b-slave1、c-slave1
机器2:namesever-2、master-b、a-slave1、c-slave2
机器3:namesever-3、master-c、a-slave2、b-slave2
注意:将二进制包解压后的文件,复制到以上12个文件夹中,使用独立的二进制包进行部署,因为需要将各自的日志文件、数据文件单独存储,如果不需要分离,则可使用同一个包
部署nameserver
创建两个目录:data、logs,用于存放数据和日志,rocketmq471是提前解压好的二进制文件目录:
修改配置文件
RocketMQ主目录:/neworiental/rocketmq01/rocketmq-nameserver/rocketmq471
新建nameserver配置文件:/neworiental/rocketmq01/rocketmq-nameserver/rocketmq471/conf/nameserver.conf
修改nameserver.conf文件:
#数据保存目录,需要配置,不配置的话默认在/root/namesrv
kvConfigPath=/neworiental/rocketmq01/rocketmq-nameserver/data/kvConfig.json
#是否启用Epoll IO模型
useEpollNativeSelector=true
#端口
listenPort=9876
修改日志路径
日志文件路径:/neworiental/rocketmq01/rocketmq-nameserver/rocketmq471/conf/logback_namesrv.xml
使用命令批量替换:
sed -i ‘s?${user.home}/logs/rocketmqlogs?/neworiental/rocketmq01/rocketmq-nameserver/logs?’ logback_namesrv.xml
打开logback_namesrv.xml,可以看到路径已全部替换:
修改启动脚本
可根据需求修改启动脚本,如:JVM配置、堆栈大小等,路径:
/neworiental/rocketmq01/rocketmq-nameserver/rocketmq471/bin/mqnamesrv
启动nameserver
nohup sh /neworiental/rocketmq01/rocketmq-nameserver/rocketmq471/bin/mqnamesrv -c /neworiental/rocketmq01/rocketmq-nameserver/rocketmq471/conf/nameserver.conf >/dev/null 2>&1 &
根据以上步骤启动剩下两个nameserver,配置文件直接复制使用
部署broker
以master-a节点为示例
修改日志路径
日志配置文件路径:
/neworiental/rocketmq01/rocketmq-a/rocketmq471/conf/logback_broker.xml
/neworiental/rocketmq01/rocketmq-a/rocketmq471/conf/logback_namesrv.xml
/neworiental/rocketmq01/rocketmq-a/rocketmq471/conf/logback_tools.xml
执行命令:(注意不同的Broker目录不一样)
sed -i 's?${user.home}/logs/rocketmqlogs?/neworiental/rocketmq01/rocketmq-a/logs?' logback_broker.xml
sed -i 's?${user.home}/logs/rocketmqlogs?/neworiental/rocketmq01/rocketmq-a/logs?' logback_namesrv.xml
sed -i 's?${user.home}/logs/rocketmqlogs?/neworiental/rocketmq01/rocketmq-a/logs?' logback_tools.xml
修改配置文件
配置文件路径:
RoketMQ的端口配置需要注意,详情见:集群端口配置讲解
#端口(注意:broker启动后,会占用3个端口,分别在listenPort基础上-2,+1,供内部程序使用,所以集群一定要规划好端口,避免冲突)
listenPort = 10911
#slave连接端口,配置之后无效,所以不用配
#haListenPort = 20911
#RocketMQ主目录
rocketmqHome = /neworiental/rocketmq01/rocketmq-a/rocketmq471
#集群名称
brokerClusterName = Rocketmq01
#nameserver地址,多个以分号隔开
namesrvAddr=172.22.36.68:9876;172.22.36.69:9876;172.22.36.70:9876
#broker名称(每组主从节点使用相同的brokerName,方便查看)
brokerName = broker-a
#Broker对外提供服务地址
brokerIP1 = 172.22.36.68
#供slave同步消息的地址
brokerIP2= 172.22.36.68
#broker的id,0为主,正数0为从
brokerId = 0
#数据存放的根目录
storePathRootDir = /neworiental/rocketmq01/rocketmq-a/store
#commit log保存目录
storePathCommitLog = /neworiental/rocketmq01/rocketmq-a/store/commitlog
#消费队列存储路径存储路径
storePathConsumerQueue = /neworiental/rocketmq01/rocketmq-a/store/consumequeue
#删除文件时间点,默认凌晨4点。24小时制,单位小时
deleteWhen = 04
#文件保存小时数
fileReservedTime = 168
#SYNC_MASTER/ASYNC_MASTER/SLAVE 节点角色
brokerRole = ASYNC_MASTER
#刷盘方式
flushDiskType = ASYNC_FLUSH
#是否允许Broker自动创建Topic
autoCreateTopicEnable = false
#是否允许 Broker 自动创建订阅组
autoCreateSubscriptionGroup = false
#linux开启epoll
useEpollNativeSelector = true
修改启动脚本
文件路径:/neworiental/rocketmq01/rocketmq-a/rocketmq471/bin/runbroker.sh
可根据需求,修改jvm参数、GC日志目录等:
启动Broker
nohup sh /neworiental/rocketmq01/rocketmq-a/rocketmq471/bin/mqbroker -c /neworiental/rocketmq01/rocketmq-a/rocketmq471/conf/broker.conf >/dev/null 2>&1 &
根据以上步骤,部署其余8个broker。
友情提示:对于这种中间件集群的搭建,基本就是不同的节点,要设置不同的存储目录,不同的配置文件,由于节点比较多,操作比较繁琐,很容易出现失误,所以可以先搭建好一个节点,确保可行之后,在本地模拟线上每个机器、每个节点的目录结构,把所有的节点对应的配置文件先改好了、检查好了,最后再每个节点逐个上传、启动,一定会事半功倍!
部署完成之后,可以进入任意broker的RocketMQ二进制包的bin目录,用命令查看集群信息:
sh mqadmin clusterList -n 127.0.0.1:9876
可以看到3主3从的RocketMQ集群搭建成功!
部署Console
RocketMQ-Console是RocketMQ的控制台页面,方便查看集群的具体信息和使用情况
下载源码
地址:https://github.com/apache/rocketmq-externals.git
下载后,使用IDEA打开rocketmq-console子模块
修改配置文件
修改日志路径
检查pom.xml,修改RocketMQ版本:
设置用户
需在application.properties中开启用户登录 rocketmq.config.loginRequired=true:(如果不需要设置用户登录,可忽略)
users.properties文件需要拷贝一份到console的data目录,方便修改,热加载:/neworiental/rocketmq01/rocketmq-console/data
打包部署
使用maven命令或IDEA将console打包,生成jar文件:rocketmq-console-ng-2.0.0.jar
上传到服务器后,java -jar rocketmq-console-ng-2.0.0.jar 启动服务即可
部署Exporter
需下载源码,自己打包:https://github.com/apache/rocketmq-exporter
exporter部署一个就可以
下载后,修改几个地方
nameserver地址:
日志目录:
pom.xml 修改rocketmq版本
这段注释掉:
脚本:
shutdown.sh
#!/bin/bash
. /etc/profile
PID=`ps -ef | grep '/neworiental/rocketmq02/rocketmq-exporter' | grep -v grep | awk '{print $2}'`
if [[ "" != "$PID" ]]; then
echo "killing rocketmq-console : $PID"
kill $PID
fi
start.sh
#!/bin/sh
. /etc/profile
LOGS_DIR=/neworiental/rocketmq02/rocketmq-expoter/logs
MAIN_JAR="-jar /neworiental/rocketmq02/rocketmq-expoter/rocketmq-exporter-0.0.2-SNAPSHOT.jar "
JAVA_ARGS="-server -Xms4096m -Xmx4096m -XX:NewSize=1500m -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=70 -XX:+PrintGCDetails -XX:+PrintHeapAtGC -XX:ThreadStackSize=512 -Xloggc:${LOGS_DIR}/gc.log "
if [ ! -d ${LOGS_DIR} ]
then
mkdir -p ${LOGS_DIR}
fi
echo ${JAVA_ARGS} ${MAIN_JAR} ${CLASSPATH}
nohup java ${JAVA_ARGS} ${MAIN_JAR} 1>>${LOGS_DIR}/stdout.log 2>>${LOGS_DIR}/stderr.log &
echo "deploying rocketmq-exporter-0.0.2-SNAPSHOT.jar now ..."
restart.sh
#!/bin/bash
PID=`ps -ef | grep '/neworiental/rocketmq02/rocketmq-expoter' | grep -v grep | awk '{print $2}'`
if [[ "" != "$PID" ]]; then
echo "killing rocketmq-exporter : $PID"
kill $PID
fi
sleep 1
LOGS_DIR=/neworiental/rocketmq02/rocketmq-exporter/logs
MAIN_JAR="-jar /neworiental/rocketmq02/rocketmq-expoter/rocketmq-exporter-0.0.2-SNAPSHOT.jar "
JAVA_ARGS="-server -Xms4096m -Xmx4096m -XX:NewSize=1500m -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=70 -XX:+PrintGCDetails -XX:+PrintHeapAtGC -XX:ThreadStackSize=512 -Xloggc:${LOGS_DIR}/gc.log "
if [ ! -d ${LOGS_DIR} ]
then
mkdir -p ${LOGS_DIR}
fi
echo ${JAVA_ARGS} ${MAIN_JAR} ${CLASSPATH}
nohup java ${JAVA_ARGS} ${MAIN_JAR} 1>>${LOGS_DIR}/stdout.log 2>>${LOGS_DIR}/stderr.log &
echo "deploying rocketmq-exporter-0.0.2-SNAPSHOT.jar now ..."
附录
所有Broker配置文件
机器1:
master-a
listenPort = 10911
rocketmqHome = /neworiental/rocketmq01/rocketmq-a/rocketmq471
brokerClusterName = Rocketmq01
namesrvAddr=172.22.36.68:9876;172.22.36.69:9876;172.22.36.70:9876
brokerName = broker-a
brokerIP1 = 172.22.36.68
brokerIP2= 172.22.36.68
brokerId = 0
storePathRootDir = /neworiental/rocketmq01/rocketmq-a/store
storePathCommitLog = /neworiental/rocketmq01/rocketmq-a/store/commitlog
storePathConsumerQueue = /neworiental/rocketmq01/rocketmq-a/store/consumequeue
deleteWhen = 04
fileReservedTime = 168
brokerRole = ASYNC_MASTER
flushDiskType = ASYNC_FLUSH
autoCreateTopicEnable = false
autoCreateSubscriptionGroup = false
useEpollNativeSelector = true
b-slave1:
listenPort = 10915
rocketmqHome = /neworiental/rocketmq01/rocketmq-b-s1/rocketmq471
brokerClusterName = Rocketmq01
namesrvAddr=172.22.36.68:9876;172.22.36.69:9876;172.22.36.70:9876
brokerName = broker-b
brokerIP1 = 172.22.36.68
brokerIP2 = 172.22.36.68
brokerId = 1
storePathRootDir = /neworiental/rocketmq01/rocketmq-b-s1/store
storePathCommitLog = /neworiental/rocketmq01/rocketmq-b-s1/store/commitlog
storePathConsumerQueue = /neworiental/rocketmq01/rocketmq-b-s1/store/consumequeue
deleteWhen = 04
fileReservedTime = 168
brokerRole = SLAVE
flushDiskType = ASYNC_FLUSH
autoCreateTopicEnable = false
autoCreateSubscriptionGroup = false
useEpollNativeSelector = true
c-slave1:
listenPort = 10919
rocketmqHome = /neworiental/rocketmq01/rocketmq-c-s1/rocketmq471
brokerClusterName = Rocketmq01
namesrvAddr=172.22.36.68:9876;172.22.36.69:9876;172.22.36.70:9876
brokerName = broker-c
brokerIP1 = 172.22.36.68
brokerIP2 = 172.22.36.68
brokerId = 1
storePathRootDir = /neworiental/rocketmq01/rocketmq-c-s1/store
storePathCommitLog = /neworiental/rocketmq01/rocketmq-c-s1/store/commitlog
storePathConsumerQueue = /neworiental/rocketmq01/rocketmq-c-s1/store/consumequeue
deleteWhen = 04
fileReservedTime = 168
brokerRole = SLAVE
flushDiskType = ASYNC_FLUSH
autoCreateTopicEnable = false
autoCreateSubscriptionGroup = false
useEpollNativeSelector = true
机器2:
master-b:
listenPort = 10911
rocketmqHome = /neworiental/rocketmq01/rocketmq-b/rocketmq471
brokerClusterName = Rocketmq01
namesrvAddr=172.22.36.68:9876;172.22.36.69:9876;172.22.36.70:9876
brokerName = broker-b
brokerIP1 = 172.22.36.69
brokerIP2= 172.22.36.69
brokerId = 0
storePathRootDir = /neworiental/rocketmq01/rocketmq-b/store
storePathCommitLog = /neworiental/rocketmq01/rocketmq-b/store/commitlog
storePathConsumerQueue = /neworiental/rocketmq01/rocketmq-b/store/consumequeue
deleteWhen = 04
fileReservedTime = 168
brokerRole = ASYNC_MASTER
flushDiskType = ASYNC_FLUSH
autoCreateTopicEnable = false
autoCreateSubscriptionGroup = false
useEpollNativeSelector = true
a-slave1:
listenPort = 10915
rocketmqHome = /neworiental/rocketmq01/rocketmq-a-s1/rocketmq471
brokerClusterName = Rocketmq01
namesrvAddr=172.22.36.68:9876;172.22.36.69:9876;172.22.36.70:9876
brokerName = broker-a
brokerIP1 = 172.22.36.69
brokerIP2 = 172.22.36.69
brokerId = 1
storePathRootDir = /neworiental/rocketmq01/rocketmq-a-s1/store
storePathCommitLog = /neworiental/rocketmq01/rocketmq-a-s1/store/commitlog
storePathConsumerQueue = /neworiental/rocketmq01/rocketmq-a-s1/store/consumequeue
deleteWhen = 04
fileReservedTime = 168
brokerRole = SLAVE
flushDiskType = ASYNC_FLUSH
autoCreateTopicEnable = false
autoCreateSubscriptionGroup = false
useEpollNativeSelector = true
c-slave2:
listenPort = 10919
rocketmqHome = /neworiental/rocketmq01/rocketmq-c-s2/rocketmq471
brokerClusterName = Rocketmq01
namesrvAddr=172.22.36.68:9876;172.22.36.69:9876;172.22.36.70:9876
brokerName = broker-c
brokerIP1 = 172.22.36.69
brokerIP2= 172.22.36.69
brokerId = 2
storePathRootDir = /neworiental/rocketmq01/rocketmq-c-s2/store
storePathCommitLog = /neworiental/rocketmq01/rocketmq-c-s2/store/commitlog
storePathConsumerQueue = /neworiental/rocketmq01/rocketmq-c-s2/store/consumequeue
deleteWhen = 04
fileReservedTime = 168
brokerRole = SLAVE
flushDiskType = ASYNC_FLUSH
autoCreateTopicEnable = false
autoCreateSubscriptionGroup = false
useEpollNativeSelector = true
机器3:
master-c:
listenPort = 10911
rocketmqHome = /neworiental/rocketmq01/rocketmq-c/rocketmq471
brokerClusterName = Rocketmq01
namesrvAddr=172.22.36.68:9876;172.22.36.69:9876;172.22.36.70:9876
brokerName = broker-c
brokerIP1 = 172.22.36.70
brokerIP2= 172.22.36.70
brokerId = 0
storePathRootDir = /neworiental/rocketmq01/rocketmq-c/store
storePathCommitLog = /neworiental/rocketmq01/rocketmq-c/store/commitlog
storePathConsumerQueue = /neworiental/rocketmq01/rocketmq-c/store/consumequeue
deleteWhen = 04
fileReservedTime = 168
brokerRole = ASYNC_MASTER
flushDiskType = ASYNC_FLUSH
autoCreateTopicEnable = false
autoCreateSubscriptionGroup = false
useEpollNativeSelector = true
a-slave2:
listenPort = 10915
rocketmqHome = /neworiental/rocketmq01/rocketmq-a-s2/rocketmq471
brokerClusterName = Rocketmq01
namesrvAddr=172.22.36.68:9876;172.22.36.69:9876;172.22.36.70:9876
brokerName = broker-a
brokerIP1 = 172.22.36.70
brokerIP1 = 172.22.36.70
brokerId = 2
storePathRootDir = /neworiental/rocketmq01/rocketmq-a-s2/store
storePathCommitLog = /neworiental/rocketmq01/rocketmq-a-s2/store/commitlog
storePathConsumerQueue = /neworiental/rocketmq01/rocketmq-a-s2/store/consumequeue
deleteWhen = 04
fileReservedTime = 168
brokerRole = SLAVE
flushDiskType = ASYNC_FLUSH
autoCreateTopicEnable = false
autoCreateSubscriptionGroup = false
useEpollNativeSelector = true
b-slave2:
listenPort = 10919
rocketmqHome = /neworiental/rocketmq01/rocketmq-b-s2/rocketmq471
brokerClusterName = Rocketmq01
namesrvAddr=172.22.36.68:9876;172.22.36.69:9876;172.22.36.70:9876
brokerName = broker-b
brokerIP1 = 172.22.36.70
brokerIP1 = 172.22.36.70
brokerId = 2
storePathRootDir = /neworiental/rocketmq01/rocketmq-b-s2/store
storePathCommitLog = /neworiental/rocketmq01/rocketmq-b-s2/store/commitlog
storePathConsumerQueue = /neworiental/rocketmq01/rocketmq-b-s2/store/consumequeue
deleteWhen = 04
fileReservedTime = 168
brokerRole = SLAVE
flushDiskType = ASYNC_FLUSH
autoCreateTopicEnable = false
autoCreateSubscriptionGroup = false
useEpollNativeSelector = true
第一次搭建,端口号分配不是很合理,应该每一组主从使用相同的端口号。下面是第二次搭建分配的情况,强迫症看着舒服多了:
Broker
节点 | 机器 | 端口 |
---|---|---|
broker-a | 172.22.36.68 | 20911 |
broker-a-s1 | 172.22.36.69 | 20911 |
broker-a-s2 | 172.22.36.70 | 20911 |
broker-b | 172.22.36.69 | 20915 |
broker-b-s1 | 172.22.36.68 | 20915 |
broker-b-s2 | 172.22.36.70 | 20915 |
broker-c | 172.22.36.70 | 20919 |
broker-c-s1 | 172.22.36.68 | 20919 |
broker-c-s2 | 172.22.36.69 | 20919 |
nameserver
节点 | 机器 | 端口 |
---|---|---|
nameserver | 172.22.36.68 | 9976 |
nameserver | 172.22.36.69 | 9976 |
nameserver | 172.22.36.70 | 9976 |
console
节点 | 机器 | 端口 |
---|---|---|
console | 172.22.36.69 | 8282 |
后续的话,还可以加入Systemd管理,注意Systemd需要对每个broker单独配置。