RocketMQ集群部署
主机名 | 主机ip |
---|---|
node1 | 192.168.1.1 |
node2 | 192.168.1.2 |
node3 | 192.168.1.3 |
配置Java环境变量
vi /etc/profile
添加以下配置:
ROCKETMQ_HOME=/usr/local/rocketmq/
export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk #你的Java安装路径
export JRE_HOME=/usr/lib/jvm/java-1.8.0-openjdk/jre
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar:$JRE_HOME/lib:$CLASSPATH
export PATH=$JAVA_HOME/bin:$PATH:$ROCKETMQ_HOME/bin:$PATH
刷新使配置生效
source /etc/profile
下载解压rocketmq压缩包
wget https://mirrors.tuna.tsinghua.edu.cn/apache/rocketmq/4.9.8/rocketmq-all-4.9.8-bin-release.zip
解压压缩包
unzip rocketmq-all-4.9.8-bin-release.zip
重命名为rocketmq
mv rocketmq-all-4.9.8-bin-release rocketmq
创建消息存储路径
Rocketmq的默认存储路径在根目录,会在根目录生成store
# node1
mkdir -p /usr/local/rocketmq/store
mkdir /usr/local/rocketmq/store/commitlog
mkdir /usr/local/rocketmq/store/consumequeue
mkdir /usr/local/rocketmq/store/index
mkdir -p /usr/local/rocketmq/store-s1
mkdir /usr/local/rocketmq/store-s1/commitlog
mkdir /usr/local/rocketmq/store-s1/consumequeue
mkdir /usr/local/rocketmq/store-s1/index
# node2
mkdir -p /usr/local/rocketmq/store
mkdir /usr/local/rocketmq/store/commitlog
mkdir /usr/local/rocketmq/store/consumequeue
mkdir /usr/local/rocketmq/store/index
mkdir -p /usr/local/rocketmq/store-s1
mkdir /usr/local/rocketmq/store-s1/commitlog
mkdir /usr/local/rocketmq/store-s1/consumequeue
mkdir /usr/local/rocketmq/store-s1/index
# node3
mkdir -p /usr/local/rocketmq/store
mkdir /usr/local/rocketmq/store/commitlog
mkdir /usr/local/rocketmq/store/consumequeue
mkdir /usr/local/rocketmq/store/index
mkdir -p /usr/local/rocketmq/store-s1
mkdir /usr/local/rocketmq/store-s1/commitlog
mkdir /usr/local/rocketmq/store-s1/consumequeue
mkdir /usr/local/rocketmq/store-s1/index
Broker配置文件
node1 Master1
vi /usr/local/rocketmq/conf/2m-2s-sync/broker-a.properties
配置如下:
#所属集群名字
brokerClusterName=rocketmq-cluster
#broker名字,不同节点名字不同,主从节点名字相同
brokerName=broker-a
#0: Master; >0: Slave
brokerId=0
#nameServer地址,分号分割
namesrvAddr=192.168.1.1:9876;192.168.1.2:9876;192.168.1.3:9876
#broker启动地址,rocketmq默认内网启动
brokerIP1=192.168.1.1
#broker的HAIP地址(供Slave同步消息的地址)
brokerIP2=192.168.1.2
#在发送消息时,自动创建服务器不存在的topic,默认创建的队列数
defaultTopicQueueNums=4
#是否允许Broker自动创建Topic,建议线下开启,线上关闭
autoCreateTopicEnable=true
#是否允许Broker自动创建订阅组,建议线下开启,线上关闭
autoCreateSubscriptionGroup=true
#Broker对外服务的监听端口
listenPort=10911
#删除文件时间点,默认凌晨4点
deleteWhen=04
#文件保留时间,默认48h
fileReservedTime=48
#commitLog每个文件的大小默认1G
mapedFileSizeCommitLog=1073741824
#ConsumeQueue每个文件默认存30W条,根据业务情况调整
mapedFileSizeConsumeQueue=300000
#检测物理文件磁盘空间
diskMaxUsedSpaceRatio=88
#存储路径
storePathRootDir=/usr/local/rocketmq/store
#commitLog 存储路径
storePathCommitLog=/usr/local/rocketmq/store/commitlog
#消费队列存储路径存储路径
storePathConsumeQueue=/usr/local/rocketmq/store/consumequeue
#消息索引存储路径
storePathIndex=/usr/local/rocketmq/store/index
#checkpoint 文件存储路径
storeCheckpoint=/usr/local/rocketmq/store/checkpoint
#abort 文件存储路径
abortFile=/usr/local/rocketmq/store/abort
#限制的消息大小
maxMessageSize=65536
#Broker角色: ASYNC_MASTER(异步复制Master)、SYNC_MASTER(同步双写Master)、SLAVE(从节点)
brokerRole=SYNC_MASTER
#刷盘方式: ASYNC_FLUSH(异步刷盘)、SYNC_FLUSH(同步刷盘)
flushDiskType=SYNC_FLUSH
node1 Master3-slave1
vi /usr/local/rocketmq/conf/2m-2s-sync/broker-c-s.properties
配置如下:
#所属集群名字
brokerClusterName=rocketmq-cluster
#broker名字,不同节点名字不同,主从节点名字相同
brokerName=broker-c
#0: Master; >0: Slave 数值小了遇到判定不大于0的问题,此处改大一点
brokerId=70
#nameServer地址,分号分割
namesrvAddr=192.168.1.1:9876;192.168.1.2:9876;192.168.1.3:9876
#broker启动地址,rocketmq默认内网启动
brokerIP1=192.168.1.1
#broker的HAIP地址(供Slave同步消息的地址)
brokerIP2=192.168.1.3
#在发送消息时,自动创建服务器不存在的topic,默认创建的队列数
defaultTopicQueueNums=4
#是否允许Broker自动创建Topic,建议线下开启,线上关闭
autoCreateTopicEnable=true
#是否允许Broker自动创建订阅组,建议线下开启,线上关闭
autoCreateSubscriptionGroup=true
#Broker对外服务的监听端口
listenPort=11011
#删除文件时间点,默认凌晨4点
deleteWhen=04
#文件保留时间,默认48h
fileReservedTime=48
#commitLog每个文件的大小默认1G
mapedFileSizeCommitLog=1073741824
#ConsumeQueue每个文件默认存30W条,根据业务情况调整
mapedFileSizeConsumeQueue=300000
#检测物理文件磁盘空间
diskMaxUsedSpaceRatio=88
#存储路径
storePathRootDir=/usr/local/rocketmq/store-s
#commitLog 存储路径
storePathCommitLog=/usr/local/rocketmq/store-s/commitlog
#消费队列存储路径存储路径
storePathConsumeQueue=/usr/local/rocketmq/store-s/consumequeue
#消息索引存储路径
storePathIndex=/usr/local/rocketmq/store-s/index
#checkpoint 文件存储路径
storeCheckpoint=/usr/local/rocketmq/store-s/checkpoint
#abort 文件存储路径
abortFile=/usr/local/rocketmq/store-s/abort
#限制的消息大小
maxMessageSize=65536
#Broker角色: ASYNC_MASTER(异步复制Master)、SYNC_MASTER(同步双写Master)、SLAVE(从节点)
brokerRole=SLAVE
#刷盘方式: ASYNC_FLUSH(异步刷盘)、SYNC_FLUSH(同步刷盘)
flushDiskType=ASYNC_FLUSH
node2 Master1-slave1
vi /usr/local/rocketmq/conf/2m-2s-sync/broker-a-s.properties
配置如下:
#所属集群名字
brokerClusterName=rocketmq-cluster
#broker名字,不同节点名字不同,主从节点名字相同
brokerName=broker-a
#0: Master; >0: Slave 数值小了遇到判定不大于0的问题,此处改大一点
brokerId=50
#nameServer地址,分号分割
namesrvAddr=192.168.1.1:9876;192.168.1.2:9876;192.168.1.3:9876
#broker启动地址,rocketmq默认内网启动
brokerIP1=192.168.1.2
#broker的HAIP地址(供Slave同步消息的地址)
brokerIP2=192.168.1.1
#在发送消息时,自动创建服务器不存在的topic,默认创建的队列数
defaultTopicQueueNums=4
#是否允许Broker自动创建Topic,建议线下开启,线上关闭
autoCreateTopicEnable=true
#是否允许Broker自动创建订阅组,建议线下开启,线上关闭
autoCreateSubscriptionGroup=true
#Broker对外服务的监听端口
listenPort=11011
#删除文件时间点,默认凌晨4点
deleteWhen=04
#文件保留时间,默认48h
fileReservedTime=48
#commitLog每个文件的大小默认1G
mapedFileSizeCommitLog=1073741824
#ConsumeQueue每个文件默认存30W条,根据业务情况调整
mapedFileSizeConsumeQueue=300000
#检测物理文件磁盘空间
diskMaxUsedSpaceRatio=88
#存储路径
storePathRootDir=/usr/local/rocketmq/store-s
#commitLog 存储路径
storePathCommitLog=/usr/local/rocketmq/store-s/commitlog
#消费队列存储路径存储路径
storePathConsumeQueue=/usr/local/rocketmq/store-s/consumequeue
#消息索引存储路径
storePathIndex=/usr/local/rocketmq/store-s/index
#checkpoint 文件存储路径
storeCheckpoint=/usr/local/rocketmq/store-s/checkpoint
#abort 文件存储路径
abortFile=/usr/local/rocketmq/store-s/abort
#限制的消息大小
maxMessageSize=65536
#Broker角色: ASYNC_MASTER(异步复制Master)、SYNC_MASTER(同步双写Master)、SLAVE(从节点)
brokerRole=SLAVE
#刷盘方式: ASYNC_FLUSH(异步刷盘)、SYNC_FLUSH(同步刷盘)
flushDiskType=ASYNC_FLUSH
node2 Master2
vi /usr/local/rocketmq/conf/2m-2s-sync/broker-b.properties
配置如下:
#所属集群名字
brokerClusterName=rocketmq-cluster
#broker名字,不同节点名字不同,主从节点名字相同
brokerName=broker-b
#0: Master; >0: Slave
brokerId=0
#nameServer地址,分号分割
namesrvAddr=192.168.1.1:9876;192.168.1.2:9876;192.168.1.3:9876
#broker启动地址,rocketmq默认内网启动
brokerIP1=192.168.1.2
#broker的HAIP地址(供Slave同步消息的地址)
brokerIP2=192.168.1.3
#在发送消息时,自动创建服务器不存在的topic,默认创建的队列数
defaultTopicQueueNums=4
#是否允许Broker自动创建Topic,建议线下开启,线上关闭
autoCreateTopicEnable=true
#是否允许Broker自动创建订阅组,建议线下开启,线上关闭
autoCreateSubscriptionGroup=true
#Broker对外服务的监听端口
listenPort=10911
#删除文件时间点,默认凌晨4点
deleteWhen=04
#文件保留时间,默认48h
fileReservedTime=48
#commitLog每个文件的大小默认1G
mapedFileSizeCommitLog=1073741824
#ConsumeQueue每个文件默认存30W条,根据业务情况调整
mapedFileSizeConsumeQueue=300000
#检测物理文件磁盘空间
diskMaxUsedSpaceRatio=88
#存储路径
storePathRootDir=/usr/local/rocketmq/store
#commitLog 存储路径
storePathCommitLog=/usr/local/rocketmq/store/commitlog
#消费队列存储路径存储路径
storePathConsumeQueue=/usr/local/rocketmq/store/consumequeue
#消息索引存储路径
storePathIndex=/usr/local/rocketmq/store/index
#checkpoint 文件存储路径
storeCheckpoint=/usr/local/rocketmq/store/checkpoint
#abort 文件存储路径
abortFile=/usr/local/rocketmq/store/abort
#限制的消息大小
maxMessageSize=65536
#Broker角色: ASYNC_MASTER(异步复制Master)、SYNC_MASTER(同步双写Master)、SLAVE(从节点)
brokerRole=SYNC_MASTER
#刷盘方式: ASYNC_FLUSH(异步刷盘)、SYNC_FLUSH(同步刷盘)
flushDiskType=SYNC_FLUSH
node3 Master2-slave1
vi /usr/local/rocketmq/conf/2m-2s-sync/broker-b-s.properties
配置如下:
#所属集群名字
brokerClusterName=rocketmq-cluster
#broker名字,不同节点名字不同,主从节点名字相同
brokerName=broker-b
#0: Master; >0: Slave 数值小了遇到判定不大于0的问题,此处改大一点
brokerId=60
#nameServer地址,分号分割
namesrvAddr=192.168.1.1:9876;192.168.1.2:9876;192.168.1.3:9876
#broker启动地址,rocketmq默认内网启动
brokerIP1=192.168.1.3
#broker的HAIP地址(供Slave同步消息的地址)
brokerIP2=192.168.1.2
#在发送消息时,自动创建服务器不存在的topic,默认创建的队列数
defaultTopicQueueNums=4
#是否允许Broker自动创建Topic,建议线下开启,线上关闭
autoCreateTopicEnable=true
#是否允许Broker自动创建订阅组,建议线下开启,线上关闭
autoCreateSubscriptionGroup=true
#Broker对外服务的监听端口
listenPort=11011
#删除文件时间点,默认凌晨4点
deleteWhen=04
#文件保留时间,默认48h
fileReservedTime=48
#commitLog每个文件的大小默认1G
mapedFileSizeCommitLog=1073741824
#ConsumeQueue每个文件默认存30W条,根据业务情况调整
mapedFileSizeConsumeQueue=300000
#检测物理文件磁盘空间
diskMaxUsedSpaceRatio=88
#存储路径
storePathRootDir=/usr/local/rocketmq/store-s
#commitLog 存储路径
storePathCommitLog=/usr/local/rocketmq/store-s/commitlog
#消费队列存储路径存储路径
storePathConsumeQueue=/usr/local/rocketmq/store-s/consumequeue
#消息索引存储路径
storePathIndex=/usr/local/rocketmq/store-s/index
#checkpoint 文件存储路径
storeCheckpoint=/usr/local/rocketmq/store-s/checkpoint
#abort 文件存储路径
abortFile=/usr/local/rocketmq/store-s/abort
#限制的消息大小
maxMessageSize=65536
#Broker角色: ASYNC_MASTER(异步复制Master)、SYNC_MASTER(同步双写Master)、SLAVE(从节点)
brokerRole=SLAVE
#刷盘方式: ASYNC_FLUSH(异步刷盘)、SYNC_FLUSH(同步刷盘)
flushDiskType=ASYNC_FLUSH
node3 Master3
vi /usr/local/rocketmq/conf/2m-2s-sync/broker-c.properties
配置如下:
#所属集群名字
brokerClusterName=rocketmq-cluster
#broker名字,不同节点名字不同,主从节点名字相同
brokerName=broker-c
#0: Master; >0: Slave
brokerId=0
#nameServer地址,分号分割
namesrvAddr=192.168.1.1:9876;192.168.1.2:9876;192.168.1.3:9876
#broker启动地址,rocketmq默认内网启动
brokerIP1=192.168.1.3
#broker的HAIP地址(供Slave同步消息的地址)
brokerIP2=192.168.1.1
#在发送消息时,自动创建服务器不存在的topic,默认创建的队列数
defaultTopicQueueNums=4
#是否允许Broker自动创建Topic,建议线下开启,线上关闭
autoCreateTopicEnable=true
#是否允许Broker自动创建订阅组,建议线下开启,线上关闭
autoCreateSubscriptionGroup=true
#Broker对外服务的监听端口
listenPort=10911
#删除文件时间点,默认凌晨4点
deleteWhen=04
#文件保留时间,默认48h
fileReservedTime=48
#commitLog每个文件的大小默认1G
mapedFileSizeCommitLog=1073741824
#ConsumeQueue每个文件默认存30W条,根据业务情况调整
mapedFileSizeConsumeQueue=300000
#检测物理文件磁盘空间
diskMaxUsedSpaceRatio=88
#存储路径
storePathRootDir=/usr/local/rocketmq/store
#commitLog 存储路径
storePathCommitLog=/usr/local/rocketmq/store/commitlog
#消费队列存储路径存储路径
storePathConsumeQueue=/usr/local/rocketmq/store/consumequeue
#消息索引存储路径
storePathIndex=/usr/local/rocketmq/store/index
#checkpoint 文件存储路径
storeCheckpoint=/usr/local/rocketmq/store/checkpoint
#abort 文件存储路径
abortFile=/usr/local/rocketmq/store/abort
#限制的消息大小
maxMessageSize=65536
#Broker角色: ASYNC_MASTER(异步复制Master)、SYNC_MASTER(同步双写Master)、SLAVE(从节点)
brokerRole=SYNC_MASTER
#刷盘方式: ASYNC_FLUSH(异步刷盘)、SYNC_FLUSH(同步刷盘)
flushDiskType=SYNC_FLUSH
修改启动脚本
根据实际情况,修改/usr/local/rocketmq/bin/runbroker.sh和/usr/local/rocketmq/bin/runserver.sh脚本的JVM参数,默认为4g,根据主机配修改:
JAVA_OPT="${JAVA_OPT} -server -Xms512m -Xmx512m -Xmn128m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m"
启动服务
启动NameServer集群
分别在三台机器上启动NameServer服务:
# 启动NameServer并指定日志输出位置
nohup sh /usr/local/rocketmq/bin/mqnamesrv &> /usr/local/rocketmq/namesrv.log &
看到如下内容,说明启动成功了
注意事项:
如果出现以下报错
ERROR: Please set the JAVA_HOME variable in your environment, We need java(x64)! !!
修改/usr/local/rocketmq/bin/runbroker.sh和/bin/runserver.sh脚本的如下内容:
# 屏蔽以下三行
# [ ! -e "$JAVA_HOME/bin/java" ] && JAVA_HOME=$HOME/jdk/java
# [ ! -e "$JAVA_HOME/bin/java" ] && JAVA_HOME=/usr/java
# [ ! -e "$JAVA_HOME/bin/java" ] && error_exit "Please set the JAVA_HOME variable in your environment, We need java(x64)!"
export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk #你的Java安装路径
其他脚本报错误同上操作
启动Broker集群
在node1上启动master1和master3-slave1
nohup sh /usr/local/rocketmq/bin/mqbroker -c /usr/local/rocketmq/conf/2m-2s-sync/broker-a.properties &> /usr/local/rocketmq/broker-a.log &
nohup sh /usr/local/rocketmq/bin/mqbroker -c /usr/local/rocketmq/conf/2m-2s-sync/broker-c-s.properties &> /usr/local/rocketmq/broker-c-s.log &
在node2上启动master2和master1-slave1
nohup sh /usr/local/rocketmq/bin/mqbroker -c /usr/local/rocketmq/conf/2m-2s-sync/broker-b.properties &> /usr/local/rocketmq/broker-b.log &
nohup sh /usr/local/rocketmq/bin/mqbroker -c /usr/local/rocketmq/conf/2m-2s-sync/broker-a-s.properties &> /usr/local/rocketmq/broker-a-s.log &
在node3上启动master3和master2-slave1
nohup sh /usr/local/rocketmq/bin/mqbroker -c /usr/local/rocketmq/conf/2m-2s-sync/broker-c.properties &> /usr/local/rocketmq/broker-c.log &
nohup sh /usr/local/rocketmq/bin/mqbroker -c /usr/local/rocketmq/conf/2m-2s-sync/broker-b-s.properties &> /usr/local/rocketmq/broker-b-s.log &
看到如下内容,说明启动成功了
关闭RocketMQ
1.关闭NameServer
sh /usr/local/rocketmq/bin/mqshutdown namesrv
#2.关闭Broker
sh /usr/local/rocketmq/bin/mqshutdown broker
防火墙配置
查看开放的端口
sudo firewall-cmd --zone=public --list-ports
开放需要的端口
#RocketMQ默认使用3个端口:9876 、10911 、11011
firewall-cmd --permanent --zone=public --add-port=9876/tcp
firewall-cmd --permanent --zone=public --add-port=10911/tcp
firewall-cmd --permanent --zone=public --add-port=11011/tcp
#部署dashboard可视化需要多开两个端口
#firewall-cmd --permanent --zone=public --add-port=10909/tcp
#firewall-cmd --permanent --zone=public --add-port=11009/tcp
重启防火墙
firewall-cmd --reload
开机自启动配置
vi /etc/rc.local
node1在文件最后添加:
nohup sh /usr/local/rocketmq/bin/mqnamesrv &> /usr/local/rocketmq/namesrv.log &
nohup sh /usr/local/rocketmq/bin/mqbroker -c /usr/local/rocketmq/conf/2m-2s-sync/broker-a.properties &> /usr/local/rocketmq/broker-a.log &
nohup sh /usr/local/rocketmq/bin/mqbroker -c /usr/local/rocketmq/conf/2m-2s-sync/broker-c-s.properties &> /usr/local/rocketmq/broker-c-s.log &
node2在文件最后添加:
nohup sh /usr/local/rocketmq/bin/mqnamesrv &> /usr/local/rocketmq/namesrv.log &
nohup sh /usr/local/rocketmq/bin/mqbroker -c /usr/local/rocketmq/conf/2m-2s-sync/broker-b.properties &> /usr/local/rocketmq/broker-b.log &
nohup sh /usr/local/rocketmq/bin/mqbroker -c /usr/local/rocketmq/conf/2m-2s-sync/broker-a-s.properties &> /usr/local/rocketmq/broker-a-s.log &
node3在文件最后添加:
nohup sh /usr/local/rocketmq/bin/mqnamesrv &> /usr/local/rocketmq/namesrv.log &
nohup sh /usr/local/rocketmq/bin/mqbroker -c /usr/local/rocketmq/conf/2m-2s-sync/broker-c.properties &> /usr/local/rocketmq/broker-c.log &
nohup sh /usr/local/rocketmq/bin/mqbroker -c /usr/local/rocketmq/conf/2m-2s-sync/broker-b-s.properties &> /usr/local/rocketmq/broker-b-s.log &
编辑完成后执行
chmod +x /etc/rc.d/rc.local