rocketmq集群安装部署过程(4.0.0版本)、安装中的常见问题举例

本篇博文参考:http://www.cnblogs.com/quchunhui/p/7126868.html,然后按照自己的环境进行重新编写

准备工作:

下载rocketmq-all-4.0.0-incubating-bin-release.zip,下载地址:http://rocketmq.apache.org/dowloading/releases/
这里写图片描述

环境:hadoop11,hadoop12,hadoop13 (Centos6.7)
软件:rocketmq安装包(v4.0.0)  rocketmq-all-4.0.0-incubating-bin-release.zip
工具:XShell,XFTP
hadoop11:    192.168.106.101
hadoop12:    192.168.106.102
hadoop13:    192.168.106.103

3个虚拟节点的构成如下:
这里写图片描述

安装步骤

步骤内容备注
下载安装包从Apache RocketMq官网上下载安装包http://rocketmq.apache.org/dowloading/releases/
上传并解压使用XFTP工具,将安装包上传至服务器
编写配置文件完成各节点的配置文件修改http://rocketmq.apache.org/docs/rmq-deployment/
集群配置启动rocketmq集群
集群验证验证rocketmq集群是否正常

操作过程

1、安装包已经上传至其中3个节点:

hadoop11,hadoop12,hadoop13都执行以下命令:

unzip rocketmq-all-4.0.0-incubating-bin-release.zip

这里写图片描述

[root@hadoop11 software]# pwd
/home/tuzq/software
修改文件夹名称:
[root@hadoop11 software]# mv apache-rocketmq-all/ rocketmq

查看最终的效果:
这里写图片描述

2、修改配置文件

我这里已经将配置文件提前准备好了,只呈现以下配置文件的结果。
节点1(192.168.106.101)配置文件:
配置文件放在:/home/tuzq/software/rocketmq/conf中
文件名称是:broker-a-m.properties

cd /home/tuzq/software/rocketmq/conf
vim broker-a-m.properties

文件中的内容如下:

brokerClusterName=post
brokerName=broker-a
namesrvAddr=192.168.106.101:9876;192.168.106.102:9876
brokerId=0
listenPort=10911
brokerIP1=192.168.106.101
deleteWhen=04
fileReservedTime=72
brokerRole=ASYNC_MASTER
storePathRootDir=/home/tuzq/software/rocketmq/rootdir
storePathCommitLog=/home/tuzq/software/rocketmq/commitlog
defaultTopicQueueNums=4
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
flushDiskType=ASYNC_FLUSH
vim broker-c-s.properties

文件的内容是:

brokerClusterName=post
brokerName=broker-c
namesrvAddr=192.168.106.101:9876;192.168.106.102:9876
brokerId=1
listenPort=10920
brokerIP1=192.168.106.101
deleteWhen=04
fileReservedTime=72
brokerRole=SLAVE
storePathRootDir=/home/tuzq/software/rocketmq/rootdir
storePathCommitLog=/home/tuzq/software/rocketmq/commitlog
defaultTopicQueueNums=4
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
flushDiskType=ASYNC_FLUSH

节点2(192.168.106.102)配置文件:

cd /home/tuzq/software/rocketmq/conf
vim broker-a-s.properties
brokerClusterName=post
brokerName=broker-a
namesrvAddr=192.168.106.101:9876;192.168.106.102:9876
brokerId=1
listenPort=10920
deleteWhen=04
brokerIP1=192.168.106.102
fileReservedTime=72
brokerRole=SLAVE
storePathRootDir=/home/tuzq/software/rocketmq/rootdir
storePathCommitLog=/home/tuzq/software/rocketmq/commitlog
defaultTopicQueueNums=4
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
flushDiskType=ASYNC_FLUSH
vim broker-b-m.properties

文件内容是:

brokerClusterName=post
brokerName=broker-b
namesrvAddr=192.168.106.101:9876;192.168.106.102:9876
brokerId=0
listenPort=10911
brokerIP1=192.168.106.102
deleteWhen=04
fileReservedTime=72
brokerRole=ASYNC_MASTER
storePathRootDir=/home/tuzq/software/rocketmq/rootdir
storePathCommitLog=/home/tuzq/software/rocketmq/commitlog
defaultTopicQueueNums=4
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
flushDiskType=ASYNC_FLUSH

节点3(192.168.106.103)配置文件:

cd /home/tuzq/software/rocketmq/conf
vim broker-b-s.properties

文件内容是:

brokerClusterName=post
brokerName=broker-b
namesrvAddr=192.168.106.101:9876;192.168.106.102:9876
brokerId=1
listenPort=10920
brokerIP1=192.168.106.103
deleteWhen=04
fileReservedTime=72
brokerRole=SLAVE
storePathRootDir=/home/tuzq/software/rocketmq/rootdir
storePathCommitLog=/home/tuzq/software/rocketmq/commitlog
defaultTopicQueueNums=4
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
flushDiskType=ASYNC_FLUSH
vim broker-c-m.properties

文件内容是:

brokerClusterName=post
brokerName=broker-c
namesrvAddr=192.168.106.101:9876;192.168.106.102:9876
brokerId=0
listenPort=10911
brokerIP1=192.168.106.103
deleteWhen=04
fileReservedTime=72
brokerRole=ASYNC_MASTER
storePathRootDir=/home/tuzq/software/rocketmq/rootdir
storePathCommitLog=/home/tuzq/software/rocketmq/commitlog
defaultTopicQueueNums=4
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
flushDiskType=ASYNC_FLUSH

3、启动nameserver

三台机器上都创建logs文件夹(rocketmq中没有这个logs文件夹):
创建文件夹命令:

mkdir -p /home/tuzq/software/rocketmq/logs

节点1和节点2的启动命令:

nohup sh /home/tuzq/software/rocketmq/bin/mqnamesrv >/home/tuzq/software/rocketmq/logs/mqnamesrv.log 2>&1 &

注意:我这里将启动日志重定义到了logs路径下,需要提前手动创建logs文件夹,以便于统一管理日志,方便查看。

下图中展示了启动命令,日志中可以看到NameServer成功启动的日志。
NameServer - 节点1
请注意hostname,下面是hadoop11

[root@hadoop11 conf]# nohup sh /home/tuzq/software/rocketmq/bin/mqnamesrv >/home/tuzq/software/rocketmq/logs/mqnamesrv.log 2>&1 &
[1] 9676
[root@hadoop11 conf]# cat /home/tuzq/software/rocketmq/logs/mqnamesrv.log
nohup: 忽略输入
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=320m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: Using the DefNew young collector with the CMS collector is deprecated and will likely be removed in a future release
Java HotSpot(TM) 64-Bit Server VM warning: UseCMSCompactAtFullCollection is deprecated and will likely be removed in a future release.
The Name Server boot success. serializeType=JSON
[root@hadoop11 conf]#

截图如下:
这里写图片描述
在下面的演示过程中将不再进行截图,所有的过程都把代码粘贴出来,在进行查看的时候注意机器的hostname

NameServer = 节点2

[root@hadoop12 rocketmq]# nohup sh /home/tuzq/software/rocketmq/bin/mqnamesrv >/home/tuzq/software/rocketmq/logs/mqnamesrv.log 2>&1 &
[1] 9558
[root@hadoop12 rocketmq]# cat /home/tuzq/software/rocketmq/logs/mqnamesrv.log
nohup: 忽略输入
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=320m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: Using the DefNew young collector with the CMS collector is deprecated and will likely be removed in a future release
Java HotSpot(TM) 64-Bit Server VM warning: UseCMSCompactAtFullCollection is deprecated and will likely be removed in a future release.
The Name Server boot success. serializeType=JSON
[root@hadoop12 rocketmq]#

这里写图片描述

在节点1和节点2上分别执行jps命令,可以看到类似如下的结果:

[root@hadoop11 conf]# jps
9682 NamesrvStartup
9774 Jps

4、启动Broker-a(Master位于节点1、Slave位于节点2)

broker-a分为Master和slave,分别位于节点1和节点2上,需要分别启动。
注意:需要根据启动角色,为broker指定一个配置文件

在执行下面的命令前,先修改文件路径:{ROCKET_HOME}/bin/runbroker.sh中的JVM参数配置,如若不改,将会因为内存不够启动不了broker.

修改前:JAVA_OPT="${JAVA_OPT} -server -Xms8g -Xmx8g -Xmn4g"
修改后:JAVA_OPT="${JAVA_OPT} -server -Xms256m -Xmx512m -Xmn512m -XX:PermSize=256m -XX:MaxPermSize=640m"

如图所示:
这里写图片描述

broker-a的master - 节点1
命令:

[root@hadoop11 bin]# nohup sh /home/tuzq/software/rocketmq/bin/mqbroker -c /home/tuzq/software/rocketmq/conf/broker-a-m.properties >/home/tuzq/software/rocketmq/logs/broker-a-m.log 2>&1 &
[2] 10081
[root@hadoop11 bin]# jps
9682 NamesrvStartup
10088 BrokerStartup
10154 Jps
[root@hadoop11 bin]# cat /home/tuzq/software/rocketmq/logs/broker-a-m.log
nohup: 忽略输入
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=256m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=640m; support was removed in 8.0
[root@hadoop11 bin]#

broker-a的slave - 节点2:
命令如下:

[root@hadoop12 bin]# nohup sh /home/tuzq/software/rocketmq/bin/mqbroker -c /home/tuzq/software/rocketmq/conf/broker-a-s.properties >/home/tuzq/software/rocketmq/logs/broker-a-s.log 2>&1 &
[2] 9926
[root@hadoop12 bin]# jps 
9989 Jps
9564 NamesrvStartup
9933 BrokerStartup
[root@hadoop12 bin]# cat /home/tuzq/software/rocketmq/logs/broker-a-s.log
nohup: 忽略输入
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=256m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=640m; support was removed in 8.0
[root@hadoop12 bin]#

验证broker-a:
broker-a启动结束,这时候可以使用命令查看一下rocketmq集群状态:
命令:

[root@hadoop12 bin]# sh /home/tuzq/software/rocketmq/bin/mqadmin clusterList -n 192.168.106.101:9876
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
#Cluster Name     #Broker Name            #BID  #Addr                  #Version                #InTPS(LOAD)       #OutTPS(LOAD) #PCWait(ms) #Hour #SPACE
post              broker-a                0     192.168.106.101:10911  V4_0_0_SNAPSHOT          0.00(0,0ms)         0.00(0,0ms)          0 419078.85 -1.0000
post              broker-a                1     192.168.106.102:10920  V4_0_0_SNAPSHOT          0.00(0,0ms)         0.00(0,0ms)          0 419078.85 0.1532
[root@hadoop12 bin]#

5、启动broker-b(Master位于节点2、slave位于节点3)

broker-b的Master - 节点2

[root@hadoop12 bin]# nohup sh /home/tuzq/software/rocketmq/bin/mqbroker -c /home/tuzq/software/rocketmq/conf/broker-b-m.properties >/home/tuzq/software/rocketmq/logs/broker-b-m.log 2>&1 &
[3] 10175
[root@hadoop12 bin]# jps
10182 BrokerStartup
9564 NamesrvStartup
9933 BrokerStartup
10238 Jps
[root@hadoop12 bin]# cat /home/tuzq/software/rocketmq/logs/broker-b-m.log
nohup: 忽略输入
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=256m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=640m; support was removed in 8.0
[root@hadoop12 bin]#

broker-b的slave - 节点3:

[root@hadoop13 bin]# nohup sh /home/tuzq/software/rocketmq/bin/mqbroker -c /home/tuzq/software/rocketmq/conf/broker-b-s.properties >/home/tuzq/software/rocketmq/logs/broker-b-s.log 2>&1 &
[1] 9953
[root@hadoop13 bin]# jps
9969 Jps
9960 BrokerStartup
[root@hadoop13 bin]# cat /home/tuzq/software/rocketmq/logs/broker-b-s.log 
nohup: 忽略输入
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=256m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=640m; support was removed in 8.0
[root@hadoop13 bin]#

验证broker-b

[root@hadoop13 bin]# sh /home/tuzq/software/rocketmq/bin/mqadmin clusterList -n 192.168.106.101:9876
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
#Cluster Name     #Broker Name            #BID  #Addr                  #Version                #InTPS(LOAD)       #OutTPS(LOAD) #PCWait(ms) #Hour #SPACE
post              broker-a                0     192.168.106.101:10911  V4_0_0_SNAPSHOT          0.00(0,0ms)         0.00(0,0ms)          0 419078.98 -1.0000
post              broker-a                1     192.168.106.102:10920  V4_0_0_SNAPSHOT          0.00(0,0ms)         0.00(0,0ms)          0 419078.98 0.1532
post              broker-b                0     192.168.106.102:10911  V4_0_0_SNAPSHOT          0.00(0,0ms)         0.00(0,0ms)          0 419078.98 0.1532
post              broker-b                1     192.168.106.103:10920  V4_0_0_SNAPSHOT          0.00(0,0ms)         0.00(0,0ms)          0 419078.98 0.0190
[root@hadoop13 bin]# 

6、启动broker-b(Master位于节点3,Slave位于节点1)

broker-c的Master - 节点3

[root@hadoop13 bin]# nohup sh /home/tuzq/software/rocketmq/bin/mqbroker -c /home/tuzq/software/rocketmq/conf/broker-c-m.properties >/home/tuzq/software/rocketmq/logs/broker-b-s.log 2>&1 &
[2] 10201
[root@hadoop13 bin]# cat /home/tuzq/software/rocketmq/logs/broker-b-s.log
nohup: 忽略输入
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=256m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=640m; support was removed in 8.0
[root@hadoop13 bin]# 

broker-c的Slave - 节点1

[root@hadoop11 bin]# nohup sh /home/tuzq/software/rocketmq/bin/mqbroker -c /home/tuzq/software/rocketmq/conf/broker-c-s.properties >/home/tuzq/software/rocketmq/logs/broker-a-m.log 2>&1 &
[3] 10458
[root@hadoop11 bin]# jps
10465 BrokerStartup
9682 NamesrvStartup
10519 Jps
10088 BrokerStartup
[root@hadoop11 bin]# cat /home/tuzq/software/rocketmq/logs/broker-a-m.log
nohup: 忽略输入
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=256m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=640m; support was removed in 8.0
[root@hadoop11 bin]#

验证broker-c

[root@hadoop11 bin]# sh /home/tuzq/software/rocketmq/bin/mqadmin clusterList -n 192.168.106.101:9876
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
#Cluster Name     #Broker Name            #BID  #Addr                  #Version                #InTPS(LOAD)       #OutTPS(LOAD) #PCWait(ms) #Hour #SPACE
post              broker-a                0     192.168.106.101:10911  V4_0_0_SNAPSHOT          0.00(0,0ms)         0.00(0,0ms)          0 419079.12 0.0203
post              broker-a                1     192.168.106.102:10920  V4_0_0_SNAPSHOT          0.00(0,0ms)         0.00(0,0ms)          0 419079.12 0.1532
post              broker-b                0     192.168.106.102:10911  V4_0_0_SNAPSHOT          0.00(0,0ms)         0.00(0,0ms)          0 419079.12 0.1532
post              broker-b                1     192.168.106.103:10920  V4_0_0_SNAPSHOT          0.00(0,0ms)         0.00(0,0ms)          0 419079.12 0.0190
post              broker-c                0     192.168.106.103:10911  V4_0_0_SNAPSHOT          0.00(0,0ms)         0.00(0,0ms)          0 419079.12 0.0190
post              broker-c                1     192.168.106.101:10920  V4_0_0_SNAPSHOT          0.00(0,0ms)         0.00(0,0ms)          0 419079.12 0.0203
[root@hadoop11 bin]#

7、常见问题

问题1
启动broker的时候提示内存不够的错误
这里写图片描述
解决方法:由于个人电脑配置不够,无法为虚拟机申请更大的内存。所以,采用修改broker启动内存的方式解决了。
修改文件路径:{ROCKET_HOME}/bin/runbroker.sh
修改前:JAVA_OPT=” JAVAOPTserverXms8gXmx8gXmn4gJAVAOPT= {JAVA_OPT} -server -Xms256m -Xmx512m -Xmn512m -XX:PermSize=256m -XX:MaxPermSize=640m”
如图所示:
这里写图片描述

重新启动broker,日志中没有错误,jps进程中也可以看到broker正常启动。
这里写图片描述

问题2
在同一台机器上启动多个broker的时候提示如下错误:
这里写图片描述
问题原因:
按照本文最开始的图所示,同一台机器上会存在两个Broker,如果不进行特殊指定,broker的默认端口是10911
所以一台机器上启动两个broker时,第二个broker就会出现端口被占用的错误。
解决办法:
修改rocketmq的配置文件,增加listenPort配置。配置之后如下所示:
同一台机器的配置文件:
这里写图片描述

–>broker-a-m.properties
这里写图片描述
–>broker-c-s.properties
这里写图片描述
使用JPS查看一下进程是否有问题
这里写图片描述
在查看一下启动日志是否有问题

最后使用clusterList命令来验证一下集群健康状态
命令:sh bin/mqadmin clusterList -n 192.168.106.101:9876
这里写图片描述

成功实现了,如本文最开始图片所示的安装部署

  • 2
    点赞
  • 8
    收藏
    觉得还不错? 一键收藏
  • 打赏
    打赏
  • 1
    评论
评论 1
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

涂作权的博客

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付:¥1
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值