linux下搭建mq集群,linux下RocketMQ的集群搭建

部署的模式为:2m-2s-async 两主两从-异步同步

节点1:10.10.94.43【a master】【 b slave】

节点2:10.10.94.44【b master】【 a slave】

1.下载mq的bin包,并上传到两个不同的linux服务器上。

2.解压后移动到/usr/local/rocketmq目录下。

mv rocketmq-all-4.3.2-bin-release /usr/local/rocketmq

3.创建需要的目录。

节点1【master】执行:

mkdir -p /usr/local/rocketmq/store/broker-a /usr/local/rocketmq/store/broker-a/nsumequeue /usr/local/rocketmq/store/broker-a/commitlog /usr/local/rocketmq/store/broker-a/index /usr/local/rocketmq/logs /usr/local/rocketmq/store/broker-b-s /usr/local/rocketmq/store/broker-b-s/nsumequeue /usr/local/rocketmq/store/broker-b-s/commitlog /usr/local/rocketmq/store/broker-b-s/index

节点2【slave】执行:

mkdir -p /usr/local/rocketmq/store/broker-a-s /usr/local/rocketmq/store/broker-a-s/nsumequeue /usr/local/rocketmq/store/broker-a-s/commitlog /usr/local/rocketmq/store/broker-a-s/index /usr/local/rocketmq/logs /usr/local/rocketmq/store/broker-b /usr/local/rocketmq/store/broker-b/nsumequeue /usr/local/rocketmq/store/broker-b/commitlog /usr/local/rocketmq/store/broker-b/index

4.启动nameserver服务器。

nameserver没有需要修改的配置文件,默认端口为9876,且nameserver服务器互相之间是不通信的。节点1,2分别执行如下命令:

nohup sh /usr/local/rocketmq/bin/mqnamesrv >/usr/local/rocketmq/logs/mqnamesrv.log 2>&1 &

jps显示NamesrvStartup已启动如下:

# jps -l

15707 org.apache.rocketmq.namesrv.NamesrvStartup

29499 sun.tools.jps.Jps

5.修改两个节点的配置文件,位于/usr/local/rocketmq/conf/2m-2s-async目录,目的是为了启动broker服务。

节点1【a master】【b slave】如下:

vim broker-a.properties

#所属集群名字

brokerClusterName=rocketmq-cluster

#broker名字,名字可重复,为了管理,每个master起一个名字,他的slave同他,eg:Amaster叫broker-a,他的slave也叫broker-a

brokerName=broker-a

#0 表示 Master,>0 表示 Slave

brokerId=0

#nameServer地址,分号分割

namesrvAddr=10.10.94.43:9876;10.10.94.44:9876

brokerIP1=10.10.94.43

#在发送消息时,自动创建服务器不存在的topic,默认创建的队列数

defaultTopicQueueNums=4

#是否允许 Broker 自动创建Topic,建议线下开启,线上关闭

autoCreateTopicEnable=true

#是否允许 Broker 自动创建订阅组,建议线下开启,线上关闭

autoCreateSubscriptionGroup=true

#Broker 对外服务的监听端口,

listenPort=10911

#删除文件时间点,默认凌晨 4点

deleteWhen=04

#文件保留时间,默认 48 小时

fileReservedTime=120

#commitLog每个文件的大小默认1G

mapedFileSizeCommitLog=1073741824

#ConsumeQueue每个文件默认存30W条,根据业务情况调整

mapedFileSizeConsumeQueue=300000

#destroyMapedFileIntervalForcibly=120000

#redeleteHangedFileInterval=120000

#检测物理文件磁盘空间

diskMaxUsedSpaceRatio=88

#存储路径

storePathRootDir=/usr/local/rocketmq/store/broker-a

#commitLog 存储路径

storePathCommitLog=/usr/local/rocketmq/store/broker-a/commitlog

#消费队列存储路径存储路径

storePathConsumeQueue=/usr/local/rocketmq/store/broker-a/consumequeue

#消息索引存储路径

storePathIndex=/usr/local/rocketmq/store/broker-a/index

#checkpoint 文件存储路径

storeCheckpoint=/usr/local/rocketmq/store/checkpoint

#abort 文件存储路径

abortFile=/usr/local/rocketmq/store/abort

#限制的消息大小

maxMessageSize=65536

#flushCommitLogLeastPages=4

#flushConsumeQueueLeastPages=2

#flushCommitLogThoroughInterval=10000

#flushConsumeQueueThoroughInterval=60000

#Broker 的角色

#- ASYNC_MASTER 异步复制Master

#- SYNC_MASTER 同步双写Master

#- SLAVE

brokerRole=ASYNC_MASTER

#刷盘方式

#- ASYNC_FLUSH 异步刷盘

#- SYNC_FLUSH 同步刷盘

flushDiskType=ASYNC_FLUSH

#checkTransactionMessageEnable=false

#发消息线程池数量

#sendMessageThreadPoolNums=128

#拉消息线程池数量

#pullMessageThreadPoolNums=128

vim broker-b-s.properties

#所属集群名字

brokerClusterName=rocketmq-cluster

brokerName=broker-b

#broker名字,名字可重复,为了管理,每个master起一个名字,他的slave同他,eg:Amaster叫broker-b,他的slave也叫broker-b

#0 表示 Master,>0 表示 Slave

brokerId=1

#nameServer地址,分号分割

namesrvAddr=10.10.94.43:9876;10.10.94.44:9876

brokerIP1=10.10.94.43

#在发送消息时,自动创建服务器不存在的topic,默认创建的队列数

defaultTopicQueueNums=4

#是否允许 Broker 自动创建Topic,建议线下开启,线上关闭

autoCreateSubscriptionGroup=true

#Broker 对外服务的监听端口,

listenPort=10920

#删除文件时间点,默认凌晨 4点

deleteWhen=04

#文件保留时间,默认 48 小时

fileReservedTime=120

#commitLog每个文件的大小默认1G

mapedFileSizeCommitLog=1073741824

#ConsumeQueue每个文件默认存30W条,根据业务情况调整

mapedFileSizeConsumeQueue=300000

#destroyMapedFileIntervalForcibly=120000

#redeleteHangedFileInterval=120000

#检测物理文件磁盘空间

diskMaxUsedSpaceRatio=88

#存储路径

storePathRootDir=/usr/local/rocketmq/store/broker-b-s

#commitLog 存储路径

storePathCommitLog=/usr/local/rocketmq/store/broker-b-s/commitlog

#消费队列存储路径存储路径

storePathConsumeQueue=/usr/local/rocketmq/store/broker-b-s/consumequeue

#消息索引存储路径

storePathIndex=/usr/local/rocketmq/store/broker-b-s/index

#checkpoint 文件存储路径

storeCheckpoint=/usr/local/rocketmq/store/checkpoint

#abort 文件存储路径

abortFile=/usr/local/rocketmq/store/abort

#限制的消息大小

maxMessageSize=65536

#flushCommitLogLeastPages=4

#flushConsumeQueueLeastPages=2

#flushCommitLogThoroughInterval=10000

#flushConsumeQueueThoroughInterval=60000

#Broker 的角色

#- ASYNC_MASTER 异步复制Master

#- SYNC_MASTER 同步双写Master

#- SLAVE

brokerRole=SLAVE

#刷盘方式

#- ASYNC_FLUSH 异步刷盘

#- SYNC_FLUSH 同步刷盘

flushDiskType=ASYNC_FLUSH

#checkTransactionMessageEnable=false

#发消息线程池数量

#sendMessageThreadPoolNums=128

#拉消息线程池数量

#pullMessageThreadPoolNums=128

节点2【b master】【a slave】

vim broker-b.properties #基本和节点1的相同除了ip和brokerId

#所属集群名字

brokerClusterName=rocketmq-cluster

#broker名字,名字可重复,为了管理,每个master起一个名字,他的slave同他,eg:Amaster叫broker-a,他的slave也叫broker-a

brokerName=broker-b

#0 表示 Master,>0 表示 Slave

brokerId=0

#nameServer地址,分号分割

namesrvAddr=10.10.94.43:9876;10.10.94.44:9876

brokerIP1=10.10.94.44

#在发送消息时,自动创建服务器不存在的topic,默认创建的队列数

defaultTopicQueueNums=4

#是否允许 Broker 自动创建Topic,建议线下开启,线上关闭

autoCreateTopicEnable=true

#是否允许 Broker 自动创建订阅组,建议线下开启,线上关闭

autoCreateSubscriptionGroup=true

#Broker 对外服务的监听端口,

listenPort=10911

#删除文件时间点,默认凌晨 4点

deleteWhen=04

#文件保留时间,默认 48 小时

fileReservedTime=120

#commitLog每个文件的大小默认1G

mapedFileSizeCommitLog=1073741824

#ConsumeQueue每个文件默认存30W条,根据业务情况调整

mapedFileSizeConsumeQueue=300000

#destroyMapedFileIntervalForcibly=120000

#redeleteHangedFileInterval=120000

#检测物理文件磁盘空间

diskMaxUsedSpaceRatio=88

#存储路径

storePathRootDir=/usr/local/rocketmq/store/broker-b

#commitLog 存储路径

storePathCommitLog=/usr/local/rocketmq/store/broker-b/commitlog

#消费队列存储路径存储路径

storePathConsumeQueue=/usr/local/rocketmq/store/broker-b/consumequeue

#消息索引存储路径

storePathIndex=/usr/local/rocketmq/store/broker-b/index

#checkpoint 文件存储路径

storeCheckpoint=/usr/local/rocketmq/store/checkpoint

#abort 文件存储路径

abortFile=/usr/local/rocketmq/store/abort

#限制的消息大小

maxMessageSize=65536

#flushCommitLogLeastPages=4

#flushConsumeQueueLeastPages=2

#flushCommitLogThoroughInterval=10000

#flushConsumeQueueThoroughInterval=60000

#Broker 的角色

#- ASYNC_MASTER 异步复制Master

#- SYNC_MASTER 同步双写Master

#- SLAVE

brokerRole=ASYNC_MASTER

#刷盘方式

#- ASYNC_FLUSH 异步刷盘

#- SYNC_FLUSH 同步刷盘

flushDiskType=ASYNC_FLUSH

#checkTransactionMessageEnable=false

#发消息线程池数量

#sendMessageThreadPoolNums=128

#拉消息线程池数量

#pullMessageThreadPoolNums=128

vim broker-a-s.properties #内容和节点1的broker-b-s.properties配置相同,此处忽略

vim broker-a-s.properties #基本和节点1的相同除了ip和brokerId

brokerClusterName=rocketmq-cluster

brokerName=broker-a

#broker名字,名字可重复,为了管理,每个master起一个名字,他的slave同他,eg:Amaster叫broker-a,他的slave也叫broker-a

brokerId=1

#nameServer地址,分号分割

namesrvAddr=10.10.94.43:9876;10.10.94.44:9876

brokeIP1=10.10.94.44

#在发送消息时,自动创建服务器不存在的topic,默认创建的队列数

defaultTopicQueueNums=4

#是否允许 Broker 自动创建Topic,建议线下开启,线上关闭

autoCreateSubscriptionGroup=true

#Broker 对外服务的监听端口,

listenPort=10920

#删除文件时间点,默认凌晨 4点

deleteWhen=04

#文件保留时间,默认 48 小时

fileReservedTime=120

#commitLog每个文件的大小默认1G

mapedFileSizeCommitLog=1073741824

#ConsumeQueue每个文件默认存30W条,根据业务情况调整

mapedFileSizeConsumeQueue=300000

#destroyMapedFileIntervalForcibly=120000

#redeleteHangedFileInterval=120000

#检测物理文件磁盘空间

diskMaxUsedSpaceRatio=88

#存储路径

storePathRootDir=/usr/local/rocketmq/store/broker-a-s

#commitLog 存储路径

storePathCommitLog=/usr/local/rocketmq/store/broker-a-s/commitlog

#消费队列存储路径存储路径

storePathConsumeQueue=/usr/local/rocketmq/store/broker-a-s/consumequeue

#消息索引存储路径

storePathIndex=/usr/local/rocketmq/store/broker-a-s/index

#checkpoint 文件存储路径

storeCheckpoint=/usr/local/rocketmq/store/checkpoint

#abort 文件存储路径

abortFile=/usr/local/rocketmq/store/abort

#限制的消息大小

maxMessageSize=65536

#flushCommitLogLeastPages=4

#flushConsumeQueueLeastPages=2

#flushCommitLogThoroughInterval=10000

#flushConsumeQueueThoroughInterval=60000

#Broker 的角色

#- ASYNC_MASTER 异步复制Master

#- SYNC_MASTER 同步双写Master

#- SLAVE

brokerRole=SLAVE

#刷盘方式

#- ASYNC_FLUSH 异步刷盘

#- SYNC_FLUSH 同步刷盘

flushDiskType=ASYNC_FLUSH

#checkTransactionMessageEnable=false

#发消息线程池数量

#sendMessageThreadPoolNums=128

#拉消息线程池数量

#pullMessageThreadPoolNums=128 under the License.

6.启动broker服务。

节点1,启动broker-a:

nohup sh /usr/local/rocketmq/bin/mqbroker -c /usr/local/rocketmq/conf/2m-2s-async/broker-a.properties > /usr/local/rocketmq/logs/broker-a.log 2>&1 &

ps查看启动成功

# ps -ef|grep broker

root 31704 22886 0 15:23 pts/1 00:00:00 sh /usr/local/rocketmq/bin/mqbroker -c /usr/local/rocketmq/conf/2m-2s-async/broker-a.properties

root 31708 31704 0 15:23 pts/1 00:00:00 sh /usr/local/rocketmq/bin/runbroker.sh org.apache.rocketmq.broker.BrokerStartup -c /usr/local/rocketmq/conf/2m-2s-async/broker-a.properties

root 31711 31708 47 15:23 pts/1 00:00:05 /usr/java/jdk1.8.0_131/bin/java -server -Xms8g -Xmx8g -Xmn4g -XX:+UseG1GC -XX:G1HeapRegionSize=16m -XX:G1ReservePercent=25 -XX:InitiatingHeapOccupancyPercent=30 -XX:SoftRefLRUPolicyMSPerMB=0 -XX:SurvivorRatio=8 -verbose:gc -Xloggc:/dev/shm/mq_gc_%p.log -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCApplicationStoppedTime -XX:+PrintAdaptiveSizePolicy -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=30m -XX:-OmitStackTraceInFastThrow -XX:+AlwaysPreTouch -XX:MaxDirectMemorySize=15g -XX:-UseLargePages -XX:-UseBiasedLocking -Djava.ext.dirs=/usr/java/jdk1.8.0_131/jre/lib/ext:/usr/local/rocketmq/bin/../lib -cp .:/usr/local/rocketmq/bin/../conf:.:/usr/java/jdk1.8.0_131/jre/lib/rt.jar:/usr/java/jdk1.8.0_131/lib/tools.jar org.apache.rocketmq.broker.BrokerStartup -c /usr/local/rocketmq/conf/2m-2s-async/broker-a.properties

root 31865 22886 0 15:23 pts/1 00:00:00 grep --color=auto broker

节点1,启动broker-b-s:

nohup sh /usr/local/rocketmq/bin/mqbroker -c /usr/local/rocketmq/conf/2m-2s-async/broker-b-s.properties > /usr/local/rocketmq/logs/broker-b-s.log 2>&1 &

# ps -ef|grep mq

root 27614 10167 0 14:40 pts/0 00:00:00 sh /usr/local/rocketmq/bin/mqnamesrv

root 27618 27614 0 14:40 pts/0 00:00:00 sh /usr/local/rocketmq/bin/runserver.sh org.apache.rocketmq.namesrv.NamesrvStartup

root 27620 27618 0 14:40 pts/0 00:00:10 /home/jdk1.8.0_131/bin/java -server -Xms4g -Xmx4g -Xmn2g -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m -XX:+UseConcMarkSweepGC -XX:+UseCMSCompactAtFullCollection -XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSParallelRemarkEnabled -XX:SoftRefLRUPolicyMSPerMB=0 -XX:+CMSClassUnloadingEnabled -XX:SurvivorRatio=8 -XX:-UseParNewGC -verbose:gc -Xloggc:/dev/shm/rmq_srv_gc.log -XX:+PrintGCDetails -XX:-OmitStackTraceInFastThrow -XX:-UseLargePages -Djava.ext.dirs=/home/jdk1.8.0_131/jre/lib/ext:/usr/local/rocketmq/bin/../lib -cp .:/usr/local/rocketmq/bin/../conf::/home/jdk1.8.0_131/lib/dt.jar:/home/jdk1.8.0_131/lib/tools.jar:/home/jdk1.8.0_131/jre/lib/dt.jar org.apache.rocketmq.namesrv.NamesrvStartup

root 28454 10167 0 16:07 pts/0 00:00:00 sh /usr/local/rocketmq/bin/mqbroker -c /usr/local/rocketmq/conf/2m-2s-async/broker-a-s.properties

root 28458 28454 0 16:07 pts/0 00:00:00 sh /usr/local/rocketmq/bin/runbroker.sh org.apache.rocketmq.broker.BrokerStartup -c /usr/local/rocketmq/conf/2m-2s-async/broker-a-s.properties

root 28461 28458 82 16:07 pts/0 00:00:05 /home/jdk1.8.0_131/bin/java -server -Xms8g -Xmx8g -Xmn4g -XX:+UseG1GC -XX:G1HeapRegionSize=16m -XX:G1ReservePercent=25 -XX:InitiatingHeapOccupancyPercent=30 -XX:SoftRefLRUPolicyMSPerMB=0 -XX:SurvivorRatio=8 -verbose:gc -Xloggc:/dev/shm/mq_gc_%p.log -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCApplicationStoppedTime -XX:+PrintAdaptiveSizePolicy -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=30m -XX:-OmitStackTraceInFastThrow -XX:+AlwaysPreTouch -XX:MaxDirectMemorySize=15g -XX:-UseLargePages -XX:-UseBiasedLocking -Djava.ext.dirs=/home/jdk1.8.0_131/jre/lib/ext:/usr/local/rocketmq/bin/../lib -cp .:/usr/local/rocketmq/bin/../conf::/home/jdk1.8.0_131/lib/dt.jar:/home/jdk1.8.0_131/lib/tools.jar:/home/jdk1.8.0_131/jre/lib/dt.jar org.apache.rocketmq.broker.BrokerStartup -c /usr/local/rocketmq/conf/2m-2s-async/broker-a-s.properties

root 28536 10167 0 16:07 pts/0 00:00:00 grep --color=auto mq

节点2,启动broker-b:

nohup sh /usr/local/rocketmq/bin/mqbroker -c /usr/local/rocketmq/conf/2m-2s-async/broker-b.properties > /usr/local/rocketmq/logs/broker-b.log 2>&1 &

节点2,启动broker-a-s:

nohup sh /usr/local/rocketmq/bin/mqbroker -c /usr/local/rocketmq/conf/2m-2s-async/broker-a-s.properties > /usr/local/rocketmq/logs/broker-a-s.log 2>&1 &

7.查看集群列表。

sh /usr/local/rocketmq/bin/mqadmin clusterList -n 10.10.94.44:9876

Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0

Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0

#Cluster Name #Broker Name #BID #Addr #Version #InTPS(LOAD) #OutTPS(LOAD) #PCWait(ms) #Hour #SPACE

rocketmq-cluster broker-a 0 10.10.94.43:10911 V4_3_2 0.00(0,0ms) 0.00(0,0ms) 0 443185.33 0.6434

rocketmq-cluster broker-a 1 10.10.94.44:10920 V4_3_2 0.00(0,0ms) 0.00(0,0ms) 0 443185.33 0.6869

rocketmq-cluster broker-b 0 10.10.94.44:10911 V4_3_2 0.00(0,0ms) 0.00(0,0ms) 0 443185.33 0.6869

rocketmq-cluster broker-b 1 10.10.94.43:10920 V4_3_2 0.00(0,0ms) 0.00(0,0ms) 0 443185.33 0.6434

最后:

启动时可能会出现内存不足的错误,如下,因为mq默认分配的很大:

Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00000005c0000000, 8589934592, 0) failed; error='Cannot allocate memory' (errno=12)

#

# There is insufficient memory for the Java Runtime Environment to continue.

# Native memory allocation (mmap) failed to map 8589934592 bytes for committing reserved memory.

# An error report file with more information is saved as:

# /usr/local/rocketmq/conf/2m-2s-async/hs_err_pid28713.log

调小内存,即可解决:

如果单台启动了多个broker,先关闭

# ./bin/mqshutdown broker

# vim bin/runbroker.sh

JAVA_OPT="${JAVA_OPT} -server -Xms512m -Xmx512m -Xmn256m"

总结:网上的文章很多,但还是自己亲手搭建一下,才能发现问题,更深刻的理解,最好是先了解了RMQ的架构,才能轻车熟路。

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值
>