RocketMQ集群搭建-双主双从

2 篇文章 0 订阅
1 篇文章 0 订阅

环境:vmware fusion centos7 64位 两台  内存4G。ip:172.16.147.2,172.16.147.3

集群部署方式选择:2namesrv,2broker+2slave,1console。

逻辑拓扑:

物理拓扑:

ip

部署服务

角色

172.16.147.2

nameserver

 

172.16.147.3

nameserver

 

172.16.147.2

broker-a

broker master

172.16.147.3

broker-b

broker master

172.16.147.2

broker-b-s

broker slave

172.16.147.3

broker-a-s

broker slave

 

部署流程:

1.下载并解压rocketmq,安装Java,telnet等

scp /Users/bjhl/Downloads/rocketmq-all-4.2.0-bin-release.zip root@ip:/root/rocketmq/rocketmq-2m2s

unzip rocketmq-all-4.2.0-bin-release.zip

2.关闭防火墙,或放开端口

systemctl status firewalld
systemctl disable firewalld
systemctl stop firewalld
systemctl status firewalld

3.配置文件:

broker双主双从配置目录为:/conf/2m-2s-async

172.16.147.2:

broker-a.properties:

# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
brokerClusterName=DefaultCluster
brokerName=broker-a
brokerId=0
deleteWhen=04
fileReservedTime=48
brokerRole=ASYNC_MASTER
flushDiskType=ASYNC_FLUSH

listenPort=10911
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
namesrvAddr=172.16.147.2:9876;172.16.147.3:9876
defaultTopicQueueNums=4
storePathRootDir=/root/rocketmq/rocketmq-2m2s/store
storePathCommitLog=/root/rocketmq/rocketmq-2m2s/store/commitlog
storePathConsumeQueue=/root/rocketmq/rocketmq-2m2s/store/consumequeue
maxMessageSize=65536

broker-b-s.properties:

# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
brokerClusterName=DefaultCluster
brokerName=broker-b
brokerId=1
deleteWhen=04
fileReservedTime=48
brokerRole=SLAVE
flushDiskType=ASYNC_FLUSH


listenPort=10921
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
namesrvAddr=172.16.147.2:9876;172.16.147.3:9876
defaultTopicQueueNums=4
storePathRootDir=/root/rocketmq/rocketmq-2m2s/store-s
storePathCommitLog=/root/rocketmq/rocketmq-2m2s/store-s/commitlog
storePathConsumeQueue=/root/rocketmq/rocketmq-2m2s/store-s/consumequeue
maxMessageSize=65536

172.16.147.3:

broker-b.properties:

# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
brokerClusterName=DefaultCluster
brokerName=broker-b
brokerId=0
deleteWhen=04
fileReservedTime=48
brokerRole=ASYNC_MASTER
flushDiskType=ASYNC_FLUSH


listenPort=10911
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
namesrvAddr=172.16.147.2:9876;172.16.147.3:9876
defaultTopicQueueNums=4
storePathRootDir=/root/rocketmq/rocketmq-2m2s/store
storePathCommitLog=/root/rocketmq/rocketmq-2m2s/store/commitlog
storePathConsumeQueue=/root/rocketmq/rocketmq-2m2s/store/consumequeue
maxMessageSize=65536

broker-a-s.properties:

# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
brokerClusterName=DefaultCluster
brokerName=broker-a
brokerId=1
deleteWhen=04
fileReservedTime=48
brokerRole=SLAVE
flushDiskType=ASYNC_FLUSH

listenPort=10921
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
namesrvAddr=172.16.147.2:9876;172.16.147.3:9876
defaultTopicQueueNums=4
storePathRootDir=/root/rocketmq/rocketmq-2m2s/store-s
storePathCommitLog=/root/rocketmq/rocketmq-2m2s/store-s/commitlog
storePathConsumeQueue=/root/rocketmq/rocketmq-2m2s/store-s/consumequeue
maxMessageSize=65536

每个VM的namesrv和broker启动时jvm参数配置目录:/bin

runserver.sh:

#!/bin/sh

# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

#===========================================================================================
# Java Environment Setting
#===========================================================================================
error_exit ()
{
    echo "ERROR: $1 !!"
    exit 1
}

[ ! -e "$JAVA_HOME/bin/java" ] && JAVA_HOME=$HOME/jdk/java
[ ! -e "$JAVA_HOME/bin/java" ] && JAVA_HOME=/usr/java
[ ! -e "$JAVA_HOME/bin/java" ] && error_exit "Please set the JAVA_HOME variable in your environment, We need java(x64)!"

export JAVA_HOME
export JAVA="$JAVA_HOME/bin/java"
export BASE_DIR=$(dirname $0)/..
export CLASSPATH=.:${BASE_DIR}/conf:${CLASSPATH}

#===========================================================================================
# JVM Configuration
#===========================================================================================
JAVA_OPT="${JAVA_OPT} -server -Xms512m -Xmx512m -Xmn256m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m"
JAVA_OPT="${JAVA_OPT} -XX:+UseConcMarkSweepGC -XX:+UseCMSCompactAtFullCollection -XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSParallelRemarkEnabled -XX:SoftRefLRUPolicyMSPerMB=0 -XX:+CMSClassUnloadingEnabled -XX:SurvivorRatio=8  -XX:-UseParNewGC"
JAVA_OPT="${JAVA_OPT} -verbose:gc -Xloggc:/dev/shm/rmq_srv_gc.log -XX:+PrintGCDetails"
JAVA_OPT="${JAVA_OPT} -XX:-OmitStackTraceInFastThrow"
JAVA_OPT="${JAVA_OPT}  -XX:-UseLargePages"
JAVA_OPT="${JAVA_OPT} -Djava.ext.dirs=${JAVA_HOME}/jre/lib/ext:${BASE_DIR}/lib"
#JAVA_OPT="${JAVA_OPT} -Xdebug -Xrunjdwp:transport=dt_socket,address=9555,server=y,suspend=n"
JAVA_OPT="${JAVA_OPT} ${JAVA_OPT_EXT}"
JAVA_OPT="${JAVA_OPT} -cp ${CLASSPATH}"

$JAVA ${JAVA_OPT} $@

runbroker.sh:

#!/bin/sh

# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

#===========================================================================================
# Java Environment Setting
#===========================================================================================
error_exit ()
{
    echo "ERROR: $1 !!"
    exit 1
}

[ ! -e "$JAVA_HOME/bin/java" ] && JAVA_HOME=$HOME/jdk/java
[ ! -e "$JAVA_HOME/bin/java" ] && JAVA_HOME=/usr/java
[ ! -e "$JAVA_HOME/bin/java" ] && error_exit "Please set the JAVA_HOME variable in your environment, We need java(x64)!"

export JAVA_HOME
export JAVA="$JAVA_HOME/bin/java"
export BASE_DIR=$(dirname $0)/..
export CLASSPATH=.:${BASE_DIR}/conf:${CLASSPATH}

#===========================================================================================
# JVM Configuration
#===========================================================================================
JAVA_OPT="${JAVA_OPT} -server -Xms512m -Xmx512m -Xmn256m"
JAVA_OPT="${JAVA_OPT} -XX:+UseG1GC -XX:G1HeapRegionSize=16m -XX:G1ReservePercent=25 -XX:InitiatingHeapOccupancyPercent=30 -XX:SoftRefLRUPolicyMSPerMB=0 -XX:SurvivorRatio=8"
JAVA_OPT="${JAVA_OPT} -verbose:gc -Xloggc:/dev/shm/mq_gc_%p.log -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCApplicationStoppedTime -XX:+PrintAdaptiveSizePolicy"
JAVA_OPT="${JAVA_OPT} -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=30m"
JAVA_OPT="${JAVA_OPT} -XX:-OmitStackTraceInFastThrow"
JAVA_OPT="${JAVA_OPT} -XX:+AlwaysPreTouch"
JAVA_OPT="${JAVA_OPT} -XX:MaxDirectMemorySize=15g"
JAVA_OPT="${JAVA_OPT} -XX:-UseLargePages -XX:-UseBiasedLocking"
JAVA_OPT="${JAVA_OPT} -Djava.ext.dirs=${JAVA_HOME}/jre/lib/ext:${BASE_DIR}/lib"
#JAVA_OPT="${JAVA_OPT} -Xdebug -Xrunjdwp:transport=dt_socket,address=9555,server=y,suspend=n"
JAVA_OPT="${JAVA_OPT} ${JAVA_OPT_EXT}"
JAVA_OPT="${JAVA_OPT} -cp ${CLASSPATH}"

numactl --interleave=all pwd > /dev/null 2>&1
if [ $? -eq 0 ]
then
        if [ -z "$RMQ_NUMA_NODE" ] ; then
                numactl --interleave=all $JAVA ${JAVA_OPT} $@
        else
                numactl --cpunodebind=$RMQ_NUMA_NODE --membind=$RMQ_NUMA_NODE $JAVA ${JAVA_OPT} $@
        fi
else
        $JAVA ${JAVA_OPT} $@
fi

步骤:

# 集群启动

# 172.16.147.2

nohup sh mqnamesrv -c ……/conf/namesrv.properties &

nohup sh mqbroker -c ……/conf/2m-2s-sync/broker-a.properties &

nohup sh mqbroker -c ……/conf/2m-2s-sync/broker-b-s.properties &

# 172.16.147.3

nohup sh mqnamesrv -c ……/conf/namesrv.properties &

nohup sh mqbroker -c ……/conf/2m-2s-sync/broker-b.properties &

nohup sh mqbroker -c ……/conf/2m-2s-sync/broker-a-s.properties &


# 关闭集群

# 172.16.147.2,172.16.147.3

sh mqshutdown broker

sh mqshutdown namesrv

监控部署:

1.下载:https://gitee.com/zziamalei/rocketmq-externals?_from=gitee_search

cd rocketmq-console

# 在rocketmq-console/src/main/resources/application.properties 配置namesrv地址

rocketmq.config.namesrvAddr=192.168.25.135:9876;192.168.25.138:9876

mvn clean package -Dmaven.test.skip=true

# 打包后在rocketmq-console/target下执行

java -jar rocketmq-console-ng-1.0.0.jar


2.启动后访问localhost:8080进入监控页面

监控相关参考:https://juejin.cn/post/6854573214988222472

测试:

# 配置环境变量,仅对当前终端有效
export NAMESRV_ADDR=172.16.147.2:9876;172.16.147.3:9876
#测试发送端
sh bin/tools.sh org.apache.rocketmq.example.quickstart.Producer
#测试消费端
sh bin/tools.sh org.apache.rocketmq.example.quickstart.Consumer

关于集群搭建和测试消息的发送和消费可参考文章:https://cloud.tencent.com/developer/article/1412653

出现的问题和卡住的点:

1.namesrv和broker的jvm参数配置不明确,一开始运行broker和namesrv后启动测试demo的producer显示内存不足启动失败,后来二者都改为最大512m。

2.broker的配置文件多余配置了abort,index,checkpoint文件夹,正确路径应该配置文件而不是文件夹,导致运行mqbroker后broker无反应,broker.log无启动成功的日志,无报错日志。最后在store.log日志中发现报错。

3.下载rocketmq监控时git过慢,没有尽早使用码云。

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值