RocketMQ集群搭建

链接:https://pan.baidu.com/s/1SbS5BEmKX-DLf3SzC3NjyA?pwd=xt4p

  1. 知识准备
    https://blog.csdn.net/tongxin_tongmeng/article/details/126342599
  2.  RocketMQ入门篇
    https://blog.csdn.net/tongxin_tongmeng/article/details/126541285
  3. VMware创建虚拟机





  4. FinalShell连接虚拟机
    1.启动并登陆4台虚拟机
    用户名:root 密码:123456
    
    2.通过ip addr命令查询4台虚拟机ip地址
    Broker-a-m:192.168.1.109
    Broker-a-s:192.168.1.240
    Broker-b-m:192.168.1.115
    Broker-b-s:192.168.1.58
    
    3.FinalShell连接4台虚拟机
    用户名:root 密码:123456
    
    4.4台虚拟机创建rocketmq需要的目录结构
    /home/mq/rocketmq/logs
    /home/mq/rocketmq/store
    /home/mq/rocketmq/store/commitlog
    /home/mq/rocketmq/store/consumequeue
    /home/mq/rocketmq/store/index




  5. 修改runserver.sh和runbroker.sh

    JAVA_OPT="${JAVA_OPT} -server -Xms512m -Xmx512m -Xmn512m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m"

    JAVA_OPT="${JAVA_OPT} -server -Xms512m -Xmx512m -Xmn512m"
  6. 替换/home/mq/rocketmq/conf目录下所有xml中的${user.home}为/home/mq/rocketmq

    [root@localhost conf]# cd /home/mq/rocketmq/conf
    [root@localhost conf]# sed -i 's#${user.home}#/home/mq/rocketmq#g' *.xml
    
  7. 启动NameServer

    [root@localhost bin]# cd /home/mq/rocketmq/bin
    [root@localhost bin]# nohup sh mqnamesrv >/dev/null 2>&1 &
  8. 启动Broker
    1.Broker启动时需要指定/home/mq/rocketmq/conf/2m-2s-sync目录下不同的配置文件
    Broker-a-m: /home/mq/rocketmq/conf/2m-2s-sync/broker-a.properties
    Broker-a-s: /home/mq/rocketmq/conf/2m-2s-sync/broker-a-s.properties
    Broker-b-m: /home/mq/rocketmq/conf/2m-2s-sync/broker-b.properties
    Broker-b-s: /home/mq/rocketmq/conf/2m-2s-sync/broker-b-s.properties
    
    2.Broker-a-m: /home/mq/rocketmq/conf/2m-2s-sync/broker-a.properties
    brokerClusterName=rocketmq-cluster
    brokerName=broker-a
    brokerId=0
    #nameServer地址,分号分割,四个ip地址为自己创建的4台虚拟机ip地址
    namesrvAddr=192.168.1.109:9876;192.168.1.240:9876;192.168.1.115:9876;192.168.1.58:9876;
    brokerRole=SYNC_MASTER
    flushDiskType=SYNC_FLUSH
    
    3.Broker-a-s: /home/mq/rocketmq/conf/2m-2s-sync/broker-a-s.properties
    brokerClusterName=rocketmq-cluster
    brokerName=broker-a
    brokerId=1
    #nameServer地址,分号分割,四个ip地址为自己创建的4台虚拟机ip地址
    namesrvAddr=192.168.1.109:9876;192.168.1.240:9876;192.168.1.115:9876;192.168.1.58:9876;
    brokerRole=SLAVE
    flushDiskType=SYNC_FLUSH
    
    4.Broker-b-m: /home/mq/rocketmq/conf/2m-2s-sync/broker-b.properties
    brokerClusterName=rocketmq-cluster
    brokerName=broker-b
    brokerId=0
    #nameServer地址,分号分割,四个ip地址为自己创建的4台虚拟机ip地址
    namesrvAddr=192.168.1.109:9876;192.168.1.240:9876;192.168.1.115:9876;192.168.1.58:9876;
    brokerRole=SYNC_MASTER
    flushDiskType=SYNC_FLUSH
    
    5.Broker-b-s: /home/mq/rocketmq/conf/2m-2s-sync/broker-b-s.properties
    brokerClusterName=rocketmq-cluster
    brokerName=broker-b
    brokerId=1
    #nameServer地址,分号分割,四个ip地址为自己创建的4台虚拟机ip地址
    namesrvAddr=192.168.1.109:9876;192.168.1.240:9876;192.168.1.115:9876;192.168.1.58:9876;
    brokerRole=SLAVE
    flushDiskType=SYNC_FLUSH
    
    6.Broker启动命令
    Broker-a-m: nohup sh mqbroker -c /home/mq/rocketmq/conf/2m-2s-sync/broker-a.properties  >/dev/null 2>&1 &
    Broker-a-s: nohup sh mqbroker -c /home/mq/rocketmq/conf/2m-2s-sync/broker-a-s.properties  >/dev/null 2>&1 &
    Broker-b-m: nohup sh mqbroker -c /home/mq/rocketmq/conf/2m-2s-sync/broker-b.properties  >/dev/null 2>&1 &
    Broker-b-s: nohup sh mqbroker -c /home/mq/rocketmq/conf/2m-2s-sync/broker-b-s.properties  >/dev/null 2>&1 &


    # Licensed to the Apache Software Foundation (ASF) under one or more
    # contributor license agreements.  See the NOTICE file distributed with
    # this work for additional information regarding copyright ownership.
    # The ASF licenses this file to You under the Apache License, Version 2.0
    # (the "License"); you may not use this file except in compliance with
    # the License.  You may obtain a copy of the License at
    #
    #     http://www.apache.org/licenses/LICENSE-2.0
    #
    # Unless required by applicable law or agreed to in writing, software
    # distributed under the License is distributed on an "AS IS" BASIS,
    # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    # See the License for the specific language governing permissions and
    # 集群名称,同一个集群下的Broker,他们的brokerClusterName相同
    brokerClusterName=rocketmq-cluster
    #broker名字,同一组下的master和slave,他们的brokerName相同,eg:Amaster叫broker-a,同组下的slave也叫broker-a
    brokerName=broker-a
    #0 表示 Master,>0 表示 Slave
    brokerId=0
    #nameServer地址,分号分割,四个ip地址为自己创建的4台虚拟机ip地址
    namesrvAddr=192.168.1.109:9876;192.168.1.240:9876;192.168.1.115:9876;192.168.1.58:9876;
    #在发送消息时,自动创建服务器不存在的topic,默认创建的队列数
    defaultTopicQueueNums=4
    #是否允许 Broker 自动创建Topic,建议线下开启,线上关闭
    autoCreateTopicEnable=true
    #是否允许 Broker 自动创建订阅组,建议线下开启,线上关闭
    autoCreateSubscriptionGroup=true
    #Broker 对外服务的监听端口,
    listenPort=10911
    #删除文件时间点,默认凌晨 4点
    deleteWhen=04
    #文件保留时间,默认 48 小时
    fileReservedTime=120
    #commitLog每个文件的大小默认1G
    mapedFileSizeCommitLog=1073741824
    #ConsumeQueue每个文件默认存30W条,根据业务情况调整
    mapedFileSizeConsumeQueue=300000
    #destroyMapedFileIntervalForcibly=120000
    #redeleteHangedFileInterval=120000
    #检测物理文件磁盘空间
    diskMaxUsedSpaceRatio=88
    #存储路径
    storePathRootDir=/home/mq/rocketmq/store
    #commitLog 存储路径
    storePathCommitLog=/home/mq/rocketmq/store/commitlog
    #消费队列存储路径存储路径
    storePathConsumeQueue=/home/mq/rocketmq/store/consumequeue
    #消息索引存储路径
    storePathIndex=/home/mq/rocketmq/store/index
    #checkpoint 文件存储路径
    storeCheckpoint=/home/mq/rocketmq/store/checkpoint
    #abort 文件存储路径
    abortFile=/usr/local/rocketmq/store/abort
    #限制的消息大小
    maxMessageSize=65536
    #flushCommitLogLeastPages=4
    #flushConsumeQueueLeastPages=2
    #flushCommitLogThoroughInterval=10000
    #flushConsumeQueueThoroughInterval=60000
    #Broker 的角色
    #- ASYNC_MASTER 异步复制Master
    #- SYNC_MASTER 同步双写Master
    #- SLAVE
    brokerRole=SYNC_MASTER
    #刷盘方式
    #- ASYNC_FLUSH 异步刷盘 对应吞吐量要求较高的场景
    #- SYNC_FLUSH 同步刷盘 消息可靠性要求较高的场景
    flushDiskType=SYNC_FLUSH
    #checkTransactionMessageEnable=false
    #发消息线程池数量
    #sendMessageThreadPoolNums=128
    #拉消息线程池数量
    #pullMessageThreadPoolNums=128
    # Licensed to the Apache Software Foundation (ASF) under one or more
    # contributor license agreements.  See the NOTICE file distributed with
    # this work for additional information regarding copyright ownership.
    # The ASF licenses this file to You under the Apache License, Version 2.0
    # (the "License"); you may not use this file except in compliance with
    # the License.  You may obtain a copy of the License at
    #
    #     http://www.apache.org/licenses/LICENSE-2.0
    #
    # Unless required by applicable law or agreed to in writing, software
    # distributed under the License is distributed on an "AS IS" BASIS,
    # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    # See the License for the specific language governing permissions and
    # 集群名称,同一个集群下的Broker,他们的brokerClusterName相同
    brokerClusterName=rocketmq-cluster
    #broker名字,同一组下的master和slave,他们的brokerName相同,eg:Amaster叫broker-a,同组下的slave也叫broker-a
    brokerName=broker-a
    #0 表示 Master,>0 表示 Slave
    brokerId=1
    #nameServer地址,分号分割,四个ip地址为自己创建的4台虚拟机ip地址
    namesrvAddr=192.168.1.109:9876;192.168.1.240:9876;192.168.1.115:9876;192.168.1.58:9876;
    #在发送消息时,自动创建服务器不存在的topic,默认创建的队列数
    defaultTopicQueueNums=4
    #是否允许 Broker 自动创建Topic,建议线下开启,线上关闭
    autoCreateTopicEnable=true
    #是否允许 Broker 自动创建订阅组,建议线下开启,线上关闭
    autoCreateSubscriptionGroup=true
    #Broker 对外服务的监听端口,
    listenPort=10911
    #删除文件时间点,默认凌晨 4点
    deleteWhen=04
    #文件保留时间,默认 48 小时
    fileReservedTime=120
    #commitLog每个文件的大小默认1G
    mapedFileSizeCommitLog=1073741824
    #ConsumeQueue每个文件默认存30W条,根据业务情况调整
    mapedFileSizeConsumeQueue=300000
    #destroyMapedFileIntervalForcibly=120000
    #redeleteHangedFileInterval=120000
    #检测物理文件磁盘空间
    diskMaxUsedSpaceRatio=88
    #存储路径
    storePathRootDir=/home/mq/rocketmq/store
    #commitLog 存储路径
    storePathCommitLog=/home/mq/rocketmq/store/commitlog
    #消费队列存储路径存储路径
    storePathConsumeQueue=/home/mq/rocketmq/store/consumequeue
    #消息索引存储路径
    storePathIndex=/home/mq/rocketmq/store/index
    #checkpoint 文件存储路径
    storeCheckpoint=/home/mq/rocketmq/store/checkpoint
    #abort 文件存储路径
    abortFile=/usr/local/rocketmq/store/abort
    #限制的消息大小
    maxMessageSize=65536
    #flushCommitLogLeastPages=4
    #flushConsumeQueueLeastPages=2
    #flushCommitLogThoroughInterval=10000
    #flushConsumeQueueThoroughInterval=60000
    #Broker 的角色
    #- ASYNC_MASTER 异步复制Master
    #- SYNC_MASTER 同步双写Master
    #- SLAVE
    brokerRole=SLAVE
    #刷盘方式
    #- ASYNC_FLUSH 异步刷盘 对应吞吐量要求较高的场景
    #- SYNC_FLUSH 同步刷盘 消息可靠性要求较高的场景
    flushDiskType=SYNC_FLUSH
    #checkTransactionMessageEnable=false
    #发消息线程池数量
    #sendMessageThreadPoolNums=128
    #拉消息线程池数量
    #pullMessageThreadPoolNums=128
    # Licensed to the Apache Software Foundation (ASF) under one or more
    # contributor license agreements.  See the NOTICE file distributed with
    # this work for additional information regarding copyright ownership.
    # The ASF licenses this file to You under the Apache License, Version 2.0
    # (the "License"); you may not use this file except in compliance with
    # the License.  You may obtain a copy of the License at
    #
    #     http://www.apache.org/licenses/LICENSE-2.0
    #
    # Unless required by applicable law or agreed to in writing, software
    # distributed under the License is distributed on an "AS IS" BASIS,
    # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    # See the License for the specific language governing permissions and
    # 集群名称,同一个集群下的Broker,他们的brokerClusterName相同
    brokerClusterName=rocketmq-cluster
    #broker名字,同一组下的master和slave,他们的brokerName相同,eg:Amaster叫broker-a,同组下的slave也叫broker-a
    brokerName=broker-b
    #0 表示 Master,>0 表示 Slave
    brokerId=0
    #nameServer地址,分号分割,四个ip地址为自己创建的4台虚拟机ip地址
    namesrvAddr=192.168.1.109:9876;192.168.1.240:9876;192.168.1.115:9876;192.168.1.58:9876;
    #在发送消息时,自动创建服务器不存在的topic,默认创建的队列数
    defaultTopicQueueNums=4
    #是否允许 Broker 自动创建Topic,建议线下开启,线上关闭
    autoCreateTopicEnable=true
    #是否允许 Broker 自动创建订阅组,建议线下开启,线上关闭
    autoCreateSubscriptionGroup=true
    #Broker 对外服务的监听端口,
    listenPort=10911
    #删除文件时间点,默认凌晨 4点
    deleteWhen=04
    #文件保留时间,默认 48 小时
    fileReservedTime=120
    #commitLog每个文件的大小默认1G
    mapedFileSizeCommitLog=1073741824
    #ConsumeQueue每个文件默认存30W条,根据业务情况调整
    mapedFileSizeConsumeQueue=300000
    #destroyMapedFileIntervalForcibly=120000
    #redeleteHangedFileInterval=120000
    #检测物理文件磁盘空间
    diskMaxUsedSpaceRatio=88
    #存储路径
    storePathRootDir=/home/mq/rocketmq/store
    #commitLog 存储路径
    storePathCommitLog=/home/mq/rocketmq/store/commitlog
    #消费队列存储路径存储路径
    storePathConsumeQueue=/home/mq/rocketmq/store/consumequeue
    #消息索引存储路径
    storePathIndex=/home/mq/rocketmq/store/index
    #checkpoint 文件存储路径
    storeCheckpoint=/home/mq/rocketmq/store/checkpoint
    #abort 文件存储路径
    abortFile=/usr/local/rocketmq/store/abort
    #限制的消息大小
    maxMessageSize=65536
    #flushCommitLogLeastPages=4
    #flushConsumeQueueLeastPages=2
    #flushCommitLogThoroughInterval=10000
    #flushConsumeQueueThoroughInterval=60000
    #Broker 的角色
    #- ASYNC_MASTER 异步复制Master
    #- SYNC_MASTER 同步双写Master
    #- SLAVE
    brokerRole=SYNC_MASTER
    #刷盘方式
    #- ASYNC_FLUSH 异步刷盘 对应吞吐量要求较高的场景
    #- SYNC_FLUSH 同步刷盘 消息可靠性要求较高的场景
    flushDiskType=SYNC_FLUSH
    #checkTransactionMessageEnable=false
    #发消息线程池数量
    #sendMessageThreadPoolNums=128
    #拉消息线程池数量
    #pullMessageThreadPoolNums=128
    # Licensed to the Apache Software Foundation (ASF) under one or more
    # contributor license agreements.  See the NOTICE file distributed with
    # this work for additional information regarding copyright ownership.
    # The ASF licenses this file to You under the Apache License, Version 2.0
    # (the "License"); you may not use this file except in compliance with
    # the License.  You may obtain a copy of the License at
    #
    #     http://www.apache.org/licenses/LICENSE-2.0
    #
    # Unless required by applicable law or agreed to in writing, software
    # distributed under the License is distributed on an "AS IS" BASIS,
    # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    # See the License for the specific language governing permissions and
    # 集群名称,同一个集群下的Broker,他们的brokerClusterName相同
    brokerClusterName=rocketmq-cluster
    #broker名字,同一组下的master和slave,他们的brokerName相同,eg:Amaster叫broker-a,同组下的slave也叫broker-a
    brokerName=broker-b
    #0 表示 Master,>0 表示 Slave
    brokerId=1
    #nameServer地址,分号分割,四个ip地址为自己创建的4台虚拟机ip地址
    namesrvAddr=192.168.1.109:9876;192.168.1.240:9876;192.168.1.115:9876;192.168.1.58:9876;
    #在发送消息时,自动创建服务器不存在的topic,默认创建的队列数
    defaultTopicQueueNums=4
    #是否允许 Broker 自动创建Topic,建议线下开启,线上关闭
    autoCreateTopicEnable=true
    #是否允许 Broker 自动创建订阅组,建议线下开启,线上关闭
    autoCreateSubscriptionGroup=true
    #Broker 对外服务的监听端口,
    listenPort=10911
    #删除文件时间点,默认凌晨 4点
    deleteWhen=04
    #文件保留时间,默认 48 小时
    fileReservedTime=120
    #commitLog每个文件的大小默认1G
    mapedFileSizeCommitLog=1073741824
    #ConsumeQueue每个文件默认存30W条,根据业务情况调整
    mapedFileSizeConsumeQueue=300000
    #destroyMapedFileIntervalForcibly=120000
    #redeleteHangedFileInterval=120000
    #检测物理文件磁盘空间
    diskMaxUsedSpaceRatio=88
    #存储路径
    storePathRootDir=/home/mq/rocketmq/store
    #commitLog 存储路径
    storePathCommitLog=/home/mq/rocketmq/store/commitlog
    #消费队列存储路径存储路径
    storePathConsumeQueue=/home/mq/rocketmq/store/consumequeue
    #消息索引存储路径
    storePathIndex=/home/mq/rocketmq/store/index
    #checkpoint 文件存储路径
    storeCheckpoint=/home/mq/rocketmq/store/checkpoint
    #abort 文件存储路径
    abortFile=/usr/local/rocketmq/store/abort
    #限制的消息大小
    maxMessageSize=65536
    #flushCommitLogLeastPages=4
    #flushConsumeQueueLeastPages=2
    #flushCommitLogThoroughInterval=10000
    #flushConsumeQueueThoroughInterval=60000
    #Broker 的角色
    #- ASYNC_MASTER 异步复制Master
    #- SYNC_MASTER 同步双写Master
    #- SLAVE
    brokerRole=SLAVE
    #刷盘方式
    #- ASYNC_FLUSH 异步刷盘 对应吞吐量要求较高的场景
    #- SYNC_FLUSH 同步刷盘 消息可靠性要求较高的场景
    flushDiskType=SYNC_FLUSH
    #checkTransactionMessageEnable=false
    #发消息线程池数量
    #sendMessageThreadPoolNums=128
    #拉消息线程池数量
    #pullMessageThreadPoolNums=128
  9. 关闭NameServer和Broker

    1.启动时:先启动NameServer再启动Broker
    
    2.关闭时:先关闭Broker再关闭NameServer
    
    3.Broker关闭命令:sh mqshutdown broker
    
    4.NameServer关闭命令:sh mqshutdown namesrv
  10. RocketMQ控制台启动(RocketMQ入门篇)


    D:\program\RocketMQ\rocketmq-console>mvn clean package -Dmaven.test.skip=true


    D:\program\RocketMQ\rocketmq-console\target>java -jar rocketmq-console-ng-2.0.0.jar
  11. RocketMQ控制台访问(RocketMQ入门篇)
     http://localhost:8888 端口8888为application.properties设置
  12. 集群测试(RocketMQ入门篇)
    1.rocketmq-provider的配置文件application.yml
    server:
      port: 8081
      servlet:
        context-path: /
    
    rocketmq:
    #  name-server: 127.0.0.1:9876
      name-server: 192.168.1.109:9876;192.168.1.240:9876;192.168.1.115:9876;192.168.1.58:9876;
      producer:
        group: producer-demo
        send-message-timeout: 60000
    
    2.rocketmq-consumer的配置文件application.yml
    server:
      port: 8082
      servlet:
        context-path: /
    
    rocketmq:
      name-server: 192.168.1.109:9876;192.168.1.240:9876;192.168.1.115:9876;192.168.1.58:9876;
      consumer:
        group_1: consumer-demo1
        group_2: consumer-demo2
        group_3: consumer-demo3
        group_4: consumer-demo4
        group_5: consumer-demo5
        group_6: consumer-demo6
        group_7: consumer-demo7
        group_8: consumer-demo8
        group_9: consumer-demo9
        group_10: consumer-demo10
        group_11: consumer-demo11
    1.发送简单消息-->消息发送成功
    结论:消息优先被主节点Broker-a-m和Broker-b-m消费
    
    2.关闭Broker-a-m、Broker-a-s、Broker-b-m、Broker-b-s任意三个NameServer服务-->消息发送成功
    结论:4台虚拟机运行了4个NameServer服务,只要有一个运行,就可以发送消息
    
    3.关闭Broker-a-m、Broker-a-s、Broker-b-m、Broker-b-s的NameServer服务-->消息发送失败
    报错:Caused by: org.apache.rocketmq.remoting.exception.RemotingConnectException: connect to [192.168.1.115:9876, 192.168.1.58:9876, 192.168.1.240:9876, 192.168.1.109:9876] failed
    结论:4台虚拟机运行了4个NameServer服务,如果都不运行,则不可以发送消息
    
    4.关闭Broker-a-s、Broker-b-s的Broker服务-->消息发送成功
    结论:关闭从节点Broker-a-s和Broker-b-s的Broker服务,只要Broker-a-m和Broker-b-m有一个主节点的Broker服务正常运行,就可以发送消息
    
    4.关闭Broker-a-m、Broker-b-m的Broker服务-->消息发送失败
    报错:Exception in thread "main" org.springframework.messaging.MessagingException: No route info of this topic: rocketmq_1
    结论:关闭所有主节点的Broker服务,即使从节点的Broker服务都正常运行,也无法发送消息





    ​​​​​​​

 

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 打赏
    打赏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

童心同萌

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付:¥1
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值