rocketmq同步双写

转载 2018年04月15日 11:30:19

配置域名

1
2
192.168.241.101 paascloud-rocketmq-001
192.168.241.102 paascloud-rocketmq-002

下载

1
2
wget http://ftp.jaist.ac.jp/pub/apache/rocketmq/4.2.0/rocketmq-all-4.2.0-bin-release.zip
scp rocketmq-all-4.2.0-bin-release.zip root@192.168.241.101:/root

解压

1
unzip rocketmq-all-4.2.0-bin-release.zip -d /usr/local/rocketmq

配置环境变量

1
vim /etc/profile
1
2
export ROCKETMQ_HOME=/usr/local/rocketmq
export PATH=$PATH::$ROCKETMQ_HOME/bin
1
# source /etc/profile

修改配置

broker-a.properties

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
brokerClusterName=paascloud-rocketmq-cluster
brokerName=broker-a
brokerId=0
namesrvAddr=paascloud-rocketmq-001:9876;paascloud-rocketmq-002:9876;
defaultTopicQueueNums=4
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
listenPort=10911
deleteWhen=04
fileReservedTime=120
mapedFileSizeCommitLog=1073741824
mapedFileSizeConsumeQueue=300000
destroyMapedFileIntervalForcibly=120000
redeleteHangedFileInterval=120000
diskMaxUsedSpaceRatio=88
storePathRootDir=/usr/local/rocketmq/data/master/store
storePathCommitLog=/usr/local/rocketmq/data/master/store/commitlog
maxMessageSize=65536
flushCommitLogLeastPages=4
flushConsumeQueueLeastPages=2
flushCommitLogThoroughInterval=10000
flushConsumeQueueThoroughInterval=60000
checkTransactionMessageEnable=false
sendMessageThreadPoolNums=128
pullMessageThreadPoolNums=128
brokerRole=SYNC_MASTER
flushDiskType=SYNC_FLUSH

broker-a-s.properties

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
brokerClusterName=paascloud-rocketmq-cluster
brokerName=broker-a
brokerId=1
namesrvAddr=paascloud-rocketmq-001:9876;paascloud-rocketmq-002:9876;
defaultTopicQueueNums=4
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
listenPort=10921
deleteWhen=04
fileReservedTime=120
mapedFileSizeCommitLog=1073741824
mapedFileSizeConsumeQueue=300000
destroyMapedFileIntervalForcibly=120000
redeleteHangedFileInterval=120000
diskMaxUsedSpaceRatio=88
storePathRootDir=/usr/local/rocketmq/data/slave/store
storePathCommitLog=/usr/local/rocketmq/data/slave/store/commitlog
maxMessageSize=65536
flushCommitLogLeastPages=4
flushConsumeQueueLeastPages=2
flushCommitLogThoroughInterval=10000
flushConsumeQueueThoroughInterval=60000
checkTransactionMessageEnable=false
sendMessageThreadPoolNums=128
pullMessageThreadPoolNums=128
brokerRole=SLAVE
flushDiskType=SYNC_FLUSH

broker-b.properties

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
brokerClusterName=paascloud-rocketmq-cluster
brokerName=broker-b
brokerId=0
namesrvAddr=paascloud-rocketmq-001:9876;paascloud-rocketmq-002:9876;
defaultTopicQueueNums=4
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
listenPort=10911
deleteWhen=04
fileReservedTime=120
mapedFileSizeCommitLog=1073741824
mapedFileSizeConsumeQueue=300000
destroyMapedFileIntervalForcibly=120000
redeleteHangedFileInterval=120000
diskMaxUsedSpaceRatio=88
storePathRootDir=/usr/local/rocketmq/data/master/store
storePathCommitLog=/usr/local/rocketmq/data/master/store/commitlog
maxMessageSize=65536
flushCommitLogLeastPages=4
flushConsumeQueueLeastPages=2
flushCommitLogThoroughInterval=10000
flushConsumeQueueThoroughInterval=60000
checkTransactionMessageEnable=false
sendMessageThreadPoolNums=128
pullMessageThreadPoolNums=128
brokerRole=SYNC_MASTER
flushDiskType=SYNC_FLUSH

broker-b-s.properties

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
brokerClusterName=paascloud-rocketmq-cluster
brokerName=broker-b
brokerId=1
namesrvAddr=paascloud-rocketmq-001:9876;paascloud-rocketmq-002:9876;
defaultTopicQueueNums=4
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
listenPort=10921
deleteWhen=04
fileReservedTime=120
mapedFileSizeCommitLog=1073741824
mapedFileSizeConsumeQueue=300000
destroyMapedFileIntervalForcibly=120000
redeleteHangedFileInterval=120000
diskMaxUsedSpaceRatio=88
storePathRootDir=/usr/local/rocketmq/data/slave/store
storePathCommitLog=/usr/local/rocketmq/data/slave/store/commitlog
maxMessageSize=65536
flushCommitLogLeastPages=4
flushConsumeQueueLeastPages=2
flushCommitLogThoroughInterval=10000
flushConsumeQueueThoroughInterval=60000
checkTransactionMessageEnable=false
sendMessageThreadPoolNums=128
pullMessageThreadPoolNums=128
brokerRole=SLAVE
flushDiskType=SYNC_FLUSH

创建存储&日志文件

1
2
3
4
5
6
7
# mkdir -p /usr/local/rocketmq/data/master/store/commitlog
# mkdir -p /usr/local/rocketmq/data/slave/store/commitlog
# mkdir -p /usr/local/rocketmq/data/master/store/consumequeue
# mkdir -p /usr/local/rocketmq/data/slave/store/consumequeue
# mkdir -p /usr/local/rocketmq/data/master/store/index
# mkdir -p /usr/local/rocketmq/data/slave/store/index
# mkdir -p /usr/local/rocketmq/logs

修改日志配置文件

1
cd /usr/local/rocketmq/conf && sed -i 's#${user.home}#/usr/local/rocketmq#g' *.xml

修改启动脚本参数

1
2
vim /usr/local/rocketmq/bin/runbroker.sh
JAVA_OPT="${JAVA_OPT} -server -Xms1g -Xmx1g -Xmn512m"
1
2
vim /usr/local/rocketmq/bin/runserver.sh
JAVA_OPT="${JAVA_OPT} -server -Xms1g -Xmx1g -Xmn512m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m"

启动服务

  1. 启动NameServer A 192.168.241.101
1
nohup sh /usr/local/rocketmq/bin/mqnamesrv &
  1. 启动NameServer A 192.168.241.101
1
nohup sh /usr/local/rocketmq/bin/mqnamesrv &
  1. 启动BrokerServer A-master 192.168.241.101
1
nohup sh mqbroker -c /usr/local/rocketmq/conf/2m-2s-sync/broker-a.properties&
  1. 启动BrokerServer A-slave 192.168.241.101
1
nohup sh mqbroker -c /usr/local/rocketmq/conf/2m-2s-sync/broker-a-s.properties&
  1. 启动BrokerServer B-master 192.168.241.102
1
nohup sh mqbroker -c /usr/local/rocketmq/conf/2m-noslave/broker-b.properties&
  1. 启动BrokerServer B-slave 192.168.241.102
1
nohup sh mqbroker -c /usr/local/rocketmq/conf/2m-noslave/broker-b-s.properties&

是否启动成功

这里写图片描述这里写图片描述

1
2
3
4
# netstat -ntlp
# jps
# tail -f -n 500 /usr/local/rocketmq/logs/rocketmqlogs/broker.log
# tail -f -n 500 /usr/local/rocketmq/logs/rocketmqlogs/namesrv.log

查看集群监控状态

1
# sh /usr/local/rocketmq/bin/mqadmin clusterlist -n 192.168.241.101:9876

这里写图片描述这里写图片描述

停止服务

1
2
# sh /usr/local/rocketmq/bin/mqshutdown namesrv
# sh /usr/local/rocketmq/bin/mqshutdown broker

清理数据

1
2
3
4
5
6
7
8
# rm -rf /usr/local/rocketmq/data/master
# rm -rf /usr/local/rocketmq/data/slave
# mkdir -p /usr/local/rocketmq/data/master/store/commitlog
# mkdir -p /usr/local/rocketmq/data/slave/store/commitlog
# mkdir -p /usr/local/rocketmq/data/master/store/consumequeue
# mkdir -p /usr/local/rocketmq/data/slave/store/consumequeue
# mkdir -p /usr/local/rocketmq/data/master/store/index
# mkdir -p /usr/local/rocketmq/data/slave/store/index

rocketmq-console-ng

rocketmq-console-ng.bat

1
2
3
@echo off
java -jar rocketmq-console-ng-1.0.0.jar --server.port=12581 --rocketmq.config.namesrvAddr=192.168.241.101:9876;192.168.241.102:9876;
@pause

效果图
这里写图片描述这里写图片描述

参考文章

https://www.jianshu.com/p/9d4e0ff358c6
http://blog.csdn.net/lovesomnus/article/details/51769977

rocketmq异步复制or同步双写

原文链接:http://www.cnblogs.com/xiaodf/p/5075167.html 参考链接:rocketmq与kafka http://blog.csdn.net/chunlon...
  • zengxiaosen
  • zengxiaosen
  • 2017-12-25 20:59:22
  • 645

rocketMq双Master双Slave,同步双写,异步刷盘

首先,我们准备7台linux服务器,尽可能的在每个服务器上部署一个服务。当然,在实际工作中如果服务器数量不足,那么可以同一台机器部署两个节点,比如:broker-a的主节点与broker-b的从节点部...
  • www_wangzheguilai
  • www_wangzheguilai
  • 2017-08-02 10:12:53
  • 1660

rocketmq 同步双写

配置域名 192.168.241.101 paascloud-rocketmq-001 192.168.241.102 paascloud-rocketmq-002 下载 wget ...
  • liu_zhaoming
  • liu_zhaoming
  • 2018-02-22 16:48:36
  • 44

RocketMQ源码分析之Broker概述与同步消息发送原理与高可用设计及思考

本文初步介绍了RocketMQ的存储架构原理,源码分析了同步消息发送实现原理,已经消息发送的高可用设计原理,分析并总结了在NameServer、Broker 宕机的情况下,消息发送端的容错机制。...
  • prestigeding
  • prestigeding
  • 2017-07-22 18:58:17
  • 1709

Rocketmq 配置双主双从

Rocketmq 配置双Master双Slave 这个配置基本流程和Rocketmq 配置双master 是一样的。 具体可参考:http://www.lrshuai.top/atc/sho...
  • qq_32923745
  • qq_32923745
  • 2017-10-18 11:20:22
  • 114

rocketmq 一主多从 的同步机制

一个master 多个slave下,数据是怎么同步的。 其实是master 维持一个 变量,如果是同步双写的的时候,将这个变量置为对应的offset。然后不停地查看slave上报的进度有没有超过这个...
  • watertofish
  • watertofish
  • 2017-11-10 01:21:34
  • 264

RocketMQ-3.5.8双主双从配置及部署说明

  • 2016年10月29日 22:44
  • 12.89MB
  • 下载

rocketmq 双master模式

Linux环境快速搭建RocketMQ双Master模式给机器命名 vi /etc/hosts 172.23.21.222 dev02.haijk.zhhl 重启网卡 service network ...
  • chang_li
  • chang_li
  • 2017-01-09 15:14:47
  • 1305

RocketMQ多Master模式

了解了RocketMQ的基本知识后,我们先来学习一下RocketMQ的多Master模式。   多Master模式:集群中不存在Slave节点,集群中所有节点都是Master节点。优点是配置简单,单...
  • u013036278
  • u013036278
  • 2017-06-25 15:17:39
  • 946

【RocketMQ】7.RocketMQ 关键特性之刷盘策略

RocketMQ 的所有消息都是持久化的, 先写入系统 PAGECACHE, 然后刷盘, 可以保证内存与磁盘都有一份数据,访问时,直接从内存读取。 7.2.1 异步刷盘 在有 RAID 卡...
  • Vchen_hao
  • Vchen_hao
  • 2016-12-14 10:59:44
  • 2138
收藏助手
不良信息举报
您举报文章:rocketmq同步双写
举报原因:
原因补充:

(最多只允许输入30个字)