zookeeper+activemq配置消息中间件集群服务器配置

最新推荐文章于 2023-05-19 14:52:30 发布

白开水挺好喝

最新推荐文章于 2023-05-19 14:52:30 发布

阅读量405

点赞数

分类专栏：开发组件文章标签： zookeeper activemq

开发组件专栏收录该内容

10 篇文章 0 订阅

订阅专栏

需求：

1、将大量的WebService请求报文发送到mq集群之中，并保持消息先后顺序

2、保证每个消息的可靠性

3、维护MQ服务器的可扩展性

综合考虑，决定使用Apache的 activemq。接触的时候，新的activemq 5.10已经出，所以，本项目使用的是activemq5.10。activemq5.10要求jdk6+，由于项目立项比较老，所以将项目整体迁移到jdk1.7进行部署，并引入activemq服务器。OK，废话不多说，开始。

首先，选三台服务器。

之所以选三台，是因为zookeeper推荐最低三台配置，这样可以保持最大的可用服务器数，具体的这里不多说，可以查阅zookeeper的安装配置，以后有机会，详细讲解zookeeper。

我这里选择三台linux服务器，ip地址分别为：192.168.120.241,192.168.120.242,192.168.120.171。系统均使用centos6.4

1、创建账号

useradd amqbroker; //默认会创建amqbroker的群组，并且amqbroker会自动加入amqbroker群组。

passwd amqbroker;修改密码

cd /home/amqbroker 切换目录

2、下载zookeeper

wget http://mirror.bit.edu.cn/apache/zookeeper/zookeeper-3.3.6/zookeeper-3.3.6.tar.gz

tar -xvf zookeeper-3.3.6.tar.gz

修改配置，mv zoo_sample.cfg zoo.cfg

vi zoo.cfg,内容如下：

[java] view plain copy

print ?

1 # The number of milliseconds of each tick
2 tickTime=2000
3 # The number of ticks that the initial
4 # synchronization phase can take
5 initLimit=10
6 # The number of ticks that can pass between
7 # sending a request and getting an acknowledgement
8 syncLimit=5
9 # the directory where the snapshot is stored.
10 # do not use /tmp for storage, /tmp here is just
11 # example sakes.
12
13 dataDir=/home/amqbroker/zkdir/data
14 dataLogDir=/home/amqbroker/zkdir/log
15 # the port at which the clients will connect
16 clientPort=2181
17 # the maximum number of client connections.
18 # increase this if you need to handle more clients
19 #maxClientCnxns=60
20 #
21 # Be sure to read the maintenance section of the
22 # administrator guide before turning on autopurge.
23 #
24 # http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
25 #
26 # The number of snapshots to retain in dataDir
27 #autopurge.snapRetainCount=3
28 # Purge task interval in hours
29 # Set to “0” to disable auto purge feature
30 #autopurge.purgeInterval=1
31
32
33 ##three servers of this cluster
34 server.1=192.168.120.241:2888:3888
35 server.2=192.168.120.242:2888:3888
36 server.3=192.168.120.171:2888:3888

  1 # The number of milliseconds of each tick
  2 tickTime=2000
  3 # The number of ticks that the initial 
  4 # synchronization phase can take
  5 initLimit=10
  6 # The number of ticks that can pass between 
  7 # sending a request and getting an acknowledgement
  8 syncLimit=5
  9 # the directory where the snapshot is stored.
 10 # do not use /tmp for storage, /tmp here is just 
 11 # example sakes.
 12 
 13 dataDir=/home/amqbroker/zkdir/data
 14 dataLogDir=/home/amqbroker/zkdir/log
 15 # the port at which the clients will connect
 16 clientPort=2181
 17 # the maximum number of client connections.
 18 # increase this if you need to handle more clients
 19 #maxClientCnxns=60
 20 #
 21 # Be sure to read the maintenance section of the 
 22 # administrator guide before turning on autopurge.
 23 #
 24 # http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
 25 #
 26 # The number of snapshots to retain in dataDir
 27 #autopurge.snapRetainCount=3
 28 # Purge task interval in hours
 29 # Set to "0" to disable auto purge feature
 30 #autopurge.purgeInterval=1
 31 
 32 
 33 ##three servers of this cluster
 34 server.1=192.168.120.241:2888:3888
 35 server.2=192.168.120.242:2888:3888
 36 server.3=192.168.120.171:2888:3888

dataDir=/home/amqbroker/zkdir/data
dataLogDir=/home/amqbroker/zkdir/log 这两行说明了，需要创建一个文件夹，分别存放data以及log，所以，在/home/amqbroker下创建文件夹zkdir，也可以创建在其他地方，只要配置中指定就好

在zkdir/data下，创建myid文件，并写入与ip地址相称的服务器编号，比如，192.168.120.241,写入 1;echo 1>myid;

其他两台机器配置相同，不过myid文件写入相应的id号

启动服务：

cd /home/amqbroker/zookeeper-3.3.6/bin

./zkServer.sh start

可以使用zookeeper自带的客户端测试：

./zkCli.sh -server 192.168.120.241:2181

其他两台一样配置

3、下载activemq-5.10

wget http://archive.apache.org/dist/activemq/5.10.1/apache-activemq-5.10.1-bin.tar.gz

tar -xvf apache-activemq-5.10.1-bin.tar.gz

vi apache-activemq-5.10.1/conf/activemq.xml

[java] view plain copy

print ?

<pre name=“code” class=“java”> 1 <!–
2 Licensed to the Apache Software Foundation (ASF) under one or more
3 contributor license agreements. See the NOTICE file distributed with
4 this work for additional information regarding copyright ownership.
5 The ASF licenses this file to You under the Apache License, Version 2.0
6 (the “License”); you may not use this file except in compliance with
7 the License. You may obtain a copy of the License at
8
9 http://www.apache.org/licenses/LICENSE-2.0
10
11 Unless required by applicable law or agreed to in writing, software
12 distributed under the License is distributed on an “AS IS” BASIS,
13 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14 See the License for the specific language governing permissions and
15 limitations under the License.
16 –>
17 <!– START SNIPPET: example –>
18 <beans
19 xmlns=“http://www.springframework.org/schema/beans”
20 xmlns:xsi=“http://www.w3.org/2001/XMLSchema-instance”
21 xsi:schemaLocation=”http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
22 http://activemq.apache.org/schema/core http://activemq.apache.org/schema/core/activemq-core.xsd”>
23
24 <!– Allows us to use system properties as variables in this configuration file –>
25 <bean class=“org.springframework.beans.factory.config.PropertyPlaceholderConfigurer”>
26 <property name=“locations”>
<value>file: ${activemq.conf}/credentials.properties</value>  </li><li class=""> 28         </property>  </li><li class="alt"> 29     </bean>  </li><li class=""> 30   </li><li class="alt"> 31      </li><li class=""> 32       </li><li class=""> 38       </li><li class="alt"> 41     <broker xmlns="http://activemq.apache.org/schema/core" brokerName="job-broker" dataDirectory="$ {activemq.data}”>
42
43 <destinationPolicy>
44 <policyMap>
45 <policyEntries>

<pre name="code" class="java">  1 <!--
  2     Licensed to the Apache Software Foundation (ASF) under one or more
  3     contributor license agreements.  See the NOTICE file distributed with
  4     this work for additional information regarding copyright ownership.
  5     The ASF licenses this file to You under the Apache License, Version 2.0
  6     (the "License"); you may not use this file except in compliance with
  7     the License.  You may obtain a copy of the License at
  8 
  9     http://www.apache.org/licenses/LICENSE-2.0
 10 
 11     Unless required by applicable law or agreed to in writing, software
 12     distributed under the License is distributed on an "AS IS" BASIS,
 13     WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 14     See the License for the specific language governing permissions and
 15     limitations under the License.
 16 -->
 17 <!-- START SNIPPET: example -->
 18 <beans
 19   xmlns="http://www.springframework.org/schema/beans"
 20   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 21   xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
 22   http://activemq.apache.org/schema/core http://activemq.apache.org/schema/core/activemq-core.xsd">
 23 
 24     <!-- Allows us to use system properties as variables in this configuration file -->
 25     <bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
 26         <property name="locations">
 27             <value>file:${activemq.conf}/credentials.properties</value>
 28         </property>
 29     </bean>
 30 
 31    <!-- Allows accessing the server log -->
 32     <!--
 33     <bean id="logQuery" class="org.fusesource.insight.log.log4j.Log4jLogQuery"
 34           lazy-init="false" scope="singleton"
 35           init-method="start" destroy-method="stop">
 36     </bean>
 37     -->
 38     <!--
 39         The <broker> element is used to configure the ActiveMQ broker.
 40     -->
 41     <broker xmlns="http://activemq.apache.org/schema/core" brokerName="job-broker" dataDirectory="${activemq.data}">
 42 
 43         <destinationPolicy>
 44             <policyMap>
 45               <policyEntries>

[java] view plain copy

print ?

46 <!– Set the following policy on all queues using the ‘>’ wildcard –>
47 <policyEntry queue=“QUEUE.SH.LTE.IMPORT”>
48 <deadLetterStrategy>
49 <!–Use the prefix ‘DLQ.’ for the destination name, and make the DLQ a queue rather than a topic –>
50 <individualDeadLetterStrategy queuePrefix=“SH.LTE.DLQ.” useQueueForQueueMessages=“true” />
51 </deadLetterStrategy>
52 </policyEntry>
53 <policyEntry queue=“QUEUE.STTTTT”>
54 <deadLetterStrategy>
55 <!–Use the prefix ‘DLQ.’ for the destination name, and make the DLQ a queue rather than a topic –>
56 <individualDeadLetterStrategy queuePrefix=“TTT.LTE.SH.DLQ.” useQueueForQueueMessages=“true” />
57 </deadLetterStrategy>
58 </policyEntry>
59 <policyEntry topic=“>” >
60 <!– The constantPendingMessageLimitStrategy is used to prevent
61 slow topic consumers to block producers and affect other consumers
62 by limiting the number of messages that are retained
63 For more information, see:
64
65 http://activemq.apache.org/slow-consumer-handling.html
66
67 –>
68 <pendingMessageLimitStrategy>
69 <constantPendingMessageLimitStrategy limit=“1000”/>
70 </pendingMessageLimitStrategy>
71 </policyEntry>
72 </policyEntries>
73 </policyMap>
74 </destinationPolicy>
75
76
77 <!–
78 The managementContext is used to configure how ActiveMQ is exposed in
79 JMX. By default, ActiveMQ uses the MBean server that is started by
80 the JVM. For more information, see:
81
82 http://activemq.apache.org/jmx.html
83 –>
84 <managementContext>
85 <managementContext createConnector=“false”/>
86 </managementContext>
87
88 <!–
89 Configure message persistence for the broker. The default persistence

 46              <!-- Set the following policy on all queues using the '>' wildcard -->
 47                 <policyEntry queue="QUEUE.SH.LTE.IMPORT">
 48                     <deadLetterStrategy>
 49                             <!--Use the prefix 'DLQ.' for the destination name, and make the DLQ a queue rather than a topic -->
 50                         <individualDeadLetterStrategy queuePrefix="SH.LTE.DLQ." useQueueForQueueMessages="true" />
 51                     </deadLetterStrategy>
 52                 </policyEntry>
 53                 <policyEntry queue="QUEUE.STTTTT">
 54                     <deadLetterStrategy>
 55                          <!--Use the prefix 'DLQ.' for the destination name, and make the DLQ a queue rather than a topic -->
 56                           <individualDeadLetterStrategy queuePrefix="TTT.LTE.SH.DLQ." useQueueForQueueMessages="true" />
 57                       </deadLetterStrategy>
 58                   </policyEntry>
 59                 <policyEntry topic=">" >
 60                     <!-- The constantPendingMessageLimitStrategy is used to prevent
 61                          slow topic consumers to block producers and affect other consumers
 62                          by limiting the number of messages that are retained
 63                          For more information, see:
 64 
 65                          http://activemq.apache.org/slow-consumer-handling.html
 66 
 67                     -->
 68                   <pendingMessageLimitStrategy>
 69                     <constantPendingMessageLimitStrategy limit="1000"/>
 70                   </pendingMessageLimitStrategy>
 71                 </policyEntry>
 72               </policyEntries>
 73             </policyMap>
 74         </destinationPolicy>
 75 
 76 
 77         <!--
 78             The managementContext is used to configure how ActiveMQ is exposed in
 79             JMX. By default, ActiveMQ uses the MBean server that is started by
 80             the JVM. For more information, see:
 81 
 82             http://activemq.apache.org/jmx.html
 83         -->
 84         <managementContext>
 85             <managementContext createConnector="false"/>
 86         </managementContext>
 87 
 88         <!--
 89             Configure message persistence for the broker. The default persistence

[java] view plain copy

print ?

89 Configure message persistence for the broker. The default persistence
90 mechanism is the KahaDB store (identified by the kahaDB tag).
91 For more information, see:
92
93 http://activemq.apache.org/persistence.html
94 –>
95
96 <!–
97 <persistenceAdapter>
98 <kahaDB directory=“ ${activemq.data}/kahadb"/>  </li><li class="alt"> 99          </persistenceAdapter>  </li><li class="">100      -->  </li><li class="alt">101      <persistenceAdapter>  </li><li class="">102         <replicatedLevelDB   </li><li class="alt">103              directory="$ {activemq.data}/leveldb”
104 replicas=“3”
105 bind=“tcp://0.0.0.0:0”
106 zkAddress=“192.168.120.171:2181,192.168.120.241:2181,192.168.120.242:2181”
107 zkPassword=“”
108 hostname=“192.168.120.241”
109 sync=“local_disk”
110 zkPath=“/activemq/leveldb-stores”/>
111 </persistenceAdapter>
112
113 <!–
114 The systemUsage controls the maximum amount of space the broker will
115 use before disabling caching and/or slowing down producers. For more information, see:
116 http://activemq.apache.org/producer-flow-control.html
117 –>
118 <systemUsage>
119 <systemUsage>
120 <memoryUsage>
121 <memoryUsage percentOfJvmHeap=“70” />
122 </memoryUsage>
123 <storeUsage>
124 <storeUsage limit=“100 gb”/>
125 </storeUsage>
126 <tempUsage>
127 <tempUsage limit=“50 gb”/>
128 </tempUsage>
129 </systemUsage>
130 </systemUsage>
131
132 <!–
133 The transport connectors expose ActiveMQ over a given protocol to

 89             Configure message persistence for the broker. The default persistence
 90             mechanism is the KahaDB store (identified by the kahaDB tag).
 91             For more information, see:
 92 
 93             http://activemq.apache.org/persistence.html
 94         -->
 95 
 96      <!--
 97      <persistenceAdapter>
 98          <kahaDB directory="${activemq.data}/kahadb"/>
 99          </persistenceAdapter>
100      -->
101      <persistenceAdapter>
102         <replicatedLevelDB 
103              directory="${activemq.data}/leveldb"
104              replicas="3"
105              bind="tcp://0.0.0.0:0"
106              zkAddress="192.168.120.171:2181,192.168.120.241:2181,192.168.120.242:2181"
107              zkPassword=""
108              hostname="192.168.120.241"
109              sync="local_disk"
110              zkPath="/activemq/leveldb-stores"/>
111      </persistenceAdapter>
112 
113           <!--
114             The systemUsage controls the maximum amount of space the broker will
115             use before disabling caching and/or slowing down producers. For more information, see:
116             http://activemq.apache.org/producer-flow-control.html
117           -->
118           <systemUsage>
119             <systemUsage>
120                 <memoryUsage>
121                     <memoryUsage percentOfJvmHeap="70" />
122                 </memoryUsage>
123                 <storeUsage>
124                     <storeUsage limit="100 gb"/>
125                 </storeUsage>
126                 <tempUsage>
127                     <tempUsage limit="50 gb"/>
128                 </tempUsage>
129             </systemUsage>
130         </systemUsage>
131 
132         <!--
133             The transport connectors expose ActiveMQ over a given protocol to

[java] view plain copy

print ?

134 clients and other brokers. For more information, see:
135
136 http://activemq.apache.org/configuring-transports.html
137 –>
138 <transportConnectors>
139 <!– DOS protection, limit concurrent connections to 1000 and frame size to 100MB –>
140 <transportConnector name=“openwire” uri=“tcp://0.0.0.0:61616?maximumConnections=1000&wireFormat.maxFrameSize=104 857600”/>
141 <transportConnector name=“amqp” uri=“amqp://0.0.0.0:5672?maximumConnections=1000&wireFormat.maxFrameSize=1048576 00”/>
142 <transportConnector name=“stomp” uri=“stomp://0.0.0.0:61613?maximumConnections=1000&wireFormat.maxFrameSize=1048 57600”/>
143 <transportConnector name=“mqtt” uri=“mqtt://0.0.0.0:1883?maximumConnections=1000&wireFormat.maxFrameSize=1048576 00”/>
144 <transportConnector name=“ws” uri=“ws://0.0.0.0:61614?maximumConnections=1000&wireFormat.maxFrameSize=104857600” />
145 </transportConnectors>
146
147 <!– destroy the spring context on shutdown to stop jetty –>
148 <shutdownHooks>
149 <bean xmlns=“http://www.springframework.org/schema/beans” class=“org.apache.activemq.hooks.SpringContextHook” />
150 </shutdownHooks>
151
152 </broker>
153
154 <!–
155 Enable web consoles, REST and Ajax APIs and demos
156 The web consoles requires by default login, you can disable this in the jetty.xml file
157
158 Take a look at ${ACTIVEMQ_HOME}/conf/jetty.xml for more details
159 –>
160 <import resource=“jetty.xml”/>
161
162 </beans>

134             clients and other brokers. For more information, see:
135 
136             http://activemq.apache.org/configuring-transports.html
137         -->
138         <transportConnectors>
139             <!-- DOS protection, limit concurrent connections to 1000 and frame size to 100MB -->
140             <transportConnector name="openwire" uri="tcp://0.0.0.0:61616?maximumConnections=1000&amp;wireFormat.maxFrameSize=104    857600"/>
141             <transportConnector name="amqp" uri="amqp://0.0.0.0:5672?maximumConnections=1000&amp;wireFormat.maxFrameSize=1048576    00"/>
142             <transportConnector name="stomp" uri="stomp://0.0.0.0:61613?maximumConnections=1000&amp;wireFormat.maxFrameSize=1048    57600"/>
143             <transportConnector name="mqtt" uri="mqtt://0.0.0.0:1883?maximumConnections=1000&amp;wireFormat.maxFrameSize=1048576    00"/>
144             <transportConnector name="ws" uri="ws://0.0.0.0:61614?maximumConnections=1000&amp;wireFormat.maxFrameSize=104857600"    />
145         </transportConnectors>
146 
147         <!-- destroy the spring context on shutdown to stop jetty -->
148         <shutdownHooks>
149             <bean xmlns="http://www.springframework.org/schema/beans" class="org.apache.activemq.hooks.SpringContextHook" />
150         </shutdownHooks>
151 
152     </broker>
153 
154     <!--
155         Enable web consoles, REST and Ajax APIs and demos
156         The web consoles requires by default login, you can disable this in the jetty.xml file
157 
158         Take a look at ${ACTIVEMQ_HOME}/conf/jetty.xml for more details
159     -->
160     <import resource="jetty.xml"/>
161 
162 </beans>

以上是192.168.120.241的activemq的activemq.xml配置文件配置。因为使用的是zookeeper，所以注意persistenceAdapter这个节点配置，如果有zkpassword，则填写，否则不要填写。按照上边的配置，zkpassword是不需要写的。

还有broker配置中，有一些policyEntity，是定制的，比如DLQ的配置，是因为针对不同的队列，重发机制的最大次数重发完以后，如果还有异常，则进入该队列定制的DLQ之中，详细的一些参数设置，请参考activemq官方文档。

另外，配置中应该有安全配置，目前没有写入，后续在说

另外两台服务器的配置与该文档相似，唯一不同就是 hostname=”192.168.120.241”这里，需要写入各自的ip地址

启动activemq服务器

cd /home/amqbroker/activemq***/bin

./activemq start

三台服务器启动类似

然后，在浏览器地址栏里输入：

http://192.168.120.241:8161/admin/queues.jsp

http://192.168.120.242:8161/admin/queues.jsp

http://192.168.120.171:8161/admin/queues.jsp

因为使用zookeeper做负载均衡，三台只有一台是master，其他两台处于等待状态，所以只有其中一台提供服务，但一旦这台服务器宕机以后，会有另外一台顶替上来，所以其他几个ip地址是打不开的，只有一台能打开

在客户端使用的时候，使用 failover进行配置的，如下：

##使用replicaDB方式来搭建activemq，使用zookeeper集群进行负载均衡以及数据同步
mq.broker.url=failover:(tcp://192.168.120.241:61616,tcp://192.168.120.242:61616,tcp://192.168.120.171:61616)?initialReconnectDelay=1000