docker安装RocketMQ5.1.4【提供官方镜像】【超详细版】
前言:由于目前docker的种种原因,导致下载镜像成了头疼的事情,所以在此篇博客会提供基于apache的RocketMQ的镜像附件,保证是官方的,原汁原味,当然不信任不下载就好。
获取到附件镜像之后 直接 执行docker load
去加载镜像就好。本篇博客不会提供docker的使用,如需docker 的使用自行百度
拉取镜像
[root@anhao ~]# docker pull apache/rocketmq:5.1.4
启动 NameServer
在这里特别说明一下:
官方的文档提供在docker里面部署的情况下提供的命令为
docker run -d --name rmqnamesrv -p 9876:9876 --network rocketmq apache/rocketmq:5.2.0 sh mqnamesrv
但是我们可以看到,只提供了端口的映射,一般生产环境下肯定要配置挂载的目录之类的,通过看其他人的博客发现有的设置很多东西,还有环境变量之类的,那么这些东西总得有一个获取的途径,但是我没看到,那么我就决定先运行起来容器,运行起来容器之后就可以进到容器里面看一些配置文件。但是也要有自己的思想,类似于官方提供的命令很明显的最后是通过脚本启动的,那么肯定要看一下这个脚本文件了。
首先就根据官方命令启动NameServer,但是我没有用到--network
这个参数
[root@anhao ~]# docker run -d --name rmqnamesrv -p 9876:9876 apache/rocketmq:5.1.4 sh mqnamesrv
3308b27b4fd0ac709bdc1c8ba66a13264a81f5fe699836081eed2505304b59db
[root@anhao ~]#
看一下日志
[root@anhao ~]# docker logs -f rmqnamesrv
OpenJDK 64-Bit Server VM warning: Using the DefNew young collector with the CMS collector is deprecated and will likely be removed in a future release
OpenJDK 64-Bit Server VM warning: UseCMSCompactAtFullCollection is deprecated and will likely be removed in a future release.
The Name Server boot success. serializeType=JSON, address 0.0.0.0:9876
可以看到NameServer已经启动成功了
刚刚也说到了,看其他博客还有设置环境变量之类的 类似于 -e "MAX_POSSIBLE_HEAP=100000000"
看到HEAP肯定堆内存了,那咱们去容器看看脚本,看看能不能找找蛛丝马迹
[root@anhao ~]# docker exec -it rmqnamesrv bash
[rocketmq@3308b27b4fd0 bin]$ ls
cachedog.sh dledger mqadmin.cmd mqbroker.numanode0 mqcontroller mqproxy os.sh runbroker.cmd setcache.sh
cleancache.sh docker-entrypoint.sh mqbroker mqbroker.numanode1 mqcontroller.cmd mqproxy.cmd play.cmd runbroker.sh startfsrv.sh
cleancache.v1.sh export.sh mqbroker.cmd mqbroker.numanode2 mqnamesrv mqshutdown play.sh runserver.cmd tools.cmd
controller mqadmin mqbrokercontainer mqbroker.numanode3 mqnamesrv.cmd mqshutdown.cmd README.md runserver.sh tools.sh
[rocketmq@3308b27b4fd0 bin]$ cat mqnamesrv
#!/bin/sh
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
if [ -z "$ROCKETMQ_HOME" ] ; then
## resolve links - $0 may be a link to maven's home
PRG="$0"
# need this for relative symlinks
while [ -h "$PRG" ] ; do
ls=`ls -ld "$PRG"`
link=`expr "$ls" : '.*-> \(.*\)$'`
if expr "$link" : '/.*' > /dev/null; then
PRG="$link"
else
PRG="`dirname "$PRG"`/$link"
fi
done
saveddir=`pwd`
ROCKETMQ_HOME=`dirname "$PRG"`/..
# make it fully qualified
ROCKETMQ_HOME=`cd "$ROCKETMQ_HOME" && pwd`
cd "$saveddir"
fi
export ROCKETMQ_HOME
sh ${ROCKETMQ_HOME}/bin/runserver.sh -Drmq.logback.configurationFile=$ROCKETMQ_HOME/conf/rmq.namesrv.logback.xml org.apache.rocketmq.namesrv.NamesrvStartup $@
[rocketmq@3308b27b4fd0 bin]$
看最后的命令,实际是运行runserver.sh
脚本
那咱们在看看呗
[rocketmq@3308b27b4fd0 bin]$ cat runserver.sh
#!/bin/bash
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#===========================================================================================
# Java Environment Setting
#===========================================================================================
error_exit ()
{
echo "ERROR: $1 !!"
exit 1
}
find_java_home()
{
case "`uname`" in
Darwin)
JAVA_HOME=$(/usr/libexec/java_home)
;;
*)
JAVA_HOME=$(dirname $(dirname $(readlink -f $(which javac))))
;;
esac
}
find_java_home
[ ! -e "$JAVA_HOME/bin/java" ] && JAVA_HOME=$HOME/jdk/java
[ ! -e "$JAVA_HOME/bin/java" ] && JAVA_HOME=/usr/java
[ ! -e "$JAVA_HOME/bin/java" ] && error_exit "Please set the JAVA_HOME variable in your environment, We need java(x64)!"
export JAVA_HOME
export JAVA="$JAVA_HOME/bin/java"
export BASE_DIR=$(dirname $0)/..
export CLASSPATH=.:${BASE_DIR}/conf:${CLASSPATH}
#===========================================================================================
# JVM Configuration
#===========================================================================================
calculate_heap_sizes()
{
case "`uname`" in
Linux)
system_memory_in_mb=`free -m| sed -n '2p' | awk '{print $2}'`
system_cpu_cores=`egrep -c 'processor([[:space:]]+):.*' /proc/cpuinfo`
;;
FreeBSD)
system_memory_in_bytes=`sysctl hw.physmem | awk '{print $2}'`
system_memory_in_mb=`expr $system_memory_in_bytes / 1024 / 1024`
system_cpu_cores=`sysctl hw.ncpu | awk '{print $2}'`
;;
SunOS)
system_memory_in_mb=`prtconf | awk '/Memory size:/ {print $3}'`
system_cpu_cores=`psrinfo | wc -l`
;;
Darwin)
system_memory_in_bytes=`sysctl hw.memsize | awk '{print $2}'`
system_memory_in_mb=`expr $system_memory_in_bytes / 1024 / 1024`
system_cpu_cores=`sysctl hw.ncpu | awk '{print $2}'`
;;
*)
# assume reasonable defaults for e.g. a modern desktop or
# cheap server
system_memory_in_mb="2048"
system_cpu_cores="2"
;;
esac
# some systems like the raspberry pi don't report cores, use at least 1
if [ "$system_cpu_cores" -lt "1" ]
then
system_cpu_cores="1"
fi
# set max heap size based on the following
# max(min(1/2 ram, 1024MB), min(1/4 ram, 8GB))
# calculate 1/2 ram and cap to 1024MB
# calculate 1/4 ram and cap to 8192MB
# pick the max
half_system_memory_in_mb=`expr $system_memory_in_mb / 2`
quarter_system_memory_in_mb=`expr $half_system_memory_in_mb / 2`
if [ "$half_system_memory_in_mb" -gt "1024" ]
then
half_system_memory_in_mb="1024"
fi
if [ "$quarter_system_memory_in_mb" -gt "8192" ]
then
quarter_system_memory_in_mb="8192"
fi
if [ "$half_system_memory_in_mb" -gt "$quarter_system_memory_in_mb" ]
then
max_heap_size_in_mb="$half_system_memory_in_mb"
else
max_heap_size_in_mb="$quarter_system_memory_in_mb"
fi
MAX_HEAP_SIZE="${max_heap_size_in_mb}M"
# Young gen: min(max_sensible_per_modern_cpu_core * num_cores, 1/4 * heap size)
max_sensible_yg_per_core_in_mb="100"
max_sensible_yg_in_mb=`expr $max_sensible_yg_per_core_in_mb "*" $system_cpu_cores`
desired_yg_in_mb=`expr $max_heap_size_in_mb / 4`
if [ "$desired_yg_in_mb" -gt "$max_sensible_yg_in_mb" ]
then
HEAP_NEWSIZE="${max_sensible_yg_in_mb}M"
else
HEAP_NEWSIZE="${desired_yg_in_mb}M"
fi
}
calculate_heap_sizes
# Dynamically calculate parameters, for reference.
Xms=$MAX_HEAP_SIZE
Xmx=$MAX_HEAP_SIZE
Xmn=$HEAP_NEWSIZE
# Set for `JAVA_OPT`.
JAVA_OPT="${JAVA_OPT} -server -Xms${Xms} -Xmx${Xmx} -Xmn${Xmn}"
JAVA_OPT="${JAVA_OPT} -XX:+UseConcMarkSweepGC -XX:+UseCMSCompactAtFullCollection -XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSParallelRemarkEnabled -XX:SoftRefLRUPolicyMSPerMB=0 -XX:+CMSClassUnloadingEnabled -XX:SurvivorRatio=8 -XX:-UseParNewGC"
JAVA_OPT="${JAVA_OPT} -verbose:gc -Xloggc:/dev/shm/rmq_srv_gc.log -XX:+PrintGCDetails"
JAVA_OPT="${JAVA_OPT} -XX:-OmitStackTraceInFastThrow"
JAVA_OPT="${JAVA_OPT} -XX:-UseLargePages"
JAVA_OPT="${JAVA_OPT} -Djava.ext.dirs=${JAVA_HOME}/jre/lib/ext:${BASE_DIR}/lib"
#JAVA_OPT="${JAVA_OPT} -Xdebug -Xrunjdwp:transport=dt_socket,address=9555,server=y,suspend=n"
JAVA_OPT="${JAVA_OPT} ${JAVA_OPT_EXT}"
JAVA_OPT="${JAVA_OPT} -cp ${CLASSPATH}"
$JAVA ${JAVA_OPT} $@
[rocketmq@3308b27b4fd0 bin]$
通过这个脚本我们可以看出来整体的逻辑,第一步找JAVA_HOME
,第二步 计算内存 ,第三步启动,其实也没咱需要设置啥的,无非在rmqnamesrv
脚本中看到了一个logback,搞java的都懂,这个是日志,如果想挂载的那就看一下位置挂载出来就是了。
启动broker
官方的文档是这样子的
# 配置 Broker 的IP地址 echo "brokerIP1=127.0.0.1" > broker.conf #启动 Broker 和 Proxy docker run -d \ --name rmqbroker \ --network rocketmq \ -p 10912:10912 -p 10911:10911 -p 10909:10909 \ -p 8080:8080 -p 8081:8081 \ -e "NAMESRV_ADDR=rmqnamesrv:9876" \ -v ./broker.conf:/home/rocketmq/rocketmq-5.2.0/conf/broker.conf \ apache/rocketmq:5.2.0 sh mqbroker --enable-proxy \ -c /home/rocketmq/rocketmq-5.2.0/conf/broker.conf # 验证 Broker 是否启动成功 docker exec -it rmqbroker bash -c "tail -n 10 /home/rocketmq/logs/rocketmqlogs/proxy.log"
里面有挂载目录之类的,那么我就需要准备一个broker.conf
,但是宝宝没有呀。不着急,先把容器启动起来,之后去容器里面看看默认的。开整!!!
修改成自己的命令去启动broker
[root@anhao ~]# docker run -d --name rmqbroker -p 10912:10912 -p 10911:10911 -p 10909:10909 -p 8080:8080 -p 8081:8081 -e "NAMESRV_ADDR=192.168.31.13:9876" apache/rocketmq:5.1.4 sh mqbroker --enable-proxy -c /home/rocketmq/rocketmq-5.1.4/conf/broker.conf
d81d50122fda2dd66814232d0774205a89eb340aadcfb4b4f23e26ca6884f866
[root@anhao ~]# docker exec -it rmqbroker bash -c "tail -n 10 /home/rocketmq/logs/rocketmqlogs/proxy.log"
}
2024-06-30 02:07:59 INFO main - ServiceProvider loaded no AccessValidator, using default org.apache.rocketmq.acl.plain.PlainAccessValidator
2024-06-30 02:08:01 INFO main - grpc server has built. port: 8081, tlsKeyPath: 1, tlsCertPath: 16, threadPool: 136314880, queueCapacity: {}, boosLoop: {}, workerLoop: {}, maxInboundMessageSize: {}
2024-06-30 02:08:01 INFO main - Server is running in TLS permissive mode
2024-06-30 02:08:01 INFO main - Using OpenSSL provider
2024-06-30 02:08:01 INFO main - SSLContext created for server
2024-06-30 02:08:02 INFO main - The broker[broker-a, 172.17.0.12:10911] boot success. serializeType=JSON and name server is 192.168.31.13:9876
2024-06-30 02:08:02 INFO main - user specified name server address: 192.168.31.13:9876
2024-06-30 02:08:02 INFO main - grpc server start successfully.
2024-06-30 02:08:02 INFO main - Sat Jun 29 18:08:02 UTC 2024 rocketmq-proxy startup successfully
[root@anhao ~]#
可以看到我们的broker也启动成功了,在此可能很多人把
-e "NAMESRV_ADDR=192.168.31.13:9876"
ip地址设置为127.0.0.1
这里注意一下可能是容器的内部地址,有可能会链接不到。
那么我们现在进入容器看一看
[root@anhao ~]# docker exec -it rmqbroker bash
[rocketmq@d81d50122fda bin]$ ls
cachedog.sh dledger mqadmin.cmd mqbroker.numanode0 mqcontroller mqproxy os.sh runbroker.cmd setcache.sh
cleancache.sh docker-entrypoint.sh mqbroker mqbroker.numanode1 mqcontroller.cmd mqproxy.cmd play.cmd runbroker.sh startfsrv.sh
cleancache.v1.sh export.sh mqbroker.cmd mqbroker.numanode2 mqnamesrv mqshutdown play.sh runserver.cmd tools.cmd
controller mqadmin mqbrokercontainer mqbroker.numanode3 mqnamesrv.cmd mqshutdown.cmd README.md runserver.sh tools.sh
[rocketmq@d81d50122fda bin]$
我们看看mqbroker
脚本
[rocketmq@d81d50122fda bin]$ cat mqbroker
#!/bin/sh
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
if [ -z "$ROCKETMQ_HOME" ] ; then
## resolve links - $0 may be a link to maven's home
PRG="$0"
# need this for relative symlinks
while [ -h "$PRG" ] ; do
ls=`ls -ld "$PRG"`
link=`expr "$ls" : '.*-> \(.*\)$'`
if expr "$link" : '/.*' > /dev/null; then
PRG="$link"
else
PRG="`dirname "$PRG"`/$link"
fi
done
saveddir=`pwd`
ROCKETMQ_HOME=`dirname "$PRG"`/..
# make it fully qualified
ROCKETMQ_HOME=`cd "$ROCKETMQ_HOME" && pwd`
cd "$saveddir"
fi
export ROCKETMQ_HOME
other_args=" "
enable_proxy=false
while [ $# -gt 0 ]; do
case $1 in
--enable-proxy)
enable_proxy=true
shift
;;
-c|--configFile)
broker_config="$2"
shift
shift
;;
*)
other_args=${other_args}" "${1}
shift
;;
esac
done
if [ "$enable_proxy" = true ]; then
args_for_proxy=$other_args" -pm local"
if [ "$broker_config" != "" ]; then
args_for_proxy=${args_for_proxy}" -bc "${broker_config}
fi
sh ${ROCKETMQ_HOME}/bin/runbroker.sh -Drmq.logback.configurationFile=$ROCKETMQ_HOME/conf/rmq.proxy.logback.xml org.apache.rocketmq.proxy.ProxyStartup ${args_for_proxy}
else
args_for_broker=$other_args
if [ "$broker_config" != "" ]; then
args_for_broker=${args_for_broker}" -c "${broker_config}
fi
sh ${ROCKETMQ_HOME}/bin/runbroker.sh -Drmq.logback.configurationFile=$ROCKETMQ_HOME/conf/rmq.broker.logback.xml org.apache.rocketmq.broker.BrokerStartup ${args_for_broker}
fi
[rocketmq@d81d50122fda bin]$
通过脚本可以看出实际运行的是runbroker.sh
脚本
[rocketmq@d81d50122fda bin]$ cat runbroker.sh
#!/bin/bash
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#===========================================================================================
# Java Environment Setting
#===========================================================================================
error_exit ()
{
echo "ERROR: $1 !!"
exit 1
}
find_java_home()
{
case "`uname`" in
Darwin)
JAVA_HOME=$(/usr/libexec/java_home)
;;
*)
JAVA_HOME=$(dirname $(dirname $(readlink -f $(which javac))))
;;
esac
}
find_java_home
[ ! -e "$JAVA_HOME/bin/java" ] && JAVA_HOME=$HOME/jdk/java
[ ! -e "$JAVA_HOME/bin/java" ] && JAVA_HOME=/usr/java
[ ! -e "$JAVA_HOME/bin/java" ] && error_exit "Please set the JAVA_HOME variable in your environment, We need java(x64)!"
export JAVA_HOME
export JAVA="$JAVA_HOME/bin/java"
export BASE_DIR=$(dirname $0)/..
export CLASSPATH=.:${BASE_DIR}/conf:${CLASSPATH}
#===========================================================================================
# JVM Configuration
#===========================================================================================
calculate_heap_sizes()
{
case "`uname`" in
Linux)
system_memory_in_mb=`free -m| sed -n '2p' | awk '{print $2}'`
system_cpu_cores=`egrep -c 'processor([[:space:]]+):.*' /proc/cpuinfo`
;;
FreeBSD)
system_memory_in_bytes=`sysctl hw.physmem | awk '{print $2}'`
system_memory_in_mb=`expr $system_memory_in_bytes / 1024 / 1024`
system_cpu_cores=`sysctl hw.ncpu | awk '{print $2}'`
;;
SunOS)
system_memory_in_mb=`prtconf | awk '/Memory size:/ {print $3}'`
system_cpu_cores=`psrinfo | wc -l`
;;
Darwin)
system_memory_in_bytes=`sysctl hw.memsize | awk '{print $2}'`
system_memory_in_mb=`expr $system_memory_in_bytes / 1024 / 1024`
system_cpu_cores=`sysctl hw.ncpu | awk '{print $2}'`
;;
*)
# assume reasonable defaults for e.g. a modern desktop or
# cheap server
system_memory_in_mb="2048"
system_cpu_cores="2"
;;
esac
# some systems like the raspberry pi don't report cores, use at least 1
if [ "$system_cpu_cores" -lt "1" ]
then
system_cpu_cores="1"
fi
# set max heap size based on the following
# max(min(1/2 ram, 1024MB), min(1/4 ram, 8GB))
# calculate 1/2 ram and cap to 1024MB
# calculate 1/4 ram and cap to 8192MB
# pick the max
half_system_memory_in_mb=`expr $system_memory_in_mb / 2`
quarter_system_memory_in_mb=`expr $half_system_memory_in_mb / 2`
if [ "$half_system_memory_in_mb" -gt "1024" ]
then
half_system_memory_in_mb="1024"
fi
if [ "$quarter_system_memory_in_mb" -gt "8192" ]
then
quarter_system_memory_in_mb="8192"
fi
if [ "$half_system_memory_in_mb" -gt "$quarter_system_memory_in_mb" ]
then
max_heap_size_in_mb="$half_system_memory_in_mb"
else
max_heap_size_in_mb="$quarter_system_memory_in_mb"
fi
MAX_HEAP_SIZE="${max_heap_size_in_mb}M"
# Young gen: min(max_sensible_per_modern_cpu_core * num_cores, 1/4 * heap size)
max_sensible_yg_per_core_in_mb="100"
max_sensible_yg_in_mb=`expr $max_sensible_yg_per_core_in_mb "*" $system_cpu_cores`
desired_yg_in_mb=`expr $max_heap_size_in_mb / 4`
if [ "$desired_yg_in_mb" -gt "$max_sensible_yg_in_mb" ]
then
HEAP_NEWSIZE="${max_sensible_yg_in_mb}M"
else
HEAP_NEWSIZE="${desired_yg_in_mb}M"
fi
}
calculate_heap_sizes
# Dynamically calculate parameters, for reference.
Xms=$MAX_HEAP_SIZE
Xmx=$MAX_HEAP_SIZE
Xmn=$HEAP_NEWSIZE
MaxDirectMemorySize=$MAX_HEAP_SIZE
# Set for `JAVA_OPT`.
JAVA_OPT="${JAVA_OPT} -server -Xms${Xms} -Xmx${Xmx} -Xmn${Xmn}"
JAVA_OPT="${JAVA_OPT} -XX:+UseG1GC -XX:G1HeapRegionSize=16m -XX:G1ReservePercent=25 -XX:InitiatingHeapOccupancyPercent=30 -XX:SoftRefLRUPolicyMSPerMB=0 -XX:SurvivorRatio=8"
JAVA_OPT="${JAVA_OPT} -verbose:gc -Xloggc:/dev/shm/mq_gc_%p.log -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCApplicationStoppedTime -XX:+PrintAdaptiveSizePolicy"
JAVA_OPT="${JAVA_OPT} -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=30m"
JAVA_OPT="${JAVA_OPT} -XX:-OmitStackTraceInFastThrow"
JAVA_OPT="${JAVA_OPT} -XX:+AlwaysPreTouch"
JAVA_OPT="${JAVA_OPT} -XX:MaxDirectMemorySize=${MaxDirectMemorySize}"
JAVA_OPT="${JAVA_OPT} -XX:-UseLargePages -XX:-UseBiasedLocking"
JAVA_OPT="${JAVA_OPT} -Djava.ext.dirs=${JAVA_HOME}/jre/lib/ext:${BASE_DIR}/lib"
#JAVA_OPT="${JAVA_OPT} -Xdebug -Xrunjdwp:transport=dt_socket,address=9555,server=y,suspend=n"
JAVA_OPT="${JAVA_OPT} ${JAVA_OPT_EXT}"
JAVA_OPT="${JAVA_OPT} -cp ${CLASSPATH}"
numactl --interleave=all pwd > /dev/null 2>&1
if [ $? -eq 0 ]
then
if [ -z "$RMQ_NUMA_NODE" ] ; then
numactl --interleave=all $JAVA ${JAVA_OPT} $@
else
numactl --cpunodebind=$RMQ_NUMA_NODE --membind=$RMQ_NUMA_NODE $JAVA ${JAVA_OPT} $@
fi
else
$JAVA ${JAVA_OPT} $@
fi
[rocketmq@d81d50122fda bin]$
其实我们看了这么多的脚本发现了,这玩意用的是G1的垃圾回收器,可以看看人家怎么配置的,学习学习。
通过脚本文件我们看看broker.conf
[rocketmq@d81d50122fda bin]$ cd ../conf/
[rocketmq@d81d50122fda conf]$ ls
2m-2s-async 2m-noslave container dledger rmq.broker.logback.xml rmq.controller.logback.xml rmq-proxy.json rmq.tools.logback.xml
2m-2s-sync broker.conf controller plain_acl.yml rmq.client.logback.xml rmq.namesrv.logback.xml rmq.proxy.logback.xml tools.yml
[rocketmq@d81d50122fda conf]$ cat broker.conf
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
brokerClusterName = DefaultCluster
brokerName = broker-a
brokerId = 0
deleteWhen = 04
fileReservedTime = 48
brokerRole = ASYNC_MASTER
flushDiskType = ASYNC_FLUSH
[rocketmq@d81d50122fda conf]$
看完是不是蒙了,我也蒙了,为啥啊,咋都不给个解释啊,我也翻了好多官方文档,最终翻到了个东西就是在官方的4.x版本里面有个参数说明,可以看看就知道啥意思了
下方是从官方文档复制过来的
客户端配置
客户端配置
相对于RocketMQ的Broker集群,生产者和消费者都是客户端。本小节主要描述生产者和消费者公共的行为配置。
客户端寻址方式
RocketMQ可以令客户端找到Name Server, 然后通过Name Server再找到Broker。如下所示有多种配置方式,优先级由高到低,高优先级会覆盖低优先级。
代码中指定Name Server地址,多个namesrv地址之间用分号分割
producer.setNamesrvAddr("192.168.0.1:9876;192.168.0.2:9876");
consumer.setNamesrvAddr("192.168.0.1:9876;192.168.0.2:9876");
Java启动参数中指定Name Server地址
-Drocketmq.namesrv.addr=192.168.0.1:9876;192.168.0.2:9876
环境变量指定Name Server地址
export NAMESRV_ADDR=192.168.0.1:9876;192.168.0.2:9876
HTTP静态服务器寻址(默认)
客户端启动后,会定时访问一个静态HTTP服务器,地址如下:http://jmenv.tbsite.net:8080/rocketmq/nsaddr,这个URL的返回内容如下:
192.168.0.1:9876;192.168.0.2:9876
客户端默认每隔2分钟访问一次这个HTTP服务器,并更新本地的Name Server地址。URL已经在代码中硬编码,可通过修改/etc/hosts文件来改变要访问的服务器,例如在/etc/hosts增加如下配置:
10.232.22.67 jmenv.taobao.net
推荐使用HTTP静态服务器寻址方式,好处是客户端部署简单,且Name Server集群可以热升级。
客户端配置
DefaultMQProducer、TransactionMQProducer、DefaultMQPushConsumer、DefaultMQPullConsumer都继承于ClientConfig类,ClientConfig为客户端的公共配置类。客户端的配置都是get、set形式,每个参数都可以用spring来配置,也可以在代码中配置,例如namesrvAddr这个参数可以这样配置,producer.setNamesrvAddr("192.168.0.1:9876"),其他参数同理。
ClientConfig配置
名称 描述 参数类型 默认值 有效值 重要性
namesrvAddr NameServer的地址列表 String 从-D系统参数rocketmq.namesrv.addr或环境变量。NAMESRV_ADDR
instanceName 客户端实例名称 String 从-D系统参数rocketmq.client.name获取,否则就是DEFAULT
clientIP 客户端IP String RemotingUtil.getLocalAddress()
namespace 客户端命名空间 String
accessChannel 设置访问通道 AccessChannel LOCAL
clientCallbackExecutorThreads 客户端通信层接收到网络请求的时候,处理器的核数 int Runtime.getRuntime().availableProcessors()
pollNameServerInterval 轮询从NameServer获取路由信息的时间间隔 int 30000,单位毫秒
heartbeatBrokerInterval 定期发送注册心跳到broker的间隔 int 30000,单位毫秒
persistConsumerOffsetInterval 作用于Consumer,持久化消费进度的间隔 int 默认值5000,单位毫秒
pullTimeDelayMillsWhenException 拉取消息出现异常的延迟时间设置 long 1000,单位毫秒
unitName 单位名称 String
unitMode 单位模式 boolean false
vipChannelEnabled 是否启用vip netty通道以发送消息 boolean 从-D com.rocketmq.sendMessageWithVIPChannel参数的值,若无则是true
useTLS 是否使用安全传输。 boolean 从-D系统参数tls.enable获取,否则就是false
mqClientApiTimeout mq客户端api超时设置 int 3000,单位毫秒
language 客户端实现语言 LanguageCode LanguageCode.JAVA
DefaultMQProducer配置
名称 描述 参数类型 默认值 有效值 重要性
producerGroup 生产组的名称,一类Producer的标识 String DEFAULT_PRODUCER
createTopicKey 发送消息的时候,如果没有找到topic,若想自动创建该topic,需要一个key topic,这个值即是key topic的值 String TopicValidator.AUTO_CREATE_TOPIC_KEY_TOPIC
defaultTopicQueueNums 自动创建topic的话,默认queue数量是多少 int 4
sendMsgTimeout 默认的发送超时时间 int 3000,单位毫秒
compressMsgBodyOverHowmuc 消息body需要压缩的阈值 int 1024 * 4,4K
retryTimesWhenSendFailed 同步发送失败的话,rocketmq内部重试多少次 int 2
retryTimesWhenSendAsyncFailed 异步发送失败的话,rocketmq内部重试多少次 int 2
retryAnotherBrokerWhenNotStoreOK 发送的结果如果不是SEND_OK状态,是否当作失败处理而尝试重发 boolean false
maxMessageSize 客户端验证,允许发送的最大消息体大小 int 1024 1024 4,4M
traceDispatcher 异步传输数据接口 TraceDispatcher null
DefaultMQPushConsumer配置
名称 描述 参数类型 默认值 有效值 重要性
consumerGroup 消费组的名称,用于标识一类消费者 String
messageModel 消费模式 MessageModel MessageModel.CLUSTERINGallocateMessageQueueStrategy CLUSTERING(集群消費模式) / ROADCASTING (广播消费模式)
consumeFromWhere 启动消费点策略 ConsumeFromWhere ConsumeFromWhere.CONSUME_FROM_LAST_OFFSET
consumeTimestamp CONSUME_FROM_LAST_OFFSET的时候使用,从哪个时间点开始消费 String 半小时前
allocateMessageQueueStrategy 负载均衡策略算法 AllocateMessageQueueStrategy AllocateMessageQueueAveragely(取模平均分配)
subscription 订阅关系 Map<String, String> {}
messageListener 消息处理监听器(回调) MessageListener null
offsetStore 消息消费进度存储器 OffsetStore null 不建议设置,offsetStore 有两个策略:LocalFileOffsetStore 和RemoteBrokerOffsetStore.若沒有显示设置的情況下,广播模式將使用LocalFileOffsetStore,集群模式將使用RemoteBrokerOffsetStore,不建议修改.
consumeThreadMin 消费线程池的core size int 20
consumeThreadMax 消费线程池的max size int 64
adjustThreadPoolNumsThreshold 动态扩线程核数的消费堆积阈值 long 100000
consumeConcurrentlyMaxSpan 并发消费下,单条consume queue队列允许的最大offset跨度,达到则触发流控 int 2000pullInterval
pullThresholdForQueue consume queue流控的阈值 int 100
pullInterval 拉取的间隔 long 0,单位毫秒
pullThresholdForTopic 主题级别的流控制阈值 int -1
pullThresholdSizeForTopic 限制主题级别的缓存消息大小 int -1
pullBatchSize 一次最大拉取的批量大小 int 32
consumeMessageBatchMaxSize 批量消费的最大消息条数 int 1
postSubscriptionWhenPull 每次拉取的时候是否更新订阅关系 boolean false
unitMode 订阅组的单位 boolean false
maxReconsumeTimes 一个消息如果消费失败的话,最多重新消费多少次才投递到死信队列 int -1 由于PullConsumer没有管理消费的线程池和管理器,需要用户自己处理各种消费结果和拉取结果,故需要投递到重试队列或死信队列的时候需要显示调用sendMessageBack.回传消息的时候会带上maxReconsumeTimes的值,broker发现此消息已经消费超过此值,则投递到死信队列,否则投递到重试队列。此逻辑和DefaultPushConsumer是一致的,只是PushConsumer无需用户显示调用.
suspendCurrentQueueTimeMillis 串行消费使用,如果返回ROLLBACK或者SUSPEND_CURRENT_QUEUE_A_MOMENT,再次消费的时间间隔 long 1000
consumeTimeout 消费的最长超时时间 long 15,单位分钟
awaitTerminationMillisWhenShutdown 关闭使用者时等待消息的最长时间,0表示无等待。 long 0
traceDispatcher 异步传输数据接口 TraceDispatcher null
registerTopics 消費者需要監聽的topic Collection 默認值:空集合
DefaultLitePullConsumer配置
名称 描述 参数类型 默认值 有效值 重要性
consumerGroup 消费组的名称,用于标识一类消费者 String
brokerSuspendMaxTimeMillis broker在长轮询下,连接最长挂起的时间 long 20000,单位毫秒
consumerTimeoutMillisWhenSuspend broker在长轮询下,客户端等待broker响应的最长等待超时时间 long 30000,单位毫秒
consumerPullTimeoutMillis pull的socket 超时时间 long 10000,单位毫秒
messageModel 消费模式 MessageModel MessageModel.CLUSTERING
messageQueueListener 负载均衡consume queue分配变化的通知监听器 MessageQueueListener
offsetStore 消息消费进度存储器 OffsetStore
allocateMessageQueueStrategy 负载均衡策略算法 AllocateMessageQueueStrategy AllocateMessageQueueAveragely(取模平均分配)
unitMode 订阅组的单位设置 boolean false
autoCommit 自动提交偏移的标志设置 boolean true
pullThreadNums 拉取线程数设置 int 20
MIN_AUTOCOMMIT_INTERVAL_MILLIS 最小提交偏移间隔时间 long 1000,单位为毫秒
autoCommitIntervalMillis 最大提交偏移间隔时间 long 5000,单位为毫秒
pullBatchSize 每次拉出的信息的最大数量 long 10
pullThresholdForAll 消耗请求的流量控制阈值 int 10000
consumeMaxSpan 消耗最大跨度偏移量 int 2000
pullThresholdForQueue 队列级别的流量控制阈值 int 1000
pullThresholdSizeForQueue 队列级别上限制缓存的消息大小 int 100MiB
pollTimeoutMillis 轮询超时设置 long 5000,以毫秒为单位
topicMetadataCheckIntervalMillis 检查主题元数据变化的间隔时间 long 30000,单位为毫秒
consumeFromWhere 消费方式设置 ConsumeFromWhere ConsumeFromWhere.CONSUME_FROM_LAST_OFFSET
consumeTimestamp 回溯消费时间 String 默认回溯消耗时间为半小时前
traceDispatcher 异步传输数据的接口 TraceDispatcher null
enableMsgTrace 信息跟踪的标志 boolean false
customizedTraceTopic 消息跟踪主题的名称 String
服务端配置
服务端配置
NameServer配置
名称 描述 参数类型 默认值 有效值 重要性
rocketmqHome RocketMQ主目录,默认用户主目录 String
namesrvAddr NameServer地址 String
kvConfigpath kv配置文件路径,包含顺序消息主题的配置信息 String
configStorePath NameServer配置文件路径,建议使用-c指定NameServer配置文件路径 String
clusterTest 是否支持集群测试,默认为false boolean
orderMessageEnable 是否支持顺序消息,默认为false boolean
网络配置属性
名称 描述 参数类型 默认值 有效值 重要性
accessMessageInMemorymaxRatio 访问消息在内存中的比率 int 40(%)
adminBrokerThreadPoolNums 服务端处理控制台管理命令线程池线程数量 int 16
autoCreateSubscriptionGroup 是否自动创建消费组 boolean true true/false
autoCreateTopicEnable 是否自动创建主题 boolean
bitMapLengthConsumeQueueExt ConsumeQueue扩展过滤bitmap大小 int 112
brokerClusterName Broker集群名称 String TestCluster
brokerFastFailureEnable 是否支持broker快速失败 如果为true表示会立即清除发送消息线程池,消息拉取线程池中排队任务 ,直接返回系统错误 boolean true
brokerId brokerID 0表示主节点 大于0表示从节点 int 0
brokerIP1 Broker服务地址 String
brokerIP2 BrokerHAIP地址,供slave同步消息的地址 String
brokerName Broker服务器名称morning服务器hostname String broker-a
brokerPermission Broker权限 默认为6表示可读可写 int 6
brokerRole broker角色,分为 ASYNC_MASTER SYNC_MASTER, SLAVE enum ASYNC_MASTER
brokerTopicEnable broker名称是否可以用做主体使用 boolean
channelNotActiveInterval long
checkCRCOnRecover 文件恢复时是否校验CRC boolean
cleanFileForciblyEnable 是否支持强行删除过期文件 boolean
cleanResourceInterval 清除过期文件线程调度频率 int
clientAsyncSemaphoreValue 客户端对invokeAsyncImpl方法的调用频率 int
clientCallbackExecutorThreads 客户端执行回调线程数 int
clientChannelMaxIdleTimeSeconds 客户端每个channel最大等待时间 int
clientCloseSocketIfTimeout 客户端关闭socket是否需要等待 boolean false
clientManagerThreadPoolQueueCapacity 客户端管理线程池任务队列初始大小 int 1000000
clientManageThreadPoolNums 服务端处理客户端管理(心跳 注册 取消注册线程数量) int 32
clientOnewaySemaphoreValue 客户端对invokeOnewayImpl方法的调用控制 int
clientPooledByteBufAllocatorEnable 客户端池化内存是否开启 boolean
clientSocketRcvBufSize 客户端socket接收缓冲区大小 long
clientSocketSndBufSize 客户端socket发送缓冲区大小 long
clientWorkerThreads worker线程数 int
clusterTopicEnable 集群名称是否可用在主题使用 boolean
commercialBaseCount
commercialBigCount
commercialEnable
commercialTimerCount
commitCommitLogLeastPages 一次提交至少需要脏页的数量,默认4页,针对 commitlog文件 int
commitCommitLogThoroughInterval Commitlog两次提交的最大间隔,如果超过该间隔,将忽略commitCommitLogLeastPages直接提交 int 200
commitIntervalCommitLog commitlog提交频率 int 200
compressedRegister 是否开启消息压缩 boolean
connectTimeoutMillis 链接超时时间 long 3000
consumerFallbehindThreshold 消息消费堆积阈值默认16GB在disableConsumeifConsumeIfConsumerReadSlowly为true时生效 long 17179869184
consumerManagerThreadPoolQueueCapacity 消费管理线程池任务队列大小 int 1000000
consumerManageThreadPoolNums 服务端处理消费管理 获取消费者列表 更新消费者进度查询消费进度等 int 32
debugLockEnable 是否支持 PutMessage Lock锁打印信息 boolean false
defaultQueryMaxNum 查询消息默认返回条数,默认为32 int 32
defaultTopicQueueNums 主体在一个broker上创建队列数量 int 8
deleteCommitLogFilesInterval 删除commitlog文件的时间间隔,删除一个文件后等一下再删除一个文件 int 100
deleteConsumeQueueFilesInterval 删除consumequeue文件时间间隔 int 100
deleteWhen 磁盘文件空间充足情况下,默认每天什么时候执行删除过期文件,默认04表示凌晨4点 string 04
destroyMapedFileIntervalForcibly 销毁MappedFile被拒绝的最大存活时间,默认120s。清除过期文件线程在初次销毁mappedfile时,如果该文件被其他线程引用,引用次数大于0.则设置MappedFile的可用状态为false,并设置第一次删除时间,下一次清理任务到达时,如果系统时间大于初次删除时间加上本参数,则将ref次数一次减1000,直到引用次数小于0,则释放物理资源 int 120000
disableConsumeIfConsumerReadSlowly 如果消费组消息消费堆积是否禁用该消费组继续消费消息 boolean false
diskFallRecorded 是否统计磁盘的使用情况,默认为true boolean true
diskMaxUsedSpaceRatio commitlog目录所在分区的最大使用比例,如果commitlog目录所在的分区使用比例大于该值,则触发过期文件删除 int 75
duplicationEnable 是否允许重复复制,默认为 false boolean false
enableCalcFilterBitMap 是否开启比特位映射 boolean false
enableConsumeQueueExt 是否启用ConsumeQueue扩展属性 boolean false
enablePropertyFilter 是否支持根据属性过滤 如果使用基于标准的sql92模式过滤消息则改参数必须设置为true boolean false
endTransactionPoolQueueCapacity 处理提交和回滚消息线程池线程队列大小 int
endTransactionThreadPoolNums 处理提交和回滚消息线程池 int 24
expectConsumerNumUseFilter 布隆过滤器参数 int 32
fastFailIfNoBufferInStorePool 从 transientStorepool中获取 ByteBuffer是否支持快速失败 boolean false
fetchNamesrvAddrByAddressServer 是否支持从服务器获取nameServer boolean false
fileReservedTime 文件保留时间,默认72小时,表示非当前写文件最后一次更新时间加上filereservedtime小与当前时间,该文件将被清理 String 120
filterDataCleanTimeSpan 清除过滤数据的时间间隔 long 86400000
filterServerNums broker服务器过滤服务器数量 int 0
filterSupportRetry 消息过滤是否支持重试 boolean false
flushCommitLogLeastPages 一次刷盘至少需要脏页的数量,针对commitlog文件 int 4
flushCommitLogTimed 表示await方法等待FlushIntervalCommitlog,如果为true表示使用Thread.sleep方法等待 boolean false
flushConsumeQueueLeastPages 一次刷盘至少需要脏页的数量,默认2页,针对 Consume文件 int 2
flushConsumeQueueThoroughInterval Consume两次刷盘的最大间隔,如果超过该间隔,将忽略 int 60000
flushConsumerOffsetHistoryInterval fushConsumeQueueLeastPages直接刷盘 int 60000
flushConsumerOffsetInterval 持久化消息消费进度 consumerOffse.json文件的频率ms int 5000
flushDelayOffsetInterval 延迟队列拉取进度刷盘间隔。默认10s long 10000
flushDiskType 刷盘方式,默认为 ASYNC_FLUSH(异步刷盘),可选值SYNC_FLUSH(同步刷盘) enum ASYNC_FLUSH
flushIntervalCommitLog commitlog刷盘频率 int 500
flushIntervalConsumeQueue consumuQueue文件刷盘频率 int 1000
flushLeastPagesWhenWarmMapedFile 用字节0填充整个文件的,每多少页刷盘一次。默认4096页,异步刷盘模式生效 int 4096
下面的配置实在官方文档的最佳实践中获取的
Broker 配置
参数名 默认值 说明
listenPort 10911 接受客户端连接的监听端口
namesrvAddr null nameServer 地址
brokerIP1 网卡的 InetAddress 当前 broker 监听的 IP
brokerIP2 跟 brokerIP1 一样 存在主从 broker 时,如果在 broker 主节点上配置了 brokerIP2 属性,broker 从节点会连接主节点配置的 brokerIP2 进行同步
brokerName null broker 的名称
brokerClusterName DefaultCluster 本 broker 所属的 Cluser 名称
brokerId 0 broker id, 0 表示 master, 其他的正整数表示 slave
storePathRootDir 存储根路径
storePathIndex 消息索引存储路径
storeCheckpoint checkpoint 文件存储路径
abortFile abort 文件存储路径
storePathCommitLog $HOME/store/commitlog/ 存储 commit log 的路径
storePathConsumerQueue $HOME/store/consumequeue/ 存储 consume queue 的路径
mapedFileSizeCommitLog 1024 * 1024 * 1024(1G) commit log 的映射文件大小
deleteWhen 04 在每天的什么时间删除已经超过文件保留时间的 commit log
fileReservedTime 72 以小时计算的文件保留时间
brokerRole ASYNC_MASTER SYNC_MASTER/ASYNC_MASTER/SLAVE
flushDiskType ASYNC_FLUSH SYNC_FLUSH/ASYNC_FLUSH SYNC_FLUSH 模式下的 broker 保证在收到确认生产者之前将消息刷盘。ASYNC_FLUSH 模式下的 broker 则利用刷盘一组消息的模式,可以取得更好的性能。
maxMessageSize 限制的消息大小
sendMessageThreadPoolNums 发消息线程池数量
pullMessageThreadPoolNums 拉消息线程池数量
好了,可以回归正题了,我目前的打算是在NameServer的bin文件夹挂载出来,这样方便以后直接修改内存。还有就是log也挂载出来
broker的差不多也同理,准备挂载bin以及conf还有log
所以开始进行拷贝
这里说明一下,我的个人习惯是在 /home/soft/rocketMQ 下进行配置挂载的目录,根据自己的磁盘空间以及目录配置
NameServer挂载
[root@anhao rocketMQ]# docker cp 3308b27b4fd0:/home/rocketmq/rocketmq-5.1.4/bin/ nameserver/
Successfully copied 110kB to /home/soft/rocketMQ/nameserver/
[root@anhao rocketMQ]# docker cp 3308b27b4fd0:/home/rocketmq/rocketmq-5.1.4/conf nameserver/
Successfully copied 147kB to /home/soft/rocketMQ/nameserver/
[root@anhao rocketMQ]# docker cp 3308b27b4fd0:/home/rocketmq/logs nameserver/
Successfully copied 2.07MB to /home/soft/rocketMQ/nameserver/
[root@anhao rocketMQ]#
Broker挂载
[root@anhao rocketMQ]# docker cp 3308b27b4fd0:/home/rocketmq/rocketmq-5.1.4/bin brocker/
Successfully copied 110kB to /home/soft/rocketMQ/brocker/
[root@anhao rocketMQ]# docker cp 3308b27b4fd0:/home/rocketmq/rocketmq-5.1.4/conf brocker/
Successfully copied 147kB to /home/soft/rocketMQ/brocker/
[root@anhao rocketMQ]# docker cp 3308b27b4fd0:/home/rocketmq/logs brocker/
Successfully copied 2.15MB to /home/soft/rocketMQ/brocker/
[root@anhao rocketMQ]#
接下来修改一下NameServer的运行脚本,毕竟是不大的环境,让内存小一点
[root@anhao bin]# pwd
/home/soft/rocketMQ/nameserver/bin
[root@anhao bin]# vim runserver.sh
找到文件中的这一行修改适合自己的内存
JAVA_OPT="${JAVA_OPT} -server -Xms1024M -Xmx1024M -Xmn512M"
接下来修改broker的运行脚本
[root@anhao bin]# pwd
/home/soft/rocketMQ/brocker/bin
[root@anhao bin]# vim runbroker.sh
找到文件中的这一行修改适合自己的内存
JAVA_OPT="${JAVA_OPT} -server -Xms1024M -Xmx1024M -Xmn512M"
接下来修改broker.conf文件
[root@anhao conf]# pwd
/home/soft/rocketMQ/brocker/conf
[root@anhao conf]# vim broker.conf
[root@anhao conf]#
我的配置文件如下
[root@anhao conf]# cat broker.conf
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
listenPort=10911
brokerClusterName = DefaultCluster
brokerName = broker-a
brokerId = 0
brokerIP1= 192.168.31.13
deleteWhen = 04
fileReservedTime = 48
brokerRole = ASYNC_MASTER
flushDiskType = ASYNC_FLUSH
storePathRootDir=/home/rocketmq/store
[root@anhao conf]#
那么接下来就正式启动服务
docker run -d --name rmqnamesrv -p 6789:9876 -v /home/soft/rocketMQ/nameserver/bin/:/home/rocketmq/rocketmq-5.1.4/bin -v /home/soft/rocketMQ/nameserver/conf/:/home/rocketmq/rocketmq-5.1.4/conf -v /home/soft/rocketMQ/nameserver/logs/:/home/rocketmq/logs apache/rocketmq:5.1.4 sh mqnamesrv
这个端口我刚开始改了,但是发现好多东西链接不上,也没找到客户端可以更换链接端口的地方,但是我大概有猜测,就是10911 端口 是基准 进行加1 以及 减2就是10909 并且broker向NamServer注册的时候虽然是容器映射端口改了,但是注册的还是容器内部端口,所以不行,这个后期有时间可以进行修改一下,在尝试一下,如果不是docker部署的话就应该没有这个问题
docker run -d --name rmqbroker -p 10912:10912 -p 10911:10911 -p 10909:10909 -p 8080:8080 -p 8081:8081 -v /home/soft/rocketMQ/broker/bin/:/home/rocketmq/rocketmq-5.1.4/bin -v /home/soft/rocketMQ/broker/conf/:/home/rocketmq/rocketmq-5.1.4/conf -v /home/soft/rocketMQ/broker/logs/:/home/rocketmq/logs -v /home/soft/rocketMQ/broker/store/:/home/rocketmq/store -e "NAMESRV_ADDR=192.168.31.13:6789" apache/rocketmq:5.1.4 sh mqbroker --enable-proxy -c /home/rocketmq/rocketmq-5.1.4/conf/broker.conf
查看是否启动成功
[root@anhao rocketMQ]# docker logs -f rmqnamesrv
OpenJDK 64-Bit Server VM warning: Using the DefNew young collector with the CMS collector is deprecated and will likely be removed in a future release
OpenJDK 64-Bit Server VM warning: UseCMSCompactAtFullCollection is deprecated and will likely be removed in a future release.
The Name Server boot success. serializeType=JSON, address 0.0.0.0:9876
[root@anhao rocketMQ]# docker logs -f rmqbroker
Sat Jun 29 21:34:44 UTC 2024 rocketmq-proxy startup successfully
[root@anhao rocketMQ]# docker exec -it rmqbroker bash -c "tail -n 10 /home/rocketmq/logs/rocketmqlogs/proxy.log"
}
2024-06-30 05:34:42 INFO main - ServiceProvider loaded no AccessValidator, using default org.apache.rocketmq.acl.plain.PlainAccessValidator
2024-06-30 05:34:44 INFO main - grpc server has built. port: 8081, tlsKeyPath: 1, tlsCertPath: 16, threadPool: 136314880, queueCapacity: {}, boosLoop: {}, workerLoop: {}, maxInboundMessageSize: {}
2024-06-30 05:34:44 INFO main - Server is running in TLS permissive mode
2024-06-30 05:34:44 INFO main - Using OpenSSL provider
2024-06-30 05:34:44 INFO main - SSLContext created for server
2024-06-30 05:34:44 INFO main - The broker[broker-a, 192.168.31.13:10911] boot success. serializeType=JSON and name server is 192.168.31.13:9876
2024-06-30 05:34:44 INFO main - user specified name server address: 192.168.31.13:9876
2024-06-30 05:34:44 INFO main - grpc server start successfully.
2024-06-30 05:34:44 INFO main - Sat Jun 29 21:34:44 UTC 2024 rocketmq-proxy startup successfully
说一下,关于使用Dashboard连接
首先配置NameServer的认证
在conf的plain_acl.yml文件,打开这个文件
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
globalWhiteRemoteAddresses:
- 10.10.103.*
- 192.168.0.*
accounts:
- accessKey: RocketMQ
secretKey: 12345678
whiteRemoteAddress:
admin: false
defaultTopicPerm: DENY
defaultGroupPerm: SUB
topicPerms:
- topicA=DENY
- topicB=PUB|SUB
- topicC=SUB
groupPerms:
# the group should convert to retry topic
- groupA=DENY
- groupB=PUB|SUB
- groupC=SUB
- accessKey: rocketmq2
secretKey: 12345678
whiteRemoteAddress: 192.168.1.*
# if it is admin, it could access all resources
admin: true
可以看到accessKey
, secretKey
配置成自己想要的,还有就是关于白名单的配置
docker安装apacherocketmq/rocketmq-dashboard
docker run -d --name rocketmq-dashboard -e "JAVA_OPTS=-Drocketmq.namesrv.addr=192.168.31.13:6789" -p 18888:8080 -t apacherocketmq/rocketmq-dashboard:1.0.0
以下是我个人的一些配置文件还有官方docker镜像的安装包,需要的可以直接下载
[root@anhao imageBuild]# docker save -o apache-rocketmq.tar apache/rocketmq:5.1.4
[root@anhao imageBuild]# ll
总用量 746088
-rw-------. 1 root root 763991552 6月 30 16:42 apache-rocketmq.tar
[root@anhao imageBuild]# docker save -o apacherocketmq-rocketmq-dashboard.tar apacherocketmq/rocketmq-dashboard:1.0.0
[root@anhao imageBuild]# ll
总用量 1482312
-rw-------. 1 root root 763991552 6月 30 16:42 apache-rocketmq.tar
-rw-------. 1 root root 753891840 6月 30 16:44 apacherocketmq-rocketmq-dashboard.tar
[root@anhao imageBuild]#