消息队列 RocketMq 安装(单机版)

消息队列 RocketMq 安装（单机版）

一、linux下载与安装

1. 官网下载RocketMq
   官网地址：https://rocketmq.apache.org/
   linux中cd到cd /opt下使用

   wget https://archive.apache.org/dist/rocketmq/4.7.1/rocketmq-all-4.7.1-bin-release.zip
   

   完成后解压，执行

   unzip rocketmq-all-4.7.1-bin-release.zip
   

   运行nameServer服务器

   cd rocketmq-all-4.7.1-bin-release/bin
   ./mqnamesrv
   ERROR: Please set the JAVA_HOME variable in your environment, We need java(x64)! !!
   

2. 提供jdk1.8的环境
   linux 官网下载参考王彦清的博客

   wget --no-cookies --no-check-certificate --header "Cookie: gpw_e24=http%3A%2F%2Fwww.oracle.com%2F; oraclelicense=accept-securebackup-cookie" "http://download.oracle.com/otn-pub/java/jdk/8u141-b15/336fa29ff2bb4ef291e347e091f7f4a7/jdk-8u141-linux-x64.tar.gz"
   tar -xzvf jdk-8u141-linux-x64.tar.gz
   

   移动到/usr/local/software下

   cd /usr/local
   mkdir software
   cd software
   mv /opt/jdk1.8.0_141/ ./
   

   配置环境变量

   	export JAVA_HOME=/usr/local/software/jdk1.8.0_141
   	export JRE_HOME=/usr/local/software/jdk1.8.0_141/jre
   	export CLASSPATH=.:$JAVA_HOME/lib:$JRE_HOME/lib:$CLASSPATH
   	export PATH=$JAVA_HOME/bin:$JRE_HOME/bin:$PATH
   

   vi /etc/profile
   

   
   重新加载配置文件

   source /etc/profile
   

   查看jdk版本

   java -version
   
   java version "1.8.0_141"
   Java(TM) SE Runtime Environment (build 1.8.0_141-b15)
   Java HotSpot(TM) 64-Bit Server VM (build 25.141-b15, mixed mode)
   

3. 重新启动nameServer服务

   [root@iZuf66kf0lsrq5yfg4ymjlZ bin]# ./mqnamesrv
   Java HotSpot(TM) 64-Bit Server VM warning: Using the DefNew young collector with the CMS collector is deprecated and will likely be removed in a future release
   Java HotSpot(TM) 64-Bit Server VM warning: UseCMSCompactAtFullCollection is deprecated and will likely be removed in a future release.
   Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00000006c0000000, 2147483648, 0) failed; error='Cannot allocate memory' (errno=12)
   # There is insufficient memory for the Java Runtime Environment to continue.
   # Native memory allocation (mmap) failed to map 2147483648 bytes for committing reserved memory.
   

   There is insufficient memory for the Java Runtime内存不足

4. 修改rocketMq默认内存大小配置
   修改runserver.sh
   默认：JAVA_OPT="${JAVA_OPT} -server -Xms4g -Xmx4g -Xmn2g -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m"
   修改：JAVA_OPT="${JAVA_OPT} -server -Xms256M -Xmx256M -Xmn128M -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m"
   修改runbroker.sh
   默认：JAVA_OPT="${JAVA_OPT} -server -Xms8g -Xmx8g -Xmn4g"
   修改：JAVA_OPT="${JAVA_OPT} -server -Xms256M -Xmx256M -Xmn128M"
   设置 tools.sh 中此项配置 为：
   JAVA_OPT="${JAVA_OPT} -server -Xms256m -Xmx256m -Xmn256m -XX:PermSize=128m -XX:MaxPermSize=128m"
   重启./mqnamesrv,成功~

5. 后台启动nameserver:顺序:先namesrv后broker
   使用nohup命令
   #如果不存在nohup,使用命令安装:yum install coreutils
   #如果 which nohup在usr/bin/nohup目录下存在，只是没有配置环境变量，要进行配置
   #nohup必须配置vi ~/.bash_profile文件，在环境变量PATH中加入/usr/bin #source ~/.bash_profile
   #查询nohup的版本 nohup --v
   #后台启动mqnamesrv
   nohup ./mqnamesrv > /dev/null 2>&1 &

6. 启动broker
   遇到的两个坑：
   1.mq中设置了brokerIP1,为什么不起作用？
   由于需要设置公网ip，需要在conf/broker.conf加入brokerIP1=x.x.x.x，由于手写，p写成了小写。导致一直没有生效。。。能复制粘贴的最好不要手打
   2. aliyun服务器端口开放
   **开放9876/10911，10909等与mq相关的端口，**本来前一天已经开了，第二天排查完问题仍无法解决后，打开阿里云后台安全组，发现还是最初的配置。。。加入上述的端口即可
   执行命令：这里的localhost换成本机ip也是可以的,localhost和本机ip的关系

   ./mqbroker -n localhost:9876 -c ../conf/broker.conf &
   

   //后台启动

   nohup ./mqbroker -n localhost:9876 -c ../conf/broker.conf &
   

   其中broker.conf中新增如下

   namesrvAddr=139.xxx.xxx.253:9876
   #brokerIp1=139.xxx.xxx.253这条p小写了，已注释。难受
   brokerIP1=139.xxx.xxx.253
   brokerIP2=139.xxx.xxx.253
   autoCreateTopicEnable=true
   

7. 查看日志

   tail -f ~/logs/rocketmqlogs/namesrv.log
   tail -f ~/logs/rocketmqlogs/broker.log
   

   在启动日志中,列出了很多参数，有些是我们指定的，如brokerIP1等。这些参数可以在源码分析的时候去找对应的设置位置及顺序，对我们理解启动过程有帮助

   2020-12-14 11:25:35 INFO main - rocketmqHome=/opt/rocketmq-all-4.7.1-bin-release
   2020-12-14 11:25:35 INFO main - namesrvAddr=localhost:9876
   2020-12-14 11:25:35 INFO main - brokerIP1=139.xxx.xxx.253
   2020-12-14 11:25:35 INFO main - brokerIP2=139.xxx.xxx.253
   2020-12-14 11:25:35 INFO main - brokerName=broker-a
   2020-12-14 11:25:35 INFO main - brokerClusterName=DefaultCluster
   2020-12-14 11:25:35 INFO main - brokerId=0
   2020-12-14 11:25:35 INFO main - brokerPermission=6
   2020-12-14 11:25:35 INFO main - defaultTopicQueueNums=8
   2020-12-14 11:25:35 INFO main - autoCreateTopicEnable=true
   2020-12-14 11:25:35 INFO main - clusterTopicEnable=true
   2020-12-14 11:25:35 INFO main - brokerTopicEnable=true
   2020-12-14 11:25:35 INFO main - autoCreateSubscriptionGroup=true
   2020-12-14 11:25:35 INFO main - messageStorePlugIn=
   2020-12-14 11:25:35 INFO main - msgTraceTopicName=RMQ_SYS_TRACE_TOPIC
   2020-12-14 11:25:35 INFO main - traceTopicEnable=false
   2020-12-14 11:25:35 INFO main - sendMessageThreadPoolNums=1
   2020-12-14 11:25:35 INFO main - pullMessageThreadPoolNums=18
   2020-12-14 11:25:35 INFO main - processReplyMessageThreadPoolNums=18
   2020-12-14 11:25:35 INFO main - queryMessageThreadPoolNums=9
   2020-12-14 11:25:35 INFO main - adminBrokerThreadPoolNums=16
   2020-12-14 11:25:35 INFO main - clientManageThreadPoolNums=32
   2020-12-14 11:25:35 INFO main - consumerManageThreadPoolNums=32
   2020-12-14 11:25:35 INFO main - heartbeatThreadPoolNums=1
   2020-12-14 11:25:35 INFO main - endTransactionThreadPoolNums=10
   2020-12-14 11:25:35 INFO main - flushConsumerOffsetInterval=5000
   2020-12-14 11:25:35 INFO main - flushConsumerOffsetHistoryInterval=60000
   2020-12-14 11:25:35 INFO main - rejectTransactionMessage=false
   2020-12-14 11:25:35 INFO main - fetchNamesrvAddrByAddressServer=false                                         
   2020-12-14 11:25:35 INFO main - sendThreadPoolQueueCapacity=10000                                             
   2020-12-14 11:25:35 INFO main - pullThreadPoolQueueCapacity=100000                                            
   2020-12-14 11:25:35 INFO main - replyThreadPoolQueueCapacity=10000
   2020-12-14 11:25:35 INFO main - queryThreadPoolQueueCapacity=20000                                            
   2020-12-14 11:25:35 INFO main - clientManagerThreadPoolQueueCapacity=1000000                                  
   2020-12-14 11:25:35 INFO main - consumerManagerThreadPoolQueueCapacity=1000000                                
   2020-12-14 11:25:35 INFO main - heartbeatThreadPoolQueueCapacity=50000
   2020-12-14 11:25:35 INFO main - endTransactionPoolQueueCapacity=100000                                        
   2020-12-14 11:25:35 INFO main - filterServerNums=0                                                            
   2020-12-14 11:25:35 INFO main - longPollingEnable=true                                                        
   2020-12-14 11:25:35 INFO main - shortPollingTimeMills=1000
   2020-12-14 11:25:35 INFO main - notifyConsumerIdsChangedEnable=true                                           
   2020-12-14 11:25:35 INFO main - highSpeedMode=false                                                           
   2020-12-14 11:25:35 INFO main - commercialEnable=true                                                         
   2020-12-14 11:25:35 INFO main - commercialTimerCount=1                                                        
   2020-12-14 11:25:35 INFO main - commercialTransCount=1
   2020-12-14 11:25:35 INFO main - commercialBigCount=1                                                          
   2020-12-14 11:25:35 INFO main - commercialBaseCount=1
   2020-12-14 11:25:35 INFO main - transferMsgByHeap=true                                                        
   2020-12-14 11:25:35 INFO main - maxDelayTime=40                                                               
   2020-12-14 11:25:35 INFO main - regionId=DefaultRegion                                                        
   2020-12-14 11:25:35 INFO main - registerBrokerTimeoutMills=6000                                               
   2020-12-14 11:25:35 INFO main - slaveReadEnable=false                                                         
   2020-12-14 11:25:35 INFO main - disableConsumeIfConsumerReadSlowly=false                                      
   2020-12-14 11:25:35 INFO main - consumerFallbehindThreshold=17179869184                                       
   2020-12-14 11:25:35 INFO main - brokerFastFailureEnable=true                                                  
   2020-12-14 11:25:35 INFO main - waitTimeMillsInSendQueue=200                                                  
   2020-12-14 11:25:35 INFO main - waitTimeMillsInPullQueue=5000                                                 
   2020-12-14 11:25:35 INFO main - waitTimeMillsInHeartbeatQueue=31000                                           
   2020-12-14 11:25:35 INFO main - waitTimeMillsInTransactionQueue=3000                                          
   2020-12-14 11:25:35 INFO main - startAcceptSendRequestTimeStamp=0                                             
   2020-12-14 11:25:35 INFO main - traceOn=true                                                                  
   2020-12-14 11:25:35 INFO main - enableCalcFilterBitMap=false                                                  
   2020-12-14 11:25:35 INFO main - expectConsumerNumUseFilter=32                                                 
   2020-12-14 11:25:35 INFO main - maxErrorRateOfBloomFilter=20                                                  
   2020-12-14 11:25:35 INFO main - filterDataCleanTimeSpan=86400000                                              
   2020-12-14 11:25:35 INFO main - filterSupportRetry=false                                                      
   2020-12-14 11:25:35 INFO main - enablePropertyFilter=false                                                    
   2020-12-14 11:25:35 INFO main - compressedRegister=false                                                      
   2020-12-14 11:25:35 INFO main - forceRegister=true                                                            
   2020-12-14 11:25:35 INFO main - registerNameServerPeriod=30000                                                
   2020-12-14 11:25:35 INFO main - transactionTimeOut=6000                                                       
   2020-12-14 11:25:35 INFO main - transactionCheckMax=15                                                        
   2020-12-14 11:25:35 INFO main - transactionCheckInterval=60000                                                
   2020-12-14 11:25:35 INFO main - aclEnable=false                                                               
   2020-12-14 11:25:35 INFO main - storeReplyMessageEnable=true                                                  
   2020-12-14 11:25:35 INFO main - autoDeleteUnusedStats=false                                                   
   2020-12-14 11:25:35 INFO main - listenPort=10911                                                              
   2020-12-14 11:25:35 INFO main - serverWorkerThreads=8                                                         
   2020-12-14 11:25:35 INFO main - serverCallbackExecutorThreads=0                                               
   2020-12-14 11:25:35 INFO main - serverSelectorThreads=3                                                       
   2020-12-14 11:25:35 INFO main - serverOnewaySemaphoreValue=256                                                
   2020-12-14 11:25:35 INFO main - serverAsyncSemaphoreValue=64                                                  
   2020-12-14 11:25:35 INFO main - serverChannelMaxIdleTimeSeconds=120                                           
   2020-12-14 11:25:35 INFO main - serverSocketSndBufSize=131072                                                 
   2020-12-14 11:25:35 INFO main - serverSocketRcvBufSize=131072                                                 
   2020-12-14 11:25:35 INFO main - serverPooledByteBufAllocatorEnable=true                                       
   2020-12-14 11:25:35 INFO main - useEpollNativeSelector=false                                                  
   2020-12-14 11:25:35 INFO main - clientWorkerThreads=4                                                         
   2020-12-14 11:25:35 INFO main - clientCallbackExecutorThreads=1                                               
   2020-12-14 11:25:35 INFO main - clientOnewaySemaphoreValue=65535                                              
   2020-12-14 11:25:35 INFO main - clientAsyncSemaphoreValue=65535                                               
   2020-12-14 11:25:35 INFO main - connectTimeoutMillis=3000                                                     
   2020-12-14 11:25:35 INFO main - channelNotActiveInterval=60000                                                
   2020-12-14 11:25:35 INFO main - clientChannelMaxIdleTimeSeconds=120                                           
   2020-12-14 11:25:35 INFO main - clientSocketSndBufSize=131072                                                 
   2020-12-14 11:25:35 INFO main - clientSocketRcvBufSize=131072                                                 
   2020-12-14 11:25:35 INFO main - clientPooledByteBufAllocatorEnable=false                                      
   2020-12-14 11:25:35 INFO main - clientCloseSocketIfTimeout=false                                              
   2020-12-14 11:25:35 INFO main - useTLS=false                                                                  
   2020-12-14 11:25:35 INFO main - storePathRootDir=/root/store                                                  
   2020-12-14 11:25:35 INFO main - storePathCommitLog=/root/store/commitlog                                      
   2020-12-14 11:25:35 INFO main - mappedFileSizeCommitLog=1073741824                                            
   2020-12-14 11:25:35 INFO main - mappedFileSizeConsumeQueue=6000000                                            
   2020-12-14 11:25:35 INFO main - enableConsumeQueueExt=false                                                   
   2020-12-14 11:25:35 INFO main - mappedFileSizeConsumeQueueExt=50331648                                        
   2020-12-14 11:25:35 INFO main - bitMapLengthConsumeQueueExt=64                                                
   2020-12-14 11:25:35 INFO main - flushIntervalCommitLog=500                                                    
   2020-12-14 11:25:35 INFO main - commitIntervalCommitLog=200                                                   
   2020-12-14 11:25:35 INFO main - useReentrantLockWhenPutMessage=false                                          
   2020-12-14 11:25:35 INFO main - flushCommitLogTimed=false                                                     
   2020-12-14 11:25:35 INFO main - flushIntervalConsumeQueue=1000                                                
   2020-12-14 11:25:35 INFO main - cleanResourceInterval=10000                                                   
   2020-12-14 11:25:35 INFO main - deleteCommitLogFilesInterval=100                                              
   2020-12-14 11:25:35 INFO main - deleteConsumeQueueFilesInterval=100                                           
   2020-12-14 11:25:35 INFO main - destroyMapedFileIntervalForcibly=120000                                       
   2020-12-14 11:25:35 INFO main - redeleteHangedFileInterval=120000                                             
   2020-12-14 11:25:35 INFO main - flushCommitLogLeastPages=4                                                    
   2020-12-14 11:25:35 INFO main - commitCommitLogLeastPages=4                                                   
   2020-12-14 11:25:35 INFO main - flushLeastPagesWhenWarmMapedFile=4096                                         
   2020-12-14 11:25:35 INFO main - flushConsumeQueueLeastPages=2                                                 
   2020-12-14 11:25:35 INFO main - flushCommitLogThoroughInterval=10000                                          
   2020-12-14 11:25:35 INFO main - commitCommitLogThoroughInterval=200                                           
   2020-12-14 11:25:35 INFO main - flushConsumeQueueThoroughInterval=60000                                       
   2020-12-14 11:25:35 INFO main - maxTransferBytesOnMessageInMemory=262144                                      
   2020-12-14 11:25:35 INFO main - maxTransferCountOnMessageInMemory=32                                          
   2020-12-14 11:25:35 INFO main - maxTransferBytesOnMessageInDisk=65536                                         
   2020-12-14 11:25:35 INFO main - maxTransferCountOnMessageInDisk=8                                             
   2020-12-14 11:25:35 INFO main - accessMessageInMemoryMaxRatio=40                                              
   2020-12-14 11:25:35 INFO main - messageIndexEnable=true                                                       
   2020-12-14 11:25:35 INFO main - maxHashSlotNum=5000000                                                        
   2020-12-14 11:25:35 INFO main - maxIndexNum=20000000                                                          
   2020-12-14 11:25:35 INFO main - maxMsgsNumBatch=64                                                            
   2020-12-14 11:25:35 INFO main - messageIndexSafe=false                                                        
   2020-12-14 11:25:35 INFO main - haListenPort=10912                                                            
   2020-12-14 11:25:35 INFO main - haSendHeartbeatInterval=5000                                                  
   2020-12-14 11:25:35 INFO main - haHousekeepingInterval=20000                                                  
   2020-12-14 11:25:35 INFO main - haTransferBatchSize=32768                                                     
   2020-12-14 11:25:35 INFO main - haMasterAddress=                                                              
   2020-12-14 11:25:35 INFO main - haSlaveFallbehindMax=268435456                                                
   2020-12-14 11:25:35 INFO main - brokerRole=ASYNC_MASTER                                                       
   2020-12-14 11:25:35 INFO main - flushDiskType=ASYNC_FLUSH                                                     
   2020-12-14 11:25:35 INFO main - syncFlushTimeout=5000                                                         
   2020-12-14 11:25:35 INFO main - messageDelayLevel=1s 5s 10s 30s 1m 2m 3m 4m 5m 6m 7m 8m 9m 10m 20m 30m 1h 2h  
   2020-12-14 11:25:35 INFO main - flushDelayOffsetInterval=10000                                                
   2020-12-14 11:25:35 INFO main - cleanFileForciblyEnable=true                                                  
   2020-12-14 11:25:35 INFO main - warmMapedFileEnable=false                                                     
   2020-12-14 11:25:35 INFO main - offsetCheckInSlave=false                                                      
   2020-12-14 11:25:35 INFO main - debugLockEnable=false                                                         
   2020-12-14 11:25:35 INFO main - duplicationEnable=false                                                       
   2020-12-14 11:25:35 INFO main - diskFallRecorded=true                                                         
   2020-12-14 11:25:35 INFO main - osPageCacheBusyTimeOutMills=1000                                              
   2020-12-14 11:25:35 INFO main - defaultQueryMaxNum=32                                                         
   2020-12-14 11:25:35 INFO main - transientStorePoolEnable=false                                                
   2020-12-14 11:25:35 INFO main - transientStorePoolSize=5                                                      
   2020-12-14 11:25:35 INFO main - fastFailIfNoBufferInStorePool=false                                           
   2020-12-14 11:25:35 INFO main - enableDLegerCommitLog=false                                                   
   2020-12-14 11:25:35 INFO main - dLegerGroup=                                                                  
   2020-12-14 11:25:35 INFO main - dLegerPeers=                                                                  
   2020-12-14 11:25:35 INFO main - dLegerSelfId= 
   

   启动成功标志：

   2020-12-14 11:25:37 INFO main - The broker[broker-a, 139.xxx.xxx.253:10911] boot success. serializeType=JSON and name server is localhost:9876
   

8. 发送官方示例
   cd 到bin目录下执行生产者

   sh tools.sh org.apache.rocketmq.example.quickstart.Producer
   

   结果：

    SendResult [sendStatus=SEND_OK, msgId=AC110001120C4D7E1886457176CB03E6, 
    offsetMsgId=8BC4C2FD00002A9F00000000000316F4, messageQueue=MessageQueue [topic=TopicTest, 
    brokerName=broker-a, queueId=3], queueOffset=249]
   

   执行消费者

   sh tools.sh org.apache.rocketmq.example.quickstart.Consumer 
   

   结果：

   ConsumeMessageThread_20 Receive New Messages: [MessageExt [brokerName=broker-a, queueId=3, 
   storeSize=203, queueOffset=52, sysFlag=0, bornTimestamp=1607917058477, bornHost=/139.xxx.xxx.253:59934, 
   storeTimestamp=1607917058479, storeHost=/139.xxx.xxx.253:10911, 
   msgId=8BC4C2FD00002A9F000000000000A618, commitLogOffset=42520, bodyCRC=458842710, 
   reconsumeTimes=0, preparedTransactionOffset=0, toString()=Message{topic='TopicTest', flag=0, properties=
   {MIN_OFFSET=0, MAX_OFFSET=250, CONSUME_START_TIME=1607921665936, 
   UNIQ_KEY=AC110001120C4D7E1886457161AD00D2, CLUSTER=DefaultCluster, WAIT=true, TAGS=TagA}, body=
   [72, 101, 108, 108, 111, 32, 82, 111, 99, 107, 101, 116, 77, 81, 32, 50, 49, 48], transactionId='null'}]] 
   

9. 关闭服务

   sh mqshutdown namesrv
   sh mqshutdown mybroker
   

---