Storm上下级依赖系统的介绍及搭建（MetaQ以及Mysql）

一 消息队列MQ的作用

二 MetaQ简介

三 MetaQ的安装部署

1 依赖于Zookeeper集群，必须先安装Zookeeper

2 准备Mta的安装包 metaq-server-1.4.6.2.tar.gz

3 解压metaQ

[root@master opt]# tar -zvxf metaq-server-1.4.6.2.tar.gz

4 配置环境变量

[root@master metamorphosis-server-wrapper]# vi /etc/profile
export METAQ_HOME=/opt/taobao/metamorphosis-server-wrapper
export PATH=$PATH:$METAQ_HOME/bin
[root@master metamorphosis-server-wrapper]# source /etc/profile

5 配置server.ini

[root@master conf]# cat server.ini
[system]
brokerId=0
hostName=master
numPartitions=1
serverPort=8123
dashboardHttpPort=8120
unflushThreshold=0
unflushInterval=10000
maxSegmentSize=1073741824
maxTransferSize=1048576
deletePolicy=delete,168
deleteWhen=0 0 6,18 * * ?
flushTxLogAtCommit=1
stat=true
;; Update consumers offsets to current max offsets when consumers offsets are out of range of current broker's messages.
;; It must be false in production.But recommend to be true in development or test.
updateConsumerOffsets=true
[zookeeper]
zk.zkConnect=master:2181
zk.zkSessionTimeoutMs=30000
zk.zkConnectionTimeoutMs=30000
zk.zkSyncTimeMs=5000
;; Topics section
[topic=test]
[topic=meta-test]

6 启动metaQ

[root@master conf]# metaServer.sh start &

7 查看

[root@master conf]# jps
2777 ServerStartup
2844 Jps
2686 QuorumPeerMain

8 用同样的方法部署其他节点

四 MySQL的安装部署

1 通用安装方法

http://blog.csdn.net/chengqiuming/article/details/78956066

2 Storm中结合使用Mysql条件

数据量小，数据重要，速度快，成本低，通用

3 注意事项

3.1 远程连接配置方法如下：

mysql> GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY 'waDY820828' WITH GRANT OPTION;
Query OK, 0 rows affected (0.00 sec)

3.2 远程连接测试

[root@slave1 ~]# mysql -u root -h master -p
Enter password:
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 9
Server version: 5.6.38 MySQL Community Server (GPL)
Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql>

五 参考

http://www.jikexueyuan.com/course/1724.html

https://my.oschina.net/MrMichael/blog/518991#OSC_h2_2