线上kafka版本 0.8.2.0
线上zookeeper版本:zookeeper-3.4.5
1.配置三台服务的host
127.0.0.1 storm01
127.0.0.2 storm02
127.0.0.3 storm03
开通防火墙
#zookeeper,storm
-A INPUT -s 127.0.0.1/32 -p tcp -m multiport --dports 22:65535 -j ACCEPT
-A INPUT -s 127.0.0.2/32 -p tcp -m multiport --dports 22:65535 -j ACCEPT
-A INPUT -s 127.0.0.3/32 -p tcp -m multiport --dports 22:65535 -j ACCEPT
2.配置jdk
wget rpm包url
配置环境变量
export JAVA_HOME=/home/storm/jdk1.8.0_111
export ZOOKEEPER_HOME=/home/storm/zookeeper-3.4.10
export STORM_HOME=/home/storm/apache-storm-1.1.0
PATH=$JAVA_HOME/bin:$ZOOKEEPER_HOME/bin:$STORM_HOME/bin:$PATH:$HOME/bin
3.配置zookeeper
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
dataDir=/home/storm/zookeeper-3.4.10/data
dataLogDir=/home/storm/zookeeper-3.4.10/datalog
# the port at which the clients will connect
clientPort=2181
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1
zookeeper.log.maxbackupindex=7
server.0=storm01:2888:3888
server.1=storm02:2888:3888
server.2=storm03:2888:3888
在dataDir下新建一个myid文件,写上对应的id
启动zookeeper
zkServer.sh start /home/storm/zookeeper-3.4.10/conf/zoo.cfg
查看zookeeper状态
zkServer.sh status
4.配置storm
storm.zookeeper.servers:
- "storm01"
- "storm02"
- "storm03"
#
nimbus.seeds: ["storm01"]
storm.local.dir: "/home/storm/apache-storm-1.1.0/data"
#slot数量一般以每个服务器的CPU线程数来计算。
supervisor.slots.ports:
- 6700
- 6701
- 6702
- 6703
- 6704
- 6705
同步配置
scp storm.yaml storm@storm02:/home/storm/apache-storm-1.1.0/conf/
scp storm.yaml storm@storm03:/home/storm/apache-storm-1.1.0/conf/
scp zoo.cfg storm@storm02:/home/storm/zookeeper-3.4.10/conf
scp zoo.cfg storm@storm03:/home/storm/zookeeper-3.4.10/conf
启动storm
storm nimbus >/dev/null 2>&1 &
storm ui >/dev/null 2>&1 &
storm supervisor >/dev/null 2>&1 &
错误1:
[storm@storm01 gavin]$storm nimbus
Need python version > 2.6
storm启动需要python2.6以上版本,storm中启动脚本python路径指的不正确,如果/usr/bin/python2.6不存在,$PYTHON也不存在,则使用PYTHON=/usr/bin/python这个
在第二个if后面手动指定PYTHON=/usr/local/Python-2.7.10/bin/python