Install Zookeeper+Storm cluster
1.Set up a Zookeeper cluster
2.Install dependencies on Nimbus and worker machines
3.Download and extract a Storm release to Nimbus and worker machines
4.Fill in mandatory configurations into storm.yaml
5.Launch daemons under supervision using "storm" script and a supervisor of your choice
server information
ubuntu-s-storm-1 137.184.63.172 Zookeeper+Storm
ubuntu-s-storm-2 167.172.138.180 Zookeeper+Storm
ubuntu-s-storm-3 143.244.165.32 Zookeeper+Storm
一、Set up a Zookeeper cluster
https://zookeeper.apache.org/releases.html
1.download and install apache-zookeeper
cd /usr/local/src
wget https://dlcdn.apache.org/zookeeper/zookeeper-3.7.1/apache-zookeeper-3.7.1-bin.tar.gz
2.Decompress the package
tar -zxvf apache-zookeeper-3.7.1-bin.tar.gz
mv apache-zookeeper-3.7.1-bin ../zookeeper
3. install JDK9
tar -zxvf jdk-8u333-linux-x64.tar.gz && mv jdk1.8.0_333 ../jdk_1.8
cat >> /etc/profile << EOF
export JAVA_HOME=/usr/local/jdk_1.8/
export JRE_HOME=$JAVA_HOME/jre
export PATH=$JAVA_HOME/bin:$JRE_HOME/bin:$PATH
export CLASSPATH=$JAVA_HOME/lib:$JRE_HOME/lib:./
EOF
source /etc/profile
4.createa data and logs
cd /usr/local/zookeeper && mkdir data logs
5.change hosts
cat >> /etc/hosts << EOF
137.184.63.172 ubuntu-s-storm-1
167.172.138.180 ubuntu-s-storm-2
143.244.165.32 ubuntu-s-storm-3
EOF
6.cp conf bak
cd /usr/local/zookeeper/conf && cp zoo_sample.cfg zoo.cfg
7.configure the zoo.cfg file
sed -i 's/dataDir=.*/dataDir=\/usr\/local\/zookeeper\/data/g' /usr/local/zookeeper/conf/zoo.cfg
cat >> /usr/local/zookeeper/conf/zoo.cfg << EOF
dataLogDir=/usr/local/zookeeper/logs
server.1=137.184.63.172:2881:3881
server.2=167.172.138.180:2881:3881
server.3=143.244.165.32:2881:3881
EOF
Explanation: server. Server ID=server IP address: communication port between servers: voting port between servers
Default port for communication between client and server: 2181
Default port for communication between servers: 2881
Default port for voting between servers: 3881
8.Change zookeeper defalut port ,to avoid conflcts whit the storm
echo "admin.serverPort=8800" >> /usr/local/zookeeper/conf/zoo.cfg
9.config Record the ID of each server,Execute separately
ubuntu-s-storm-1
echo 1 >/usr/local/zookeeper/data/myid
ubuntu-s-storm-2
echo 2 >/usr/local/zookeeper/data/myid
ubuntu-s-storm-13
echo 3 >/usr/local/zookeeper/data/myid
10.start zookeeper
/usr/local/zookeeper/bin/zkServer.sh start/status/restart/stop
二.Install dependencies on Nimbus and worker machines
1.Java 7+ (Apache Storm 1.x is tested through travis ci against both java 7 and java 8 JDKs)
2.Python 2.6.6 (Python 3.x should work too, but is not tested as part of our CI enviornment)
apt install python-is-python3
三.Download and extract a Storm release to Nimbus and worker machines
1.install Storm
wget https://dlcdn.apache.org/storm/apache-storm-2.4.0/apache-storm-2.4.0.tar.gz
tar -zxvf apache-storm-2.4.0.tar.gz
mv apache-storm-2.4.0 ../storm
四.Fill in mandatory configurations into storm.yaml
1.config storm.yaml
cat >> storm.yaml << EOF
storm.zookeeper.servers:
- "ubuntu-s-storm-1"
- "ubuntu-s-storm-2"
- "ubuntu-s-storm-3"
storm.local.dir: "/usr/local/storm/data"
nimbus.seeds: ["ubuntu-s-storm-1", "ubuntu-s-storm-2", "ubuntu-s-storm-2"]
supervisor.slots.ports:
- 6700
- 6701
- 6702
- 6703
EOF
2.start Storm
nohup /usr/local/storm/bin/storm nimbus &
nohup /usr/local/storm/bin/storm supervisor &
nohup /usr/local/storm/bin/storm ui &