storm on yarn 部署

1. 环境介绍

1.1 节点与服务映射关系

ip	host	服务
192.168.40.132	master	Namenode、NodeManager、DataNode、zookeeper
192.168.40.133	slave1	ResurceManager、NodeManager、DataNode、zookeeper
192.168.40.134	slave2	NodeManager、DataNode、zookeeper

1.2 jdk版本

[root@slave2 ~]# java -version
java version "1.7.0_79"
Java(TM) SE Runtime Environment (build 1.7.0_79-b15)
Java HotSpot(TM) 64-Bit Server VM (build 24.79-b02, mixed mode)
 
 
1
2
3
4
 
 
1
2
3
4

1.3 hadoop版本

[root@slave2 ~]# hadoop version
Hadoop 2.6.0
Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git -r e3496499ecb8d220fba99dc5ed4c99c8f9e33bb1
Compiled by jenkins on 2014-11-13T21:10Z
Compiled with protoc 2.5.0
From source with checksum 18e43357c8f927c0695f1e9522859d6a
This command was run using /application/hadoop-2.6.0/share/hadoop/common/hadoop-common-2.6.0.jar
 
 
1
2
3
4
5
6
7
 
 
1
2
3
4
5
6
7

1.4 各节点进程 jps

2. 安装maven

2.1 下载、解压maven

 wget http://mirrors.tuna.tsinghua.edu.cn/apache/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz

tar -xzvf apache-maven-3.3.9-bin.tar.gz

ln -s apache-maven-3.3.9/ maven
 
 
1
2
3
4
5
 
 
1
2
3
4
5

2.2 配置maven环境变量

export MAVEN_HOME=/application/maven
export PATH=.:$MAVEN_HOME/bin:$JAVA_HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$PATH

source /etc/profile
 
 
1
2
3
4
 
 
1
2
3
4

2.3 验证，查看maven版本号

mvn -v
Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-11T00:41:47+08:00)
Maven home: /application/maven
Java version: 1.7.0_79, vendor: Oracle Corporation
Java home: /application/jdk1.7.0_79/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "2.6.32-504.el6.x86_64", arch: "amd64", family: "unix"
 
 
1
2
3
4
5
6
7
 
 
1
2
3
4
5
6
7

3. 安装storm-yarn

3.1 下载storm-yarn

wget https://github.com/yahoo/storm-yarn/archive/master.zip
 
 
1
 
 
1

重命名下载的压缩包为storm-yarn.zip

mv master.zip storm-yarn.zip
 
 
1
 
 
1

解压

unzip storm-yarn.zip
 
 
1
 
 
1

3.2 修改pom.xml

根据Hadoop的版本号修改storm-yarn-master/pom.xml

<properties>
    <storm.version>0.9.6</storm.version>
    <hadoop.version>2.6.0</hadoop.version>
</properties>
 
 
1
2
3
4
 
 
1
2
3
4

因为我的环境中hadoop的版本是2.6.0，没有安装storm（也不用安装），所以pom.xml没有修改。

3.3 编译打包storm-yarn-master

mvn package -DskipTests
 
 
1
 
 
1

打包成功

[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ storm-yarn ---
[INFO] ----------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ----------------------------------------------------------------------
[INFO] Total time: 08:41 min
[INFO] Finished at: 2016-08-26T10:16:27+08:00
[INFO] Final Memory: 14M/34M
[INFO] ----------------------------------------------------------------------
 
 
1
2
3
4
5
6
7
8
 
 
1
2
3
4
5
6
7
8

3.4 解压lib/storm.zip并移动解压后的目录

解压storm-yarn-master/lib/storm.zip

unzip storm-yarn-master/lib/storm.zip 

[root@master lib]# ls
storm-0.9.0-wip21  storm-0.9.0-wip21.zip  storm-starter-0.0.1-SNAPSHOT.jar  storm.zip
 
 
1
2
3
4
 
 
1
2
3
4

将解压后生成的storm-0.9.0-wip21移到和storm-yarn-master同级的目录

[root@master lib]mv storm-0.9.0-wip21 ../..
[root@master lib]# cd ../..
[root@master application]# ls
maven   storm-yarn.zip  apache-maven-3.3.9  tools   hadoop  zookeeper   hadoop-2.6.0    java    storm-0.9.0-wip21   zookeeper-3.3.6 jdk1.7.0_79 storm-yarn-master   
 
 
1
2
3
4
 
 
1
2
3
4

3.5 配置环境变量

export PATH=.:/application/storm-0.9.0-wip21/bin:/application/storm-yarn-master/bin:$MAVEN_HOME/bin::$JAVA_HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$PATH
 
 
1
 
 
1

使用source /etc/profile使之生效

3.6 编辑 storm-0.9.0-wip21/conf/storm.yaml

修改zookeeper地址

 storm.zookeeper.servers:
     - "master"
     - "slave1"
     - "slave2"
 
 
1
2
3
4
 
 
1
2
3
4

3.7 压缩storm.zip并上传到hdfs上

压缩storm-0.9.0-wip21

zip -r storm.zip storm-0.9.0-wip21 
 
 
1
 
 
1

将压缩后的文件storm.zip上传到hdfs上的目录/lib/storm/0.9.0-wip21/

hadoop fs -mkdir -p /lib/storm/0.9.0-wip21
hdfs dfs -put storm.zip /lib/storm/0.9.0-wip21/
hdfs dfs -ls /lib/storm/0.9.0-wip21/
Found 1 items
-rw-r--r--   3 root supergroup   17141097 2016-08-26 10:27 /lib/storm/0.9.0-wip21/storm.zip
 
 
1
2
3
4
5
 
 
1
2
3
4
5

4. 运行storm-yarn

4.1 启动storm-yarn

[root@master application]# storm-yarn launch storm-0.9.0-wip21/conf/storm.yaml 
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/application/hadoop-2.6.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/application/storm-0.9.0-wip21/lib/logback-classic-1.0.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
16/08/26 10:28:28 INFO client.RMProxy: Connecting to ResourceManager at slave1/192.168.40.133:8032
16/08/26 10:28:29 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
16/08/26 10:28:30 INFO yarn.StormOnYarn: Copy App Master jar from local filesystem and add to local environment
16/08/26 10:28:32 INFO yarn.StormOnYarn: Set the environment for the application master
16/08/26 10:28:32 INFO yarn.StormOnYarn: YARN CLASSPATH COMMAND = [[yarn, classpath]]
16/08/26 10:28:32 INFO yarn.StormOnYarn: Using JAVA_HOME = [/application/java]
16/08/26 10:28:32 INFO yarn.StormOnYarn: Setting up app master command:[/application/java/bin/java, -Dstorm.home=./storm/storm-0.9.0-wip21/, -Dlogfile.name=<LOG_DIR>/master.log, com.yahoo.storm.yarn.MasterServer, 1><LOG_DIR>/stdout, 2><LOG_DIR>/stderr]
16/08/26 10:28:32 INFO impl.YarnClientImpl: Submitted application application_1472175545791_0002
application_1472175545791_0002
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

通过http://slave2:8088查看yarn的appplication运行情况

4.2 获取nimbus地址

得到storm的属性信息，并保存到/root/.storm/storm.yaml中

[root@master application]# mkdir -p /root/.storm/
[root@master application]# storm-yarn getStormConfig -appId application_1472175545791_0002  -output ~/.storm/storm.yaml
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/application/hadoop-2.6.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/application/storm-0.9.0-wip21/lib/logback-classic-1.0.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
16/08/26 10:30:13 INFO client.RMProxy: Connecting to ResourceManager at slave1/192.168.40.133:8032
16/08/26 10:30:13 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
16/08/26 10:30:14 INFO yarn.StormOnYarn: application report for application_1472175545791_0002 :slave2:9000
16/08/26 10:30:14 INFO yarn.StormOnYarn: Attaching to slave2:9000 to talk to app master application_1472175545791_0002
{10:35:18}16/08/26 10:30:14 INFO yarn.StormMasterCommand: storm.yaml downloaded into /root/.storm/storm.yaml
 
 
1
2
3
4
5
6
7
8
9
10
11
12
 
 
1
2
3
4
5
6
7
8
9
10
11
12

根据/root/.storm/storm.yaml获取nimbus地址

[root@master .storm]# cat ~/.storm/storm.yaml | grep nimbus.host
nimbus.host: 192.168.40.134
 
 
1
2
 
 
1
2

4.3 通过nimbus访问storm UI

通过访问http://192.168.40.134:7070访问storm UI

4.4 提交任务wordcount

[root@master storm-yarn-master]# storm jar lib/storm-starter-0.0.1-SNAPSHOT.jar storm.starter.WordCountTopology WordCountTopology -c nimbus.host=192.168.40.134
561  [main] INFO  backtype.storm.StormSubmitter - Jar not uploaded to master yet. Submitting jar...
573  [main] INFO  backtype.storm.StormSubmitter - Uploading topology jar lib/storm-starter-0.0.1-SNAPSHOT.jar to assigned location: storm-local/nimbus/inbox/stormjar-12a737f5-82e9-43a6-b422-c1740cc36635.jar
596  [main] INFO  backtype.storm.StormSubmitter - Successfully uploaded topology jar to assigned location: storm-local/nimbus/inbox/stormjar-12a737f5-82e9-43a6-b422-c1740cc36635.jar
596  [main] INFO  backtype.storm.StormSubmitter - Submitting topology WordCountTopology in distributed mode with conf {"nimbus.host":"192.168.40.134","topology.workers":3,"topology.debug":true}
948  [main] INFO  backtype.storm.StormSubmitter - Finished submitting topology: WordCountTopology
 
 
1
2
3
4
5
6
 
 
1
2
3
4
5
6

查看Storm UI

查看topology wordcount的运行情况

查看各节点jps

5. 关闭

5.1 关闭storm 提交的任务

[root@master ~]# storm kill WordCountTopology
2277 [main] INFO  backtype.storm.thrift - Connecting to Nimbus at 192.168.40.134:6627
2387 [main] INFO  backtype.storm.command.kill-topology - Killed topology: WordCountTopology
 
 
1
2
3
 
 
1
2
3

5.2 关闭storm on yarn

[root@master ~]# yarn application -kill application_1472175545791_0002
Killing application application_1472175545791_0002