storm安装包:apache-storm-1.0.3.tar.gz
解压命令:tar -zxvf apache-storm-1.0.3.tar.gz -C ~/training/
配置环境变量:vi ~/.bash_profile
STORM_HOME=/root/training/apache-storm-1.0.3
export STORM_HOME
PATH=$STORM_HOME/bin:$PATH
export PATH
生效:source ~/.bash_profile
其中核心配置文件:/root/training/apache-storm-1.0.3/conf storm.yaml 行号打开配置一下环境变量(伪分布模式)
18 storm.zookeeper.servers:
19 - "bigdata111"
23 nimbus.seeds: ["bigdata111"]
25 storm.local.dir: "/root/training/apache-storm-1.0.3/tmp"
27 supervisor.slots.ports:
28 - 6700
29 - 6701
30 - 6702
31 - 6703
完整配置文件:
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
########### These MUST be filled in for a storm configuration
storm.zookeeper.servers:
- "bigdata111"
# - "server2"
#
# nimbus.seeds: ["host1", "host2", "host3"]
nimbus.seeds: ["bigdata111"]
storm.local.dir: "/root/training/apache-storm-1.0.3/tmp"
supervisor.slots.ports:
- 6700
- 6701
- 6702
- 6703
#
#
# ##### These may optionally be filled in:
#
## List of custom serializations
# topology.kryo.register:
# - org.mycompany.MyType
# - org.mycompany.MyType2: org.mycompany.MyType2Serializer
#
## List of custom kryo decorators
# topology.kryo.decorators:
# - org.mycompany.MyDecorator
#
## Locations of the drpc servers
# drpc.servers:
# - "server1"
# - "server2"
## Metrics Consumers
# topology.metrics.consumer.register:
# - class: "org.apache.storm.metric.LoggingMetricsConsumer"
# parallelism.hint: 1
# - class: "org.mycompany.MyMetricsConsumer"
# parallelism.hint: 1
# argument:
# - endpoint: "metrics-collector.mycompany.org"
配置结束storm 组件启动命令
启动: 主节点 storm nimbus &
从节点 storm supervisor &
UI网页 storm ui & 地址: http://ip:8080
异常:启动storm nimbus &
Exception in thread "main" java.lang.ExceptionInInitializerError
at org.apache.storm.config$read_storm_config.invoke(config.clj:78)
at org.apache.storm.config$fn__908.invoke(config.clj:100)
at org.apache.storm.config__init.load(Unknown Source)
at org.apache.storm.config__init.<clinit>(Unknown Source)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at clojure.lang.RT.classForName(RT.java:2154)
at clojure.lang.RT.classForName(RT.java:2163)
at clojure.lang.RT.loadClassForName(RT.java:2182)
at clojure.lang.RT.load(RT.java:436)
at clojure.lang.RT.load(RT.java:412)
at clojure.core$load$fn__5448.invoke(core.clj:5866)
at clojure.core$load.doInvoke(core.clj:5865)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.core$load_one.invoke(core.clj:5671)
at clojure.core$load_lib$fn__5397.invoke(core.clj:5711)
at clojure.core$load_lib.doInvoke(core.clj:5710)
at clojure.lang.RestFn.applyTo(RestFn.java:142)
at clojure.core$apply.invoke(core.clj:632)
at clojure.core$load_libs.doInvoke(core.clj:5753)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.core$apply.invoke(core.clj:634)
at clojure.core$use.doInvoke(core.clj:5843)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at org.apache.storm.command.config_value$loading__5340__auto____11787.invoke(config_value.clj:16)
at org.apache.storm.command.config_value__init.load(Unknown Source)
at org.apache.storm.command.config_value__init.<clinit>(Unknown Source)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at clojure.lang.RT.classForName(RT.java:2154)
at clojure.lang.RT.classForName(RT.java:2163)
at clojure.lang.RT.loadClassForName(RT.java:2182)
at clojure.lang.RT.load(RT.java:436)
at clojure.lang.RT.load(RT.java:412)
at clojure.core$load$fn__5448.invoke(core.clj:5866)
at clojure.core$load.doInvoke(core.clj:5865)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.lang.Var.invoke(Var.java:379)
at org.apache.storm.command.config_value.<clinit>(Unknown Source)
Caused by: expected '<document start>', but found BlockMappingStart
in 'reader', line 23, column 1:
nimbus.seeds: ["bigdata111"]
line 23 行抛出异常 因为忘记在配置前文件前漏配了一个空格;
23nimbus.seeds: ["bigdata111"]
坑:配置storm.yaml时一定要注意了在需要修改的地方一定要注意前后的空格,少一个都无法启动。。。
最后执行启动各项组件的命令,成功启动;
[root@bigdata111 conf]storm nimbus &
[1] 26326
[root@bigdata111 conf]# storm supervisor &
[2] 26485
[root@bigdata111 conf]#storm ui &
[3] 26583
[root@bigdata111 conf]# jps
22945 QuorumPeerMain //这个是之前启动的zooke 服务器 storm配置启动之前必须安装Zookeeper。
26485 Supervisor
26677 Jps
26326 nimbus
26583 core