文章目录
1、前提
Storm 安装前,确保 zookeeper 的单机或集群版已经配置好,并启动成功。
2、集群说明
zookeeper :
IP地址 | 名称 | 说明 |
---|---|---|
192.168.1.102 | zookeeper | 单机版的(如果是集群版的会更好) |
storm :
IP地址 | 名称 | 说明 |
---|---|---|
192.168.1.104 | nimbus、ui 的服务 | 主节点 |
192.168.1.105 | supervisor | 工作节点1 |
192.168.1.106 | supervisor | 工作节点2 |
3、安装步骤
3.1、下载 storm
清华大学镜像地址: https://mirrors.cnnic.cn/apache/storm/
我下载的是 apache-storm-0.9.2-incubating.tar.gz
3.2、上传、解压(3台)
上传 apache-storm-0.9.2-incubating.tar.gz 到 centos7 的 /mysoft
中
解压命令:
tar -zxvf apache-storm-0.9.2-incubating.tar.gz
重命名为 /apache-storm-0.9.2
,移动 /usr/local/
目录下:
mv apache-storm-0.9.2-incubating /usr/local/apache-storm-0.9.2
3.3、创建 data 目录(3台)
此目录用于存放 storm 的数据
cd /usr/local/apache-storm-0.9.2/
/usr/local/apache-storm-0.9.2/data
3.4、配置 /etc/profile(3台)
将 storm_home 配置到 /etc/profile
上。
vi /etc/profile
内容:
STORM_HOME=/usr/local/apache-storm-0.9.2/
PATH=$PATH:$STORM_HOME/bin
export PATH STORM_HOME
重新加载 profile:
source /etc/profile
3.5、配置 storm(3台)
编辑 conf 目录下的 storm.yaml
配置文件:
vi /usr/local/apache-storm-0.9.2/conf/storm.yaml
添加 或修改内容如下:
storm.zookeeper.servers:
- "192.168.1.102"
nimbus.host: "192.168.1.104"
storm.local.dir: "/usr/local/apache-storm-0.9.2/data"
ui.port: 18080
supervisor.slots.ports:
- 6700
- 6701
- 6702
- 6703
说明 :
(3台) 是指此步骤的操作,在3台服务器上完全相同。
3.6、启动
3.6.1、在 192.168.1.104 启动 nimbus
由于 storm 已经在 /etc/profile
中配置全局命令,所以 启动命令 可以在任何目录下操作
启动 nimbus :
storm nimbus &
也可以使用下面的命令:
nohup bin/storm nimbus >/dev/null 2>&1 &
3.6.2、在 192.168.1.105、106 启动 supervisor
启动 nimbus :
storm supervisor &
3.7、检查是否启动成功
192.168.1.104 查看 nimbus 的启动状态 :
[root@rocketmq-nameserver1 conf]# jps
5425 Jps
5252 nimbus
192.168.1.105、106 上 查看 supervisor 的启动状态:
[root@rocketmq-nameserver3 logs]# jps
4677 supervisor
6795 Jps
3.8、查看日志,检查是否启动成功:
192.168.1.104 查看 nimbus 的日志 :
进入 logs 目录:
[root@rocketmq-nameserver1 apache-storm-0.9.2]# cd logs/
[root@rocketmq-nameserver1 logs]# ls
access.log metrics.log nimbus.log ui.log
查看 nimbus.log 日志(192.168.1.104 启动的是 nimbus )
[root@rocketmq-nameserver1 logs]# tail -300f nimbus.log
2019-05-19 19:07:28 o.a.z.ZooKeeper [INFO] Client environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT
2019-05-19 19:07:28 o.a.z.ZooKeeper [INFO] Client environment:host.name=rocketmq-nameserver1
2019-05-19 19:07:28 o.a.z.ZooKeeper [INFO] Client environment:java.version=1.8.0_201
2019-05-19 19:07:28 o.a.z.ZooKeeper [INFO] Client environment:java.vendor=Oracle Corporation
2019-05-19 19:07:28 o.a.z.ZooKeeper [INFO] Client environment:java.home=/mysoft/jdk1.8.0_201/jre
.....(省略)
2019-05-19 19:07:29 o.a.z.ZooKeeper [INFO] Initiating client connection, connectString=192.168.1.102:2181 sessionTimeout=20000 watcher=org.apache.curator.ConnectionState@400d912a
2019-05-19 19:07:29 o.a.z.ClientCnxn [INFO] Opening socket connection to server 192.168.1.102/192.168.1.102:2181. Will not attempt to authenticate using SASL (unknown error)
2019-05-19 19:07:30 o.a.z.ClientCnxn [INFO] Socket connection established to 192.168.1.102/192.168.1.102:2181, initiating session
2019-05-19 19:07:30 o.a.z.ClientCnxn [INFO] Session establishment complete on server 192.168.1.102/192.168.1.102:2181, sessionid = 0x16acfa5816d0000, negotiated timeout = 20000
2019-05-19 19:07:30 o.a.c.f.s.ConnectionStateManager [INFO] State change: CONNECTED
2019-05-19 19:07:30 o.a.c.f.s.ConnectionStateManager [WARN] There are no ConnectionStateListeners registered.
2019-05-19 19:07:30 b.s.zookeeper [INFO] Zookeeper state update: :connected:none
2019-05-19 19:07:31 o.a.z.ZooKeeper [INFO] Session: 0x16acfa5816d0000 closed
2019-05-19 19:07:31 o.a.z.ClientCnxn [INFO] EventThread shut down
2019-05-19 19:07:31 o.a.c.f.i.CuratorFrameworkImpl [INFO] Starting
2019-05-19 19:07:31 o.a.z.ZooKeeper [INFO] Initiating client connection, connectString=192.168.1.102:2181/storm sessionTimeout=20000 watcher=org.apache.curator.ConnectionState@6ac97b84
2019-05-19 19:07:31 o.a.z.ClientCnxn [INFO] Opening socket connection to server 192.168.1.102/192.168.1.102:2181. Will not attempt to authenticate using SASL (unknown error)
2019-05-19 19:07:31 o.a.z.ClientCnxn [INFO] Socket connection established to 192.168.1.102/192.168.1.102:2181, initiating session
2019-05-19 19:07:31 o.a.z.ClientCnxn [INFO] Session establishment complete on server 192.168.1.102/192.168.1.102:2181, sessionid = 0x16acfa5816d0001, negotiated timeout = 20000
2019-05-19 19:07:31 o.a.c.f.s.ConnectionStateManager [INFO] State change: CONNECTED
2019-05-19 19:07:31 o.a.c.f.s.ConnectionStateManager [WARN] There are no ConnectionStateListeners registered.
2019-05-19 19:07:32 b.s.d.nimbus [INFO] Starting Nimbus server...
192.168.1.105、106 查看 nimbus 的日志 :
[root@rocketmq-nameserver2 apache-storm-0.9.2]# cd logs/
[root@rocketmq-nameserver2 logs]# ls
access.log metrics.log supervisor.log
查看 supervisor.log 日志:(192.168.1.105 、106 启动的是 supervisor )
[root@rocketmq-nameserver3 logs]# tail -300f supervisor.log
2019-05-19 19:08:38 o.a.z.ZooKeeper [INFO] Client environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT
2019-05-19 19:08:38 o.a.z.ZooKeeper [INFO] Client environment:host.name=rocketmq-nameserver3
2019-05-19 19:08:38 o.a.z.ZooKeeper [INFO] Client environment:java.version=1.8.0_201
2019-05-19 19:08:38 o.a.z.ZooKeeper [INFO] Client environment:java.vendor=Oracle Corporation
2019-05-19 19:08:38 o.a.z.ZooKeeper [INFO] Client environment:java.home=/mysoft/jdk1.8.0_201/jre
.....(省略)
2019-05-19 19:08:39 o.a.z.ZooKeeper [INFO] Initiating client connection, connectString=192.168.1.102:2181 sessionTimeout=20000 watcher=org.apache.curator.ConnectionState@6579c3d9
2019-05-19 19:08:39 o.a.z.ClientCnxn [INFO] Opening socket connection to server 192.168.1.102/192.168.1.102:2181. Will not attempt to authenticate using SASL (unknown error)
2019-05-19 19:08:39 o.a.z.ClientCnxn [INFO] Socket connection established to 192.168.1.102/192.168.1.102:2181, initiating session
2019-05-19 19:08:40 o.a.z.ClientCnxn [INFO] Session establishment complete on server 192.168.1.102/192.168.1.102:2181, sessionid = 0x16acfa5816d0002, negotiated timeout = 20000
2019-05-19 19:08:40 o.a.c.f.s.ConnectionStateManager [INFO] State change: CONNECTED
2019-05-19 19:08:40 o.a.c.f.s.ConnectionStateManager [WARN] There are no ConnectionStateListeners registered.
2019-05-19 19:08:40 b.s.zookeeper [INFO] Zookeeper state update: :connected:none
2019-05-19 19:08:41 o.a.z.ZooKeeper [INFO] Session: 0x16acfa5816d0002 closed
2019-05-19 19:08:41 o.a.c.f.i.CuratorFrameworkImpl [INFO] Starting
2019-05-19 19:08:41 o.a.z.ClientCnxn [INFO] EventThread shut down
2019-05-19 19:08:41 o.a.z.ZooKeeper [INFO] Initiating client connection, connectString=192.168.1.102:2181/storm sessionTimeout=20000 watcher=org.apache.curator.ConnectionState@59712875
2019-05-19 19:08:41 o.a.z.ClientCnxn [INFO] Opening socket connection to server 192.168.1.102/192.168.1.102:2181. Will not attempt to authenticate using SASL (unknown error)
2019-05-19 19:08:41 o.a.z.ClientCnxn [INFO] Socket connection established to 192.168.1.102/192.168.1.102:2181, initiating session
2019-05-19 19:08:41 o.a.z.ClientCnxn [INFO] Session establishment complete on server 192.168.1.102/192.168.1.102:2181, sessionid = 0x16acfa5816d0003, negotiated timeout = 20000
2019-05-19 19:08:41 o.a.c.f.s.ConnectionStateManager [INFO] State change: CONNECTED
2019-05-19 19:08:41 o.a.c.f.s.ConnectionStateManager [WARN] There are no ConnectionStateListeners registered.
2019-05-19 19:08:41 b.s.d.supervisor [INFO] Starting supervisor with id 0ee53c84-1eb4-4a4d-b9be-f7f72ba8e9b2 at host rocketmq-nameserver2
3.9、storm UI
如果前面的步骤检查没有问题了,启动 storm ui
。
3.9.1、启动 storm ui
storm ui &
也可以使用下面的命令:
nohup bin/storm supervisor >/dev/null 2>&1 &
3.9.2、查看 storm ui
的日志
[root@rocketmq-nameserver1 apache-storm-0.9.2]# cd logs/
[root@rocketmq-nameserver1 logs]# ls
access.log metrics.log nimbus.log ui.log
查看日志:
[root@rocketmq-nameserver1 logs]# tail -500f ui.log
2019-05-19 19:18:26 o.m.log [INFO] Logging to Logger[org.mortbay.log] via org.mortbay.log.Slf4jLog
2019-05-19 19:18:26 o.m.log [INFO] jetty-6.1.26
2019-05-19 19:18:26 o.m.log [INFO] Started SocketConnector@0.0.0.0:18080
2019-05-19 19:18:28 b.s.thrift [INFO] Connecting to Nimbus at 192.168.1.104:6627
2019-05-19 19:18:28 b.s.thrift [INFO] Connecting to Nimbus at 192.168.1.104:6627
从日志中可以看出 UI 的端口号是 18080
。
3.9.3、登录 storm ui
地址:http://192.168.1.104:18080