启动Heron集群的常用命令

最新推荐文章于 2019-11-10 11:54:03 发布

Ethan_pika

最新推荐文章于 2019-11-10 11:54:03 发布

阅读量186

点赞数

分类专栏：流式计算Heron 文章标签： Heron Aurora Mesos Zookeeper HDFS

本文链接：https://blog.csdn.net/yitian_z/article/details/102952083

版权

流式计算Heron 专栏收录该内容

50 篇文章 0 订阅

订阅专栏

关闭防火墙

yitian@heron01:~$ sudo ufw disable
[sudo] password for yitian:
Firewall stopped and disabled on system startup

查看/启动ZooKeeper运行状态

Master（heron01）启动zookeeper

yitian@heron01:~$ ./zookeeper/zookeeper-3.4.10/bin/zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /home/yitian/zookeeper/zookeeper-3.4.10/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED

yitian@heron01:~$ ./zookeeper/zookeeper-3.4.10/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /home/yitian/zookeeper/zookeeper-3.4.10/bin/../conf/zoo.cfg
Error contacting service. It is probably not running.

yitian@heron01:~$ jps
5971 Jps
5913 QuorumPeerMain

yitian@heron01:~$ sudo ufw disable
[sudo] password for yitian:
Firewall stopped and disabled on system startup

yitian@heron01:~$ ./zookeeper/zookeeper-3.4.10/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /home/yitian/zookeeper/zookeeper-3.4.10/bin/../conf/zoo.cfg
Mode: follower

Slave（heron02）启动zookeeper

yitian@heron02:~$ ./zookeeper/zookeeper-3.4.10/bin/zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /home/yitian/zookeeper/zookeeper-3.4.10/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED

yitian@heron02:~$ ./zookeeper/zookeeper-3.4.10/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /home/yitian/zookeeper/zookeeper-3.4.10/bin/../conf/zoo.cfg
Error contacting service. It is probably not running.

yitian@heron02:~$ jps
3026 Jps
2971 QuorumPeerMain

yitian@heron02:~$ sudo ufw disable
[sudo] password for yitian:
Firewall stopped and disabled on system startup

yitian@heron02:~$ ./zookeeper/zookeeper-3.4.10/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /home/yitian/zookeeper/zookeeper-3.4.10/bin/../conf/zoo.cfg
Mode: leader

Slave（heron03）启动zookeeper

yitian@heron03:~$ ./zookeeper/zookeeper-3.4.10/bin/zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /home/yitian/zookeeper/zookeeper-3.4.10/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
 
yitian@heron03:~$ ./zookeeper/zookeeper-3.4.10/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /home/yitian/zookeeper/zookeeper-3.4.10/bin/../conf/zoo.cfg
Mode: follower

启动HDFS

Master（heron01）主机上启动HDFS

yitian@heron01:~$ ./hadoop/hadoop-2.7.4/sbin/start-dfs.sh
Starting namenodes on [heron01]
heron01: starting namenode, logging to /home/yitian/hadoop/hadoop-2.7.4/logs/hadoop-yitian-namenode-heron01.out
heron02: starting datanode, logging to /home/yitian/hadoop/hadoop-2.7.4/logs/hadoop-yitian-datanode-heron02.out
heron03: starting datanode, logging to /home/yitian/hadoop/hadoop-2.7.4/logs/hadoop-yitian-datanode-heron03.out
Starting secondary namenodes [heron01]
heron01: starting secondarynamenode, logging to /home/yitian/hadoop/hadoop-2.7.4/logs/hadoop-yitian-secondarynamenode-heron01.out

Master（heron01）主机上查看运行状态

yitian@heron01:~$ jps
6834 SecondaryNameNode
6613 NameNode
5913 QuorumPeerMain
7983 Jps

heron01浏览器查看：heron01:50070

Slave（heron02，heron03）主机上查看运行状态

yitian@heron02:~$ jps
3322 DataNode
3450 Jps
2971 QuorumPeerMain

启动Mesos集群

在Master（heron01）主机上运行Mesos集群（root用户）

root@heron01:/home/yitian# ./mesosinstall/sbin/mesos-start-cluster.sh
Starting mesos-master on heron01
ssh -o StrictHostKeyChecking=no -o ConnectTimeout=2 heron01 /home/yitian/mesosinstall/sbin/mesos-daemon.sh mesos-master </dev/null >/dev/null
Starting mesos-agent on heron02
ssh -o StrictHostKeyChecking=no -o ConnectTimeout=2 heron02 /home/yitian/mesosinstall/sbin/mesos-daemon.sh mesos-agent </dev/null >/dev/null
Starting mesos-agent on heron03
ssh -o StrictHostKeyChecking=no -o ConnectTimeout=2 heron03 /home/yitian/mesosinstall/sbin/mesos-daemon.sh mesos-agent </dev/null >/dev/null
Everything's started!

Master（heron01）查看mesos-master运行状态

yitian@ubuntu:~$ ps -e |grep mesos
   4062 ?        00:00:15 mesos-master

Slave（heron02，heron03）查看mesos-slave运行状态

yitian@ubuntu:~$ ps -e |grep mesos
   2308 ?        00:00:08 mesos-agent

heron01浏览器查看：heron01:5050

查看/启动Aurora Scheduler运行状态

Master（heron01）查看Aurora运行状态

yitian@ubuntu:~$ service aurora-scheduler status
● aurora-scheduler.service - Aurora Scheduler
    Loaded: loaded (/lib/systemd/system/aurora-scheduler.service; enabled; vendor
    Active: active (running) since Mon 2018-02-19 01:10:26 PST; 59s ago
  Main PID: 2315 (java)
    CGroup: /system.slice/aurora-scheduler.service
            └─2315 java -Djava.library.path=/usr/local/lib -classpath /usr/share/

Master（heron01）启动Aurora Scheduler

service aurora-scheduler start

Master（heron01）浏览器查看：heron01:8081

Master（heron01）浏览器查看：heron01:5050

启动Heron Tracker

Master（heron01）启动heron-tracker

yitian@ubuntu:~$ heron-tracker
[2018-02-19 01:14:53 +0000] [INFO]: Connecting to zk hostports: [('heron01', 2181)] rootpath: /home/yitian/heron/state
[2018-02-19 01:14:53 +0000] [INFO]: Connecting to heron01:2181
[2018-02-19 01:14:53 +0000] [INFO]: Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=10000, session_id=0, passwd='\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None)
[2018-02-19 01:14:53 +0000] [INFO]: Zookeeper connection established, state: CONNECTED
[2018-02-19 01:14:53 +0000] [INFO]: Sending request(xid=1): Exists(path='/home/yitian/heron/state/topologies', watcher=None)
[2018-02-19 01:14:53 +0000] [INFO]: Received response(xid=1): ZnodeStat(czxid=143, mzxid=143, ctime=1518514432729, mtime=1518514432729, version=0, cversion=7, aversion=0, ephemeralOwner=0, dataLength=0, numChildren=1, pzxid=3682)
[2018-02-19 01:14:53 +0000] [INFO]: Adding children watch for path: /home/yitian/heron/state/topologies
[2018-02-19 01:14:53 +0000] [INFO]: Sending request(xid=2): GetChildren(path='/home/yitian/heron/state/topologies', watcher=<bound method ChildrenWatch._watcher of <kazoo.recipe.watchers.ChildrenWatch object at 0x7f8ad45cfb10>>)
[2018-02-19 01:14:53 +0000] [INFO]: Received response(xid=2): [u'WordCountTopology']
[2018-02-19 01:14:53 +0000] [INFO]: State watch triggered for topologies.
[2018-02-19 01:14:53 +0000] [INFO]: Adding new topology: WordCountTopology, state_manager: zk
[2018-02-19 01:14:53 +0000] [INFO]: Registering a watch with uid: 134b02ac-c216-4ed9-a39f-27a7677b69fd
[2018-02-19 01:14:53 +0000] [INFO]: No execution state found for: WordCountTopology
[2018-02-19 01:14:53 +0000] [INFO]: Adding data watch for path: /home/yitian/heron/state/pplans/WordCountTopology
[2018-02-19 01:14:53 +0000] [INFO]: Sending request(xid=3): GetData(path=u'/home/yitian/heron/state/pplans/WordCountTopology', watcher=<bound method DataWatch._watcher of <kazoo.recipe.watchers.DataWatch object at 0x7f8ad45cfc50>>)
[2018-02-19 01:14:53 +0000] [INFO]: Received error(xid=3) NoNodeError((), {})
[2018-02-19 01:14:53 +0000] [INFO]: Sending request(xid=4): Exists(path=u'/home/yitian/heron/state/pplans/WordCountTopology', watcher=<bound method DataWatch._watcher of <kazoo.recipe.watchers.DataWatch object at 0x7f8ad45cfc50>>)
[2018-02-19 01:14:53 +0000] [INFO]: Watch triggered for topology pplan: WordCountTopology
[2018-02-19 01:14:53 +0000] [INFO]: No execution state found for: WordCountTopology
[2018-02-19 01:14:53 +0000] [INFO]: Adding data watch for path: /home/yitian/heron/state/executionstate/WordCountTopology
[2018-02-19 01:14:53 +0000] [INFO]: Sending request(xid=5): GetData(path=u'/home/yitian/heron/state/executionstate/WordCountTopology', watcher=<bound method DataWatch._watcher of <kazoo.recipe.watchers.DataWatch object at 0x7f8ad45cfd90>>)
[2018-02-19 01:14:53 +0000] [INFO]: Received response(xid=5): ('\n\x11WordCountTopology\x125WordCountTopologycdd8eed3-0fe0-4276-90f6-c7b4bf04edf0\x18\x8c\xb8\xa6\xd4\x05"\x06yitian*\x06aurora2\x05devel:\x06yitianB\x1a\n\x0erelease-agent1\x12\x00\x1a\x060.17.1', ZnodeStat(czxid=3684, mzxid=3684, ctime=1518967820958, mtime=1518967820958, version=0, cversion=0, aversion=0, ephemeralOwner=0, dataLength=139, numChildren=0, pzxid=3684))
[2018-02-19 01:14:53 +0000] [INFO]: Watch triggered for topology execution state: WordCountTopology
[2018-02-19 01:14:53 +0000] [INFO]: Setting topology info for topology: WordCountTopology
[2018-02-19 01:14:53 +0000] [INFO]: Adding data watch for path: /home/yitian/heron/state/tmasters/WordCountTopology
[2018-02-19 01:14:53 +0000] [INFO]: Sending request(xid=6): GetData(path=u'/home/yitian/heron/state/tmasters/WordCountTopology', watcher=<bound method DataWatch._watcher of <kazoo.recipe.watchers.DataWatch object at 0x7f8ad45cff90>>)
[2018-02-19 01:14:53 +0000] [INFO]: Received error(xid=6) NoNodeError((), {})
[2018-02-19 01:14:53 +0000] [INFO]: Sending request(xid=7): Exists(path=u'/home/yitian/heron/state/tmasters/WordCountTopology', watcher=<bound method DataWatch._watcher of <kazoo.recipe.watchers.DataWatch object at 0x7f8ad45cff90>>)
[2018-02-19 01:14:53 +0000] [INFO]: Watch triggered for topology tmaster: WordCountTopology
[2018-02-19 01:14:53 +0000] [INFO]: Setting topology info for topology: WordCountTopology
[2018-02-19 01:14:53 +0000] [INFO]: Adding data watch for path: /home/yitian/heron/state/schedulers/WordCountTopology
[2018-02-19 01:14:53 +0000] [INFO]: Sending request(xid=8): GetData(path=u'/home/yitian/heron/state/schedulers/WordCountTopology', watcher=<bound method DataWatch._watcher of <kazoo.recipe.watchers.DataWatch object at 0x7f8ad467e090>>)
[2018-02-19 01:14:53 +0000] [INFO]: Received error(xid=8) NoNodeError((), {})
[2018-02-19 01:14:53 +0000] [INFO]: Sending request(xid=9): Exists(path=u'/home/yitian/heron/state/schedulers/WordCountTopology', watcher=<bound method DataWatch._watcher of <kazoo.recipe.watchers.DataWatch object at 0x7f8ad467e090>>)
[2018-02-19 01:14:53 +0000] [INFO]: Watch triggered for topology scheduler location: WordCountTopology
[2018-02-19 01:14:53 +0000] [INFO]: Setting topology info for topology: WordCountTopology
[2018-02-19 01:14:53 +0000] [INFO]: Tracker has started
[2018-02-19 01:14:53 +0000] [INFO]: Running on port: 8888
[2018-02-19 01:14:53 +0000] [INFO]: Using config file: /home/yitian/.herontools/conf/heron_tracker.yaml
[2018-02-19 01:14:53 +0000] [INFO]: Using state manager:
     type: zookeeper
     name: zk
     hostport: heron01:2181
     rootpath: /home/yitian/heron/state
     tunnelhost: 127.0.0.1

浏览器查看

启动Heron UI

Master启动heron-ui

yitian@ubuntu:~$ heron-ui
[2018-02-19 01:14:59 +0000] [INFO]: Listening at http://0.0.0.0:8889
[2018-02-19 01:14:59 +0000] [INFO]: Using tracker url: http://127.0.0.1:8888
[2018-02-19 01:14:59 +0000] [INFO]: Using base url:
[2018-02-19 01:14:59 +0000] [INFO]: static/

Ethan_pika

关注

0
点赞
踩
0

收藏

觉得还不错? 一键收藏
0
评论
启动Heron集群的常用命令

关闭防火墙yitian@heron01:~$ sudo ufw disable[sudo] password for yitian:Firewall stopped and disabled on system startup查看/启动ZooKeeper运行状态Master（heron01）启动zookeeperyitian@heron01:~$ ./zookeeper/...
复制链接

扫一扫