Heron Cluster Topology运行及资源参数调整

Topology管理命令

Submit Topology

Activate Topology

yitian@heron04:~$ heron activate aurora/yitian/devel WordCountTopology
[2018-04-07 05:15:28 +0000] [INFO]: Using cluster definition in /home/yitian/.heron/conf/aurora
[2018-04-07 05:15:28 -0700] [INFO] com.twitter.heron.statemgr.zookeeper.curator.CuratorStateManager: Starting Curator client connecting to: heron04:2181 
[2018-04-07 05:15:28 -0700] [INFO] org.apache.curator.framework.imps.CuratorFrameworkImpl: Starting 
[2018-04-07 05:15:29 -0700] [INFO] org.apache.curator.framework.state.ConnectionStateManager: State change: CONNECTED 
[2018-04-07 05:15:29 -0700] [INFO] com.twitter.heron.statemgr.zookeeper.curator.CuratorStateManager: Directory tree initialized. 
[2018-04-07 05:15:29 -0700] [INFO] com.twitter.heron.statemgr.zookeeper.curator.CuratorStateManager: Checking existence of path: /heron/topologies/WordCountTopology 
[2018-04-07 05:15:29 -0700] [INFO] com.twitter.heron.spi.utils.TMasterUtils: Topology command ACTIVATE completed successfully. 
[2018-04-07 05:15:29 -0700] [INFO] com.twitter.heron.statemgr.zookeeper.curator.CuratorStateManager: Closing the CuratorClient to: heron04:2181 
[2018-04-07 05:15:29 -0700] [INFO] com.twitter.heron.statemgr.zookeeper.curator.CuratorStateManager: Closing the tunnel processes 
[2018-04-07 05:15:29 +0000] [INFO]: Successfully activate topology: WordCountTopology

Deactivate Topology

yitian@heron04:~$ heron deactivate aurora/yitian/devel WordCountTopology
[2018-04-07 05:58:03 +0000] [INFO]: Using cluster definition in /home/yitian/.heron/conf/aurora
[2018-04-07 05:58:04 -0700] [INFO] com.twitter.heron.statemgr.zookeeper.curator.CuratorStateManager: Starting Curator client connecting to: heron04:2181 
[2018-04-07 05:58:04 -0700] [INFO] org.apache.curator.framework.imps.CuratorFrameworkImpl: Starting 
[2018-04-07 05:58:04 -0700] [INFO] org.apache.curator.framework.state.ConnectionStateManager: State change: CONNECTED 
[2018-04-07 05:58:04 -0700] [INFO] com.twitter.heron.statemgr.zookeeper.curator.CuratorStateManager: Directory tree initialized. 
[2018-04-07 05:58:04 -0700] [INFO] com.twitter.heron.statemgr.zookeeper.curator.CuratorStateManager: Checking existence of path: /heron/topologies/WordCountTopology 
[2018-04-07 05:58:05 -0700] [INFO] com.twitter.heron.spi.utils.TMasterUtils: Topology command DEACTIVATE completed successfully. 
[2018-04-07 05:58:05 -0700] [INFO] com.twitter.heron.statemgr.zookeeper.curator.CuratorStateManager: Closing the CuratorClient to: heron04:2181 
[2018-04-07 05:58:05 -0700] [INFO] com.twitter.heron.statemgr.zookeeper.curator.CuratorStateManager: Closing the tunnel processes 
[2018-04-07 05:58:05 +0000] [INFO]: Successfully deactivate topology: WordCountTopology

Kill Topology

yitian@heron04:~$ heron kill aurora/yitian/devel WordCountTopology
[2018-04-07 07:35:12 +0000] [INFO]: Using cluster definition in /home/yitian/.heron/conf/aurora
[2018-04-07 07:35:13 -0700] [INFO] com.twitter.heron.statemgr.zookeeper.curator.CuratorStateManager: Starting Curator client connecting to: heron04:2181  
[2018-04-07 07:35:13 -0700] [INFO] org.apache.curator.framework.imps.CuratorFrameworkImpl: Starting  
[2018-04-07 07:35:13 -0700] [INFO] org.apache.curator.framework.state.ConnectionStateManager: State change: CONNECTED  
[2018-04-07 07:35:13 -0700] [INFO] com.twitter.heron.statemgr.zookeeper.curator.CuratorStateManager: Directory tree initialized.  
[2018-04-07 07:35:13 -0700] [INFO] com.twitter.heron.statemgr.zookeeper.curator.CuratorStateManager: Checking existence of path: /heron/topologies/WordCountTopology  
[2018-04-07 07:35:13 -0700] [INFO] com.twitter.heron.scheduler.aurora.AuroraLauncher: Will try 5 attempts at interval: 2000 ms  
 INFO] Killing tasks for job: aurora/yitian/devel/WordCountTopology
 INFO] Instances to be killed: [0, 1]
Successfully killed instances [0, 1]
Job killall succeeded
[2018-04-07 07:35:21 -0700] [INFO] com.twitter.heron.statemgr.zookeeper.curator.CuratorStateManager: Deleted node for path: /heron/packingplans/WordCountTopology  
[2018-04-07 07:35:21 -0700] [INFO] com.twitter.heron.statemgr.zookeeper.curator.CuratorStateManager: Deleted node for path: /heron/pplans/WordCountTopology  
[2018-04-07 07:35:21 -0700] [INFO] com.twitter.heron.statemgr.zookeeper.curator.CuratorStateManager: Deleted node for path: /heron/schedulers/WordCountTopology  
[2018-04-07 07:35:21 -0700] [INFO] com.twitter.heron.statemgr.zookeeper.curator.CuratorStateManager: Deleted node for path: /heron/executionstate/WordCountTopology  
[2018-04-07 07:35:21 -0700] [INFO] com.twitter.heron.statemgr.zookeeper.curator.CuratorStateManager: Deleted node for path: /heron/topologies/WordCountTopology  
[2018-04-07 07:35:21 -0700] [INFO] com.twitter.heron.statemgr.zookeeper.curator.CuratorStateManager: Closing the CuratorClient to: heron04:2181  
[2018-04-07 07:35:21 -0700] [INFO] com.twitter.heron.statemgr.zookeeper.curator.CuratorStateManager: Closing the tunnel processes  
[2018-04-07 07:35:21 +0000] [INFO]: Successfully kill topology: WordCountTopology

Heron Cluster集群运行情况

1. Mesos Cluster

2. Aurora Scheduler

WordCountTopology运行情况

1. Heron UI

2. Heron Topology Config