Zookeeper Error: Heron Failed to set packing plan for topology ‘WordCountTopology’

最新推荐文章于 2024-08-21 13:43:48 发布

Ethan_pika

最新推荐文章于 2024-08-21 13:43:48 发布

阅读量234

点赞数

分类专栏：流式计算Heron Zookeeper 文章标签： Heron Zookeeper

本文链接：https://blog.csdn.net/yitian_z/article/details/102932404

版权

流式计算Heron 同时被 2 个专栏收录

50 篇文章 0 订阅

订阅专栏

Zookeeper

4 篇文章 0 订阅

订阅专栏

问题描述

https://stackoverflow.com/questions/50675906/heron-failed-to-set-packing-plan-for-topology-wordcounttopology

问题解决

出现问题的原因为：集群中zookeeper集群配置出现问题，只使用了standalone的方式配置了zookeeper，重新在三个节点中使用正确的zookeeper并启动后，问题解决。

三台zookeeper主机中的zoo.cfg文件中配置为：

该问题解决之后，重新提交topo:

yitian@heron01:~$ heron submit aurora/yitian/devel --config-path ~/.heron/conf ~/.heron/examples/heron-api-examples.jar com.twitter.heron.examples.api.WordCountTopology WordCountTopology --deploy-deactivated
[2018-06-05 04:44:48 +0000] [INFO]: Using cluster definition in /home/yitian/.heron/conf/aurora
[2018-06-05 04:44:48 +0000] [INFO]: Launching topology: 'WordCountTopology'
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/yitian/.heron/lib/uploader/heron-dlog-uploader.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/yitian/.heron/lib/statemgr/heron-zookeeper-statemgr.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.JDK14LoggerFactory]
[2018-06-05 04:44:50 -0700] [INFO] com.twitter.heron.statemgr.zookeeper.curator.CuratorStateManager: Starting Curator client connecting to: heron01:2181  
[2018-06-05 04:44:50 -0700] [INFO] org.apache.curator.framework.imps.CuratorFrameworkImpl: Starting  
[2018-06-05 04:44:50 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT  
[2018-06-05 04:44:50 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Client environment:host.name=heron01  
[2018-06-05 04:44:50 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Client environment:java.version=1.8.0_151  
[2018-06-05 04:44:50 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation  
[2018-06-05 04:44:50 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Client environment:java.home=/usr/java/jdk1.8.0_151/jre  
[2018-06-05 04:44:50 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Client environment:java.class.path=:/home/yitian/.heron/lib/scheduler/heron-scheduler.jar:/home/yitian/.heron/lib/scheduler/heron-binpacking-packing.jar:/home/yitian/.heron/lib/scheduler/heron-yarn-scheduler.jar:/home/yitian/.heron/lib/scheduler/heron-slurm-scheduler.jar:/home/yitian/.heron/lib/scheduler/heron-mesos-scheduler.jar:/home/yitian/.heron/lib/scheduler/heron-marathon-scheduler.jar:/home/yitian/.heron/lib/scheduler/heron-kubernetes-scheduler.jar:/home/yitian/.heron/lib/scheduler/heron-roundrobin-packing.jar:/home/yitian/.heron/lib/scheduler/heron-local-scheduler.jar:/home/yitian/.heron/lib/scheduler/heron-aurora-scheduler.jar:/home/yitian/.heron/lib/uploader/heron-localfs-uploader.jar:/home/yitian/.heron/lib/uploader/heron-hdfs-uploader.jar:/home/yitian/.heron/lib/uploader/heron-null-uploader.jar:/home/yitian/.heron/lib/uploader/heron-scp-uploader.jar:/home/yitian/.heron/lib/uploader/heron-s3-uploader.jar:/home/yitian/.heron/lib/uploader/heron-gcs-uploader.jar:/home/yitian/.heron/lib/uploader/heron-dlog-uploader.jar:/home/yitian/.heron/lib/statemgr/heron-localfs-statemgr.jar:/home/yitian/.heron/lib/statemgr/heron-zookeeper-statemgr.jar:/home/yitian/.heron/lib/packing/heron-binpacking-packing.jar:/home/yitian/.heron/lib/packing/heron-roundrobin-packing.jar  
[2018-06-05 04:44:50 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib  
[2018-06-05 04:44:50 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp  
[2018-06-05 04:44:50 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Client environment:java.compiler=<NA>  
[2018-06-05 04:44:50 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Client environment:os.name=Linux  
[2018-06-05 04:44:50 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Client environment:os.arch=amd64  
[2018-06-05 04:44:50 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Client environment:os.version=4.10.0-28-generic  
[2018-06-05 04:44:50 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Client environment:user.name=yitian  
[2018-06-05 04:44:50 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Client environment:user.home=/home/yitian  
[2018-06-05 04:44:50 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Client environment:user.dir=/home/yitian  
[2018-06-05 04:44:50 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Initiating client connection, connectString=heron01:2181 sessionTimeout=30000 watcher=org.apache.curator.ConnectionState@2a17b7b6  
[2018-06-05 04:44:50 -0700] [INFO] org.apache.zookeeper.ClientCnxn: Opening socket connection to server heron01/218.195.241.174:2181. Will not attempt to authenticate using SASL (unknown error)  
[2018-06-05 04:44:50 -0700] [INFO] org.apache.zookeeper.ClientCnxn: Socket connection established to heron01/218.195.241.174:2181, initiating session  
[2018-06-05 04:44:50 -0700] [INFO] org.apache.zookeeper.ClientCnxn: Session establishment complete on server heron01/218.195.241.174:2181, sessionid = 0x163cfbe17d2001a, negotiated timeout = 30000  
[2018-06-05 04:44:50 -0700] [INFO] org.apache.curator.framework.state.ConnectionStateManager: State change: CONNECTED  
[2018-06-05 04:44:50 -0700] [INFO] com.twitter.heron.statemgr.zookeeper.curator.CuratorStateManager: Directory tree initialized.  
[2018-06-05 04:44:50 -0700] [INFO] com.twitter.heron.statemgr.zookeeper.curator.CuratorStateManager: Checking existence of path: /heron/topologies/WordCountTopology  
[2018-06-05 04:44:58 -0700] [INFO] com.twitter.heron.uploader.hdfs.HdfsUploader: Target topology file already exists at '/heron/topologies/aurora/WordCountTopology-yitian-tag-0--3752538403818566244.tar.gz'. Overwriting it now  
[2018-06-05 04:44:58 -0700] [INFO] com.twitter.heron.uploader.hdfs.HdfsUploader: Uploading topology package at '/tmp/tmpFHrzGK/topology.tar.gz' to target HDFS at '/heron/topologies/aurora/WordCountTopology-yitian-tag-0--3752538403818566244.tar.gz'  
[2018-06-05 04:45:05 -0700] [INFO] com.twitter.heron.statemgr.zookeeper.curator.CuratorStateManager: Created node for path: /heron/topologies/WordCountTopology  
[2018-06-05 04:45:05 -0700] [INFO] com.twitter.heron.statemgr.zookeeper.curator.CuratorStateManager: Created node for path: /heron/packingplans/WordCountTopology  
[2018-06-05 04:45:05 -0700] [INFO] com.twitter.heron.statemgr.zookeeper.curator.CuratorStateManager: Created node for path: /heron/executionstate/WordCountTopology  
[2018-06-05 04:45:05 -0700] [INFO] com.twitter.heron.scheduler.aurora.AuroraLauncher: Launching topology in aurora  
[2018-06-05 04:45:05 -0700] [INFO] com.twitter.heron.scheduler.utils.SchedulerUtils: Updating scheduled-resource in packing plan: WordCountTopology  
[2018-06-05 04:45:05 -0700] [INFO] com.twitter.heron.statemgr.zookeeper.curator.CuratorStateManager: Deleted node for path: /heron/packingplans/WordCountTopology  
[2018-06-05 04:45:05 -0700] [INFO] com.twitter.heron.statemgr.zookeeper.curator.CuratorStateManager: Created node for path: /heron/packingplans/WordCountTopology  
 INFO] Creating job WordCountTopology
^CERROR] Command interrupted by user

从而验证了新实验室计算机的配置无法搭建heron集群环境。。。