storm提交拓扑后出现Executor topology_name not alive的问题

最新推荐文章于 2021-10-07 22:00:47 发布

斗者_2013

最新推荐文章于 2021-10-07 22:00:47 发布

阅读量2.9k

点赞数

分类专栏： Hadoop 文章标签： storm 调试

Hadoop 专栏收录该内容

34 篇文章 1 订阅

订阅专栏

1 bolt不停重启，excutor无法启动。

nimbus日志类似如下（），

2014-03-12 10:55:06 b.s.d.nimbus [INFO] Executor MITAS3-74-1394565794:[34
34] not alive
2014-03-12 10:55:06 b.s.d.nimbus [INFO] Executor MITAS3-74-1394565794:[4 4]
not alive
2014-03-12 10:55:06 b.s.d.nimbus [INFO] Executor MITAS3-74-1394565794:[40
40] not alive
2014-03-12 10:55:06 b.s.d.nimbus [INFO] Executor MITAS3-74-1394565794:[10
10] not alive
2014-03-12 10:55:06 b.s.d.nimbus [INFO] Executor MITAS3-74-1394565794:[16
16] not alive
2014-03-12 10:55:06 b.s.d.nimbus [INFO] Executor MITAS3-74-1394565794:[22
22] not alive
2014-03-12 10:55:06 b.s.d.nimbus [INFO] Executor MITAS3-74-1394565794:[28
28] not alive

2014-03-12 10:55:06 b.s.s.EvenScheduler [INFO] Available slots:
(["5d105f66-1add-421b-8265-e7340a95928c" 6700]
["32ab1745-c260-4491-ae4d-92dcc5d14a62" 6700])
2014-03-12 10:55:06 b.s.d.nimbus [INFO] Reassigning MITAS3-74-1394565794 to
6 slots
2014-03-12 10:55:06 b.s.d.nimbus [INFO] Reassign executors: [[34 34] [4 4]
[40 40] [10 10] [16 16] [22 22] [28 28]]

原因是：work的内存设置太小，导致excutor无法启动

更改方法修改storm work配置，增大内存。

## to worker 
worker.childopts: "-Xmx4096m"

参考：

I have encountered the same issue in a case of out of memory in worker
process. Try increase the memory of the wokers by setting nimbus.childopts
property. Also, if you are creating short living object at higher rate
use +UseG1GC
. Since you are saying that you hold data in your memory, I'm suspecting
(as I said) an OutOfMemeory error. Don't cover this just by increasing the
heap size, but also I recommend to profile your worker and see if you have
a memory leak.
转自：http://qnalist.com/questions/4769895/topology-restart-due-to-executor-not-alive

这是网上的一般答案。

这给我提供了一个方向，那就是内存问题。首先查看虚拟机的内存是多大。分配给nimbus，supervisor，worker三者的内存不能大于虚拟机（或本机的实际内存）。                    Storm中真正干活的是各个worker，而worker由supervisor负责启动。现在我的问题是nimbus和supervisor都可以正常启动。没有报错，而提交拓扑时，出现了如上错误。                                     推断是虚拟机分配完supervisor的内存后，剩余的可用内存不足以分配给worker，导致执行器Executor始终不能激活。

 
 
  
  在配置文件storm.yaml中，有：
 
 
 
 
  
  # to nimbus 
  
  
nimbus.childopts: "-Xmx1024m"
 
 
 
 
  
  # to supervisor 
  
  
supervisor.childopts: "-Xmx1024m"
 
 
 
 
  
  # to worker 
  
  
worker.childopts: "-Xmx768m" 
  
  
如果worker在运行时，需要用指定的JVM参数，那么可以像这样配置：
  
  
worker.childopts: "-Dworker=worker -Xmx768m -Xdebug –Xnoagent -Djava.compiler=NONE -Xrunjdwp:transport=dt_socket,address=8111,suspend=y,server=y "