hbase全分布式遇到的问题3--集群中有没有配置好ip映射的节点

最新推荐文章于 2024-04-27 11:14:02 发布
最新推荐文章于 2024-04-27 11:14:02 发布
阅读量1.9k 收藏
点赞数
分类专栏： hbase 文章标签： hbase
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/beijihukk/article/details/61962104
版权

现象：启动start-abase.sh后过一小段时间，所有的hmaster和regionserver进程全部自动死掉。

这个问题因为hmaster和hregionserver进程都死掉，一直以为是什么别的原因，也没有耐心去查看日志，花了很多时间瞎弄，后来无意间才发现我这有两个节点根本无法解析另一个节点的主机名（hadoop.lsd4.com),才导致这样的问题，贴一下日志：

2017-03-13 09:18:41,194 INFO  [regionserver/hadoop.lsd2.com/192.168.56.12:16020] zookeeper.ZooKeeper: Initiating client connection, connectString=hadoop.lsd1.com:2181,hadoop.lsd2.com:2181,hadoop.lsd3.com:2181,hadoop.lsd4.com:2181 sessionTimeout=90000 watcher=regionserver:160200x0, quorum=hadoop.lsd1.com:2181,hadoop.lsd2.com:2181,hadoop.lsd3.com:2181,hadoop.lsd4.com:2181, baseZNode=/hbase

2017-03-13 09:18:41,195 WARN  [regionserver/hadoop.lsd2.com/192.168.56.12:16020] zookeeper.RecoverableZooKeeper: Unable to create ZooKeeper Connection

java.net.UnknownHostException: hadoop.lsd4.com

at java.net.InetAddress.getAllByName0(InetAddress.java:1259)

at java.net.InetAddress.getAllByName(InetAddress.java:1171)

at java.net.InetAddress.getAllByName(InetAddress.java:1105)

at org.apache.zookeeper.client.StaticHostProvider.<init>(StaticHostProvider.java:61)

at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:445)

at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:380)

at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.checkZk(RecoverableZooKeeper.java:141)

at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.delete(RecoverableZooKeeper.java:178)

at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1236)

at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1225)

at org.apache.hadoop.hbase.regionserver.HRegionServer.deleteMyEphemeralNode(HRegionServer.java:1416)

at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1090)

at java.lang.Thread.run(Thread.java:745)

2017-03-13 09:18:42,206 INFO  [regionserver/hadoop.lsd2.com/192.168.56.12:16020] zookeeper.ZooKeeper: Initiating client connection, connectString=hadoop.lsd1.com:2181,hadoop.lsd2.com:2181,hadoop.lsd3.com:2181,hadoop.lsd4.com:2181 sessionTimeout=90000 watcher=regionserver:160200x0, quorum=hadoop.lsd1.com:2181,hadoop.lsd2.com:2181,hadoop.lsd3.com:2181,hadoop.lsd4.com:2181, baseZNode=/hbase

2017-03-13 09:18:42,207 WARN  [regionserver/hadoop.lsd2.com/192.168.56.12:16020] zookeeper.RecoverableZooKeeper: Unable to create ZooKeeper Connection

java.net.UnknownHostException: hadoop.lsd4.com

at java.net.InetAddress.getAllByName0(InetAddress.java:1259)

at java.net.InetAddress.getAllByName(InetAddress.java:1171)

at java.net.InetAddress.getAllByName(InetAddress.java:1105)

at org.apache.zookeeper.client.StaticHostProvider.<init>(StaticHostProvider.java:61)

at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:445)

at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:380)

at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.checkZk(RecoverableZooKeeper.java:141)

at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.delete(RecoverableZooKeeper.java:178)

at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1236)

at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1225)

at org.apache.hadoop.hbase.regionserver.HRegionServer.deleteMyEphemeralNode(HRegionServer.java:1416)

at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1090)

at java.lang.Thread.run(Thread.java:745)

2017-03-13 09:18:44,207 INFO  [regionserver/hadoop.lsd2.com/192.168.56.12:16020] zookeeper.ZooKeeper: Initiating client connection, connectString=hadoop.lsd1.com:2181,hadoop.lsd2.com:2181,hadoop.lsd3.com:2181,hadoop.lsd4.com:2181 sessionTimeout=90000 watcher=regionserver:160200x0, quorum=hadoop.lsd1.com:2181,hadoop.lsd2.com:2181,hadoop.lsd3.com:2181,hadoop.lsd4.com:2181, baseZNode=/hbase

2017-03-13 09:18:44,208 WARN  [regionserver/hadoop.lsd2.com/192.168.56.12:16020] zookeeper.RecoverableZooKeeper: Unable to create ZooKeeper Connection

java.net.UnknownHostException: hadoop.lsd4.com

at java.net.InetAddress.getAllByName0(InetAddress.java:1259)

at java.net.InetAddress.getAllByName(InetAddress.java:1171)

at java.net.InetAddress.getAllByName(InetAddress.java:1105)

at org.apache.zookeeper.client.StaticHostProvider.<init>(StaticHostProvider.java:61)

at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:445)

at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:380)

at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.checkZk(RecoverableZooKeeper.java:141)

at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.delete(RecoverableZooKeeper.java:178)

at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1236)

at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1225)

at org.apache.hadoop.hbase.regionserver.HRegionServer.deleteMyEphemeralNode(HRegionServer.java:1416)

at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1090)

at java.lang.Thread.run(Thread.java:745)

2017-03-13 09:18:48,208 INFO  [regionserver/hadoop.lsd2.com/192.168.56.12:16020] zookeeper.ZooKeeper: Initiating client connection, connectString=hadoop.lsd1.com:2181,hadoop.lsd2.com:2181,hadoop.lsd3.com:2181,hadoop.lsd4.com:2181 sessionTimeout=90000 watcher=regionserver:160200x0, quorum=hadoop.lsd1.com:2181,hadoop.lsd2.com:2181,hadoop.lsd3.com:2181,hadoop.lsd4.com:2181, baseZNode=/hbase

2017-03-13 09:18:48,209 WARN  [regionserver/hadoop.lsd2.com/192.168.56.12:16020] zookeeper.RecoverableZooKeeper: Unable to create ZooKeeper Connection

java.net.UnknownHostException: hadoop.lsd4.com: unknown error

at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)

at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:907)

at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1302)

at java.net.InetAddress.getAllByName0(InetAddress.java:1255)

at java.net.InetAddress.getAllByName(InetAddress.java:1171)

at java.net.InetAddress.getAllByName(InetAddress.java:1105)

at org.apache.zookeeper.client.StaticHostProvider.<init>(StaticHostProvider.java:61)

at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:445)

at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:380)

at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.checkZk(RecoverableZooKeeper.java:141)

at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.delete(RecoverableZooKeeper.java:178)

at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1236)

at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1225)

at org.apache.hadoop.hbase.regionserver.HRegionServer.deleteMyEphemeralNode(HRegionServer.java:1416)

at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1090)

at java.lang.Thread.run(Thread.java:745)

2017-03-13 09:18:56,210 INFO  [regionserver/hadoop.lsd2.com/192.168.56.12:16020] zookeeper.ZooKeeper: Initiating client connection, connectString=hadoop.lsd1.com:2181,hadoop.lsd2.com:2181,hadoop.lsd3.com:2181,hadoop.lsd4.com:2181 sessionTimeout=90000 watcher=regionserver:160200x0, 

2017-03-13 09:18:56,211 ERROR [regionserver/hadoop.lsd2.com/192.168.56.12:16020] zookeeper.RecoverableZooKeeper: ZooKeeper delete failed after 4 attempts

2017-03-13 09:18:56,212 WARN  [regionserver/hadoop.lsd2.com/192.168.56.12:16020] regionserver.HRegionServer: Failed deleting my ephemeral node

org.apache.zookeeper.KeeperException$OperationTimeoutException: KeeperErrorCode = OperationTimeout

at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.checkZk(RecoverableZooKeeper.java:144)

at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.delete(RecoverableZooKeeper.java:178)

at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1236)

at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1225)

at org.apache.hadoop.hbase.regionserver.HRegionServer.deleteMyEphemeralNode(HRegionServer.java:1416)

at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1090)

at java.lang.Thread.run(Thread.java:745)

2017-03-13 09:18:56,213 INFO  [regionserver/hadoop.lsd2.com/192.168.56.12:16020] regionserver.HRegionServer: stopping server hadoop.lsd2.com,16020,1489411066501; zookeeper connection closed.

2017-03-13 09:18:56,213 INFO  [regionserver/hadoop.lsd2.com/192.168.56.12:16020] regionserver.HRegionServer: regionserver/hadoop.lsd2.com/192.168.56.12:16020 exiting

2017-03-13 09:18:56,225 ERROR [main] regionserver.HRegionServerCommandLine: Region server exiting

java.lang.RuntimeException: HRegionServer Aborted

at org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:68)

at org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:87)

at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)

at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126)

at org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.java:2677)

[root@hadoop logs]# 

其实回头来看，如果仔细查看日志的话，也不难找出问题，我这里是hadoop.lsd2.com/hadoop.lsd3.com两个节点的/etc/hosts文件中没有配置好hadoop.lsd4.com的映射（应该是以前做别的试验删掉了没及时还原），导致在通信的时候无法解析域名。

解决方法：重新把主机名映射写的最全的节点的/etc/hosts文件拷贝到各节点，保证每个节点的主机名都能解析，再重启集群。

多多-江北残刀
关注
  • 0
    点赞
  • 踩
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
专栏目录
【Hbase】【CDH】hbase shell 登陆报错：zookeeper.RecoverableZooKeeper: ZooKeeper exists failed after 4 attempt
mukvintt的博客
06-09 1366
前言 在Cloudera manager 环境中，给某一个client节点上添加一个新的hbase gateway。 但是也就是这个时候，在client节点上执行命令 [root@client ~]# hbase shell 提示文章标题的错误. Description 在安装了hbase client上 gateway ，进行登陆hbase，发现以下报错： [root@client ~]# hbase shell 2020-06-09 16:36:59,057 INFO [main]
ZooKeeper报错调试
01-20
NoNode for &hellip; 报错原因：找不到/zk-test-ephemeral-0000000001这个节点 解决：重新创建或者重新连接服务器，排查时因为不存在节点还是服务器连接失败 KeeperErrorCode = NoChildrenForEphemerals for &hellip; 报错原因：父节点是临时节点时不能创建子节点 解决：更改父节点的CreateMode KeeperErrorCode = Directory not empty for &hellip; 报错原因：父节点包含子节点，无法删除 解决：删除叶子叶子节点之后再删除 KeeperErrorCode = NodeExists for &hellip; 报错
hbase 12.1 regionserver 宕机报错记录——1
qq_23280769的专栏
07-10 6806
"Append_min" : 0,     "Append_max" : 0,     "Append_mean" : 0.0,     "Append_median" : 0.0,     "Append_75th_percentile" : 0.0,     "Append_90th_percentile" : 0.0,     "Append_95th_percentile" :
使用ElasticJob时报错:OperationTimeoutException: KeeperErrorCode = OperationTimeout
最新发布
qq_42068856的博客
04-27 510
最近在搭建ElasticJob使用环境时发现一个报错，提示zookeeper连接失败。明明yml里配置的zookeeper地址是正确的，但就是连不上。查阅相关文档和各种尝试后终于找到了问题原因。原来是因为默认连接zookeeper的时间太短了，zookeeper还没响应这边就提示报错。加上重试的间隔时间配置后，重启项目就可以正常启动了。以下是本例使用的依赖版本。
ERROR [main] zookeeper.RecoverableZooKeeper: ZooKeeper exists failed after 4 attempts
我的博客
09-02 1万+
运行hbase shell命令时报错： 2018-09-02 13:50:33,994 ERROR [main] zookeeper.RecoverableZooKeeper: ZooKeeper exists failed after 4 attempts 2018-09-02 13:50:33,995 WARN [main] zookeeper.ZKUtil: hconnection-...
zookeeper.RecoverableZooKeeper: ZooKeeper create failed after 4 attempts，Master exiting
qq_43701760的博客
05-08 3934
as closed socket, closing socket connection and attempting reconnect 2019-05-08 09:52:34,896 ERROR [main] zookeeper.RecoverableZooKeeper: ZooKeeper create failed after 4 attempts 2019-05-08 09:52:34,8...
Apache Hadoop---HBase.docx
06-11
Apache HBase是一个基于Hadoop的分布式列族数据库，它针对大规模结构化数据提供高可靠性、高性能、可伸缩的实时读写访问。HBase的设计灵感来源于Google的BigTable，采用了一个增强的稀疏排序映射表的数据模型，键由...
HBase官方文档中文版-HBase手册中文版
12-02
HBase是一种分布式、基于列族的NoSQL数据库，由Apache软件基金会开发并维护，是Hadoop生态系统中的重要组件。这份“HBase官方文档中文版”提供了全面深入的HBase知识，帮助用户理解和掌握如何在大数据场景下有效地...
hbase-1.1.3-bin.tar.gz
05-09
描述中提到"在Vmware中创建3台虚拟机master、slave、slave1，部署hadoop集群"，这意味着我们将进行一个典型的Hadoop分布式环境的搭建，其中包含一个主节点（Master）和两个工作节点（Slaves）。Hadoop集群的架构包括...
hbase-1.2.0-cdh5.12.0.tar.gz
04-24
在CDH 5.12.0环境下部署HBase 1.2.0，需要配置Hadoop集群，设置HBase的相关参数，如Master节点、RegionServer、Zookeeper等。同时，监控和调优是保证HBase高效运行的关键，包括Region分布、内存使用、延时问题等。 ...
Impala-Kudu-HBase-Spark安装文档
07-24
【Impala-Kudu-HBase-Spark安装文档】是关于在分布式计算环境中集成多个组件的详细指南，主要包括四个关键部分：Impala、Kudu、HBase和Spark。这些组件都是大数据处理和分析的重要工具，用于构建高效的数据仓库和...
zookeeper报错OperationTimeoutException: KeeperErrorCode = OperationTimeout解决方案
热门推荐
cucgyfjklx的博客
04-15 2万+
zookeeper报错exception.RegException: org.apache.zookeeper.KeeperException$OperationTimeoutException: KeeperErrorCode = OperationTimeout解决方案
hbase shell报错:zookeeper.RecoverableZooKeeper: ZooKeeper exists failed after 4 attempts
普通Gopher
11-04 3515
2019-11-04 16:03:02,954 ERROR [main] zookeeper.RecoverableZooKeeper: ZooKeeper exists failed after 4 attempts 2019-11-04 16:03:02,956 WARN [main] zookeeper.ZKUtil: hconnection-0x7412ed6b0x0, quorum=n...
【转载】elastic-job链接zookeeper报错KeeperErrorCode = OperationTimeout
weixin_38251332的博客
12-06 2232
org.apache.zookeeper.KeeperException$OperationTimeoutException: KeeperErrorCode = OperationTimeout 报错新同事本地启项目，springboot启动报这个错，原文如下: 排查过程： 网上有说防火墙的(试了不行)；有说zk版本的(这边还是第三方包间接依赖的，且有包冲突，试了去掉一个没好使);解决方案： 修改了超时时间 本地启动OK原文链接
Hbase由于GC时间过长导致 Zookeeper认为其死亡 节点自动关闭
Poche.Long的博客
05-29 4625
Hbase由于GC时间过长导致 Zookeeper认为其死亡 节点自动关闭 目录 Hbase由于GC时间过长导致 Zookeeper认为其死亡 节点自动关闭 目录 日志 官方解释 配置添加 后记 记录Hbase regionserver 经常死亡的原因。 日志 日志内容如下： 2018-05-29 10:04:20,809 ERROR [regionserv...
hbase依赖zk集群
大JAVA解决方案
02-11 832
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)2018-02-11 09:35:15,973 INFO  [main-SendThread(centoshostnameKL1:2181)] zookeeper.ClientCnxn: Opening socket connection to serve...
【集群问题解决】Hbase的"Failed deleting my ephemeral node"错误解决
terryliu98的专栏
09-26 6514
HBase的一个RegionServer莫名其妙的挂了，重启
hbase的regionserver宕机
Ran
01-30 1714
hbase的regionserver宕机
HBase：高可靠分布式列式数据库
HBase有三种运行模式：单机模式、伪分布式模式和分布式模式，以适应不同的部署环境。 访问接口： HBase提供了多种访问接口供用户操作数据，包括Java API、HBase Shell（基于JRuby的命令行工具）以及通过REST、...

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值