RegionServer进程启动没几秒就消失，查看日志，得到以下异常，解决方案

于 2019-10-09 17:04:01 发布
阅读量1k 收藏
点赞数 1
分类专栏： 大数据 Hbase 文章标签： Hbase
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/qq_44160357/article/details/102465331
版权

RegionServer进程启动没几秒就消失，查看log日志，发现报错如下：

			org.apache.hadoop.hbase.ClockOutOfSyncException: org.apache.hadoop.hbase.ClockOutOfSyncException: Server linuxfour,16020,1570638266978 has been rejected; Reported time is too far out of sync with master.  Time difference of 9122902ms > max allowed of 30000ms
	at org.apache.hadoop.hbase.master.ServerManager.checkClockSkew(ServerManager.java:342)
	at org.apache.hadoop.hbase.master.ServerManager.regionServerStartup(ServerManager.java:200)
	at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:497)
	at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:11085)
	at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:413)
	at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132)
	at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
	at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)

	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
	at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:99)
	at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:89)
	at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:358)
	at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.getRemoteException(ProtobufUtil.java:335)
	at org.apache.hadoop.hbase.regionserver.HRegionServer.reportForDuty(HRegionServer.java:2594)
	at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:942)
	at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.ClockOutOfSyncException): org.apache.hadoop.hbase.ClockOutOfSyncException: Server linuxfour,16020,1570638266978 has been rejected; Reported time is too far out of sync with master.  Time difference of 9122902ms > max allowed of 30000ms
	at org.apache.hadoop.hbase.master.ServerManager.checkClockSkew(ServerManager.java:342)
	at org.apache.hadoop.hbase.master.ServerManager.regionServerStartup(ServerManager.java:200)
	at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:497)
	at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:11085)
	at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:413)
	at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132)
	at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
	at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)

	at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:387)
	at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:95)
	at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:410)
	at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:406)
	at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:103)
	at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:118)
	at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:162)
	at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:192)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
	at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310)
	at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:284)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
	at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
	at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1359)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:935)
	at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:801)
	at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:404)
	at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:304)
	at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
	at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
	... 1 more
2019-10-10 00:24:38,778 ERROR [regionserver/Linuxfour:16020] regionserver.HRegionServer: erver linuxfour,16020,1570638266978: Unhandled: org.apache.hadoop.hbase.ClockOutOfSyncException: Server linuxfour,16020,1570638266978 has been rejected; Reported time is too far out of sync with master.  Time difference of 9122902ms > max allowed of 30000ms
	at org.apache.hadoop.hbase.master.ServerManager.checkClockSkew(ServerManager.java:342)
	at org.apache.hadoop.hbase.master.ServerManager.regionServerStartup(ServerManager.java:200)
	at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:497)
	at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:11085)
	at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:413)
	at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132)
	at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
	at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
 *****
org.apache.hadoop.hbase.ClockOutOfSyncException: org.apache.hadoop.hbase.ClockOutOfSyncException: Server linuxfour,16020,1570638266978 has been rejected; Reported time is too far out of sync with master.  Time difference of 9122902ms > max allowed of 30000ms
	at org.apache.hadoop.hbase.master.ServerManager.checkClockSkew(ServerManager.java:342)
	at org.apache.hadoop.hbase.master.ServerManager.regionServerStartup(ServerManager.java:200)
	at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:497)
	at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:11085)
	at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:413)
	at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132)
	at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
	at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)

	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
	at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:99)
	at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:89)
	at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:358)
	at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.getRemoteException(ProtobufUtil.java:335)
	at org.apache.hadoop.hbase.regionserver.HRegionServer.reportForDuty(HRegionServer.java:2594)
	at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:942)
	at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.ClockOutOfSyncException): org.apache.hadoop.hbase.ClockOutOfSyncException: Server linuxfour,16020,1570638266978 has been rejected; Reported time is too far out of sync with master.  Time difference of 9122902ms > max allowed of 30000ms
	at org.apache.hadoop.hbase.master.ServerManager.checkClockSkew(ServerManager.java:342)
	at org.apache.hadoop.hbase.master.ServerManager.regionServerStartup(ServerManager.java:200)
	at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:497)
	at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:11085)
	at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:413)
	at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132)
	at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
	at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)

	at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:387)
	at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:95)
	at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:410)
	at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:406)
	at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:103)
	at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:118)
	at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:162)
	at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:192)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
	at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310)
	at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:284)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
	at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
	at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1359)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:935)
	at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:801)
	at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:404)
	at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:304)
	at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
	at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
	... 1 more
2019-10-10 00:24:38,792 ERROR [regionserver/Linuxfour:16020] regionserver.HRegionServer: RegionServer abort: loaded coprocessors are: []
2019-10-10 00:24:38,840 INFO  [regionserver/Linuxfour:16020] regionserver.HRegionServer: Dump of metrics as JSON on abort: {
  "beans" : [ {
    "name" : "java.lang:type=Memory",
    "modelerType" : "sun.management.MemoryImpl",
    "Verbose" : false,
    "NonHeapMemoryUsage" : {
      "committed" : 53018624,
      "init" : 2555904,
      "max" : -1,
      "used" : 51900544
    },
    "ObjectPendingFinalizationCount" : 0,
    "HeapMemoryUsage" : {
      "committed" : 17186816,
      "init" : 16777216,
      "max" : 249430016,
      "used" : 11683712
    },
    "ObjectName" : "java.lang:type=Memory"
  } ],
  "beans" : [ {
    "name" : "Hadoop:service=HBase,name=RegionServer,sub=IPC",
    "modelerType" : "RegionServer,sub=IPC",
    "tag.Context" : "regionserver",
    "tag.Hostname" : "Linuxfour"
  } ],
  "beans" : [ ],
  "beans" : [ ]
}
2019-10-10 00:24:38,903 INFO  [regionserver/Linuxfour:16020] regionserver.HRegionServer: ***** STOPPING region server 'linuxfour,16020,1570638266978' *****
2019-10-10 00:24:38,903 INFO  [regionserver/Linuxfour:16020] regionserver.HRegionServer: STOPPED: Unhandled: org.apache.hadoop.hbase.ClockOutOfSyncException: Server linuxfour,16020,1570638266978 has been rejected; Reported time is too far out of sync with master.  Time difference of 9122902ms > max allowed of 30000ms
	at org.apache.hadoop.hbase.master.ServerManager.checkClockSkew(ServerManager.java:342)
	at org.apache.hadoop.hbase.master.ServerManager.regionServerStartup(ServerManager.java:200)
	at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:497)
	at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:11085)
	at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:413)
	at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132)
	at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
	at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)

2019-10-10 00:24:38,907 INFO  [regionserver/Linuxfour:16020] regionserver.HRegionServer: Stopping infoServer
2019-10-10 00:24:38,935 INFO  [regionserver/Linuxfour:16020] handler.ContextHandler: Stopped o.e.j.w.WebAppContext@2c708440{/,null,UNAVAILABLE}{file:/opt/module/hbase-2.0.6/hbase-webapps/regionserver}
2019-10-10 00:24:38,956 INFO  [regionserver/Linuxfour:16020] server.AbstractConnector: Stopped ServerConnector@7652a589{HTTP/1.1,[http/1.1]}{0.0.0.0:16030}
2019-10-10 00:24:38,957 INFO  [regionserver/Linuxfour:16020] handler.ContextHandler: Stopped o.e.j.s.ServletContextHandler@7551da2a{/static,file:///opt/module/hbase-2.0.6/hbase-webapps/static/,UNAVAILABLE}
2019-10-10 00:24:38,958 INFO  [regionserver/Linuxfour:16020] handler.ContextHandler: Stopped o.e.j.s.ServletContextHandler@250b236d{/logs,file:///opt/module/hbase-2.0.6/logs/,UNAVAILABLE}
2019-10-10 00:24:38,964 INFO  [regionserver/Linuxfour:16020] flush.RegionServerFlushTableProcedureManager: Stopping region server flush procedure manager abruptly.
2019-10-10 00:24:38,965 INFO  [regionserver/Linuxfour:16020] snapshot.RegionServerSnapshotManager: Stopping RegionServerSnapshotManager abruptly.
2019-10-10 00:24:38,965 INFO  [regionserver/Linuxfour:16020] regionserver.HRegionServer: aborting server linuxfour,16020,1570638266978
2019-10-10 00:24:38,965 INFO  [regionserver/Linuxfour:16020] zookeeper.ReadOnlyZKClient: Close zookeeper connection 0x2c58fad1 to 192.168.88.52:2181,192.168.88.53:2181,192.168.88.54:2181
2019-10-10 00:24:38,971 INFO  [ReadOnlyZKClient-192.168.88.52:2181,192.168.88.53:2181,192.168.88.54:2181@0x2c58fad1] zookeeper.ZooKeeper: Session: 0x36db136b4ac0010 closed
2019-10-10 00:24:38,971 INFO  [ReadOnlyZKClient-192.168.88.52:2181,192.168.88.53:2181,192.168.88.54:2181@0x2c58fad1-EventThread] zookeeper.ClientCnxn: EventThread shut down for session: 0x36db136b4ac0010
2019-10-10 00:24:39,004 INFO  [regionserver/Linuxfour:16020] regionserver.HRegionServer: stopping server linuxfour,16020,1570638266978; all regions closed.
2019-10-10 00:24:39,009 INFO  [regionserver/Linuxfour:16020] hbase.ChoreService: Chore service for: regionserver/Linuxfour:16020 had [] on shutdown
2019-10-10 00:24:39,010 INFO  [regionserver/Linuxfour:16020] ipc.NettyRpcServer: Stopping server on /192.168.88.54:16020
2019-10-10 00:24:39,051 INFO  [regionserver/Linuxfour:16020] zookeeper.ZooKeeper: Session: 0x16db0ab5d6e000a closed
2019-10-10 00:24:39,051 INFO  [regionserver/Linuxfour:16020] regionserver.HRegionServer: Exiting; stopping=linuxfour,16020,1570638266978; zookeeper connection closed.
2019-10-10 00:24:39,051 INFO  [main-EventThread] zookeeper.ClientCnxn: EventThread shut down for session: 0x16db0ab5d6e000a
2019-10-10 00:24:39,052 ERROR [main] regionserver.HRegionServerCommandLine: Region server exiting
java.lang.RuntimeException: HRegionServer Aborted
	at org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:67)
	at org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:87)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:149)
	at org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.java:2994)
2019-10-10 00:24:39,065 INFO  [Thread-4] regionserver.ShutdownHook: Shutdown hook starting; hbase.shutdown.hook=true; fsShutdownHook=org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer@1e6b9a95
2019-10-10 00:24:39,066 INFO  [Thread-4] regionserver.ShutdownHook: Starting fs shutdown hook thread.
2019-10-10 00:24:39,074 INFO  [Thread-4] regionserver.ShutdownHook: Shutdown hook finished.

一大推错误， 只要看几个关键的错误：

Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.ClockOutOfSyncException): org.apache.hadoop.hbase.ClockOutOfSyncException: Server linuxfour,16020,1570638266978 has been rejected; Reported time is too far out of sync with master.  Time difference of 9122902ms > max allowed of 30000ms

java.lang.RuntimeException: HRegionServer Aborted

造成的原因是: 时间同步的问题，hbase集群必须要进行时间同步， 否则响应时间超过30000ms时， 节点会自动挂掉
我这个情况是同步的问题

一共四台centos， Linuxone，Linuxtwo， Linuxthre时间是同步， 就Linuxfour时间与前三台不一样。

解决方案：

第一步： 用yum源安装ntp服务。

                                   yum  install  -y  ntp

连上某个时间服务器，同步标准时间

                         ntpdate     ntpdate  ntp1.aliyun.com

注意： 其中ntpdate ntp1.aliyun.com 这个是一个阿里云时间服务器的地址， 连上这个服务器就可以同步时间。

最后重启hbase就可以了。

搞定

你携秋水揽星河
关注
  • 1
    点赞
  • 踩
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
专栏目录
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值