Troubleshooting: java.lang.RuntimeException: HMaster Aborted

Trouble

When reinstall Cloudera cluster in the same group of nodes, you must delete the file locates at  /opt/cloudera/ from all nodes, or there will be following error when the  first run command try to start the HMaster instance:

Master server abort: loaded coprocessors are: []

Unhandled exception. Starting shutdown. org.apache.hadoop.hbase.TableExistsException: hbase:namespace at org.apache.hadoop.hbase.master.handler.CreateTableHandler.prepare(CreateTableHandler.java:120) at org.apache.hadoop.hbase.master.TableNamespaceManager.createNamespaceTable(TableNamespaceManager.java:230) at org.apache.hadoop.hbase.master.TableNamespaceManager.start(TableNamespaceManager.java:85) at org.apache.hadoop.hbase.master.HMaster.initNamespace(HMaster.java:1059) at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:920) at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:606) at java.lang.Thread.run(Thread.java:744)

Aborting

Stopping server on 60000

CatalogJanitor-sdc-r0-00:60000 exiting

RpcServer.handler=7,port=60000: exiting

Replication.RpcServer.handler=1,port=60000: exiting

RpcServer.handler=10,port=60000: exiting

RpcServer.handler=12,port=60000: exiting

Replication.RpcServer.handler=0,port=60000: exiting

RpcServer.handler=19,port=60000: exiting

RpcServer.handler=8,port=60000: exiting

RpcServer.handler=5,port=60000: exiting

RpcServer.handler=1,port=60000: exiting

RpcServer.handler=23,port=60000: exiting

RpcServer.listener,port=60000: stopping

Replication.RpcServer.handler=2,port=60000: exiting

RpcServer.handler=16,port=60000: exiting

RpcServer.handler=6,port=60000: exiting

RpcServer.handler=3,port=60000: exiting

sdc-r0-00,60000,1403595631803-BalancerChore exiting

sdc-r0-00,60000,1403595631803-ClusterStatusChore exiting

RpcServer.handler=17,port=60000: exiting

RpcServer.handler=14,port=60000: exiting

RpcServer.handler=13,port=60000: exiting

RpcServer.handler=21,port=60000: exiting

RpcServer.handler=24,port=60000: exiting

RpcServer.responder: stopped

master:sdc-r0-00:60000.archivedHFileCleaner exiting

master:sdc-r0-00:60000.oldLogCleaner exiting

RpcServer.handler=18,port=60000: exiting

Stopping infoServer

RpcServer.handler=11,port=60000: exiting

RpcServer.handler=15,port=60000: exiting

RpcServer.handler=9,port=60000: exiting

RpcServer.handler=20,port=60000: exiting

RpcServer.handler=2,port=60000: exiting

RpcServer.handler=22,port=60000: exiting

RpcServer.handler=0,port=60000: exiting

RpcServer.handler=4,port=60000: exiting

Stopping replicationLogCleaner-0x246cc103e39101f, quorum=sdc-r0-01:2181,sdc-r0-00:2181,sdc-r0-02:2181, baseZNode=/hbase

RpcServer.responder: stopping

Stopped SelectChannelConnector@0.0.0.0:60010

Session: 0x246cc103e39101f closed

EventThread shut down

Closing zookeeper sessionid=0x346cc103e561088

Session: 0x346cc103e561088 closed

EventThread shut down

sdc-r0-00,60000,1403595631803.splitLogManagerTimeoutMonitor exiting

Session: 0x246cc103e39101e closed

EventThread shut down

HMaster main thread exiting

Master exiting java.lang.RuntimeException: HMaster Aborted at org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:192) at org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:134) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126) at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2789)

Troubleshooting

Run the following commands from from one of the servers running an HBase service to do the troubleshooting[1]:

hbasezkcli #enter the ZooKeeper CLI

rmr/hbase #to remove the znode

---

[1] https://community.cloudera.com/t5/Storage-Random-Access-HDFS/HMaster-not-starting-TableExistsException/td-p/11466