HBase2.4.10 踩坑笔记

inch-whf

已于 2022-05-12 11:45:27 修改

阅读量3.5k

点赞数 5

分类专栏：大数据文章标签： java hbase

于 2022-03-21 14:59:21 首次发布

本文链接：https://blog.csdn.net/iku_whf/article/details/123598291

版权

大数据专栏收录该内容

3 篇文章 0 订阅

订阅专栏

安装

目标：安装 HBase2.4.10 ，伪分布式，使用自带的 Zookeeper

参考

确保有 Java 环境

确保有 Hadoop 环境，我的是 Hadoop3.3.1

[root@main ~]# start-all.sh
Starting namenodes on [0.0.0.0]
上一次登录：一 3月 21 18:33:26 CST 2022pts/0 上
Starting datanodes
上一次登录：一 3月 21 18:34:12 CST 2022pts/0 上
Starting secondary namenodes [main]
上一次登录：一 3月 21 18:34:14 CST 2022pts/0 上
Starting resourcemanager
上一次登录：一 3月 21 18:34:23 CST 2022pts/0 上
Starting nodemanagers
上一次登录：一 3月 21 18:34:38 CST 2022pts/0 上

[root@main ~]# jps
81285 NameNode
85095 Jps
82118 SecondaryNameNode
81593 DataNode
83017 NodeManager
82749 ResourceManager

下载并解压压缩包(建议HBASE_HOME=/usr/local/hbase)

添加环境变量

# hbase config
export HBASE_HOME=/usr/local/hbase
export PATH=$PATH:$HBASE_HOME/bin

修改配置文件hbase/conf/hbase-site.xml,hbase/conf/hbase-env.sh，按自己的需求配

hbase/conf/hbase-site.xml

<configuration>
  <property>
      <name>hbase.rootdir</name>
      <value>hdfs://localhost:9000/hbase</value>
      <description>hadoop地址和仓库</description>
  </property>
  <property>
    <name>hbase.cluster.distributed</name>
    <value>true</value>
    <description>是否使用分布式模式（是）</description>
  </property>
  
  <property>
    <name>hbase.tmp.dir</name>
    <value>./tmp</value>
    <description>临时文件位置</description>
  </property>

  <property>
    <name>hbase.unsafe.stream.capability.enforce</name>
    <value>false</value>
    <description>是否使用本地文件跑hbase（否，言下之意就是使用hdfs）</description>
  </property>

  <property>
    <name>hbase.zookeeper.quorum</name>
    <value>localhost:2181</value>
    <description>Zookeeper的节点host:port</description>
  </property>
</configuration>

hbase/conf/hbase-env.sh


export JAVA_HOME=/usr/lib/jvm/java

export HBASE_PID_DIR=/var/hbase/pids

# hbase启动时，同时启动自带的zookeeper
export HBASE_MANAGES_ZK=true

export HBASE_DISABLE_HADOOP_CLASSPATH_LOOKUP="true"

常用命令
```
# 启动
start-hbase.sh
# 关闭
stop-hbase.sh
# 查询
hbase shell
```
有任何异常记得查看日志文件，在$HBASE_HOME/logs！！！（有master，zookeeper，regionsever）

正常结果

[root@main ~]# start-hbase.sh
localhost: running zookeeper, logging to /usr/local/hbase/bin/../logs/hbase-root-zookeeper-main.out
running master, logging to /usr/local/hbase/logs/hbase-root-master-main.out
: running regionserver, logging to /usr/local/hbase/logs/hbase-root-regionserver-main.out

[root@main ~]# jps
14917 Jps
3478 SecondaryNameNode
4119 ResourceManager
4391 NodeManager
2664 NameNode
5609 HMaster
5753 HRegionServer
5450 HQuorumPeer
2939 DataNode


[root@main ~]# hbase shell
2022-03-21 19:35:24,229 WARN  [main] util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
HBase Shell
Use "help" to get list of supported commands.
Use "exit" to quit this interactive shell.
For Reference, please visit: http://hbase.apache.org/2.0/book.html#shell
Version 2.4.10, r3e5359c73d1a96dd7d2ac5bc8f987e9a89ef90ea, Mon Feb 28 10:03:15 PST 2022
Took 0.0069 seconds                                                                                                                                                                    
hbase:001:0> create 'user','name','age','addr','phone','email'
Created table user
Took 2.7152 seconds                                                                                                                                                                    
=> Hbase::Table - user
hbase:002:0> list
TABLE                                                                                                                                                                                  
user                                                                                                                                                                                   
1 row(s)
Took 0.0979 seconds                                                                                                                                                                    
=> ["user"]

[root@main ~]# stop-hbase.sh
stopping hbase...............
localhost: running zookeeper, logging to /usr/local/hbase/bin/../logs/hbase-root-zookeeper-main.out
localhost: stopping zookeeper.

Java远程开发

配置IP映射，所有那个远程服务器可能的名称全都配上，包括但不限于 [域名，服务器的原始主机名，自定义的主机名（hadoop，hbase）]

114.55.58.223 hadoop hbase zookeeper Master iZbp18y7b5jm99960ajdloZ
第一个是域名，最后一个是主机名，Master是HBase的机器名
将远程服务器上的localhost尽可能替换掉
找到一个教程，林子雨大数据课程文件，点HBase
新建Maven项目，导包，写 ↑ 里的代码并理解，运行查看结果（遇到没有任何输出但不报错，记得打日志）

遇到的问题

执行stop-hbase.sh命令，一直...；执行hbase shell命令，报错Server is not running yet

[root@main ~]# stop-hbase.sh
stopping hbase... # 一直.

hbase shell
# 省略一些启动日志，只留最后一句
hbase:001:0> create 'user','name','age','addr','phone','email'
# 报错
ERROR: org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet
        at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2821)
        at org.apache.hadoop.hbase.master.MasterRpcServices.isMasterRunning(MasterRpcServices.java:1205)
        at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
        at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:392)
        at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:133)
        at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:356)
        at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:336)

For usage try 'help "create"'

查看日志hbase/logs/hbase-root-master-main.out，后搜索

	2022-03-19 22:57:46,728 WARN  [RS-EventLoopGroup-3-1] concurrent.DefaultPromise: An exception was thrown by org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete()
java.lang.IllegalArgumentException: object is not an instance of declaring class
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.<init>(ProtobufDecoder.java:69)

原因：（我的）Hadoop3.3.1 与 HBase2.4.10 有一点不兼容

解决办法：hbase/conf/hbase-env.sh取消注释export HBASE_DISABLE_HADOOP_CLASSPATH_LOOKUP="true"

hbase shell执行命令，报错ERROR: KeeperErrorCode = ConnectionLoss for /hbase/master

[root@main ~]# stop-hbase.sh
no hbase master found

[root@main ~]# hbase shell
# 省略一些启动日志，只留最后一句
hbase:001:0> create 'user','name','age','addr','phone','email'
# 报错
ERROR: KeeperErrorCode = ConnectionLoss for /hbase/master
For usage try 'help "create"'

原因：Hadoop处于安全模式
解决办法：执行hdfs dfsadmin -safemode leave关闭安全模式，可能继续报错(下一个)

（这个问题请忽略）start-hbase.sh时报错，日志包冲突

[root@main ~]# start-hbase.sh
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/hbase/lib/client-facing-thirdparty/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
running master, logging to /usr/local/hbase/logs/hbase-root-master-main.out

原因：日志包冲突
解决：把HBase的改名，

[root@main ~]# cd /usr/local/hbase/lib/client-facing-thirdparty
[root@main client-facing-thirdparty]# mv slf4j-log4j12-1.7.30.jar slf4j-log4j12-1.7.30.jar.bak

hadoop与zookeeper连接异常
表现：进入http://hbase:16010/zk.jsp，最后一行为 stat is not executed because it is not in the whitelist.，（最前面的字符可能为另外的4字符命令）
原因：命令不在zookeeper白名单中
解决办法：使用自己的zookeeper，zookeeper配置文件为zoo.cfg（注意最后一行）

zoo.cfg
```
clientPort=2181 
dataDir=/data 
dataLogDir=/data/log 
tickTime=2000 
initLimit=5 
syncLimit=2 
autopurge.snapRetainCount=3 
autopurge.purgeInterval=0 
maxClientCnxns=60 
server.1=whfc.cc:2888:3888
4lw.commands.whitelist=*
```
Java远程操作HBase和Hadoop，admin.tableExists(tableName)语句不报错也不输出
原因：IP映射没配置