tips:最近生产环境的Hadoop大数据环境经常挂(天天人工重启,手放到生产的机器就会自动敲重启环境的命令,真想铲掉。。。),于是想着是不是某些组件资源不足,于是进行了格式化hadoop之路,格式五分钟,重启三小时
- 版本先亮出来(hadoop-3.2.0 zookeeper-3.4.9 hbase-1.2.4)
格式化的过程就不说了,能走到这一步的,相信我们一样的菜,说说要命的后遗症吧。
- 格式完成之后,hdfs的文件目录会被清除掉,可以选择重建
hadoop fs -ls /sqoop 查看dfs里面的数据文件
- hbase里面的表及结构会被全部清除,创建新表提示旧表已存在:table already exists
1.问题出现:
在格式化NameNode后,重新运行sqoop脚本提示表不存在,于是去hbase创建表提示table already exists
2.原因: hadoop重新格式化后,hdfs上没有了数据,在hbase中新建表却提示Table already exists。虽然HDFS上和Hbase相关的东西都已经删除了。但是zookeeper保存有hbase表的地址,数据访问是通过zookeeper的地址转到hdfs上,因此需要将Zookeeper中的相应的表也删除。
破案方案如下:
进入zookeeper的bin目录:
./zkcli.sh 运行zookeeper的客户端
[hadoop@slave1 bin]$ ./zkCli.sh
Connecting to localhost:2181
2021-12-15 10:08:39,371 [myid:] - INFO [main:Environment@100] - Client environment:zookeeper.version=3.4.9-1757313, built on 08/23/2016 06:50 GMT
2021-12-15 10:08:39,374 [myid:] - INFO [main:Environment@100] - Client environment:host.name=slave1
2021-12-15 10:08:39,375 [myid:] - INFO [main:Environment@100] - Client environment:java.version=1.8.0_131
2021-12-15 10:08:39,377 [myid:] - INFO [main:Environment@100] - Client environment:java.vendor=Oracle Corporation
2021-12-15 10:08:39,377 [myid:] - INFO [main:Environment@100] - Client environment:java.home=/home/hadoop/java/jdk1.8.0_131/jre
2021-12-15 10:08:39,378 [myid:] - INFO [main:Environment@100] - Client environment:java.class.path=/home/hadoop/zookeeper/zookeeper-3.4.9/bin/../build/classes:/home/hadoop/zookeeper/zookeepe
r-3.4.9/bin/../build/lib/*.jar:/home/hadoop/zookeeper/zookeeper-3.4.9/bin/../lib/slf4j-log4j12-1.6.1.jar:/home/hadoop/zookeeper/zookeeper-3.4.9/bin/../lib/slf4j-api-1.6.1.jar:/home/hadoop/zookeeper/zookeeper-3.4.9/bin/../lib/netty-3.10.5.Final.jar:/home/hadoop/zookeeper/zookeeper-3.4.9/bin/../lib/log4j-1.2.16.jar:/home/hadoop/zookeeper/zookeeper-3.4.9/bin/../lib/jline-0.9.94.jar:/home/hadoop/zookeeper/zookeeper-3.4.9/bin/../zookeeper-3.4.9.jar:/home/hadoop/zookeeper/zookeeper-3.4.9/bin/../src/java/lib/*.jar:/home/hadoop/zookeeper/zookeeper-3.4.9/bin/../conf:.:/usr/java/jdk1.8.0_181-cloudera/lib/dt.jar:/usr/java/jdk1.8.0_181-cloudera/lib/tools.jar2021-12-15 10:08:39,378 [myid:] - INFO [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2021-12-15 10:08:39,379 [myid:] - INFO [main:Environment@100] - Client environment:java.io.tmpdir=/tmp
2021-12-15 10:08:39,379 [myid:] - INFO [main:Environment@100] - Client environment:java.compiler=<NA>
2021-12-15 10:08:39,379 [myid:] - INFO [main:Environment@100] - Client environment:os.name=Linux
2021-12-15 10:08:39,379 [myid:] - INFO [main:Environment@100] - Client environment:os.arch=amd64
2021-12-15 10:08:39,379 [myid:] - INFO [main:Environment@100] - Client environment:os.version=3.10.0-693.el7.x86_64
2021-12-15 10:08:39,379 [myid:] - INFO [main:Environment@100] - Client environment:user.name=hadoop
2021-12-15 10:08:39,379 [myid:] - INFO [main:Environment@100] - Client environment:user.home=/home/hadoop
2021-12-15 10:08:39,379 [myid:] - INFO [main:Environment@100] - Client environment:user.dir=/home/hadoop/zookeeper/zookeeper-3.4.9/bin
2021-12-15 10:08:39,381 [myid:] - INFO [main:ZooKeeper@438] - Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWat
cher@277050dcWelcome to ZooKeeper!
2021-12-15 10:08:39,409 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1032] - Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2181. Will not attempt to
authenticate using SASL (unknown error)JLine support is enabled
2021-12-15 10:08:39,500 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@876] - Socket connection established to localhost/0:0:0:0:0:0:0:1:2181, initiating session
2021-12-15 10:08:39,510 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1299] - Session establishment complete on server localhost/0:0:0:0:0:0:0:1:2181, sessionid = 0x1
7db79d3f12001e, negotiated timeout = 30000
WATCHER::
WatchedEvent state:SyncConnected type:None path:null
[zk: localhost:2181(CONNECTED) 0]
ls /hbase/table 列出hbase的所有表
[zk: localhost:2181(CONNECTED) 0] ls /hbase/table
[stockQuotes, hbase:meta, hbase:namespace, omz]
rmr /hbase/table/omz 删除对应表的缓存
[zk: localhost:2181(CONNECTED) 0] ls /hbase/table
[stockQuotes, hbase:meta, hbase:namespace, omz]
[zk: localhost:2181(CONNECTED) 1] rmr /hbase/table/omz
[zk: localhost:2181(CONNECTED) 2] ls /hbase/table
[stockQuotes, hbase:meta, hbase:namespace]
[zk: localhost:2181(CONNECTED) 3]
最后去hbase重新创建表,create ‘omz’, {NAME=>‘info’,VERSIONS => 1}; 发现创建成功。
- 运行sqoop脚本发现报访问不了hbase里面的表,报错截图如下:
掉头发了,明明啥也没动,经过百度 stockflow github只能看见类似的报错,而我一直在纠结could not access Hbase table 这行报错,没注意下面类找不到才是引发这个问题的真正原因,事实证明千万不要钻牛角尖,要不然stockflow也救不了你,要学会和自己和解,于是我喝了半小时的水,饱了之后继续掉头发,忽然发现为啥不能当成一个普通的类找不到的bug来对待呢,对吧,少类那我就把相应的类给导入进来,继续破案。。。
- 找到装有sqoop服务的机器,进入sqoop安装目录下面的lib目录
我的安装目录是 /home/hadoop/sqoop1/sqoop-1.4.6-cdh5.5.2
发现里面并没有报错信息上提示所需要的类所在的jar包,于是去代码里面看下有没有用过这个类,哎,还真有,直接去本地仓库copy出来放到lib目录(实在没有的话去https://mvnrepository.com/下载)
来,最后再运行一次搞到吐的sqoop脚本,熟悉的画面出现了(长叹一口真气),破案了。。。WIN+L下班