常见错误集(持续更新)

1.多次执行hdfs namenode -format命令然后启动start-dfs.sh 然后jps后发现datanode节点(或者其它节点,比如namenode)没有启动，然后去logs目录下查看datanode的日志文件hadoop-hadoop–datanode(或者其它比如namenode).log
发现报如下错： 2019-07-03 12:39:54,640 WARN
org.apache.hadoop.hdfs.server.common.Storage: java.io.IOException:
Incompatible clusterIDs in /tmp/hadoop-hadoop/dfs/data: namenode
clusterID = CID-b8177bab-6539-47dd-aa98-c85618e1b236; datanode
clusterID = CID-2f5d2ee8-3d73-4b13-9938-66dc3bdd1931 2019-07-03
12:39:54,648 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode:
Initialization failed for Block pool (Datanode Uuid
unassigned) service to localhost/127.0.0.1:9000. Exiting.
java.io.IOException: All specified directories are failed to load.
at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:478)
at org.apache.hadoop.hdfs.server.datanode.DataNode.initStorage(DataNode.java:1394)
at org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:1355)
at org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:317)
at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:228)
at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:829)
at java.lang.Thread.run(Thread.java:745) 2019-07-03 12:39:54,650 WARN org.apache.hadoop.hdfs.server.datanode.DataNode:
Ending block pool service for: Block pool (Datanode Uuid
unassigned) service to localhost/127.0.0.1:9000 2019-07-03
12:39:54,660 INFO org.apache.hadoop.hdfs.server.datanode.DataNode:
Removed Block pool (Datanode Uuid unassigned) 2019-07-03
12:39:56,662 WARN org.apache.hadoop.hdfs.server.datanode.DataNode:
Exiting Datanode

解决办法为找到报错中有一个目录输出：/tmp/hadoop-hadoop/dfs/data
(data目录同级目录中应该还有name目录，namesecondary这些)然后里面有一个VERSION文件夹，修改里面的clusterID与其它目录中对应一致，然后再重新启动即可，，，或者还有一种办法，直接删除掉data目录下的所有目录(data目录里面可能有name,data…)，然后重新hdfs
namenode -format格式化，然后重新start-dfs.sh启动即可

2.hive中进行表删除操作时候报错，错误为以下内容：
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:For direct MetaStore DB connections, we don’t support retries at the client level.)
通过查找hive详细日志发现以下错误：
ERROR [main]: metastore.RetryingHMSHandler (RetryingHMSHandler.java:invokeInternal(203)) - Retrying HMSHandler after 2000 ms (attempt 9 of 10) with error: javax.jdo.JDODataStoreException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ‘OPTION SQL_SELECT_LIMIT=DEFAULT’ at line 1

发现是因为hive/lib下的mysql驱动版本过低导致的，so，去mysql官网下载新的mysql驱动包放置hive/lib下即可解决