- 安装完hadoop,格式化之后启动hdfs,datanode不能启动
查看日志:
2017-02-07 14:29:47,741 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting
2017-02-07 14:29:47,758 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 50020: starting
2017-02-07 14:29:53,973 INFO org.apache.hadoop.hdfs.server.common.Storage: Using 1 threads to upgrade data directories (dfs.datanode.paralle
l.volumes.load.threads.num=1, dataDirs=1)
2017-02-07 14:29:54,113 INFO org.apache.hadoop.hdfs.server.common.Storage: Lock on /opt/data/tmp/dfs/data/in_use.lock acquired by nodename 5
400@hadoop-master
2017-02-07 14:29:54,203 WARN org.apache.hadoop.hdfs.server.common.Storage: Failed to add storage directory [DISK]file:/opt/data/tmp/dfs/data
/
java.io.IOException: Incompatible clusterIDs in /opt/data/tmp/dfs/data: namenode clusterID = CID-2ca58eab-b3ef-4f08-b3f9-6246c4d6d0be; datan
ode clusterID = CID-2121b6fc-ca1c-4f87-a700-1f7314390f13
at org.apache.hadoop.hdfs.server.datanode.DataStorage.doTransition(DataStorage.java:775)
at org.apache.hadoop.hdfs.server.datanode.DataStorage.loadStorageDirectory(DataStorage.java:300)
at org.apache.hadoop.hdfs.server.datanode.DataStorage.loadDataStorage(DataStorage.java:416)
at org.apache.hadoop.hdfs.server.datanode.DataStorage.addStorageLocations(DataStorage.java:395)
at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:573)
at org.apache.hadoop.hdfs.server.datanode.DataNode.initStorage(DataNode.java:1362)
at org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:1327)
at org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:317)
at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:223)
at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:802)
at java.lang.Thread.run(Thread.java:745)
2017-02-07 14:29:54,298 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for Block pool (Datanode
Uuid unassigned) service to hadoop-master/192.168.8.88:9000. Exiting.
java.io.IOException: All specified directories are failed to load.
at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:574)
at org.apache.hadoop.hdfs.server.datanode.DataNode.initStorage(DataNode.java:1362)
at org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:1327)
at org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:317)
at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:223)
at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:802)
at java.lang.Thread.run(Thread.java:745)
2017-02-07 14:29:54,298 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Ending block pool service for: Block pool (Datan
ode Uuid unassigned) service to hadoop-master/192.168.8.88:9000
2017-02-07 14:29:54,324 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Removed Block pool (Datanode Uuid unassigned)
2017-02-07 14:29:56,324 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Exiting Datanode
2017-02-07 14:29:56,342 INFO org.apache.hadoop.util.ExitUtil: Exiting with status 0
2017-02-07 14:29:56,377 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG:
造成这个原因的是在安装完hadoop后格式化启动hdfs后,又格式化了一遍hdfs,知识后namenode的clusterID和datanode的clusterID不一致,所以造成不能正常的启动datanode.
解决方法:在配置的hadoop运行的临时文件目录中找到datanode的clusterID目录
/opt/data/tmp/dfs/data/current/VERSION clusterID和/opt/data/tmp/dfs/name/current/VERSION中的clusterID一致就可以了,把namenode中的clusterID拷贝到datanode中的clusterID。 - 开启Hadoop2.6.0出现ssh无法解析主机名等错误提示的解决办法!
报错:
You: ssh: Could not resolve hostname You: Temporary failure in name resolution
warning:: ssh: Could not resolve hostname warning:: Temporary failure in name resolution
VM: ssh: Could not resolve hostname VM: Temporary failure in name resolution
have: ssh: Could not resolve hostname have: Temporary failure in name resolution
library: ssh: Could not resolve hostname library: Temporary failure in name resolution
loaded: ssh: Could not resolve hostname loaded: Temporary failure in name resolution
might: ssh: Could not resolve hostname might: Temporary failure in name resolution
which: ssh: Could not resolve hostname which: Temporary failure in name resolution
have: ssh: Could not resolve hostname have: Temporary failure in name resolution
disabled: ssh: Could not resolve hostname disabled: Temporary failure in name resolution
stack: ssh: Could not resolve hostname stack: Temporary failure in name resolution
guard.: ssh: Could not resolve hostname guard.: Temporary failure in name resolution
VM: ssh: Could not resolve hostname VM: Temporary failure in name resolution
The: ssh: Could not resolve hostname The: Temporary failure in name resolution
try: ssh: Could not resolve hostname try: Temporary failure in name resolution
will: ssh: Could not resolve hostname will: Temporary failure in name resolution
to: ssh: Could not resolve hostname to: Temporary failure in name resolution
fix: ssh: Could not resolve hostname fix: Temporary failure in name resolution
the: ssh: Could not resolve hostname the: Temporary failure in name resolution
省略一大堆 Could not resolve hostname stack: Temporary failure in name resolution
解决办法:
出现上述问题主要是环境变量没设置好,在~/.bash_profile或者/etc/profile中加入以下语句就没问题了
vi /etc/profile或者vi ~/.bash_profile
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib"
然后用source重新编译使之生效即可!
source /etc/profile或者source ~/.bash_profile