安装hadoop2.x出现的问题

  1. 安装完hadoop,格式化之后启动hdfs,datanode不能启动
    查看日志:
    2017-02-07 14:29:47,741 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting
    2017-02-07 14:29:47,758 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 50020: starting
    2017-02-07 14:29:53,973 INFO org.apache.hadoop.hdfs.server.common.Storage: Using 1 threads to upgrade data directories (dfs.datanode.paralle
    l.volumes.load.threads.num=1, dataDirs=1)
    2017-02-07 14:29:54,113 INFO org.apache.hadoop.hdfs.server.common.Storage: Lock on /opt/data/tmp/dfs/data/in_use.lock acquired by nodename 5
    400@hadoop-master
    2017-02-07 14:29:54,203 WARN org.apache.hadoop.hdfs.server.common.Storage: Failed to add storage directory [DISK]file:/opt/data/tmp/dfs/data
    /
    java.io.IOException: Incompatible clusterIDs in /opt/data/tmp/dfs/data: namenode clusterID = CID-2ca58eab-b3ef-4f08-b3f9-6246c4d6d0be; datan
    ode clusterID = CID-2121b6fc-ca1c-4f87-a700-1f7314390f13
    at org.apache.hadoop.hdfs.server.datanode.DataStorage.doTransition(DataStorage.java:775)
    at org.apache.hadoop.hdfs.server.datanode.DataStorage.loadStorageDirectory(DataStorage.java:300)
    at org.apache.hadoop.hdfs.server.datanode.DataStorage.loadDataStorage(DataStorage.java:416)
    at org.apache.hadoop.hdfs.server.datanode.DataStorage.addStorageLocations(DataStorage.java:395)
    at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:573)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.initStorage(DataNode.java:1362)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:1327)
    at org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:317)
    at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:223)
    at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:802)
    at java.lang.Thread.run(Thread.java:745)
    2017-02-07 14:29:54,298 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for Block pool (Datanode
    Uuid unassigned) service to hadoop-master/192.168.8.88:9000. Exiting.
    java.io.IOException: All specified directories are failed to load.
    at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:574)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.initStorage(DataNode.java:1362)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:1327)
    at org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:317)
    at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:223)
    at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:802)
    at java.lang.Thread.run(Thread.java:745)
    2017-02-07 14:29:54,298 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Ending block pool service for: Block pool (Datan
    ode Uuid unassigned) service to hadoop-master/192.168.8.88:9000
    2017-02-07 14:29:54,324 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Removed Block pool (Datanode Uuid unassigned)
    2017-02-07 14:29:56,324 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Exiting Datanode
    2017-02-07 14:29:56,342 INFO org.apache.hadoop.util.ExitUtil: Exiting with status 0
    2017-02-07 14:29:56,377 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG:
    造成这个原因的是在安装完hadoop后格式化启动hdfs后,又格式化了一遍hdfs,知识后namenode的clusterID和datanode的clusterID不一致,所以造成不能正常的启动datanode.
    解决方法:在配置的hadoop运行的临时文件目录中找到datanode的clusterID目录
    /opt/data/tmp/dfs/data/current/VERSION clusterID和/opt/data/tmp/dfs/name/current/VERSION中的clusterID一致就可以了,把namenode中的clusterID拷贝到datanode中的clusterID。
  2. 开启Hadoop2.6.0出现ssh无法解析主机名等错误提示的解决办法!
    报错:
    You: ssh: Could not resolve hostname You: Temporary failure in name resolution
    warning:: ssh: Could not resolve hostname warning:: Temporary failure in name resolution
    VM: ssh: Could not resolve hostname VM: Temporary failure in name resolution
    have: ssh: Could not resolve hostname have: Temporary failure in name resolution
    library: ssh: Could not resolve hostname library: Temporary failure in name resolution
    loaded: ssh: Could not resolve hostname loaded: Temporary failure in name resolution
    might: ssh: Could not resolve hostname might: Temporary failure in name resolution
    which: ssh: Could not resolve hostname which: Temporary failure in name resolution
    have: ssh: Could not resolve hostname have: Temporary failure in name resolution
    disabled: ssh: Could not resolve hostname disabled: Temporary failure in name resolution
    stack: ssh: Could not resolve hostname stack: Temporary failure in name resolution
    guard.: ssh: Could not resolve hostname guard.: Temporary failure in name resolution
    VM: ssh: Could not resolve hostname VM: Temporary failure in name resolution
    The: ssh: Could not resolve hostname The: Temporary failure in name resolution
    try: ssh: Could not resolve hostname try: Temporary failure in name resolution
    will: ssh: Could not resolve hostname will: Temporary failure in name resolution
    to: ssh: Could not resolve hostname to: Temporary failure in name resolution
    fix: ssh: Could not resolve hostname fix: Temporary failure in name resolution
    the: ssh: Could not resolve hostname the: Temporary failure in name resolution
    省略一大堆 Could not resolve hostname stack: Temporary failure in name resolution
    解决办法:
    出现上述问题主要是环境变量没设置好,在~/.bash_profile或者/etc/profile中加入以下语句就没问题了
    vi /etc/profile或者vi ~/.bash_profile
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib"

然后用source重新编译使之生效即可!
source /etc/profile或者source ~/.bash_profile

  • 1
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值