Hadoop集群环境搭建中一个错误的解决方案

最新推荐文章于 2020-05-03 19:32:31 发布
最新推荐文章于 2020-05-03 19:32:31 发布
阅读量146 收藏
点赞数
分类专栏： Hadoop综合 文章标签： Hadoop Apache Java JVM Debian

本文转自我的ChinaUnix博客： http://blog.chinaunix.net/u3/107162/showart_2204785.html

2010-03-29 

环境已经搭好并可以启动了，如下：
  1. maohong@maohong-desktop:~/Software/Development/Hadoop/hadoop-0.20.2$ bin/start-all.sh
  2. starting namenode, logging to /home/maohong/Software/Development/Hadoop/hadoop-0.20.2/bin/../logs/hadoop-maohong-namenode-maohong-desktop.out
  3. slave1: starting datanode, logging to /home/maohong/Software/Development/Hadoop/hadoop-0.20.2/bin/../logs/hadoop-maohong-datanode-debian.out
  4. slave2: starting datanode, logging to /home/maohong/Software/Development/Hadoop/hadoop-0.20.2/bin/../logs/hadoop-maohong-datanode-node2.out
  5. master: starting datanode, logging to /home/maohong/Software/Development/Hadoop/hadoop-0.20.2/bin/../logs/hadoop-maohong-datanode-maohong-desktop.out
  6. master: starting secondarynamenode, logging to /home/maohong/Software/Development/Hadoop/hadoop-0.20.2/bin/../logs/hadoop-maohong-secondarynamenode-maohong-desktop.out
  7. starting jobtracker, logging to /home/maohong/Software/Development/Hadoop/hadoop-0.20.2/bin/../logs/hadoop-maohong-jobtracker-maohong-desktop.out
  8. slave1: starting tasktracker, logging to /home/maohong/Software/Development/Hadoop/hadoop-0.20.2/bin/../logs/hadoop-maohong-tasktracker-debian.out
  9. slave2: starting tasktracker, logging to /home/maohong/Software/Development/Hadoop/hadoop-0.20.2/bin/../logs/hadoop-maohong-tasktracker-node2.out
  10. master: starting tasktracker, logging to /home/maohong/Software/Development/Hadoop/hadoop-0.20.2/bin/../logs/hadoop-maohong-tasktracker-maohong-desktop.out
  11. maohong@maohong-desktop:~/Software/Development/Hadoop/hadoop-0.20.2$ jps
  12. 22565 SecondaryNameNode
  13. 22646 JobTracker
  14. 22342 DataNode
  15. 22907 Jps
  16. 22115 NameNode
  17. 22861 TaskTracker


但是执行wordcount程序到时候出现Error: java.lang.NullPointerException 错误如下：

  1. maohong@maohong-desktop:~/Software/Development/Hadoop/hadoop-0.20.2$ bin/hadoop jar hadoop-0.20.2-examples.jar wordcount test-in test-out
  2. 10/03/25 19:40:05 INFO input.FileInputFormat: Total input paths to process : 4
  3. 10/03/25 19:40:05 INFO mapred.JobClient: Running job: job_201003251936_0001
  4. 10/03/25 19:40:06 INFO mapred.JobClient:  map 0% reduce 0%
  5. 10/03/25 19:40:13 INFO mapred.JobClient:  map 50% reduce 0%
  6. 10/03/25 19:40:14 INFO mapred.JobClient:  map 100% reduce 0%
  7. 10/03/25 19:40:21 INFO mapred.JobClient: Task Id : attempt_201003251936_0001_r_000000_0, Status : FAILED
  8. Error: java.lang.NullPointerException
  9.         at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
  10.         at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.getMapCompletionEvents(ReduceTask.java:2683)
  11.         at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.run(ReduceTask.java:2605)

  13. 10/03/25 19:40:21 WARN mapred.JobClient: Error reading task outputhttp://localhost:50060/tasklog?plaintext=true&taskid=attempt_201003251936_0001_r_000000_0&filter=stdout
  14. 10/03/25 19:40:21 WARN mapred.JobClient: Error reading task outputhttp://localhost:50060/tasklog?plaintext=true&taskid=attempt_201003251936_0001_r_000000_0&filter=stderr
  15. 10/03/25 19:40:27 INFO mapred.JobClient: Task Id : attempt_201003251936_0001_r_000000_1, Status : FAILED
  16. Error: java.lang.NullPointerException
  17.         at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
  18.         at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.getMapCompletionEvents(ReduceTask.java:2683)
  19.         at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.run(ReduceTask.java:2605)

  21. 10/03/25 19:40:27 WARN mapred.JobClient: Error reading task outputnode2.1036dhcp
  22. 10/03/25 19:40:27 WARN mapred.JobClient: Error reading task outputnode2.1036dhcp
  23. 10/03/25 19:40:36 INFO mapred.JobClient: Task Id : attempt_201003251936_0001_r_000000_2, Status : FAILED
  24. Error: java.lang.NullPointerException
  25.         at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
  26.         at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.getMapCompletionEvents(ReduceTask.java:2683)
  27.         at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.run(ReduceTask.java:2605)

  29. 10/03/25 19:40:45 INFO mapred.JobClient: Job complete: job_201003251936_0001
  30. 10/03/25 19:40:45 INFO mapred.JobClient: Counters: 12
  31. 10/03/25 19:40:45 INFO mapred.JobClient:   Job Counters
  32. 10/03/25 19:40:45 INFO mapred.JobClient:     Launched reduce tasks=4
  33. 10/03/25 19:40:45 INFO mapred.JobClient:     Launched map tasks=4
  34. 10/03/25 19:40:45 INFO mapred.JobClient:     Data-local map tasks=4
  35. 10/03/25 19:40:45 INFO mapred.JobClient:     Failed reduce tasks=1
  36. 10/03/25 19:40:45 INFO mapred.JobClient:   FileSystemCounters
  37. 10/03/25 19:40:45 INFO mapred.JobClient:     HDFS_BYTES_READ=8637
  38. 10/03/25 19:40:45 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=11495
  39. 10/03/25 19:40:45 INFO mapred.JobClient:   Map-Reduce Framework
  40. 10/03/25 19:40:45 INFO mapred.JobClient:     Combine output records=900
  41. 10/03/25 19:40:45 INFO mapred.JobClient:     Map input records=83
  42. 10/03/25 19:40:45 INFO mapred.JobClient:     Spilled Records=900
  43. 10/03/25 19:40:45 INFO mapred.JobClient:     Map output bytes=14697
  44. 10/03/25 19:40:45 INFO mapred.JobClient:     Combine input records=1525
  45. 10/03/25 19:40:45 INFO mapred.JobClient:     Map output records=1525
  46. maohong@maohong-desktop:~/Software/Development/Hadoop/hadoop-0.20.2$


jobtracker的log文件如下：
  1. 2010-03-25 19:40:09,447 INFO org.apache.hadoop.mapred.JobInProgress: Choosing data-local task task_201003251936_0001_m_000003
  2. 2010-03-25 19:40:12,268 INFO org.apache.hadoop.mapred.JobInProgress: Task 'attempt_201003251936_0001_m_000000_0' has completed task_201003251936_0001_m_000000 successfully.
  3. 2010-03-25 19:40:12,268 INFO org.apache.hadoop.mapred.ResourceEstimator: completedMapsUpdates:1  completedMapsInputSize:4275  completedMapsOutputSize:5190
  4. 2010-03-25 19:40:12,271 INFO org.apache.hadoop.mapred.JobInProgress: Task 'attempt_201003251936_0001_m_000001_0' has completed task_201003251936_0001_m_000001 successfully.
  5. 2010-03-25 19:40:12,271 INFO org.apache.hadoop.mapred.ResourceEstimator: completedMapsUpdates:2  completedMapsInputSize:5745  completedMapsOutputSize:7302
  6. 2010-03-25 19:40:12,288 INFO org.apache.hadoop.mapred.JobTracker: Adding task 'attempt_201003251936_0001_r_000000_0' to tip task_201003251936_0001_r_000000, for tracker 'tracker_localhost:localhost/127.0.0.1:38831'
  7. 2010-03-25 19:40:12,522 INFO org.apache.hadoop.mapred.JobInProgress: Task 'attempt_201003251936_0001_m_000002_0' has completed task_201003251936_0001_m_000002 successfully.
  8. 2010-03-25 19:40:12,522 INFO org.apache.hadoop.mapred.ResourceEstimator: completedMapsUpdates:3  completedMapsInputSize:7215  completedMapsOutputSize:9414
  9. 2010-03-25 19:40:12,524 INFO org.apache.hadoop.mapred.JobInProgress: Task 'attempt_201003251936_0001_m_000003_0' has completed task_201003251936_0001_m_000003 successfully.
  10. 2010-03-25 19:40:12,524 INFO org.apache.hadoop.mapred.ResourceEstimator: completedMapsUpdates:4  completedMapsInputSize:8641  completedMapsOutputSize:11367
  11. 2010-03-25 19:40:18,300 INFO org.apache.hadoop.mapred.TaskInProgress: Error from attempt_201003251936_0001_r_000000_0: Error: java.lang.NullPointerException
  12.         at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
  13.         at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.getMapCompletionEvents(ReduceTask.java:2683)
  14.         at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.run(ReduceTask.java:2605)

  16. 2010-03-25 19:40:18,301 INFO org.apache.hadoop.mapred.JobTracker: Adding task (cleanup)'attempt_201003251936_0001_r_000000_0' to tip task_201003251936_0001_r_000000, for tracker 'tracker_localhost:localhost/127.0.0.1:38831'
  17. 2010-03-25 19:40:21,307 INFO org.apache.hadoop.mapred.JobTracker: Removed completed task 'attempt_201003251936_0001_r_000000_0' from 'tracker_localhost:localhost/127.0.0.1:38831'
  18. 2010-03-25 19:40:21,559 INFO org.apache.hadoop.mapred.JobTracker: Adding task 'attempt_201003251936_0001_r_000000_1' to tip task_201003251936_0001_r_000000, for tracker 'tracker_node2.1036dhcp:localhost/127.0.0.1:59187'
  19. 2010-03-25 19:40:24,599 INFO org.apache.hadoop.mapred.TaskInProgress: Error from attempt_201003251936_0001_r_000000_1: Error: java.lang.NullPointerException
  20.         at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
  21.         at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.getMapCompletionEvents(ReduceTask.java:2683)
  22.         at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.run(ReduceTask.java:2605)

  24. 2010-03-25 19:40:24,600 INFO org.apache.hadoop.mapred.JobTracker: Adding task (cleanup)'attempt_201003251936_0001_r_000000_1' to tip task_201003251936_0001_r_000000, for tracker 'tracker_node2.1036dhcp:localhost/127.0.0.1:59187'
  25. 2010-03-25 19:40:27,607 INFO org.apache.hadoop.mapred.JobTracker: Removed completed task 'attempt_201003251936_0001_r_000000_1' from 'tracker_node2.1036dhcp:localhost/127.0.0.1:59187'
  26. 2010-03-25 19:40:30,201 INFO org.apache.hadoop.mapred.JobTracker: Adding task 'attempt_201003251936_0001_r_000000_2' to tip task_201003251936_0001_r_000000, for tracker 'tracker_maohong-desktop:localhost/127.0.0.1:60931'
  27. 2010-03-25 19:40:33,260 INFO org.apache.hadoop.mapred.TaskInProgress: Error from attempt_201003251936_0001_r_000000_2: Error: java.lang.NullPointerException
  28.         at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
  29.         at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.getMapCompletionEvents(ReduceTask.java:2683)
  30.         at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.run(ReduceTask.java:2605)

  32. 2010-03-25 19:40:33,261 INFO org.apache.hadoop.mapred.JobTracker: Adding task (cleanup)'attempt_201003251936_0001_r_000000_2' to tip task_201003251936_0001_r_000000, for tracker 'tracker_maohong-desktop:localhost/127.0.0.1:60931'
  33. 2010-03-25 19:40:36,266 INFO org.apache.hadoop.mapred.JobTracker: Adding task 'attempt_201003251936_0001_r_000000_3' to tip task_201003251936_0001_r_000000, for tracker 'tracker_maohong-desktop:localhost/127.0.0.1:60931'
  34. 2010-03-25 19:40:36,266 INFO org.apache.hadoop.mapred.JobTracker: Removed completed task 'attempt_201003251936_0001_r_000000_2' from 'tracker_maohong-desktop:localhost/127.0.0.1:60931'
  35. 2010-03-25 19:40:39,270 INFO org.apache.hadoop.mapred.TaskInProgress: Error from attempt_201003251936_0001_r_000000_3: Error: java.lang.NullPointerException
  36.         at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
  37.         at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.getMapCompletionEvents(ReduceTask.java:2683)
  38.         at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.run(ReduceTask.java:2605)

  40. 2010-03-25 19:40:39,271 INFO org.apache.hadoop.mapred.JobTracker: Adding task (cleanup)'attempt_201003251936_0001_r_000000_3' to tip task_201003251936_0001_r_000000, for tracker 'tracker_maohong-desktop:localhost/127.0.0.1:60931'
  41. 2010-03-25 19:40:42,278 INFO org.apache.hadoop.mapred.TaskInProgress: TaskInProgress task_201003251936_0001_r_000000 has failed 4 times.
  42. 2010-03-25 19:40:42,278 INFO org.apache.hadoop.mapred.JobInProgress: Aborting job job_201003251936_0001
  43. 2010-03-25 19:40:42,279 INFO org.apache.hadoop.mapred.JobInProgress: Killing job 'job_201003251936_0001'
  44. 2010-03-25 19:40:42,279 INFO org.apache.hadoop.mapred.JobTracker: Adding task 'attempt_201003251936_0001_m_000004_0' to tip task_201003251936_0001_m_000004, for tracker 'tracker_maohong-desktop:localhost/127.0.0.1:60931'
  45. 2010-03-25 19:40:42,279 INFO org.apache.hadoop.mapred.JobTracker: Removed completed task 'attempt_201003251936_0001_r_000000_3' from 'tracker_maohong-desktop:localhost/127.0.0.1:60931'
  46. 2010-03-25 19:40:45,288 INFO org.apache.hadoop.mapred.JobInProgress: Task 'attempt_201003251936_0001_m_000004_0' has completed task_201003251936_0001_m_000004 successfully.
  47. 2010-03-25 19:40:45,333 INFO org.apache.hadoop.mapred.JobTracker: Removed completed task 'attempt_201003251936_0001_m_000004_0' from 'tracker_maohong-desktop:localhost/127.0.0.1:60931'
  48. 2010-03-25 19:40:45,333 INFO org.apache.hadoop.mapred.JobTracker: Removed completed task 'attempt_201003251936_0001_r_000000_2' from 'tracker_maohong-desktop:localhost/127.0.0.1:60931'
  49. 2010-03-25 19:40:45,334 INFO org.apache.hadoop.mapred.JobTracker: Removed completed task 'attempt_201003251936_0001_r_000000_3' from 'tracker_maohong-desktop:localhost/127.0.0.1:60931'
  50. 2010-03-25 19:40:45,334 INFO org.apache.hadoop.mapred.JobTracker: Removed completed task 'attempt_201003251936_0001_m_000000_0' from 'tracker_localhost:localhost/127.0.0.1:38831'
  51. 2010-03-25 19:40:45,334 INFO org.apache.hadoop.mapred.JobTracker: Removed completed task 'attempt_201003251936_0001_m_000001_0' from 'tracker_localhost:localhost/127.0.0.1:38831'
  52. 2010-03-25 19:40:45,334 INFO org.apache.hadoop.mapred.JobTracker: Removed completed task 'attempt_201003251936_0001_m_000005_0' from 'tracker_localhost:localhost/127.0.0.1:38831'
  53. 2010-03-25 19:40:45,335 INFO org.apache.hadoop.mapred.JobTracker: Removed completed task 'attempt_201003251936_0001_r_000000_0' from 'tracker_localhost:localhost/127.0.0.1:38831'
  54. 2010-03-25 19:40:45,694 INFO org.apache.hadoop.mapred.JobTracker: Removed completed task 'attempt_201003251936_0001_m_000002_0' from 'tracker_node2.1036dhcp:localhost/127.0.0.1:59187'
  55. 2010-03-25 19:40:45,694 INFO org.apache.hadoop.mapred.JobTracker: Removed completed task 'attempt_201003251936_0001_m_000003_0' from 'tracker_node2.1036dhcp:localhost/127.0.0.1:59187'
  56. 2010-03-25 19:40:45,694 INFO org.apache.hadoop.mapred.JobTracker: Removed completed task 'attempt_201003251936_0001_r_000000_1' from 'tracker_node2.1036dhcp:localhost/127.0.0.1:59187'


其中一个tasktracker的log如下，另外两个tasktracker的log也有与此同样的错误：
  1. 2010-03-25 19:40:30,249 INFO org.apache.hadoop.mapred.TaskTracker: Trying to launch : attempt_201003251936_0001_r_000000_2
  2. 2010-03-25 19:40:30,249 INFO org.apache.hadoop.mapred.TaskTracker: In TaskLauncher, current free slots : 2 and trying to launch attempt_201003251936_0001_r_000000_2
  3. 2010-03-25 19:40:30,587 INFO org.apache.hadoop.mapred.JvmManager: In JvmRunner constructed JVM ID: jvm_201003251936_0001_r_1711860611
  4. 2010-03-25 19:40:30,588 INFO org.apache.hadoop.mapred.JvmManager: JVM Runner jvm_201003251936_0001_r_1711860611 spawned.
  5. 2010-03-25 19:40:31,057 INFO org.apache.hadoop.mapred.TaskTracker: JVM with ID: jvm_201003251936_0001_r_1711860611 given task: attempt_201003251936_0001_r_000000_2
  6. 2010-03-25 19:40:31,437 FATAL org.apache.hadoop.mapred.TaskTracker: Task: attempt_201003251936_0001_r_000000_2 - Killed : java.lang.NullPointerException
  7.         at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
  8.         at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.getMapCompletionEvents(ReduceTask.java:2683)
  9.         at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.run(ReduceTask.java:2605)

问题的原因在于master和slave节点的/etc/hosts文件~~~~~~
/etc/hosts文件中的主机名一定要是机器名，而不是master、slave1、slave2，否则不能正确解析。这就是结症所在！
 
改了之后就解决了，呵呵
 
iteye_3065
关注
专栏目录
8.Apache Kylin build cube 报错 Error: java.lang.IllegalArgumentException: Cannot add NaN
终回首的博客
03-12 390
1 问题 构建cube第9步（#9 Step Name: Build N-Dimension Cuboid : level 2）报错Error: java.lang.IllegalArgumentException: Cannot add NaN 2 解决
Hadoop集群搭建错误总结 环境：Centos7
weixin_42871236的博客
08-10 303
1.配置好ip地址以后，端口ens33不显示ip地 解决方法：将BOOTPROTO=none改为BOOTPROTO=dchp 2.network无法重启，提示：Job for network service failed because the control process exited with error code 原因：和NetworkManager服务有冲突 解决方法： 1)关闭N...
hadoop集群搭建常见错误
liuurick的博客
03-27 502
常见错误及解决方案 1.防火墙没关闭、或者没有启动YARN *INFO client.RMProxy: Connecting to ResourceManager at hadoop02/192.168.200.135:8032* 2.主机名称配置错误 3.IP地址配置错误 4.ssh没有配置好 5.多用户启动集群不统一 6.配置文件修改不细心 7.未编译源码 *Unable to load n...
Hadoop启动及关闭
努力是最好的捷径
01-03 2133
启动Hadoop 进到Hadoop的目录例如/opt/hadoop-liang目录下的bin ~$  ./start-all.sh 执行后终端显示如下信息（四个数据节点）： starting namenode, logging to /opt/hadoop-liang/libexec/../logs/hadoop-hadoop-namenode-master.out sl
datanode启动失败
shuideyidi的专栏
11-01 8056
当我动态添加一个hadoop从节点的之后，出现了一个问题： [root@hadoop current]# hadoop-daemon.sh start datanode starting datanode, logging to /usr/local/hadoop1.1/libexec/../logs/hadoop-root-datanode-hadoop.out [root@hadoop 
大数据技术基础实验报告-Linux环境下hadoop集群的搭建与基本配置.doc
08-21
【大数据技术基础实验报告-Linux环境下hadoop...完成上述步骤后，一个基本的Hadoop集群就在Linux环境中搭建起来了，能够进行大数据的存储和处理。在实际应用中，还需考虑高可用性、性能优化以及安全策略等方面的配置。
大数据教程之搭建Hadoop集群.zip_大数据环境搭建hadoop
01-09
1. "hadoop安装实例（原创最终版）.doc"：这可能是一个详细的Hadoop安装步骤文档，包括了从下载源码到编译、安装、配置的全过程，可能还包含了遇到问题时的解决方案。 2. "细细品味Hadoop_Hadoop集群（第1期）_...
HADOOP快速入门及搭建集群环境
10-14
YARN是HADOOP的资源管理器，负责管理HADOOP集群中计算资源的分配。 1.2 HADOOP产生背景 HADOOP的产生背景是大数据时代的到来，随着数据量的急剧增长，传统的数据处理方式无法满足需求，HADOOP的出现解决了这个问题...
解决hadoop集群中有一个slaver的datanode启动失败
yaoqsm的博客
10-12 1061
个人觉得，原因很有可能是master上存在hadoop的临时文件导致的，解决办法就是删除master上的tmp临时文件和所有日志文件。然后重新将master的hadoop文件压缩打包，再传递给slaver。然后将slaver上的旧hadoop删除掉，把传过来的新的hadoop解压安装。重点：这个方法会导致HDFS中存储的所有文件丢失。 代码： 在master上执行下列代码 cd /
编写运行hadoop mapreduce程序
无锡demo的博客
08-14 906
前提： http://blog.csdn.net/wuxidemo/article/details/77115931 设置好分布式部署，并启动 idea开发mapreduce程序： 1. 新建maven project，修改pom.xml增加hadoop相关依赖 xml version="1.0" encoding="UTF-8"?> project xmlns="htt
java.lang.NullPointerException at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.init
freeRoad
12-29 3230
最近在使用Hbase MultitableInputFormat多表输入时发生下面错误： Unable to initialize MapOutputCollector org.apache.hadoop.mapred.MapTask$MapOutputBuffer java.lang.NullPointerException at org.apache.hadoop.mapred.M
配置hadoopHA（高可用集群）常见错误解决办法
盖世英雄来了
03-31 6198
乾坤未定，你我皆是黑马。 在学习hadoop过程中， 1.在启动第二个节点的namenode时候，出现错误。 InconsistentFSStateException:Directory/opt/modules/hadoopha/hadoop-2.5.2/data/tmp/dfs/nameisinaninconsistentstate:st...
hadoop常见错误及解决办法！
热门推荐
沉底的石头
05-30 1万+
1：Shuffle Error: Exceeded MAX_FAILED_UNIQUE_FETCHES; bailing-out  Answer： 程序里面需要打开多个文件，进行分析，系统一般默认数量是1024，（用ulimit -a可以看到）对于正常使用是够了，但是对于程序来讲，就太少了。 修改办法： 修改2个文件。        /etc/security/limits.
解决运行 Hadoop MapReduce 任务时错误: 找不到或无法加载主类
学而不思则忘
09-03 8773
在运行 WordCount 实例时，出现了一个错误：“错误: 找不到或无法加载主类 org.apache.hadoop.mapreduce.v2.app.MRAppMaster”。 搜了很久发现要在 mapred-site.xml 文件中添加 mapreduce 程序所用到的 classpath。 解决方法： 编辑 Hadoop 安装目录下 etc/hadoop/mapred-site.xml 文...
Linux ubuntu-hadoop-2.9.2安装与配置
Vincent_zhang1949的博客
05-03 1036
ubuntu下安装并配置hadoop2.9.2,给出执行./start-all.sh出错的解决办法,ubuntu下使用apt-get安装JDK8并配置环境变量。
hadoop启动后 jps 查不到namenode的解决办法（日志报：ulimit -a for user root）
康师傅没有眼泪
09-25 7314
问题：hadoop启动后 jps 查不到namenode start-dfs.sh [root@master ~]# start-dfs.sh Starting namenodes on [master] master: starting namenode, logging to /usr/local/export/soft/hadoop-2.7.7/logs/hadoop-root-nam...
CentOS环境下Hadoop集群详细搭建指南
这个Hadoop集群搭建教程提供了一个清晰的步骤指南，帮助用户从零开始构建自己的Hadoop环境，为后续的大数据处理和分析奠定了基础。通过实践，用户可以更好地理解和掌握Hadoop的工作原理和集群管理。
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值