hadoop1.2.1运行wordcount报错

最新推荐文章于 2023-03-02 23:07:40 发布
最新推荐文章于 2023-03-02 23:07:40 发布
阅读量919 收藏 2
点赞数 1
分类专栏： hadoop 文章标签： hadoop ubuntu wordcount
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/u010147288/article/details/50118803
版权

安装步骤参见http://www.tuicool.com/articles/NBvMv2
根据上述网站中的步骤安装好hadoop1.2.1，安装过程中一定要细心，文件位置不要随意放。
我就是文件位置设置同上述网站不一样，导致后续执行wordcount不成功，放出日志如下：

Meta VERSION=”1” .
Job JOBID=”job_201511301937_0004” JOBNAME=”word count” USER=”root” SUBMIT_TIME=”1448884207054” JOBCONF=”hdfs://localhost:9000/user/root/home/paomo/tmp/mapred/staging/root/.staging/job_201511301937_0004/job.xml” VIEW_JOB=”” MODIFY_JOB=”” JOB_QUEUE=”default” WORKFLOW_ID=”” WORKFLOW_NAME=”” WORKFLOW_NODE_NAME=”” WORKFLOW_ADJACENCIES=”” WORKFLOW_TAGS=”” .
Job JOBID=”job_201511301937_0004” JOB_PRIORITY=”NORMAL” .
Job JOBID=”job_201511301937_0004” LAUNCH_TIME=”1448884207728” TOTAL_MAPS=”1” TOTAL_REDUCES=”1” JOB_STATUS=”PREP” .
Task TASKID=”task_201511301937_0004_m_000002” TASK_TYPE=”SETUP” START_TIME=”1448884207752” SPLITS=”” .
MapAttempt TASK_TYPE=”SETUP” TASKID=”task_201511301937_0004_m_000002” TASK_ATTEMPT_ID=”attempt_201511301937_0004_m_000002_0” START_TIME=”0” TRACKER_NAME=”tracker_paomo:localhost/127.0.0.1:51068” HTTP_PORT=”50060” LOCALITY=”OFF_SWITCH” AVATAAR=”VIRGIN” .
MapAttempt TASK_TYPE=”SETUP” TASKID=”task_201511301937_0004_m_000002” TASK_ATTEMPT_ID=”attempt_201511301937_0004_m_000002_0” TASK_STATUS=”FAILED” FINISH_TIME=”0” HOSTNAME=”paomo” ERROR=”Error initializing attempt_201511301937_0004_m_000002_0:
java.io.IOException: Exception reading file:/home/paomo/tmp/mapred/local/ttprivate/taskTracker/root/jobcache/job_201511301937_0004/jobToken
at org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:135)
at org.apache.hadoop.mapreduce.security.TokenCache.loadTokens(TokenCache.java:178)
at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1289)
at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1226)
at org.apache.hadoop.mapred.TaskTracker 5.run(TaskTracker.java:2603)atjava.lang.Thread.run(Thread.java:745)Causedby:java.io.FileNotFoundException:Filefile:/home/paomo/tmp/mapred/local/ttprivate/taskTracker/root/jobcache/job2015113019370004/jobTokendoesnotexist.atorg.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:402)atorg.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:255)atorg.apache.hadoop.fs.ChecksumFileSystem ChecksumFSInputChecker.(ChecksumFileSystem.java:125)
at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:283)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:436)
at org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:129)
… 5 more
” .
MapAttempt TASK_TYPE=”SETUP” TASKID=”task_201511301937_0004_m_000002” TASK_ATTEMPT_ID=”attempt_201511301937_0004_m_000002_1” START_TIME=”0” TRACKER_NAME=”tracker_paomo:localhost/127.0.0.1:51068” HTTP_PORT=”50060” LOCALITY=”OFF_SWITCH” AVATAAR=”VIRGIN” .
MapAttempt TASK_TYPE=”SETUP” TASKID=”task_201511301937_0004_m_000002” TASK_ATTEMPT_ID=”attempt_201511301937_0004_m_000002_1” TASK_STATUS=”FAILED” FINISH_TIME=”0” HOSTNAME=”paomo” ERROR=”Error initializing attempt_201511301937_0004_m_000002_1:
java.io.IOException: Exception reading file:/home/paomo/tmp/mapred/local/ttprivate/taskTracker/root/jobcache/job_201511301937_0004/jobToken
at org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:135)
at org.apache.hadoop.mapreduce.security.TokenCache.loadTokens(TokenCache.java:178)
at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1289)
at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1226)
at org.apache.hadoop.mapred.TaskTracker 5.run(TaskTracker.java:2603)atjava.lang.Thread.run(Thread.java:745)Causedby:java.io.FileNotFoundException:Filefile:/home/paomo/tmp/mapred/local/ttprivate/taskTracker/root/jobcache/job2015113019370004/jobTokendoesnotexist.atorg.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:402)atorg.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:255)atorg.apache.hadoop.fs.ChecksumFileSystem ChecksumFSInputChecker.(ChecksumFileSystem.java:125)
at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:283)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:436)
at org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:129)
… 5 more
” .
MapAttempt TASK_TYPE=”SETUP” TASKID=”task_201511301937_0004_m_000002” TASK_ATTEMPT_ID=”attempt_201511301937_0004_m_000002_2” START_TIME=”0” TRACKER_NAME=”tracker_paomo:localhost/127.0.0.1:51068” HTTP_PORT=”50060” LOCALITY=”OFF_SWITCH” AVATAAR=”VIRGIN” .
MapAttempt TASK_TYPE=”SETUP” TASKID=”task_201511301937_0004_m_000002” TASK_ATTEMPT_ID=”attempt_201511301937_0004_m_000002_2” TASK_STATUS=”FAILED” FINISH_TIME=”0” HOSTNAME=”paomo” ERROR=”Error initializing attempt_201511301937_0004_m_000002_2:
java.io.IOException: Exception reading file:/home/paomo/tmp/mapred/local/ttprivate/taskTracker/root/jobcache/job_201511301937_0004/jobToken
at org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:135)
at org.apache.hadoop.mapreduce.security.TokenCache.loadTokens(TokenCache.java:178)
at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1289)
at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1226)
at org.apache.hadoop.mapred.TaskTracker 5.run(TaskTracker.java:2603)atjava.lang.Thread.run(Thread.java:745)Causedby:java.io.FileNotFoundException:Filefile:/home/paomo/tmp/mapred/local/ttprivate/taskTracker/root/jobcache/job2015113019370004/jobTokendoesnotexist.atorg.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:402)atorg.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:255)atorg.apache.hadoop.fs.ChecksumFileSystem ChecksumFSInputChecker.(ChecksumFileSystem.java:125)
at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:283)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:436)
at org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:129)
… 5 more
” .
MapAttempt TASK_TYPE=”SETUP” TASKID=”task_201511301937_0004_m_000002” TASK_ATTEMPT_ID=”attempt_201511301937_0004_m_000002_3” START_TIME=”0” TRACKER_NAME=”tracker_paomo:localhost/127.0.0.1:51068” HTTP_PORT=”50060” LOCALITY=”OFF_SWITCH” AVATAAR=”VIRGIN” .
MapAttempt TASK_TYPE=”SETUP” TASKID=”task_201511301937_0004_m_000002” TASK_ATTEMPT_ID=”attempt_201511301937_0004_m_000002_3” TASK_STATUS=”FAILED” FINISH_TIME=”0” HOSTNAME=”paomo” ERROR=”Error initializing attempt_201511301937_0004_m_000002_3:
java.io.IOException: Exception reading file:/home/paomo/tmp/mapred/local/ttprivate/taskTracker/root/jobcache/job_201511301937_0004/jobToken
at org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:135)
at org.apache.hadoop.mapreduce.security.TokenCache.loadTokens(TokenCache.java:178)
at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1289)
at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1226)
at org.apache.hadoop.mapred.TaskTracker 5.run(TaskTracker.java:2603)atjava.lang.Thread.run(Thread.java:745)Causedby:java.io.FileNotFoundException:Filefile:/home/paomo/tmp/mapred/local/ttprivate/taskTracker/root/jobcache/job2015113019370004/jobTokendoesnotexist.atorg.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:402)atorg.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:255)atorg.apache.hadoop.fs.ChecksumFileSystem ChecksumFSInputChecker.(ChecksumFileSystem.java:125)
at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:283)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:436)
at org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:129)
… 5 more
” .
Task TASKID=”task_201511301937_0004_m_000002” TASK_TYPE=”SETUP” TASK_STATUS=”FAILED” FINISH_TIME=”0” ERROR=”Error initializing attempt_201511301937_0004_m_000002_3:
java.io.IOException: Exception reading file:/home/paomo/tmp/mapred/local/ttprivate/taskTracker/root/jobcache/job_201511301937_0004/jobToken
at org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:135)
at org.apache.hadoop.mapreduce.security.TokenCache.loadTokens(TokenCache.java:178)
at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1289)
at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1226)
at org.apache.hadoop.mapred.TaskTracker 5.run(TaskTracker.java:2603)atjava.lang.Thread.run(Thread.java:745)Causedby:java.io.FileNotFoundException:Filefile:/home/paomo/tmp/mapred/local/ttprivate/taskTracker/root/jobcache/job2015113019370004/jobTokendoesnotexist.atorg.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:402)atorg.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:255)atorg.apache.hadoop.fs.ChecksumFileSystem ChecksumFSInputChecker.(ChecksumFileSystem.java:125)
at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:283)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:436)
at org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:129)
… 5 more
” TASK_ATTEMPT_ID=”” .
Task TASKID=”task_201511301937_0004_m_000001” TASK_TYPE=”CLEANUP” START_TIME=”1448884209118” SPLITS=”” .
MapAttempt TASK_TYPE=”CLEANUP” TASKID=”task_201511301937_0004_m_000001” TASK_ATTEMPT_ID=”attempt_201511301937_0004_m_000001_0” START_TIME=”0” TRACKER_NAME=”tracker_paomo:localhost/127.0.0.1:51068” HTTP_PORT=”50060” LOCALITY=”OFF_SWITCH” AVATAAR=”VIRGIN” .
MapAttempt TASK_TYPE=”CLEANUP” TASKID=”task_201511301937_0004_m_000001” TASK_ATTEMPT_ID=”attempt_201511301937_0004_m_000001_0” TASK_STATUS=”FAILED” FINISH_TIME=”0” HOSTNAME=”paomo” ERROR=”Error initializing attempt_201511301937_0004_m_000001_0:
java.io.IOException: Exception reading file:/home/paomo/tmp/mapred/local/ttprivate/taskTracker/root/jobcache/job_201511301937_0004/jobToken
at org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:135)
at org.apache.hadoop.mapreduce.security.TokenCache.loadTokens(TokenCache.java:178)
at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1289)
at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1226)
at org.apache.hadoop.mapred.TaskTracker 5.run(TaskTracker.java:2603)atjava.lang.Thread.run(Thread.java:745)Causedby:java.io.FileNotFoundException:Filefile:/home/paomo/tmp/mapred/local/ttprivate/taskTracker/root/jobcache/job2015113019370004/jobTokendoesnotexist.atorg.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:402)atorg.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:255)atorg.apache.hadoop.fs.ChecksumFileSystem ChecksumFSInputChecker.(ChecksumFileSystem.java:125)
at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:283)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:436)
at org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:129)
… 5 more
” .
MapAttempt TASK_TYPE=”CLEANUP” TASKID=”task_201511301937_0004_m_000001” TASK_ATTEMPT_ID=”attempt_201511301937_0004_m_000001_1” START_TIME=”0” TRACKER_NAME=”tracker_paomo:localhost/127.0.0.1:51068” HTTP_PORT=”50060” LOCALITY=”OFF_SWITCH” AVATAAR=”VIRGIN” .
MapAttempt TASK_TYPE=”CLEANUP” TASKID=”task_201511301937_0004_m_000001” TASK_ATTEMPT_ID=”attempt_201511301937_0004_m_000001_1” TASK_STATUS=”FAILED” FINISH_TIME=”0” HOSTNAME=”paomo” ERROR=”Error initializing attempt_201511301937_0004_m_000001_1:
java.io.IOException: Exception reading file:/home/paomo/tmp/mapred/local/ttprivate/taskTracker/root/jobcache/job_201511301937_0004/jobToken
at org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:135)
at org.apache.hadoop.mapreduce.security.TokenCache.loadTokens(TokenCache.java:178)
at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1289)
at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1226)
at org.apache.hadoop.mapred.TaskTracker 5.run(TaskTracker.java:2603)atjava.lang.Thread.run(Thread.java:745)Causedby:java.io.FileNotFoundException:Filefile:/home/paomo/tmp/mapred/local/ttprivate/taskTracker/root/jobcache/job2015113019370004/jobTokendoesnotexist.atorg.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:402)atorg.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:255)atorg.apache.hadoop.fs.ChecksumFileSystem ChecksumFSInputChecker.(ChecksumFileSystem.java:125)
at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:283)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:436)
at org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:129)
… 5 more
” .
MapAttempt TASK_TYPE=”CLEANUP” TASKID=”task_201511301937_0004_m_000001” TASK_ATTEMPT_ID=”attempt_201511301937_0004_m_000001_2” START_TIME=”0” TRACKER_NAME=”tracker_paomo:localhost/127.0.0.1:51068” HTTP_PORT=”50060” LOCALITY=”OFF_SWITCH” AVATAAR=”VIRGIN” .
MapAttempt TASK_TYPE=”CLEANUP” TASKID=”task_201511301937_0004_m_000001” TASK_ATTEMPT_ID=”attempt_201511301937_0004_m_000001_2” TASK_STATUS=”FAILED” FINISH_TIME=”0” HOSTNAME=”paomo” ERROR=”Error initializing attempt_201511301937_0004_m_000001_2:
java.io.IOException: Exception reading file:/home/paomo/tmp/mapred/local/ttprivate/taskTracker/root/jobcache/job_201511301937_0004/jobToken
at org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:135)
at org.apache.hadoop.mapreduce.security.TokenCache.loadTokens(TokenCache.java:178)
at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1289)
at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1226)
at org.apache.hadoop.mapred.TaskTracker 5.run(TaskTracker.java:2603)atjava.lang.Thread.run(Thread.java:745)Causedby:java.io.FileNotFoundException:Filefile:/home/paomo/tmp/mapred/local/ttprivate/taskTracker/root/jobcache/job2015113019370004/jobTokendoesnotexist.atorg.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:402)atorg.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:255)atorg.apache.hadoop.fs.ChecksumFileSystem ChecksumFSInputChecker.(ChecksumFileSystem.java:125)
at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:283)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:436)
at org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:129)
… 5 more
” .
MapAttempt TASK_TYPE=”CLEANUP” TASKID=”task_201511301937_0004_m_000001” TASK_ATTEMPT_ID=”attempt_201511301937_0004_m_000001_3” START_TIME=”0” TRACKER_NAME=”tracker_paomo:localhost/127.0.0.1:51068” HTTP_PORT=”50060” LOCALITY=”OFF_SWITCH” AVATAAR=”VIRGIN” .
MapAttempt TASK_TYPE=”CLEANUP” TASKID=”task_201511301937_0004_m_000001” TASK_ATTEMPT_ID=”attempt_201511301937_0004_m_000001_3” TASK_STATUS=”FAILED” FINISH_TIME=”0” HOSTNAME=”paomo” ERROR=”Error initializing attempt_201511301937_0004_m_000001_3:
java.io.IOException: Exception reading file:/home/paomo/tmp/mapred/local/ttprivate/taskTracker/root/jobcache/job_201511301937_0004/jobToken
at org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:135)
at org.apache.hadoop.mapreduce.security.TokenCache.loadTokens(TokenCache.java:178)
at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1289)
at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1226)
at org.apache.hadoop.mapred.TaskTracker 5.run(TaskTracker.java:2603)atjava.lang.Thread.run(Thread.java:745)Causedby:java.io.FileNotFoundException:Filefile:/home/paomo/tmp/mapred/local/ttprivate/taskTracker/root/jobcache/job2015113019370004/jobTokendoesnotexist.atorg.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:402)atorg.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:255)atorg.apache.hadoop.fs.ChecksumFileSystem ChecksumFSInputChecker.(ChecksumFileSystem.java:125)
at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:283)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:436)
at org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:129)
… 5 more
” .
Task TASKID=”task_201511301937_0004_m_000001” TASK_TYPE=”CLEANUP” TASK_STATUS=”FAILED” FINISH_TIME=”0” ERROR=”Error initializing attempt_201511301937_0004_m_000001_3:
java.io.IOException: Exception reading file:/home/paomo/tmp/mapred/local/ttprivate/taskTracker/root/jobcache/job_201511301937_0004/jobToken
at org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:135)
at org.apache.hadoop.mapreduce.security.TokenCache.loadTokens(TokenCache.java:178)
at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1289)
at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1226)
at org.apache.hadoop.mapred.TaskTracker$5.run(TaskTracker.java:2603)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.FileNotFoundException: File file:/home/paomo/tmp/mapred/local/ttprivate/taskTracker/root/jobcache/job_201511301937_0004/jobToken does not exist.
at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:402)
at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:255)
at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.(ChecksumFileSystem.java:125)
at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:283)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:436)
at org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:129)
… 5 more
” TASK_ATTEMPT_ID=”” .
Job JOBID=”job_201511301937_0004” FINISH_TIME=”1448884210837” JOB_STATUS=”FAILED” FINISHED_MAPS=”0” FINISHED_REDUCES=”0” FAIL_REASON=”JobCleanup Task Failure, Task: task_201511301937_0004_m_000001” .

看到粗体字，仔细想就明白tmp文件没有找到。在网上查阅很多资料，
报错受以下博客的启发http://www.it165.net/pro/html/201501/31366.html
tmp路径的设置在hadoop-1.2.1/conf/core-site.xml文件中，
进到我的安装路径下查看，发现tmp在下列路径中：
tmp路径
而我在core-site.xml中设置为/home/paomo/tmp，后来改成了
core-site中tmp的设置
很明显位置不符合，难怪找不到tmp下生成的文件。因为在运行过程中Job会生成一些临时文件，执行过程中需要用到。一个方法是修改core-site.xml中hadoop.tmp.dir的值为tmp实际存放的位置，如下图：
tmp现在的设置
再重新运行就可以了。输入：bin/hadoop jar hadoop-examples-1.2.1.jar wordcount input output
运行结果如下：
运行结果
终于看到map 100% reduce 100%。
输入bin/hadoop dfs -cat output/part-r-00000，可以查看结果。

泡沫9水灵灵
关注
  • 1
    点赞
  • 踩
  • 2
    收藏
    觉得还不错? 一键收藏
  • 1
    评论
专栏目录
wordcount程序中，输出文件夹中为空
小李的博客
03-23 1537
ShuffleError: error in shuffle in localfetcher#1问题解决办法 问题 WIndows10 下使用IDEA本地调试wordcount出现如下错误 2021-03-05 11:11:41,311 WARN [org.apache.hadoop.mapred.LocalJobRunner] - job_local1291814541_0001 java.lang.Exception: org.apache.hadoop.mapreduce.task.reduce.Sh
linux运行hadoop实例wordcount详解
DaiDaiHan1的专栏
02-24 1680
在服务器上配置了hadoop准备自己从wordcount.java开始到跑起wordcount程序，但是就这简单的几步遇到了很多问题，现在总结一下我碰到的问题： 整个过程步骤是： 1.首先创建文件目录： mkdir  Test 2.编译wordcount.java文件生成.class文件，存放在第一步创建的文件夹Test中： javac  -classpath /hadoop*core*
Hadoop(1)---运行Hadoop自带的wordcount出错问题。
weixin_34234721的博客
08-16 853
在hadoop2.9.0版本中，对namenode、yarn做了ha，随后在某一台namenode节点上运行自带的wordcount程序出现偶发性的错误（有时成功，有时失败），错误信息如下： 18/08/16 17:02:42 INFO client.ConfiguredRMFailoverProxyProvider: Failing over to rm2 18/08/16 17...
Hadoop wordcount报错解决方案
阿拉修
06-30 1644
Error: Could not find or load main class org.apache.hadoop.mapred.YarnChild 解决方案： 1, 在命令行执行，复制信息。冒号用逗号代替 hadoop classpath 2 , 编辑yarn-site.xml，添加信息：</configuration> <property> <name>namemapreduce.reduce.env/name</name>
hadoop wordcount出现的错误
疯狂的鸡窝
08-30 837
我配置了两台机器的集群，一台作为主机，一台作为从机，当主机启动wordcont任务时候出现这样的错误 java.io.IOException: Could not obtain block: blk_3096925682732876688_1028 file=/home/hadoop/tmp/mapred/system/job_201308300007_0001/jobToken     
Windows环境下eclipse提交到远程wordcount程序报错 at org.apache.hadoop.util.Shell.runCommand(Shell.java:545)
www_jun的专栏
10-23 5865
远程hadoop2.7，本地的windows7 程序报错如下： 2015-09-28 22:04:21,423 WARN  [main] util.NativeCodeLoader (NativeCodeLoader.java:(62)) - Unable to load native-hadoop library for your platform... using builtin-java
hadoop-1.2.1运行WordCount
08-27
基于之前“搭建hadoop-1.2.1环境”搭建的hadoop环境，运行wordcount例子
hadoop1.2.1修改WordCount并编译
08-28
hadoop1.2.1修改WordCount并编译 在其中加入控制台输出
解决hadoop本地运行报错
10-11
org.apache.hadoop.io.nativeio.NativeIO.java解决办法 将org放入项目的工程目录下 本地运行MR必备的源码包，本地运行MR必备的源码包，本地运行MR必备的源码包，
hadoop1.2.1环境
07-16
在Linux上配置的hadoop1.2.1完全分布式环境
hadoop1.2.1安装部署文档
03-13
自己整理的hadoop1.2.1安装部署文档，详细过程都以图片展示，希望对需要了解这方面技术的同学有帮助
运行hadoop报错
wangkjs的博客
08-01 917
一：找不到内部类的问题 java.lang.ClassNotFoundException:com.test.WordCount$TokenizerMapperat org.apache.hadoop.conf.Configuration.getClass(Configuration.java:809) at org.apache.had
Hadoop3.2.0HA集群搭建后运行wordcount遇到错误
吴杰
08-17 1458
异常1： Container exited with a non-zero exit code 127. Error file: prelaunch.err. Last 4096 bytes of prelaunch.err : Last 4096 bytes of stderr : /bin/bash: /bin/java: No such file or directory 解决方法：在...
Hadoop-2.3.0学习（2）——Wordcount的运行以及遇到的问题
welcome to my life
06-05 1156
时间紧迫，直接开始啦！ 第一步：
Hadoop集群运行wordcount jar包出错
weixin_30670925的博客
01-02 507
出错提示 [xiaoqiu@s151 /home/xiaoqiu]$ hadoop jar HDFSTest.jar com.cr.hdfs.wordcount1.wordcountAPP hdfs: //s150/usr/xiaoqiu/wordcount hdfs://s150/usr/xiaoqiu/wordcount/ou...
eclipse下执行wordcount报错 java.lang.ClassNotFoundException
12-07 5301
我用eclipse执行wordcout，报错如下， 网上一般都说是要加job.setJarByClass(WordCount.class);，但我代码里有这句 另外，导出jar到linux下就正常了 13/06/06 17:16:34 WARN mapred.JobClient: No job jar file set.  User classes may not be found. See
【Hadoop】在笔记本虚拟机上部署Hadoop集群后运行Wordcount程序报错Connection refused
Louis Lee's Blog
03-02 745
在笔记本虚拟机上部署Hadoop集群后运行Wordcount程序报错Connection refused
ubuntu里第二次运行wordcount程序出错
weixin_34310369的博客
05-14 170
今天想用ubuntu重新再运行下之前的hadoop中自带的wordcount程序，在hadoop安装路径下（/usr/local/hadoop）输入命令bin/start-all.sh启动成功，运行wordcount例子时报错如下：hadoop@liu:~$ hadoop jar usr/local/hadoop/hadoop-examples-1.0.4.jar wordc...
maven工程的java项目运行时报出 找不到主清单解决方法
panda_lgn的博客
11-18 641
出错是因为没有找到入口类main() 修改pom.xml文件夹 <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-shade-plugin</artifactId> <version>1....
hadoop集群运行wordcount
最新发布
06-28
Hadoop集群运行WordCount的步骤如下： 1. 准备数据：将需要统计的文本数据上传到Hadoop集群中的HDFS文件系统中。 2. 编写MapReduce程序：编写WordCount的MapReduce程序，其中包括Mapper类和Reducer类。 3. 打包...

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论 1
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值