yarn-site.xml
Add the following to etc/hadoop/yarn-site.xml.
yarn.nodemanager.aux-services
mapreduce.shuffle
这里改下:
mapreduce_shuffle
1,建立java Hadoop project的时候,建立maven project。早pom.xml里面加入对应版本的dependency。 右击project,选择 maven build,goals 里面写package,产生jar文件。
2,产生输入文件:
hadoop fs -put 输入文件路径 文件夹
example:
hadoop fs -put $HADOOP_HOME/Hadoop-WordCount/input/ input
hadoop fs -ls input
3, 运行java 文件:
hadoop jar jar文件路径 package名称.文件名 input文件 输出文件
example:
hadoop jar $HADOOP_HOME/Hadoop-WordCount/wordcount.jar WordCount input output
4, view output file
hadoop fs -ls output
hadoop fs -cat output/*
如果想要显示system.out.println 的文件:
Easy way to access the logs is http://localhost:50030/jobtracker.jsp->click on the completed job->click on map or reduce task->click on tasknumber->task logs->stdout logs.