初学者写,只为备录
MAC系统下:(可能linux也差不多)
1:编译程序
*hadoop-2.6.0(可能2以上一样)的编译环境core包被分散在了hadoop-2.6.0/share文件下
javac -classpath $HADOOP_HOME/share/hadoop/common/hadoop-common-2.6.0.jar:$HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.6.0.jar -d /Library/userDevtool/FirstJar WordCount.java
*$HADOOP_HOME也可换成hadoop-2.6.0安装目录。WordCount.java(WordCount程序所在的目录/WordCount.java,我的放在hadoop安装目录的根目录下,我又cd进入到根目录下操作的。-d 后面跟的就是编译后存放的文件目录)
2:将编译结果打成jar包:
jar -cvf FirstJar.jar /Library/userDevtool/FirstJar
*FirstJar.jar 就是打成jar包的名称,也可加目录。后面就是要打包的class所在的目录
3:在集群上创建输入路径
bin/hadoop fs -mkdir /test/wordcount_nput
4:上传本地文件到hdfs的wordcount_nput下
bin/hadoop dfs -put /Library/userDevtool/hadoop-2.6.0/input/file* /test/wordcount_nput
5:在集群上运行WorldCount程序
bin/hadoop jar FirstJar.jar WordCount test/wordcount_nput /test/wordcount_output
6:查看输出文件
直接在控制台查看:bin/hadoop fs -cat /test/wordcount_output
或者把文件拷到本地查看:bin/hadoop dfs -get /test/wordcount_output /test/wordcount_output $HADOOP_HOME/output