hadoop fs -ls output
hadoop fs -ls
1,首先先配置好JDK2,并配置SSH免密码登陆
3,安装HADOOP,这里用的是版本hadoop-0.20.2.tar.gz,可上CSDN下载那里找到这个版本。
4,安装伪分布式。
可以把伪分布式的hadoop看做只有一个节点的集群,在这个集群中,这个节点即是master,也是slave;既是NameNode,也是DATANODE;既是jobtracker,也是tasktracker。
解压之后,需修改几个文件。
5,在hadoop的安装目录下,先讲Wordcount.java这个例子打包生成JAR文件。
[root@localhost hadoop-0.20.2]# mkdir FirstJar
[root@localhost hadoop-0.20.2]# cp src/examples/org/apache/hadoop/examples/WordCount.java FirstJar
javac -classpath hadoop-0.20.2-core.jar WordCount.java Wordcount
假如是以上这个版本的话会报错,以为还缺少一个JAR
编译时需加入commons-cli-1.2.jar
命令如下由于编译时需注意包的问题,将WordCount.java放到一个文件下,编译时需注意包的问题
javac -classpath hadoop-0.20.2-core.jar:/root/hadoop-0.20.2/lib/commons-cli-1.2.jar -d FirstJar FirstJar/WordCount.java
特别注意引入commons-cli-1.2.jar请使用绝对路径!!!!
会生成已文件org/apache/hadoop/examples/WordCount 的三个CLASS文件
打包JAR文件
[root@localhost hadoop-0.20.2]# jar -cvf Wordcount.jar -C FirstJar/ .
记住后面有个 . !!!!!!!!!!成功后的页面
added manifest adding: WordCount.java(in = 2395) (out= 813)(deflated 66%) adding: org/(in = 0) (out= 0)(stored 0%) adding: org/apache/(in = 0) (out= 0)(stored 0%) adding: org/apache/hadoop/(in = 0) (out= 0)(stored 0%) adding: org/apache/hadoop/examples/(in = 0) (out= 0)(stored 0%) adding: org/apache/hadoop/examples/WordCount$TokenizerMapper.class(in = 1790) (out= 765)(deflated 57%) adding: org/apache/hadoop/examples/WordCount.class(in = 1911) (out= 996)(deflated 47%) adding: org/apache/hadoop/examples/WordCount$IntSumReducer.class(in = 1789) (out= 746)(deflated 58%)
之后查看hadoop的目录,会发现生成了Wordcount.jar这个JAR接着上传输入文件(输入文件是file01,file02,存放在input)
[root@localhost hadoop-0.20.2]# bin/hadoop dfs -mkdir input
可能会报这个错误: org.apache.hadoop.hdfs.server.namenode.SafeModeException: Cannot create directory /user/hadoop/input. Name node is in safe mode.
hadoop dfsadmin -safemode leave
即可解除安全模式,在分别创建两个file01,file02,内容自定
hadoop jar wordCount.jar org.apache.hadoop.examples.WordCount input output
注意类要加上包的路径。hadoop fs -ls
hadoop fs -ls output
hadoop fs -cat output/part-r-00000
嘿嘿,终于成功了!
hadoop编译WORDCOUNT
最新推荐文章于 2022-09-01 17:32:27 发布