hadoop use at first

最新推荐文章于 2024-06-27 11:58:56 发布

yunccll

最新推荐文章于 2024-06-27 11:58:56 发布

阅读量409

点赞数

分类专栏： Linux/Unix 文章标签： hadoop jar path input shell

本文链接：https://blog.csdn.net/yunccll/article/details/7004760

版权

Linux/Unix 专栏收录该内容

26 篇文章 0 订阅

订阅专栏

1. compile the WordCount.java

javac -classpath ./hadoop-core-0.20.203.0.jar -d ../hdp_test -s ../hdp_test ../hdp_test/WordCo
unt.java -Xlint:deprecation

2. jar it
jar -cvf ~/hdp_test/WordCount.jar .

3. change core-site.xml
add the dfs property,
because the format meta data of hdfs was save in the /tmp in default,
so it 's clear after reboot the machine
change the dfs proterty block:

    <property>
        <name>hadoop.tmp.dir</name>
        <value>/home/chenglun/tmp</value>
        <description>A base for other temporary directories</description>
    </property>

4.
hadoop namenode format

5. run it
5.0 prepare the input data
hadoop jar WordCount.jar org.myorg.WordCount /home/chenglun/input /home/chenglun/output

Appendix: hdfs shell:

hadoop fs -ls [path]
hadoop fs -cat [path]
hadoop fs -mkdir [dir]
hadoop fs -copyFromLocal local URI