MapReduce打包jar包及Xshell jar包实现

NeilVicia

已于 2022-06-19 22:47:50 修改

阅读量470

点赞数 1

于 2021-10-27 16:32:42 首次发布

本文链接：https://blog.csdn.net/liuyongsheng666/article/details/120995907

版权

打包jar包

实现连接hdfs文件系统

        //指定读取文件路径
//        Path path =new Path("E:\\hadoopstu\\in\\demo1");
//        Path path =new Path("hdfs://linux01:9000/input");
        Path path=new Path(args[0]);
        FileInputFormat.setInputPaths(job,path);


        //指定执行任务完成后的路径
//        Path pathout =new Path("E:\\hadoopstu\\in\\out1");
//        Path pathout =new Path("hdfs://linux01:9000/output1");
        Path pathout =new Path(args[1]);
        FileSystem fileSystem=FileSystem.get(pathout.toUri(),conf);

Xshell自带实现及java实现

自带实现

cd /opt/soft/hadoop260/share/hadoop/mapreduce
hdfs dfs -put /opt/soft2/hello2.txt /input
hadoop jar hadoop-mapreduce-examples-2.6.0-cdh5.14.2.jar wordcount /input/hello.txt /output

java jar包实现

[root@linux01 soft2]# hdfs dfs -chmod -R 777 /
[root@linux01 install]# hadoop jar hadoopstu-1.0-SNAPSHOT.jar nj.zb.kb15.demo1.WordCountDriver
/input/hello.txt /output