win10下用IDEA+Hadoop运行时的环境搭建

构建win10下用IDEA运行Hadoop程序

本地解压hadoop安装包：

如图

2.将hadoop添加到环境变量中

在Path变量中添加

[外链图片转存失败(img-1NtPRFvX-1567778520881)(image007.png)]

下载winutils工具,将bin目录覆盖到解压的hadoop安装包中

3.在IDEA中构建新项目：

注意：IDEA必须下载专业版

打开之后：

如图

创建完项目之后，打开porn.xml文件：

添加配置项

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>java_Hadoop</groupId>
    <artifactId>java_Hadoop</artifactId>
    <version>1.0-SNAPSHOT</version>
    <dependencies>
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-common</artifactId>
            <version>2.8.0</version>
        </dependency>
        <dependency>
    <groupId>org.apache.hadoop</groupId>
    <artifactId>hadoop-hdfs</artifactId>
            <version>2.8.0</version>
        </dependency>
    </dependencies>

相关的包自己会从官网下载

具体实现代码：

package hdfs;  //包

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileStatus;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;

import java.io.IOException;

public class ListHdfsFile {

    private static Configuration configuration;  // 静态
    private static FileSystem fs ;          //定义两个抽象对象

    static {
        configuration = new Configuration();
        configuration.set("fs.defaultFS", "172.18.74.68:9000");
        try {
            fs = FileSystem.get(configuration);
        } catch (IOException e) {
            e.printStackTrace();
        }

    }

    private static void listFileDir(Path p) throws IOException {
        FileStatus[] statuses = fs.listStatus(p);
        for (FileStatus status : statuses) {
            Path path = status.getPath();
            String name = path.getName();
            System.out.println(name);
        }
    }                  //类的封装


    public static void main(String[] args) throws IOException {
        Path path = new Path("/");   //定义的 listFileDir函数实现
        listFileDir(path);
    }
}