BigData<5>_HDFS API编程

一、开发环境搭建

1. 选择IDEA，JAVA，使用Maven来管理项目
   

2. 修改om.xml配置文件，加入Hadoop依赖包

  <!--引入cdh的仓库-->
  <repositories>
    <repository>
      <id>cloudera</id>
      <url>https://repository.cloudera.com/artifactory/cloudera-repos</url>
    </repository>
  </repositories>

  <properties>
    <!--定义Hadoop版本-->
    <hadoop.version>2.6.0-cdh5.15.1</hadoop.version>
  </properties>

  <dependencies>
    <!--添加Hadoop的依赖包-->
    <dependency>
      <groupId>org.apache.hadoop</groupId>
      <artifactId>hadoop-client</artifactId>
      <version>${hadoop.version}</version>
    </dependency>

    <!--添加junit的依赖包-->
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>4.10</version>
      <scope>test</scope>
    </dependency>
  </dependencies>

二、使用Java API 操作HDFS文件系统

1. 关键点：
   1. new 一个Configuration
   2. 获取FileSystem
   3. HDFS API操作

   Configuration configuration = new Configuration();
   FileSystem fileSystem = FileSystem.get(new URI("hdfs://hadoop000:8020"), configuration, "hadoop000");
   Path path = new Path("/hdfsapi/test");
   fileSystem.mkdirs(path);

2. jUnit 封装

转载于:https://www.cnblogs.com/isguoqiang/p/11471405.html