1.2.0Hadoop的java操作

最新推荐文章于 2020-09-15 17:09:48 发布

__Simon'

最新推荐文章于 2020-09-15 17:09:48 发布

阅读量138

点赞数

分类专栏： Hadoop

本文链接：https://blog.csdn.net/Simon_09010817/article/details/79700963

版权

Hadoop 专栏收录该内容

24 篇文章 0 订阅

订阅专栏

Hadoop的java操作(Eclipse)

一、创建一个maven工程

二、在pom文件中引入相关jar的依赖

引入三个依赖：

hadoop-common

hadoop-hdfs

hadoop-client

    <!-- 引入hadoop-common包 -->   
   <dependency>
      <groupId>org.apache.hadoop</groupId>
      <artifactId>hadoop-common</artifactId>
      <version>2.6.4</version>
   </dependency>
   <!-- 引入hadoop-hdfs包 -->  
   <dependency>
       <groupId>org.apache.hadoop</groupId>
       <artifactId>hadoop-hdfs</artifactId>
       <version>2.6.4</version>
   </dependency>
   <!--引入hadoop-client包  -->
   <dependency>
      <groupId>org.apache.hadoop</groupId>
      <artifactId>hadoop-client</artifactId>
      <version>2.6.4</version>
   </dependency>

三、编写java类连接hadoop

包的版本不能随便引入，否则会导致错误

四、阿里云代理

   <mirror>
     <!--This sends everything else to /public -->
     <id>nexus</id>
      <mirrorOf>*</mirrorOf>
     <url>http://maven.aliyun.com/nexus/content/groups/public/</url>
   </mirror>
   <mirror>
     <!--This is used to direct the public snapshots repo in the
         profile below over to a different nexus group -->
     <id>nexus-public-snapshots</id>
     <mirrorOf>public-snapshots</mirrorOf>
     <url>http://maven.aliyun.com/nexus/content/repositories/snapshots/</url>
</mirror>

五、测试连接

代码：

        //连接hadoop
      Stringuri= "hdfs://10.10.34.106:9000";
      Configurationconf  =  new Configuration();
      FileSystem fs = FileSystem.get(new URI(uri), conf);