基于CDH-6.2.0编译flink-1.12.1(Hadoop-3.0.0&Hive-2.1.1)

1、编译准备环境：

• jdk-1.8
• maven-3.9                   

2、flink-1.12.1下载

https://flink.apache.org/downloads.html

3、flink-shaded-12

https://www.apache.org/dyn/closer.lua/flink/flink-shaded-12.0/flink-shaded-12.0-src.tgz

https://flink.apache.org/downloads.html#flink-shaded

 

放flink-1.12.1的根目录，进入flink-shaded-12 ，修改pomx.ml

添加 cloudera 的maven库：

<!--添加CDH的仓库-->

<repositories>

     <repository>

           <id>cloudera</id>

         <url>https://repository.cloudera.com/artifactory/cloudera-repos</url>

      </repository>

</repositories>

编译：mvn clean install -DskipTests -Dhadoop.version=3.0.0-cdh6.2.0

4、在flink-1.12.1的pom.xml文件中也加入cloudera的库：

<hadoop.version>3.0.0</hadoop.version>

<hive.version>2.1.1</hive.version>

<hivemetastore.hadoop.version>3.0.0</hivemetastore.hadoop.version>

<repositories>
    <repository>
        <id>cloudera</id>
        <url>https://repository.cloudera.com/artifactory/cloudera-repos</url>
    </repository>
</repositories>

 

5.编译flink-12.1(可能需要多次编译才能通过)

mvn -T2C clean install -DskipTests -Dfast -Pinclude-hadoop -Pvendor-repos -Dhadoop.version=3.0.0-cdh6.2.0 -Dscala-2.11