基于HDP（ambari）的Flink1.9.0编译

我在网上一直没有找到基于HDP-Flink1.9编译的文档，这几天一直在研究，踩了不少坑，现在整理一下，希望可以帮到大家

 

一、准备

1.安装Apache Maven 3.2.5

2.下载flink-shaded-7.0

3.下载flink-release-1.9.0

 

二、准备编译

1.修改flink-shaded项目中的pom.xml，添加如下

把上述内容加入到<profiles></profiles> 里面去。

<profile>
	<id>vendor-repos</id>
	<activation>
		<property>
			<name>vendor-repos</name>
		</property>
	</activation>
	<!-- Add vendor maven repositories -->
	<repositories>
		<!-- Cloudera -->
		<repository>
			<id>cloudera-releases</id>
			<url>https://repository.cloudera.com/artifactory/cloudera-repos</url>
			<releases>
				<enabled>true</enabled>
			</releases>
			<snapshots>
				<enabled>false</enabled>
			</snapshots>
		</repository>
		<!-- Hortonworks -->
		<repository>
			<id>HDPReleases</id>
			<name>HDP Releases</name>
			<url>https://repo.hortonworks.com/content/repositories/releases/</url>
			<snapshots><enabled>false</enabled></snapshots>
			<releases><enabled>true</enabled></releases>
		</repository>
		<repository>
			<id>HortonworksJettyHadoop</id>
			<name>HDP Jetty</name>
			<url>https://repo.hortonworks.com/content/repositories/jetty-hadoop</url>
			<snapshots><enabled>false</enabled></snapshots>
			<releases><enabled>true</enabled></releases>
		</repository>
		<!-- MapR -->
		<repository>
			<id>mapr-releases</id>
			<url>https://repository.mapr.com/maven/</url>
			<snapshots><enabled>false</enabled></snapshots>
			<releases><enabled>true</enabled></releases>
		</repository>
	</repositories>
</profile>

2.编译flink-shaded-hadoop-2（我hadoop的版本为3.1.0.3.0.0.0-1634）

mvn  clean install -DskipTests -Drat.skip=true -Pvendor-repos  -Dhadoop.version=3.1.0.3.0.0.0-1634

备注：如果途中报错，jar无法下载的话，手动导入到Maven仓库中

 

3.在flink-release-1.9.0中修改pom文件，添加如下

<profile>
	<id>vendor-repos</id>
	<activation>
		<property>
			<name>vendor-repos</name>
		</property>
	</activation>
	<!-- Add vendor maven repositories -->
	<repositories>
		<!-- Cloudera -->
		<repository>
			<id>cloudera-releases</id>
			<url>https://repository.cloudera.com/artifactory/cloudera-repos</url>
			<releases>
				<enabled>true</enabled>
			</releases>
			<snapshots>
				<enabled>false</enabled>
			</snapshots>
		</repository>
		<!-- Hortonworks -->
		<repository>
			<id>HDPReleases</id>
			<name>HDP Releases</name>
			<url>https://repo.hortonworks.com/content/repositories/releases/</url>
			<snapshots><enabled>false</enabled></snapshots>
			<releases><enabled>true</enabled></releases>
		</repository>
		<repository>
			<id>HortonworksJettyHadoop</id>
			<name>HDP Jetty</name>
			<url>https://repo.hortonworks.com/content/repositories/jetty-hadoop</url>
			<snapshots><enabled>false</enabled></snapshots>
			<releases><enabled>true</enabled></releases>
		</repository>
		<!-- MapR -->
		<repository>
			<id>mapr-releases</id>
			<url>https://repository.mapr.com/maven/</url>
			<snapshots><enabled>false</enabled></snapshots>
			<releases><enabled>true</enabled></releases>
		</repository>
	</repositories>
</profile>

5.编译flink-1.9

mvn clean install -DskipTests -Pvendor-repos -Drat.skip=true -Pinclude-hadoop -Dhadoop.version=3.1.0.3.0.0.0-1634

 

完成