ARM下Flink1.8.1编译，Could not resolve dependencies for project

flink编译过程中下载的依赖包很多，在 apache-maven-3.5.4/conf/settings.xml 配置ali和huawei两个mirror,下载会加速不少。

 

<mirror>
<id>nexus-aliyun</id>
<mirrorOf>*,!hadoop</mirrorOf>
<name>Nexus aliyun</name>
<url>http://maven.aliyun.com/nexus/content/groups/public</url>
</mirror>

<mirror>
<id>huaweimaven</id>
<name>huawei maven</name>
<url>https://mirrors.huaweicloud.com/repository/maven/</url>
<mirrorOf>central</mirrorOf>
</mirror>

 

编译问题汇总：

[ERROR] After correcting the problems, you can resume the build with the command

[ERROR] mvn <goals> -rf :flink-mapr-fs

报错缺失flink-mapr-fs，需要手动下载安装。

通过手动下载maprfs-5.2.1-mapr.jar包，下载地址地址：https://repository.mapr.com/nexus/content/groups/mapr-public/com/mapr/hadoop/maprfs/5.2.1-mapr/

.上传至主节点

将下载的maprfs-5.2.1-mapr.jar包上传至主节点的/home/hadoop/downloads目录下。

3.手动安装

手动安装缺少的包到本地仓库

mvn install:install-file -DgroupId=com.mapr.hadoop -DartifactId=maprfs -Dversion=5.2.1-mapr -Dpackaging=jar -Dfile=/home/hadoop/downloads/maprfs-5.2.1-mapr.jar

4.继续编译

使用maven继续编译flink（可以排除刚刚已经安装的包）

mvn clean install -Dmaven.test.skip=true -Dhadoop.version=2.7.3 -rf :flink-mapr-fs

[ERROR] /home/hadoop/opensource/flink/flink-filesystems/flink-mapr-fs/src/main/java/org/apache/flink/runtime/fs/maprfs/MapRFileSystem.java:[73,93] cannot find symbol

[ERROR] symbol: class Configuration

缺失org.apache.hadoop.fs包，报错找不到。

flink-mapr-fs模块的pom文件中添加如下依赖：

<dependency>

<groupId>org.apache.hadoop</groupId>

<artifactId>hadoop-common</artifactId>

<version>${hadoop.version}</version>

</dependency>

又报错：

[ERROR] Failed to execute goal on project flink-avro-confluent-registry: Could not resolve dependencies for project org.apache.flink:flink-avro-confluent-registry:jar:1.6.2: Could not find artifact io.confluent:kafka-schema-registry-client:jar:3.3.1 in nexus-osc (http://maven.aliyun.com/nexus/content/repositories/central) -> [Help 1]

同样的方法结果：

wget http://packages.confluent.io/maven/io/confluent/kafka-schema-registry-client/3.3.1/kafka-schema-registry-client-3.3.1.jar

mvn install:install-file -DgroupId=io.confluent -DartifactId=kafka-schema-registry-client -Dversion=3.3.1 -Dpackaging=jar -Dfile=/opt/tools/installed/kafka-schema-registry-client-3.3.1.jar

继续编译

mvn clean install -DskipTests -Dhadoop.version=3.1.1 -Dscala-2.11 -Dpython2.7  -rf :flink-avro-confluent-registry

 

[INFO] flink-yarn-tests ................................... SUCCESS [ 42.978 s]
[INFO] flink-fs-tests ..................................... SUCCESS [ 0.595 s]
[INFO] flink-docs 1.8.1 ................................... SUCCESS [ 2.857 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 06:38 min
[INFO] Finished at: 2020-04-19T14:10:44+08:00
[INFO] ------------------------------------------------------------------------

编译完成：

flink-release-1.8.1/flink-dist/target/flink-1.8.1-bin