ARM下Flink1.8.1编译,Could not resolve dependencies for project

6 篇文章 1 订阅
3 篇文章 0 订阅

flink编译过程中下载的依赖包很多,在 apache-maven-3.5.4/conf/settings.xml 配置ali和huawei两个mirror,下载会加速不少。

 

<mirror>
<id>nexus-aliyun</id>
<mirrorOf>*,!hadoop</mirrorOf>
<name>Nexus aliyun</name>
<url>http://maven.aliyun.com/nexus/content/groups/public</url>
</mirror>

<mirror>
<id>huaweimaven</id>
<name>huawei maven</name>
<url>https://mirrors.huaweicloud.com/repository/maven/</url>
<mirrorOf>central</mirrorOf>
</mirror>

 

编译问题汇总:

[ERROR] After correcting the problems, you can resume the build with the command

[ERROR] mvn <goals> -rf :flink-mapr-fs

报错缺失flink-mapr-fs,需要手动下载安装。

通过手动下载maprfs-5.2.1-mapr.jar包,下载地址地址:https://repository.mapr.com/nexus/content/groups/mapr-public/com/mapr/hadoop/maprfs/5.2.1-mapr/

.上传至主节点

将下载的maprfs-5.2.1-mapr.jar包上传至主节点的/home/hadoop/downloads目录下。

3.手动安装

手动安装缺少的包到本地仓库

mvn install:install-file -DgroupId=com.mapr.hadoop -DartifactId=maprfs -Dversion=5.2.1-mapr -Dpackaging=jar -Dfile=/home/hadoop/downloads/maprfs-5.2.1-mapr.jar

4.继续编译

使用maven继续编译flink(可以排除刚刚已经安装的包)

mvn clean install -Dmaven.test.skip=true -Dhadoop.version=2.7.3 -rf :flink-mapr-fs

[ERROR] /home/hadoop/opensource/flink/flink-filesystems/flink-mapr-fs/src/main/java/org/apache/flink/runtime/fs/maprfs/MapRFileSystem.java:[73,93] cannot find symbol

[ERROR] symbol: class Configuration

缺失org.apache.hadoop.fs包,报错找不到。


flink-mapr-fs模块的pom文件中添加如下依赖:

<dependency>

<groupId>org.apache.hadoop</groupId>

<artifactId>hadoop-common</artifactId>

<version>${hadoop.version}</version>

</dependency>


又报错:

[ERROR] Failed to execute goal on project flink-avro-confluent-registry: Could not resolve dependencies for project org.apache.flink:flink-avro-confluent-registry:jar:1.6.2: Could not find artifact io.confluent:kafka-schema-registry-client:jar:3.3.1 in nexus-osc (http://maven.aliyun.com/nexus/content/repositories/central) -> [Help 1]

同样的方法结果:

wget http://packages.confluent.io/maven/io/confluent/kafka-schema-registry-client/3.3.1/kafka-schema-registry-client-3.3.1.jar

mvn install:install-file -DgroupId=io.confluent -DartifactId=kafka-schema-registry-client -Dversion=3.3.1 -Dpackaging=jar -Dfile=/opt/tools/installed/kafka-schema-registry-client-3.3.1.jar

继续编译

mvn clean install -DskipTests -Dhadoop.version=3.1.1 -Dscala-2.11 -Dpython2.7  -rf :flink-avro-confluent-registry

 

[INFO] flink-yarn-tests ................................... SUCCESS [ 42.978 s]
[INFO] flink-fs-tests ..................................... SUCCESS [ 0.595 s]
[INFO] flink-docs 1.8.1 ................................... SUCCESS [ 2.857 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 06:38 min
[INFO] Finished at: 2020-04-19T14:10:44+08:00
[INFO] ------------------------------------------------------------------------

编译完成:

flink-release-1.8.1/flink-dist/target/flink-1.8.1-bin

评论 1
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值