java在linux环境用maven编译_Linux下Maven编译spark源码时报错了

在Linux系统中使用Maven编译Spark源码时遇到了依赖解析失败的问题,具体为无法从中央仓库下载org.apache.hadoop:hadoop-yarn-server-web-proxy:jar:2.6.5,错误提示为repo1.maven.org域名解析失败。尝试编译无Hive功能的Spark包,但使用./make-distribution.sh也出现错误。寻求关于Maven编译Spark源码的指导。
摘要由CSDN通过智能技术生成

Linux下使用Maven编译spark源码时报了以下这个错误,知道该怎么解决

[INFO] ------------------------------------------------------------------------

[INFO] Reactor Summary:

[INFO]

[INFO] Spark Project Parent POM ........................... SUCCESS [ 9.290 s]

[INFO] Spark Project Test Tags ............................ SUCCESS [ 5.538 s]

[INFO] Spark Project Launcher ............................. SUCCESS [ 21.234 s]

[INFO] Spark Project Networking ........................... SUCCESS [ 17.278 s]

[INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [ 10.260 s]

[INFO] Spark Project Unsafe ............................... SUCCESS [ 29.558 s]

[INFO] Spark Project Core ................................. SUCCESS [09:45 min]

[INFO] Spark Project Bagel ................................ SUCCESS [ 32.979 s]

[INFO] Spark Project GraphX ............................... SUCCESS [02:51 min]

[INFO] Spark Project Streaming ............................ SUCCESS [03:54 min]

[INFO] Spark Project Catalyst ............................. SUCCESS [04:00 min]

[INFO] Spark Project SQL .................................. SUCCESS [05:26 min]

[INFO] Spark Project ML Library ........................... SUCCESS [06:28 min]

[INFO] Spark Project Tools ................................ SUCCESS [ 34.488 s]

[INFO] Spark Project Hive ................................. SUCCESS [03:03 min]

[INFO] Spark Project Docker Integration Tests ............. SUCCESS [ 18.751 s]

[INFO] Spark Project REPL ................................. SUCCESS [01:00 min]

[INFO] Spark Project YARN Shuffle Service ................. SUCCESS [ 8.247 s]

[INFO] Spark Project YARN ................................. FAILURE [06:13 min]

[INFO] Spark Project Assembly ............................. SKIPPED

[INFO] Spark Project External Twitter ..................... SKIPPED

[INFO] Spark Project External Flume Sink .................. SKIPPED

[INFO] Spark Project External Flume ....................... SKIPPED

[INFO] Spark Project External Flume Assembly .............. SKIPPED

[INFO] Spark Project External MQTT ........................ SKIPPED

[INFO] Spark Project External MQTT Assembly ............... SKIPPED

[INFO] Spark Project External ZeroMQ ...................... SKIPPED

[INFO] Spark Project External Kafka ....................... SKIPPED

[INFO] Spark Project Examples ............................. SKIPPED

[INFO] Spark Project External Kafka Assembly .............. SKIPPED

[INFO] ------------------------------------------------------------------------

[INFO] BUILD FAILURE

[INFO] ------------------------------------------------------------------------

[INFO] Total time: 45:54 min

[INFO] Finished at: 2016-12-21T16:42:26+08:00

[INFO] Final Memory: 64M/447M

[INFO] ------------------------------------------------------------------------

[ERROR] Failed to execute goal on project spark-yarn_2.10: Could not resolve dependencies for project org.apache.spark:spark-yarn_2.10:jar:1.6.0: Failed to collect dependencies at org.apache.hadoop:hadoop-yarn-server-web-proxy:jar:2.6.5: Failed to read artifact descriptor for org.apache.hadoop:hadoop-yarn-server-web-proxy:jar:2.6.5: Could not transfer artifact org.apache.hadoop:hadoop-yarn-server-web-proxy:pom:2.6.5 from/to central (https://repo1.maven.org/maven2): repo1.maven.org: Temporary failure in name resolution: Unknown host repo1.maven.org: Temporary failure in name resolution -> [Help 1]

org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal on project spark-yarn_2.10: Could not resolve dependencies for project org.apache.spark:spark-yarn_2.10:jar:1.6.0: Failed to collect dependencies at org.apache.hadoop:hadoop-yarn-server-web-proxy:jar:2.6.5

at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.getDependencies(LifecycleDependencyResolver.java:221)

at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.resolveProjectDependencies(LifecycleDependencyResolver.java:127)

Caused by: org.apache.maven.project.DependencyResolutionException: Could not resolve dependencies for project org.apache.spark:spark-yarn_2.10:jar:1.6.0: Failed to collect dependencies at org.apache.hadoop:hadoop-yarn-server-web-proxy:jar:2.6.5

at org.apache.maven.project.DefaultProjectDependenciesResolver.resolve(DefaultProjectDependenciesResolver.java:180)

at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.getDependencies(LifecycleDependencyResolver.java:195)

... 23 more

Caused by: org.eclipse.aether.collection.DependencyCollectionException: Failed to collect dependencies at org.apache.hadoop:hadoop-yarn-server-web-proxy:jar:2.6.5

at org.eclipse.aether.internal.impl.DefaultDependencyCollector.collectDependencies(DefaultDependencyCollector.java:291)

at org.eclipse.aether.internal.impl.DefaultRepositorySystem.collectDependencies(DefaultRepositorySystem.java:316)

Caused by: org.eclipse.aether.resolution.ArtifactDescriptorException: Failed to read artifact descriptor for org.apache.hadoop:hadoop-yarn-server-web-proxy:jar:2.6.5

at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:282)

at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.readArtifactDescriptor(DefaultArtifactDescriptorReader.java:198)

Caused by: org.eclipse.aether.resolution.ArtifactResolutionException: Could not transfer artifact org.apache.hadoop:hadoop-yarn-server-web-proxy:pom:2.6.5 from/to central (https://repo1.maven.org/maven2): repo1.maven.org: Temporary failure in name resolution

at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:444)

at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts(DefaultArtifactResolver.java:246)

Caused by: org.eclipse.aether.transfer.ArtifactTransferException: Could not transfer artifact org.apache.hadoop:hadoop-yarn-server-web-proxy:pom:2.6.5 from/to central (https://repo1.maven.org/maven2): repo1.maven.org: Temporary failure in name resolution

at org.eclipse.aether.connector.basic.ArtifactTransportListener.transferFailed(ArtifactTransportListener.java:43)

at org.eclipse.aether.connector.basic.BasicRepositoryConnector$TaskRunner.run(BasicRepositoryConnector.java:355)

Caused by: org.apache.maven.wagon.TransferFailedException: repo1.maven.org: Temporary failure in name resolution

at org.apache.maven.wagon.providers.http.AbstractHttpClientWagon.fillInputData(AbstractHttpClientWagon.java:1066)

at org.apache.maven.wagon.providers.http.AbstractHttpClientWagon.fillInputData(AbstractHttpClientWagon.java:960)

Caused by: java.net.UnknownHostException: repo1.maven.org: Temporary failure in name resolution

at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)

at java.net.InetAddress$1.lookupAllHostAddr(InetAddress.java:901)

[ERROR]

[ERROR]

[ERROR] For more information about the errors and possible solutions, please read the following articles:

[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException

[ERROR]

[ERROR] After correcting the problems, you can resume the build with the command

[ERROR] mvn -rf :spark-yarn_2.10

我想要的是编译一个不带Hive功能的Spakr包,试过用Spark源码自带的./make-distribution.sh编译工具编译, 但是编译出来的Spark安装包没有总是报错说缺少文件, 所以就试着用Maven来编译试试, 没想到又错误了, 第一次使用Maven, 不是很懂, 请各位前辈不吝赐教

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值