Giraph 源码编译常见错误

  • Giraph 源码编译命令:

        

 
 
  1. giraph_basedir $ mvn -Phadoop_yarn -Dhadoop.version={指定hadoop版本} -DskipTests clean package

or

 
 
  1. giraph_basedir $ mvn -Phadoop_2 -Dhadoop.version={指定hadoop版本} -DskipTests clean package

如果一切正常,在giraph_basedir/giraph-dist目录下已经生成目标 jar

  • Giraph 编译中常见错误

1. hadoop版本问题

[ERROR] /home/cyf/giraph-release/giraph-core/target/munged/main/org/apache/giraph/job/GiraphJob.java:[213,10] 错误: setPingInterval(Configuration,int)在Client中不是公共的; 无法从外部程序包中对其进行访问

[INFO] 1 error

[INFO] -------------------------------------------------------------

[INFO] ------------------------------------------------------------------------

[INFO] Reactor Summary:

[INFO] 

[INFO] Apache Giraph Parent ............................... SUCCESS [  3.658 s]

[INFO] Apache Giraph Core ................................. FAILURE [  5.584 s]

[INFO] Apache Giraph Blocks Framework ..................... SKIPPED

[INFO] Apache Giraph Examples ............................. SKIPPED

[INFO] Apache Giraph Distribution ......................... SKIPPED

[INFO] ------------------------------------------------------------------------

[INFO] BUILD FAILURE

[INFO] ------------------------------------------------------------------------

[INFO] Total time: 9.558 s

[INFO] Finished at: 2016-08-28T20:10:25+08:00

[INFO] Final Memory: 41M/709M

[INFO] ------------------------------------------------------------------------

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.0:compile (default-compile) on project giraph-core: Compilation failure

[ERROR] /home/cyf/giraph-release/giraph-core/target/munged/main/org/apache/giraph/job/GiraphJob.java:[213,10] 错误: setPingInterval(Configuration,int)在Client中不是公共的; 无法从外部程序包中对其进行访问

[ERROR] -> [Help 1]

[ERROR] 

[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.

[ERROR] Re-run Maven using the -X switch to enable full debug logging.

[ERROR] 

[ERROR] For more information about the errors and possible solutions, please read the following articles:

[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

[ERROR] 

[ERROR] After correcting the problems, you can resume the build with the command

[ERROR]   mvn <goals> -rf :giraph-core

解决方法:高版本的 Hadoop 的 hadoop/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java 文件中的 setPingInterval(Configuration,int) 方法的访问权限已经从public 变为 default, 包外已经不能直接访问。只能降低Hadoop版本,2.5.2,2.6.3,2.7.2 还都是public 类型,以上版本都变成了default


2. SASL_PROPS 变量问题

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.0:compile (default-compile) on project giraph-core: Compilation failure: Compilation failure:

[ERROR] /home/cyf/giraph-release/giraph-core/target/munged/main/org/apache/giraph/comm/netty/SaslNettyClient.java:[84,67] 错误: 找不到符号

[ERROR] 符号:   变量 SASL_PROPS

[ERROR] 位置: 类 SaslRpcServer

[ERROR] /home/cyf/giraph-release/giraph-core/target/munged/main/org/apache/giraph/comm/netty/SaslNettyServer.java:[105,61] 错误: 找不到符号

[ERROR] -> [Help 1]

[ERROR] 

[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.

[ERROR] Re-run Maven using the -X switch to enable full debug logging.

[ERROR] 

[ERROR] For more information about the errors and possible solutions, please read the following articles:

[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

[ERROR] 

[ERROR] After correcting the problems, you can resume the build with the command

[ERROR]   mvn <goals> -rf :giraph-core

解决方法:

打开pom.xml文件,定位到profilehadoop_yarn的位置

 
 
  1. <profile>
  2.      <id>hadoop_yarn</id>
  3.      <modules>
  4.        <module>giraph-dist</module>
  5.      </modules>
  6.      <properties>
  7.        <hadoop.version>SET_HADOOP_VERSION_USING_MVN_DASH_D_OPTION</hadoop.version>
  8.        <munge.symbols>PURE_YARN,STATIC_SASL_SYMBOL</munge.symbols>
  9.        <!-- TODO: add these checks eventually -->
  10.        <project.enforcer.skip>true</project.enforcer.skip>
  11.        <giraph.maven.dependency.plugin.skip>true</giraph.maven.dependency.plugin.skip>
  12.        <giraph.maven.duplicate.finder.skip>true</giraph.maven.duplicate.finder.skip>
  13.      </properties>
  14.      <dependencies>
  15.        <!-- sorted lexicographically -->
  16.        <dependency>
  17.          <groupId>org.apache.hadoop</groupId>
  18.          <artifactId>hadoop-common</artifactId>
  19.        </dependency>
  20.        <dependency>
  21.          <groupId>org.apache.hadoop</groupId>
  22.          <artifactId>hadoop-mapreduce-client-common</artifactId>
  23.        </dependency>
  24.        <dependency>
  25.          <groupId>org.apache.hadoop</groupId>
  26.          <artifactId>hadoop-mapreduce-client-core</artifactId>
  27.        </dependency>
  28.        <dependency>
  29.          <groupId>org.apache.hadoop</groupId>
  30.          <artifactId>hadoop-yarn-common</artifactId>
  31.          <version>${hadoop.version}</version>
  32.        </dependency>
  33.        <dependency>
  34.          <groupId>org.apache.hadoop</groupId>
  35.          <artifactId>hadoop-yarn-server-common</artifactId>
  36.          <version>${hadoop.version}</version>
  37.        </dependency>
  38.        <dependency>
  39.          <groupId>org.apache.hadoop</groupId>
  40.          <artifactId>hadoop-yarn-server-resourcemanager</artifactId>
  41.          <version>${hadoop.version}</version>
  42.        </dependency>
  43.        <dependency>
  44.          <groupId>org.apache.hadoop</groupId>
  45.          <artifactId>hadoop-yarn-server-nodemanager</artifactId>
  46.          <version>${hadoop.version}</version>
  47.        </dependency>
  48.        <dependency>
  49.          <groupId>org.apache.hadoop</groupId>
  50.          <artifactId>hadoop-yarn-server-tests</artifactId>
  51.          <version>${hadoop.version}</version>
  52.          <type>test-jar</type>
  53.        </dependency>
  54.      </dependencies>
  55.    </profile>

将文件中红色部分删除即可。


3. 强制遵循编码规范

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-checkstyle-plugin:2.15:check (default) on project giraph-core: You have 1 Checkstyle violation. -> [Help 1]

[ERROR] 

[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.

[ERROR] Re-run Maven using the -X switch to enable full debug logging.

[ERROR] 

[ERROR] For more information about the errors and possible solutions, please read the following articles:

[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

[ERROR] 

[ERROR] After correcting the problems, you can resume the build with the command

[ERROR]   mvn <goals> -rf :giraph-core

解决方法:使用了mvn install 命令进行编译,改成mvn package就行了。

 
 
  1. giraph_basedir $ mvn -Phadoop_yarn -Dhadoop.version={指定hadoop版本} -DskipTests clean package




4. 没有指定hbase等的版本?

[INFO] Reactor Summary:

[INFO] 

[INFO] Apache Giraph Parent ............................... SUCCESS [  3.750 s]

[INFO] Apache Giraph Core ................................. SUCCESS [ 49.851 s]

[INFO] Apache Giraph Blocks Framework ..................... SUCCESS [ 21.132 s]

[INFO] Apache Giraph Examples ............................. SUCCESS [ 19.574 s]

[INFO] Apache Giraph Distribution ......................... FAILURE [  0.024 s]

[INFO] ------------------------------------------------------------------------

[INFO] BUILD FAILURE

[INFO] ------------------------------------------------------------------------

[INFO] Total time: 01:34 min

[INFO] Finished at: 2016-08-28T19:54:07+08:00

[INFO] Final Memory: 48M/766M

[INFO] ------------------------------------------------------------------------

[ERROR] Failed to execute goal on project giraph-dist: Could not resolve dependencies for project org.apache.giraph:giraph-dist:pom:1.2.0-RC0: The following artifacts could not be resolved: org.apache.giraph:giraph-hbase:jar:1.2.0-RC0, org.apache.giraph:giraph-accumulo:jar:1.2.0-RC0, org.apache.giraph:giraph-hcatalog:jar:1.2.0-RC0, org.apache.giraph:giraph-gora:jar:1.2.0-RC0, org.apache.giraph:giraph-kibble:jar:1.2.0-RC0, org.apache.giraph:giraph-rexster-io:jar:1.2.0-RC0: Failure to find org.apache.giraph:giraph-hbase:jar:1.2.0-RC0 in http://repo1.maven.org/maven2 was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced -> [Help 1]

org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal on project giraph-dist: Could not resolve dependencies for project org.apache.giraph:giraph-dist:pom:1.2.0-RC0: The following artifacts could not be resolved: org.apache.giraph:giraph-hbase:jar:1.2.0-RC0, org.apache.giraph:giraph-accumulo:jar:1.2.0-RC0, org.apache.giraph:giraph-hcatalog:jar:1.2.0-RC0, org.apache.giraph:giraph-gora:jar:1.2.0-RC0, org.apache.giraph:giraph-kibble:jar:1.2.0-RC0, org.apache.giraph:giraph-rexster-io:jar:1.2.0-RC0: Failure to find org.apache.giraph:giraph-hbase:jar:1.2.0-RC0 in http://repo1.maven.org/maven2 was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced

at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.getDependencies(LifecycleDependencyResolver.java:221)

at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.resolveProjectDependencies(LifecycleDependencyResolver.java:127)

at org.apache.maven.lifecycle.internal.MojoExecutor.ensureDependenciesAreResolved(MojoExecutor.java:245)

at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:199)

at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)

at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)

at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)

at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)

at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)

at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)

at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)

at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)

at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)

at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)

at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)

at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:606)

at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)

at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)

at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)

at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)

Caused by: org.apache.maven.project.DependencyResolutionException: Could not resolve dependencies for project org.apache.giraph:giraph-dist:pom:1.2.0-RC0: The following artifacts could not be resolved: org.apache.giraph:giraph-hbase:jar:1.2.0-RC0, org.apache.giraph:giraph-accumulo:jar:1.2.0-RC0, org.apache.giraph:giraph-hcatalog:jar:1.2.0-RC0, org.apache.giraph:giraph-gora:jar:1.2.0-RC0, org.apache.giraph:giraph-kibble:jar:1.2.0-RC0, org.apache.giraph:giraph-rexster-io:jar:1.2.0-RC0: Failure to find org.apache.giraph:giraph-hbase:jar:1.2.0-RC0 in http://repo1.maven.org/maven2 was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced

at org.apache.maven.project.DefaultProjectDependenciesResolver.resolve(DefaultProjectDependenciesResolver.java:211)

at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.getDependencies(LifecycleDependencyResolver.java:195)

... 23 more

Caused by: org.eclipse.aether.resolution.DependencyResolutionException: The following artifacts could not be resolved: org.apache.giraph:giraph-hbase:jar:1.2.0-RC0, org.apache.giraph:giraph-accumulo:jar:1.2.0-RC0, org.apache.giraph:giraph-hcatalog:jar:1.2.0-RC0, org.apache.giraph:giraph-gora:jar:1.2.0-RC0, org.apache.giraph:giraph-kibble:jar:1.2.0-RC0, org.apache.giraph:giraph-rexster-io:jar:1.2.0-RC0: Failure to find org.apache.giraph:giraph-hbase:jar:1.2.0-RC0 in http://repo1.maven.org/maven2 was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced

at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:384)

at org.apache.maven.project.DefaultProjectDependenciesResolver.resolve(DefaultProjectDependenciesResolver.java:205)

... 24 more

Caused by: org.eclipse.aether.resolution.ArtifactResolutionException: The following artifacts could not be resolved: org.apache.giraph:giraph-hbase:jar:1.2.0-RC0, org.apache.giraph:giraph-accumulo:jar:1.2.0-RC0, org.apache.giraph:giraph-hcatalog:jar:1.2.0-RC0, org.apache.giraph:giraph-gora:jar:1.2.0-RC0, org.apache.giraph:giraph-kibble:jar:1.2.0-RC0, org.apache.giraph:giraph-rexster-io:jar:1.2.0-RC0: Failure to find org.apache.giraph:giraph-hbase:jar:1.2.0-RC0 in http://repo1.maven.org/maven2 was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced

at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:444)

at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts(DefaultArtifactResolver.java:246)

at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:367)

... 25 more

Caused by: org.eclipse.aether.transfer.ArtifactNotFoundException: Failure to find org.apache.giraph:giraph-hbase:jar:1.2.0-RC0 in http://repo1.maven.org/maven2 was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced

at org.eclipse.aether.internal.impl.DefaultUpdateCheckManager.newException(DefaultUpdateCheckManager.java:231)

at org.eclipse.aether.internal.impl.DefaultUpdateCheckManager.checkArtifact(DefaultUpdateCheckManager.java:206)

at org.eclipse.aether.internal.impl.DefaultArtifactResolver.gatherDownloads(DefaultArtifactResolver.java:585)

at org.eclipse.aether.internal.impl.DefaultArtifactResolver.performDownloads(DefaultArtifactResolver.java:503)

at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:421)

... 27 more

[ERROR] 

[ERROR] Re-run Maven using the -X switch to enable full debug logging.

[ERROR] 

[ERROR] For more information about the errors and possible solutions, please read the following articles:

[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException

[ERROR] 

[ERROR] After correcting the problems, you can resume the build with the command

[ERROR]   mvn <goals> -rf :giraph-dist

解决方法:

好像没有影响,打开giraph-dist目录,目标 jar 已经生成。不过按上面的命令好像不会出现这个问题,不知道当时当时怎么搞的遇到了这个错误。


  • 0
    点赞
  • 1
    收藏
    觉得还不错? 一键收藏
  • 1
    评论
评论 1
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值