Giraph 源码编译常见错误

原创 2016年08月28日 20:34:43
  • Giraph 源码编译命令:

        

  1. giraph_basedir $ mvn -Phadoop_yarn -Dhadoop.version={指定hadoop版本} -DskipTests clean package

or

  1. giraph_basedir $ mvn -Phadoop_2 -Dhadoop.version={指定hadoop版本} -DskipTests clean package

如果一切正常,在giraph_basedir/giraph-dist目录下已经生成目标 jar

  • Giraph 编译中常见错误

1. hadoop版本问题

[ERROR] /home/cyf/giraph-release/giraph-core/target/munged/main/org/apache/giraph/job/GiraphJob.java:[213,10] 错误: setPingInterval(Configuration,int)在Client中不是公共的; 无法从外部程序包中对其进行访问

[INFO] 1 error

[INFO] -------------------------------------------------------------

[INFO] ------------------------------------------------------------------------

[INFO] Reactor Summary:

[INFO] 

[INFO] Apache Giraph Parent ............................... SUCCESS [  3.658 s]

[INFO] Apache Giraph Core ................................. FAILURE [  5.584 s]

[INFO] Apache Giraph Blocks Framework ..................... SKIPPED

[INFO] Apache Giraph Examples ............................. SKIPPED

[INFO] Apache Giraph Distribution ......................... SKIPPED

[INFO] ------------------------------------------------------------------------

[INFO] BUILD FAILURE

[INFO] ------------------------------------------------------------------------

[INFO] Total time: 9.558 s

[INFO] Finished at: 2016-08-28T20:10:25+08:00

[INFO] Final Memory: 41M/709M

[INFO] ------------------------------------------------------------------------

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.0:compile (default-compile) on project giraph-core: Compilation failure

[ERROR] /home/cyf/giraph-release/giraph-core/target/munged/main/org/apache/giraph/job/GiraphJob.java:[213,10] 错误: setPingInterval(Configuration,int)在Client中不是公共的; 无法从外部程序包中对其进行访问

[ERROR] -> [Help 1]

[ERROR] 

[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.

[ERROR] Re-run Maven using the -X switch to enable full debug logging.

[ERROR] 

[ERROR] For more information about the errors and possible solutions, please read the following articles:

[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

[ERROR] 

[ERROR] After correcting the problems, you can resume the build with the command

[ERROR]   mvn <goals> -rf :giraph-core

解决方法:高版本的 Hadoop 的 hadoop/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java 文件中的 setPingInterval(Configuration,int) 方法的访问权限已经从public 变为 default, 包外已经不能直接访问。只能降低Hadoop版本,2.5.2,2.6.3,2.7.2 还都是public 类型,以上版本都变成了default


2. SASL_PROPS 变量问题

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.0:compile (default-compile) on project giraph-core: Compilation failure: Compilation failure:

[ERROR] /home/cyf/giraph-release/giraph-core/target/munged/main/org/apache/giraph/comm/netty/SaslNettyClient.java:[84,67] 错误: 找不到符号

[ERROR] 符号:   变量 SASL_PROPS

[ERROR] 位置: 类 SaslRpcServer

[ERROR] /home/cyf/giraph-release/giraph-core/target/munged/main/org/apache/giraph/comm/netty/SaslNettyServer.java:[105,61] 错误: 找不到符号

[ERROR] -> [Help 1]

[ERROR] 

[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.

[ERROR] Re-run Maven using the -X switch to enable full debug logging.

[ERROR] 

[ERROR] For more information about the errors and possible solutions, please read the following articles:

[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

[ERROR] 

[ERROR] After correcting the problems, you can resume the build with the command

[ERROR]   mvn <goals> -rf :giraph-core

解决方法:

打开pom.xml文件,定位到profilehadoop_yarn的位置

  1. <profile>
  2.      <id>hadoop_yarn</id>
  3.      <modules>
  4.        <module>giraph-dist</module>
  5.      </modules>
  6.      <properties>
  7.        <hadoop.version>SET_HADOOP_VERSION_USING_MVN_DASH_D_OPTION</hadoop.version>
  8.        <munge.symbols>PURE_YARN,STATIC_SASL_SYMBOL</munge.symbols>
  9.        <!-- TODO: add these checks eventually -->
  10.        <project.enforcer.skip>true</project.enforcer.skip>
  11.        <giraph.maven.dependency.plugin.skip>true</giraph.maven.dependency.plugin.skip>
  12.        <giraph.maven.duplicate.finder.skip>true</giraph.maven.duplicate.finder.skip>
  13.      </properties>
  14.      <dependencies>
  15.        <!-- sorted lexicographically -->
  16.        <dependency>
  17.          <groupId>org.apache.hadoop</groupId>
  18.          <artifactId>hadoop-common</artifactId>
  19.        </dependency>
  20.        <dependency>
  21.          <groupId>org.apache.hadoop</groupId>
  22.          <artifactId>hadoop-mapreduce-client-common</artifactId>
  23.        </dependency>
  24.        <dependency>
  25.          <groupId>org.apache.hadoop</groupId>
  26.          <artifactId>hadoop-mapreduce-client-core</artifactId>
  27.        </dependency>
  28.        <dependency>
  29.          <groupId>org.apache.hadoop</groupId>
  30.          <artifactId>hadoop-yarn-common</artifactId>
  31.          <version>${hadoop.version}</version>
  32.        </dependency>
  33.        <dependency>
  34.          <groupId>org.apache.hadoop</groupId>
  35.          <artifactId>hadoop-yarn-server-common</artifactId>
  36.          <version>${hadoop.version}</version>
  37.        </dependency>
  38.        <dependency>
  39.          <groupId>org.apache.hadoop</groupId>
  40.          <artifactId>hadoop-yarn-server-resourcemanager</artifactId>
  41.          <version>${hadoop.version}</version>
  42.        </dependency>
  43.        <dependency>
  44.          <groupId>org.apache.hadoop</groupId>
  45.          <artifactId>hadoop-yarn-server-nodemanager</artifactId>
  46.          <version>${hadoop.version}</version>
  47.        </dependency>
  48.        <dependency>
  49.          <groupId>org.apache.hadoop</groupId>
  50.          <artifactId>hadoop-yarn-server-tests</artifactId>
  51.          <version>${hadoop.version}</version>
  52.          <type>test-jar</type>
  53.        </dependency>
  54.      </dependencies>
  55.    </profile>

将文件中红色部分删除即可。


3. 强制遵循编码规范

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-checkstyle-plugin:2.15:check (default) on project giraph-core: You have 1 Checkstyle violation. -> [Help 1]

[ERROR] 

[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.

[ERROR] Re-run Maven using the -X switch to enable full debug logging.

[ERROR] 

[ERROR] For more information about the errors and possible solutions, please read the following articles:

[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

[ERROR] 

[ERROR] After correcting the problems, you can resume the build with the command

[ERROR]   mvn <goals> -rf :giraph-core

解决方法:使用了mvn install 命令进行编译,改成mvn package就行了。

  1. giraph_basedir $ mvn -Phadoop_yarn -Dhadoop.version={指定hadoop版本} -DskipTests clean package




4. 没有指定hbase等的版本?

[INFO] Reactor Summary:

[INFO] 

[INFO] Apache Giraph Parent ............................... SUCCESS [  3.750 s]

[INFO] Apache Giraph Core ................................. SUCCESS [ 49.851 s]

[INFO] Apache Giraph Blocks Framework ..................... SUCCESS [ 21.132 s]

[INFO] Apache Giraph Examples ............................. SUCCESS [ 19.574 s]

[INFO] Apache Giraph Distribution ......................... FAILURE [  0.024 s]

[INFO] ------------------------------------------------------------------------

[INFO] BUILD FAILURE

[INFO] ------------------------------------------------------------------------

[INFO] Total time: 01:34 min

[INFO] Finished at: 2016-08-28T19:54:07+08:00

[INFO] Final Memory: 48M/766M

[INFO] ------------------------------------------------------------------------

[ERROR] Failed to execute goal on project giraph-dist: Could not resolve dependencies for project org.apache.giraph:giraph-dist:pom:1.2.0-RC0: The following artifacts could not be resolved: org.apache.giraph:giraph-hbase:jar:1.2.0-RC0, org.apache.giraph:giraph-accumulo:jar:1.2.0-RC0, org.apache.giraph:giraph-hcatalog:jar:1.2.0-RC0, org.apache.giraph:giraph-gora:jar:1.2.0-RC0, org.apache.giraph:giraph-kibble:jar:1.2.0-RC0, org.apache.giraph:giraph-rexster-io:jar:1.2.0-RC0: Failure to find org.apache.giraph:giraph-hbase:jar:1.2.0-RC0 in http://repo1.maven.org/maven2 was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced -> [Help 1]

org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal on project giraph-dist: Could not resolve dependencies for project org.apache.giraph:giraph-dist:pom:1.2.0-RC0: The following artifacts could not be resolved: org.apache.giraph:giraph-hbase:jar:1.2.0-RC0, org.apache.giraph:giraph-accumulo:jar:1.2.0-RC0, org.apache.giraph:giraph-hcatalog:jar:1.2.0-RC0, org.apache.giraph:giraph-gora:jar:1.2.0-RC0, org.apache.giraph:giraph-kibble:jar:1.2.0-RC0, org.apache.giraph:giraph-rexster-io:jar:1.2.0-RC0: Failure to find org.apache.giraph:giraph-hbase:jar:1.2.0-RC0 in http://repo1.maven.org/maven2 was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced

at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.getDependencies(LifecycleDependencyResolver.java:221)

at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.resolveProjectDependencies(LifecycleDependencyResolver.java:127)

at org.apache.maven.lifecycle.internal.MojoExecutor.ensureDependenciesAreResolved(MojoExecutor.java:245)

at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:199)

at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)

at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)

at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)

at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)

at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)

at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)

at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)

at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)

at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)

at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)

at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)

at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:606)

at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)

at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)

at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)

at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)

Caused by: org.apache.maven.project.DependencyResolutionException: Could not resolve dependencies for project org.apache.giraph:giraph-dist:pom:1.2.0-RC0: The following artifacts could not be resolved: org.apache.giraph:giraph-hbase:jar:1.2.0-RC0, org.apache.giraph:giraph-accumulo:jar:1.2.0-RC0, org.apache.giraph:giraph-hcatalog:jar:1.2.0-RC0, org.apache.giraph:giraph-gora:jar:1.2.0-RC0, org.apache.giraph:giraph-kibble:jar:1.2.0-RC0, org.apache.giraph:giraph-rexster-io:jar:1.2.0-RC0: Failure to find org.apache.giraph:giraph-hbase:jar:1.2.0-RC0 in http://repo1.maven.org/maven2 was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced

at org.apache.maven.project.DefaultProjectDependenciesResolver.resolve(DefaultProjectDependenciesResolver.java:211)

at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.getDependencies(LifecycleDependencyResolver.java:195)

... 23 more

Caused by: org.eclipse.aether.resolution.DependencyResolutionException: The following artifacts could not be resolved: org.apache.giraph:giraph-hbase:jar:1.2.0-RC0, org.apache.giraph:giraph-accumulo:jar:1.2.0-RC0, org.apache.giraph:giraph-hcatalog:jar:1.2.0-RC0, org.apache.giraph:giraph-gora:jar:1.2.0-RC0, org.apache.giraph:giraph-kibble:jar:1.2.0-RC0, org.apache.giraph:giraph-rexster-io:jar:1.2.0-RC0: Failure to find org.apache.giraph:giraph-hbase:jar:1.2.0-RC0 in http://repo1.maven.org/maven2 was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced

at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:384)

at org.apache.maven.project.DefaultProjectDependenciesResolver.resolve(DefaultProjectDependenciesResolver.java:205)

... 24 more

Caused by: org.eclipse.aether.resolution.ArtifactResolutionException: The following artifacts could not be resolved: org.apache.giraph:giraph-hbase:jar:1.2.0-RC0, org.apache.giraph:giraph-accumulo:jar:1.2.0-RC0, org.apache.giraph:giraph-hcatalog:jar:1.2.0-RC0, org.apache.giraph:giraph-gora:jar:1.2.0-RC0, org.apache.giraph:giraph-kibble:jar:1.2.0-RC0, org.apache.giraph:giraph-rexster-io:jar:1.2.0-RC0: Failure to find org.apache.giraph:giraph-hbase:jar:1.2.0-RC0 in http://repo1.maven.org/maven2 was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced

at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:444)

at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts(DefaultArtifactResolver.java:246)

at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:367)

... 25 more

Caused by: org.eclipse.aether.transfer.ArtifactNotFoundException: Failure to find org.apache.giraph:giraph-hbase:jar:1.2.0-RC0 in http://repo1.maven.org/maven2 was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced

at org.eclipse.aether.internal.impl.DefaultUpdateCheckManager.newException(DefaultUpdateCheckManager.java:231)

at org.eclipse.aether.internal.impl.DefaultUpdateCheckManager.checkArtifact(DefaultUpdateCheckManager.java:206)

at org.eclipse.aether.internal.impl.DefaultArtifactResolver.gatherDownloads(DefaultArtifactResolver.java:585)

at org.eclipse.aether.internal.impl.DefaultArtifactResolver.performDownloads(DefaultArtifactResolver.java:503)

at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:421)

... 27 more

[ERROR] 

[ERROR] Re-run Maven using the -X switch to enable full debug logging.

[ERROR] 

[ERROR] For more information about the errors and possible solutions, please read the following articles:

[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException

[ERROR] 

[ERROR] After correcting the problems, you can resume the build with the command

[ERROR]   mvn <goals> -rf :giraph-dist

解决方法:

好像没有影响,打开giraph-dist目录,目标 jar 已经生成。不过按上面的命令好像不会出现这个问题,不知道当时当时怎么搞的遇到了这个错误。


版权声明:本文为博主原创文章,未经博主允许不得转载。

相关文章推荐

阿里架构师首次畅谈余额宝背后的故事

一年前的现在,在杭州支付宝大楼里有个叫“春秋书院”的闭关室,里面一群紧张而兴奋的年轻人在忙碌着。项目室巨大的落地窗前,站着一个面色凝重的人,他就是天弘基金创新事业部技术负责人樊振华,一个在金融IT领域...

安装PHP的LDAP扩展

安装PHP的LDAP扩展 Windows下 如果你的php已经有php_ldap.dll文件,你可以打开php.ini文件, 找到 “;extension=php_ldap.dll” 去掉...

Giraph源码分析(三)—— 消息通信

1. 由前文知道每个BSPServiceWorker有一个WorkerServer对象,WorkerServer对象里面又有ServerData对象,作为数据实体。ServerData中包含该Work...

Giraph 源码分析(五)—— 加载数据+同步总结

master和workers同步过程总结。 (1)master创建znode A,然后检测A的子节点数目是否等于workers数目,不等于就陷入等待。某个worker创建一个子节点后,就会唤...

Giraph源码分析(四)—— Master 如何检查Worker启动成功

本文的目的:说明Giraph如何借助ZooKeeper来实现Master与Workers间的同步(?不太确定)。 环境:在单机上(机器名:giraphx)启动了2个workers。 Giraph遵从单...

ubuntu源码编译安装php常见错误解决办法

‘./configure’ ‘–prefix=/usr/local/php’ ‘–with-config-file-path=/etc’ ‘–with-mysql=/usr/local/mysql’ ...

源码编译安装httpd及其常见错误

一、编译安装的整体步骤  1、在官网下载源码,并解压下载 2、切换到其目录中  3、执行./configure 4、编译   二、编译中及安装后配置常见的参数及其说明 编译中配...

编译Android4.0源码时常见错误及解决办法

原文转载自:编译Android4.0源码时常见错误及解决办法 在Ubuntu12.04(64位)编译Android4.0源码时,遇到各种各样的问题。不是缺这个,就是少那个。现把这些问题和解决...
  • wyw594
  • wyw594
  • 2014年03月11日 14:52
  • 856

Giraph-Eclipse安装-源码调试

Giraph-Eclipse安装-调试源码 1. 官网上下载giraph-1.0.0,放在 /home/hadoop目录下,把giraph-1.0.0文件夹重命名为giraph-1.0.0-src。 ...

Giraph源码分析(一) —— 启动ZooKeeper服务

欢迎访问: 西北工业大学 - 大数据与知识管理研究室 (Northwestern Polytechnical University - BigData and Knowledge Management...
内容举报
返回顶部
收藏助手
不良信息举报
您举报文章:Giraph 源码编译常见错误
举报原因:
原因补充:

(最多只允许输入30个字)