windows下使用maven编译flume

本文讲解如何在windows下使用maven编译，以及如何在windows下执行

1、下载 flume 的源文件，地址：http://www.apache.org/dyn/closer.cgi/flume/1.5.0.1/apache-flume-1.5.0.1-src.tar.gz

2、本地安装maven环境

3、切换到源文件路径下执行：mvn clean

                                                 mvn package -DskipTests    // -DskipTests 使编译跳过测试，不然会报错

 

4、提示编译成功后，会在*\flume-ng-dist\target 目录下产生文件：apache-flume-1.5.0.1-bin、 apache-flume-1.5.0.1-bin.tar.gz、apache-flume-1.5.0.1-src、apache-flume-1.5.0.1-src.tar.gz，apache-flume-1.5.0.1-bin下的文件就可以用了

5、编译后，发现生成的仍然是linux下的脚本，无法再windows下执行，没关系，只要自己加上windows的执行脚本就行了，简单的脚本示例如下：

 

rem flume  1.5 . 0.1  windows script. set FLUME_HOME=D:\java\apache-flume- 1.5 . 0.1 -bin set JAVA= "D:\java\JDK6_2\bin\java.exe" set JAVA_OPTS=-Xmx60m set CONF=%FLUME_HOME%\conf\kafka.properties set AGENT=kafka %JAVA%  %JAVA_OPTS%  -Dlog4j.configuration=file: ///%FLUME_HOME%\conf\log4j.properties -cp "%FLUME_HOME%\lib\*" org.apache.flume.node.Application  -f %CONF%   -n %AGENT%

 

 

6、可能遇到的问题：

内存溢出

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin: 2 . 3.2 :testCompile ( default -testCompile) on project flume-ng-elasticsearch-sink: Co mpilation failure [ERROR] Failure executing javac, but could not parse the error: [ERROR] [ERROR] [ERROR] 系统资源不足。 [ERROR] 有关详细信息, 请参阅以下堆栈跟踪。 [ERROR] java.lang.OutOfMemoryError: PermGen space [ERROR] at java.lang.ClassLoader.defineClass1(Native Method) [ERROR] at java.lang.ClassLoader.defineClass(ClassLoader.java: 791 ) [ERROR] at java.security.SecureClassLoader.defineClass(SecureClassLoader.java: 14 2 ) ……

当Maven项目很大时，maven运行需要很大的内存，在默认配置下，就可能遇到java的堆溢出，这时指定分配内存大小

windows环境下：找到maven目录\bin\mvn.bat文件，编辑，在行

@REM set MAVEN_OPTS=-Xdebug -Xnoagent -Djava.compiler=NONE... 下设置

set MAVEN_OPTS= -Xms512m -Xmx1024m -XX:PermSize=256m -XX:MaxPermSize=512m

linux环境下：设置环境变量，编辑/etc/profile 增加

export  MAVEN_OPTS=-Xmx512m

 

 

Saxon-HE-9.5.1-4.jar下载失败

[ERROR] Failed to execute goal on project flume-ng-morphline-solr-sink: Could no t resolve dependencies  for  project org.apache.flume.flume-ng-sinks:flume-ng-morp hline-solr-sink:jar: 1.5 . 0.1 : The following artifacts could not be resolved: net. sf.saxon:Saxon-HE:jar: 9.5 . 1 - 4 , ua_parser:ua-parser:jar: 1.3 . 0 : Could not transfer   artifact net.sf.saxon:Saxon-HE:jar: 9.5 . 1 - 4  from/to nexus (http: //www.ispp.com/n exus/content/groups/ public ): Error transferring file: Server returned HTTP respo nse code:  502  for  URL: http: //www.ispp.com/nexus/content/groups/public/net/sf/sa xon/Saxon-HE/ 9.5 . 1 - 4 /Saxon-HE- 9.5 . 1 - 4 .jar -> [Help  1 ] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e swit ch. [ERROR] Re-run Maven using the -X  switch  to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please rea d the following articles: [ERROR] [Help  1 ] http: //cwiki.apache.org/confluence/display/MAVEN/DependencyReso lutionException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR]   mvn <goals> -rf :flume-ng-morphline-solr-sink ……

这个包比较大，重试几把就行了

ua_parser:ua-parser:jar:1.3.0下载失败

[ERROR] Failed to execute goal on project flume-ng-morphline-solr-sink: Could no t resolve dependencies  for  project org.apache.flume.flume-ng-sinks:flume-ng-morp hline-solr-sink:jar: 1.5 . 0.1 : Failure to find ua_parser:ua-parser:jar: 1.3 . 0  in ht tp: //www.ispp.com/nexus/content/groups/public was cached in the local repository , resolution will not be reattempted until the update interval of nexus has elap sed or updates are forced -> [Help  1 ] org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal o n project flume-ng-morphline-solr-sink: Could not resolve dependencies  for  proje ct org.apache.flume.flume-ng-sinks:flume-ng-morphline-solr-sink:jar: 1.5 . 0.1 : Fai lure to find ua_parser:ua-parser:jar: 1.3 . 0  in http: //www.ispp.com/nexus/content/ groups/ public  was cached in the local repository, resolution will not be reattem pted until the update interval of nexus has elapsed or updates are forced          at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.getDe pendencies(LifecycleDependencyResolver.java: 196 )          at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.resol veProjectDependencies(LifecycleDependencyResolver.java: 108 )          at org.apache.maven.lifecycle.internal.MojoExecutor.ensureDependenciesAr eResolved(MojoExecutor.java: 258 ) ……

这个问题搞了好一会，试了几种方法，没搞定。

发现它是对flume-ng-morphline-solr-sink的编译，需要依赖 ua_parser:ua-parser:jar:1.3.0，变通一点，先绕过它再说，在*\apache-flume-1.5.0.1-src\flume-ng-sinks\pom.xml 中，

注释掉 <module>flume-ng-morphline-solr-sink</module>，最后可以愉快的看到 [INFO] BUILD SUCCESS 啦