win10编译Hadoop3.0.2源码遇到的坑

目录

背景

2、 Failed to execute goal org.codehaus.mojo:native-maven-plugin:1.0-alpha-8:javah (default) on project hadoop-common: Error running javah command: Error executing command line. Exit code:1 -> [Help 1]

org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.codehaus.mojo:native-maven-plugin:1.0-alpha-8:javah (default) on project hadoop-common: Error running javah command

3、Cannot run program “msbuild” (in directory “D:hadoop-3.0.2-srchadoop-common-projecthadoop-common”): CreateProcess error=2, 系统找不到指定的文件。

4、?Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project hadoop-hdfs-native-client: An Ant BuildException has occured: exec returned: 1

[ERROR] around Ant part … @ 11:135 in D:hadoop-3.0.2-srchadoop-hdfs-projecthadoop-hdfs-native-client argetantrunuild-main.xml

5、?Could not resolve dependencies for project org.apache.hadoop:hadoop-resourceestimator:jar:3.0.2: Could not transfer artifact org.ojalgo:ojalgo:jar:43.0 from/to nexus-aliyun (http://maven.aliyun.com/nexus/content/groups/public): GET request of: org/ojalgo/ojalgo/43.0/ojalgo-43.0.jar from nexus-aliyun failed: Connection reset

编译结果

资源下载链接

---

背景

周五尝试用hive建表时，出现了问题，需要修改hadoop源码，而后编译。

但是在编译时，遇到了不少问题，周日耗时一天，排查完毕，编译通过

所有资源网盘链接见最后，请按需使用

1、Failed to execute goal org.apache.hadoop:hadoop-maven-plugins:3.0.2:protoc (compile-protoc) on project hadoop-common: org.apache.maven.plugin.MojoExecutionException: ‘protoc --version’ did not return a version

解决方法：安装protobuf2.5-win

1、下载zip压缩包

2、解压压缩包，然后把解压路径添加到环境变量path里

2、 Failed to execute goal org.codehaus.mojo:native-maven-plugin:1.0-alpha-8:javah (default) on project hadoop-common: Error running javah command: Error executing command line. Exit code:1 -> [Help 1]

org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.codehaus.mojo:native-maven-plugin:1.0-alpha-8:javah (default) on project hadoop-common: Error running javah command

javah出现了问题，解决方法：

修改源码目录下hadoop-common-projecthadoop-commonpom.xml文件，把所有的javahPath标签内容改成javah的绝对路径

3、Cannot run program “msbuild” (in directory “D:hadoop-3.0.2-srchadoop-common-projecthadoop-common”): CreateProcess error=2, 系统找不到指定的文件。

解决方法：安装VS Code，然后把msbuild可执行文件的绝对路径添加到path里(C:Program Files (x86)MSBuild .0Bin)

4、Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project hadoop-hdfs-native-client: An Ant BuildException has occured: exec returned: 1

[ERROR] around Ant part …<exec failοnerrοr=“true” dir=“D:hadoop-3.0.2-srchadoop-hdfs-projecthadoop-hdfs-native-client arget/native” executable=“msbuild”>… @ 11:135 in D:hadoop-3.0.2-srchadoop-hdfs-projecthadoop-hdfs-native-client argetantrunuild-main.xml

cmake和MsBuild(MSB1009找不到)莫名报错，暂时在hadoop-hdfs-projecthadoop-hdfs-native-clientpom.xml中，屏蔽它们的错误

5、Could not resolve dependencies for project org.apache.hadoop:hadoop-resourceestimator:jar:3.0.2: Could not transfer artifact org.ojalgo:ojalgo:jar:43.0 from/to nexus-aliyun (http://maven.aliyun.com/nexus/content/groups/public): GET request of: org/ojalgo/ojalgo/43.0/ojalgo-43.0.jar from nexus-aliyun failed: Connection reset

解决方法：把ojalgo-43.0.jar下载下来，复制到本地maven仓库文件夹的orgojalgoojalgojar#.0目录下

编译结果

资源下载链接

链接：https://pan.baidu.com/s/10PY2fhlBqJ0XNQiwlMpQIA
提取码：l4bb

包括hadoop3.0.2源码压缩包，apache-maven-3.6.1-bin.zip，cmake-3.14.5-win32-x86.msi，ojalgo-43.0.jar，protoc-2.5.0-win32.zip，VSCodeSetup-x64-1.25.1.exe，zlib-1.2.3.exe