Ambari子模块继承两套版本号
Ambari有很多个模块,最顶层的是Ambari Main(Ambari主模块)。大部分模块的版本号继承自Ambari Main,而Ambari Metrics及其下面的模块则未继承自Ambari Main,下面姑且分别称之为Ambari Main系和Ambari Metrics系。
Ambari Main系
Ambari > Ambari Project > Ambari Web/Server/Agent ……
Ambari Web/Server/Agent等模块都继承了Ambari Project模块的版本号,参见Ambari Server的pom.xml:
/ambari-branch-2.2.1/ambari-server/pom.xml
<parent>
<groupId>org.apache.ambari</groupId>
<artifactId>ambari-project</artifactId>
<version>2.2.1.0.0</version>
<relativePath>../ambari-project</relativePath>
</parent>
Ambari Project模块则又继承了Ambari Main的版本号,参见Ambari Project的pom.xml:
/ambari-branch-2.2.1/ambari-project/pom.xml
<parent>
<groupId>org.apache.ambari</groupId>
<artifactId>ambari</artifactId>
<version>2.2.1.0.0</version>
</parent>
Ambari Metrics系
Ambari Metrics > Ambari Metrics Common/Collector/Monitor……
Ambari Metrics模块并未继承Ambari Main的版本号,Ambari Metrics的pom.xml中未配置<parent>
标签:
/ambari-branch-2.2.1/ambari-metrics//pom.xml
Ambari Metrics Common/Collector/Monitor等模块则继承了Ambari Metrics模块的版本号,参见Ambari Metrics Common的pom.xml:
/ambari-branch-2.2.1/ambari-metrics/ambari-metrics-common/pom.xm
<parent>
<artifactId>ambari-metrics</artifactId>
<groupId>org.apache.ambari</groupId>
<version>2.2.1.0.0</version>
</parent>
以我们下载的Ambari 2.2.1版本的源码为例,Ambari Main的原始版本号为2.2.1.0.0,Ambari Project/Web/Server/Agent/Metrics等模块的原始版本号也是2.2.1.0.0。
执行以下命令修改Ambari Main的版本号至2.2.1.1:
# cd /ambari/ambari-branch-2.2.1
# mvn versions:set -DnewVersion=2.2.1.1
然后查看各个模块的pom.xml发现,Ambari Project/Server/Agent等模块pom文件中版本号都已经更新为2.2.1.1,而Ambari Metrics模块的pom文件中版本号依然为2.2.1.0.0。
执行以下命令对Ambari Main进行编译:
# mvn -B install package rpm:rpm -DnewVersion=2.2.1.1
-DskipTests -Dpython.ver="python >= 2.6" -Preplaceurl
编译结束后,可以发现
Ambari Web/Views等模块已经生成了版本号为2.2.1.1的rpm包:
/ambari-branch-2.2.1/ambari-web/target/rpm/ambari-web/RPMS/noarch/ambari-web-2.2.1.1-1.noarch.rpm
而Ambari Metrics模块生成的依然是版本号为2.2.1.0的rpm包:
/ambari-branch-2.2.1/ambari-metrics/target/rpm/ambari-metrics/RPMS/noarch/ambari-metrics-2.2.1.0-0.noarch.rpm
版本号格式规范
pom.xml文件中以正则表达式限定各模块的各种版本号格式。
parse-version
<execution>
<id>regex-property</id>
<goals>
<goal>regex-property</goal>
</goals>
<configuration>
<name>ambariVersion</name>
<value>${project.version}</value>
<regex>^([0-9]+)\.([0-9]+)\.([0-9]+)\.([0-9]+)(\.|-).*</regex>
<replacement>$1.$2.$3.$4</replacement>
<failIfNoMatch>false</failIfNoMatch>
</configuration>
</execution>
parse-package-version
Ambari Main系
参见Ambari的pom.xml:
/ambari-branch-2.2.1/pom.xml
<execution>
<id>parse-package-version</id>
<goals>
<goal>regex-property</goal>
</goals>
<configuration>
<name>package-version</name>
<value>${project.version}</value>
<regex>^([0-9]+)\.([0-9]+)\.([0-9]+)\.([0-9]+)((\.|-).*)?</regex>
<replacement>$1.$2.$3.$4</replacement>
<failIfNoMatch>true</failIfNoMatch>
</configuration>
</execution>
Ambari Metrics系
参见Ambari Metrics的pom.xml:
/ambari-branch-2.2.1/ambari-metrics//pom.xml
<execution>
<id>parse-package-version</id>
<goals>
<goal>regex-property</goal>
</goals>
<configuration>
<name>package-version</name>
<value>${project.version}</value>
<regex>^([0-9]+)\.([0-9]+)\.([0-9]+)\.([0-9]+)(\.|-).*</regex>
<replacement>$1.$2.$3.$4</replacement>
<failIfNoMatch>true</failIfNoMatch>
</configuration>
</execution>
parse-package-release
Ambari Main系
参见Ambari的pom.xml:
/ambari-branch-2.2.1/pom.xml
<execution>
<id>parse-package-release</id>
<goals>
<goal>regex-property</goal>
</goals>
<configuration>
<name>package-release</name>
<value>${project.version}</value>
<regex>^([0-9]+)\.([0-9]+)\.([0-9]+)\.([0-9]+)((\.|-)(([0-9]+)|(SNAPSHOT)|(techwin)).*)?</regex>
<replacement>$7</replacement>
<failIfNoMatch>true</failIfNoMatch>
</configuration>
</execution>
Ambari Metrics系
参见Ambari Metrics的pom.xml:
/ambari-branch-2.2.1/ambari-metrics//pom.xml
<execution>
<id>parse-package-release</id>
<goals>
<goal>regex-property</goal>
</goals>
<configuration>
<name>package-release</name>
<value>${project.version}</value>
<regex>
^([0-9]+)\.([0-9]+)\.([0-9]+)\.([0-9]+)(\.|-)((([a-zA-Z]+)?([0-9]+))|(SNAPSHOT)).*
</regex>
<replacement>$7</replacement>
<failIfNoMatch>true</failIfNoMatch>
</configuration>
</execution>
版本号格式验证
set -DnewVersion
执行以下命令修改源代码版本号时,会根据pom.xml中的
parse-version进行版本号检查。
# mvn versions:set -DnewVersion=x.x.x.x
但是由于该配置项中配置了
<failIfNoMatch>false</failIfNoMatch>
所以即便格式不符合正则表达式的规范,也不会报错。
install package
执行以下命令进行模块编译时,会先根据pom.xml中的
parse-package-version进行版本号检查,如果检查通过会再根据parse-package-release进行版本号检查,只有两次检查都通过才能成功编译。
# mvn -B install package rpm:rpm -DnewVersion=2.2.1 -DskipTests -Dpython.ver="python >= 2.6" -Preplaceurl
由于这两个配置项中配置了
<failIfNoMatch>true</failIfNoMatch>
所以当pom.xml文件中声明的版本号格式不符合正则表达式的规范时,编译会报错。
未通过parse-package-version检查的情况:
[ERROR] Failed to execute goal org.codehaus.mojo:build-helper-maven-plugin:1.8:regex-property (parse-package-version) on project ambari-views: No match to regex ‘^([0-9]+).([0-9]+).([0-9]+).([0-9]+)((.|-).*)?’ found in ‘2.2.1’. -> [Help 1]
通过parse-package-version检查,而未通过parse-package-release检查的情况:
[ERROR] Failed to execute goal org.codehaus.mojo:build-helper-maven-plugin:1.8:regex-property (parse-package-release) on project ambari-metrics: No match to regex ‘^([0-9]+).([0-9]+).([0-9]+).([0-9]+)(.|-)((([a-zA-Z]+)?([0-9]+))|(SNAPSHOT)).*’ found in ‘2.2.0.0.’. -> [Help 1]
总体来讲,Ambari Metrics系与Ambari Main系相比,前者对于版本号的规则稍微复杂一点。
版本号格式导致的ambari-metrics编译报错
[INFO] ------------------------------------------------------------------------
[INFO] Building ambari-metrics 2.2.1.1
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- build-helper-maven-plugin:1.8:parse-version (parse-version) @ ambari-metrics ---
[INFO]
[INFO] --- build-helper-maven-plugin:1.8:regex-property (regex-property) @ ambari-metrics ---
[INFO] No match to regex '^([0-9]+)\.([0-9]+)\.([0-9]+)\.([0-9]+)(\.|-).*' found in '2.2.1.1'. The initial value '2.2.1.1' is left as-is...
[INFO]
[INFO] --- build-helper-maven-plugin:1.8:regex-property (parse-package-version) @ ambari-metrics ---
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] ambari-metrics .................................... FAILURE [1.604s]
[INFO] Ambari Metrics Common ............................. SKIPPED
[INFO] Ambari Metrics Hadoop Sink ........................ SKIPPED
[INFO] Ambari Metrics Flume Sink ......................... SKIPPED
[INFO] Ambari Metrics Kafka Sink ......................... SKIPPED
[INFO] Ambari Metrics Storm Sink ......................... SKIPPED
[INFO] Ambari Metrics Collector .......................... SKIPPED
[INFO] Ambari Metrics Monitor ............................ SKIPPED
[INFO] Ambari Metrics Assembly ........................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2.994s
[INFO] Finished at: Wed Jun 22 14:19:26 CST 2016
[INFO] Final Memory: 6M/15M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:build-helper-maven-plugin:1.8:regex-property (parse-package-version) on project ambari-metrics: No match to regex '^([0-9]+)\.([0-9]+)\.([0-9]+)\.([0-9]+)(\.|-).*' found in '2.2.1.1'. -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
报错信息提示版本号2.2.1.1 不符合版本号格式要求:
^([0-9]+)\.([0-9]+)\.([0-9]+)\.([0-9]+)(\.|-).*
参照正则表达式设置正确的版本号格式,如2.2.1.1.0
版本号不一致导致的Ambari Server编译报错
Ambari Server的ambari-metrics-common依赖
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Ambari Main ....................................... SUCCESS [19.820s]
[INFO] Apache Ambari Project POM ......................... SUCCESS [0.835s]
[INFO] Ambari Web ........................................ SUCCESS [37.145s]
[INFO] Ambari Views ...................................... SUCCESS [7.536s]
[INFO] Ambari Admin View ................................. SUCCESS [5:02.761s]
[INFO] ambari-metrics .................................... SUCCESS [3.859s]
[INFO] Ambari Metrics Common ............................. SUCCESS [5.188s]
[INFO] Ambari Metrics Hadoop Sink ........................ SUCCESS [10.753s]
[INFO] Ambari Metrics Flume Sink ......................... SUCCESS [5.335s]
[INFO] Ambari Metrics Kafka Sink ......................... SUCCESS [5.929s]
[INFO] Ambari Metrics Storm Sink ......................... SUCCESS [17.246s]
[INFO] Ambari Metrics Collector .......................... SUCCESS [5:49.381s]
[INFO] Ambari Metrics Monitor ............................ SUCCESS [9.156s]
[INFO] Ambari Metrics Assembly ........................... SUCCESS [5:10.913s]
[INFO] Ambari Server ..................................... FAILURE [29.396s]
[INFO] Ambari Agent ...................................... SKIPPED
[INFO] Ambari Client ..................................... SKIPPED
[INFO] Ambari Python Client .............................. SKIPPED
[INFO] Ambari Groovy Client .............................. SKIPPED
[INFO] Ambari Shell ...................................... SKIPPED
[INFO] Ambari Python Shell ............................... SKIPPED
[INFO] Ambari Groovy Shell ............................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 18:39.688s
[INFO] Finished at: Wed Jun 22 09:57:27 CST 2016
[INFO] Final Memory: 153M/366M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project ambari-server:
Could not resolve dependencies for project org.apache.ambari:ambari-server:jar:2.2.1.1:
Could not find artifact org.apache.ambari:ambari-metrics-common:jar:2.2.1.1 in oss.sonatype.org (https://oss.sonatype.org/content/groups/staging) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :ambari-server
这次实验试图把Ambari编译到2.2.1.1版本,然而编译到Ambari Server的时候失败了,报错信息中提示Ambari Server依赖的ambari-metrics-common:jar:2.2.1.1无法找到。我们进入以下目录查看发现Ambari Metrics Common模块生成的jar包是2.2.1.0.0版本的,而Ambari Server依赖需要的是2.2.1.1版本的:
/ambari-branch-2.2.1/ambari-metrics/ambari-metrics-common/target/ambari-metrics-common-2.2.1.0.0.jar
导致该问题出现的原因,就是上面提到的Ambari Metrics模块未继承Ambari Main的版本号,导致 Ambari Metrics模块版本号与其他模块不一致。
参照官方对该版本不一致问题的解释:
https://cwiki.apache.org/confluence/display/AMBARI/Ambari+Development
RHEL/CentOS 6:
mvn versions:set -DnewVersion=${AMBARI_VERSION}
#Note: The ambari-metrics project is not wired up to the main ambari project. However there is a dependency on ambari-metrics-common to build the ambari-server RPM.
#Hence you also need to set ambari-metrics project version as well.
pushd ambari-metrics
mvn versions:set -DnewVersion=${AMBARI_VERSION}
popd
mvn -B clean install package rpm:rpm -DskipTests -Dpython.ver="python >= 2.6" -Preplaceurl
解决方案
上面官方的解释中也提到了解决方案,就是在编译前对Ambari Metrics模块再单独进行一次版本更改,使Ambari Metrics模块与Ambari其他模块的版本号保持一致。
然而个人以为官方文档中对版本号的解释有问题:
Note: Replace ${AMBARI_VERSION} with a 4-digit version you want the artifacts to be
(e.g., -DnewVersion=1.6.1.1)
官方说明的版本号格式为4个数字,如1.6.1.1格式。这个格式对Ambari Main系的模块来说是没问题的,但是对Ambari Metrics系的模块来讲就是无效的格式,参考前面“版本号格式导致的ambari-metrics编译报错”。
稳妥的解决方案就是把Ambari Main系和Ambari Metrics系的版本号统一设置为5个数字格式,如2.2.1.1.0。
以我的项目为例:
# cd /ambari/ambari-branch-2.2.1
# mvn clean
修改Ambari主项目版本号:
# mvn versions:set -DnewVersion=2.2.1.1.0
修改Ambari Metrics版本号:
# pushd ambari-metrics
# mvn versions:set -DnewVersion=2.2.1.1.0
# popd
编译项目:
# cd /ambari/ambari-branch-2.2.1
# mvn -B install package rpm:rpm -DnewVersion=2.2.1.1.0 -DskipTests -Dpython.ver="python >= 2.6" -Preplaceurl
子模块版本号覆盖继承版本号
一般来说,子模块的版本号与继承的父模块版本号一致,当执行set -DnewVersion命令对父模块进行修改版本号的操作,会同时修改子模块的版本号。子模块pom.xml中,父模块和子模块的版本号会同步改变:
# mvn versions:set -DnewVersion=2.2.5.5
[INFO] Building Ambari Main 2.2.1.1.0
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- versions-maven-plugin:2.2:set (default-cli) @ ambari ---
[INFO] Searching for local aggregator root...
[INFO] Local aggregation root: /ambari/ambari-branch-2.2.1
[INFO] Processing change of org.apache.ambari:ambari:2.2.1.1.0 -> 2.2.5.5
[INFO] Processing org.apache.ambari:ambari
[INFO] Updating project org.apache.ambari:ambari
[INFO] from version 2.2.1.1.0 to 2.2.5.5
[INFO]
[INFO] Processing org.apache.ambari:ambari-project
[INFO] Updating parent org.apache.ambari:ambari
[INFO] from version 2.2.1.1.0 to 2.2.5.5
[INFO] Updating project org.apache.ambari:ambari-project
[INFO] from version 2.2.1.1.0 to 2.2.5.5
[INFO]
[INFO] Processing org.apache.ambari.contrib.views:ambari-contrib-views
[INFO] Updating parent org.apache.ambari:ambari-project
[INFO] from version 2.2.1.1.0 to 2.2.5.5
[INFO] Updating project org.apache.ambari.contrib.views:ambari-contrib-views
[INFO] from version 2.2.1.1.0 to 2.2.5.5
参考上面的日志发现,由于Ambari Project的版本号继承自Ambari,此时对Ambari进行修改版本号的操作,Ambari Project的版本号会随之改变。此时对于Ambari Project的pom.xml文件有两步操作:
更改继承自父模块的版本号:
[INFO] Updating parent org.apache.ambari:ambari
[INFO] from version 2.2.1.1.0 to 2.2.5.5
更改后的父模块版本号:
<parent>
<groupId>org.apache.ambari</groupId>
<artifactId>ambari</artifactId>
<version>2.2.5.5</version>
</parent>
更改子模块本身的版本号:
[INFO] Updating project org.apache.ambari:ambari-project
[INFO] from version 2.2.1.1.0 to 2.2.5.5
更改后的子模块版本号:
<groupId>org.apache.ambari</groupId>
<artifactId>ambari-project</artifactId>
<version>2.2.5.5/version>
但是如果执行set -DnewVersion命令对子模块进行了单独的修改版本号,就会出现如下所示的子模块版本号与父模块版本号不一致的情况,如下所示我单独将Ambari View的版本号由2.2.1.0.0改为了2.2.2.0.0。
<parent>
<groupId>org.apache.ambari</groupId>
<artifactId>ambari-project</artifactId>
<version>2.2.1.0.0</version>
<relativePath>../ambari-project</relativePath>
</parent>
<modelVersion>4.0.0</modelVersion>
<groupId>org.apache.ambari</groupId>
<artifactId>ambari-views</artifactId>
<packaging>jar</packaging>
<name>Ambari Views</name>
<version>2.2.2.0.0</version>
此时再执行set -DnewVersion命令对父模块进行修改版本号的操作,将不会修改子模块的版本号。子模块pom.xml中,只有父模块版本号会发生改变:
[INFO] Processing org.apache.ambari:ambari-shell
[INFO] Updating parent org.apache.ambari:ambari-project
[INFO] from version 2.2.1.1.0 to 2.2.5.5
[INFO] Updating project org.apache.ambari:ambari-shell
[INFO] from version 2.2.1.1.0 to 2.2.5.5
[INFO]
[INFO] Processing org.apache.ambari:ambari-views
[INFO] Updating parent org.apache.ambari:ambari-project
[INFO] from version 2.2.1.1.0 to 2.2.5.5
[INFO]
[INFO] Processing org.apache.ambari:ambari-web
[INFO] Updating parent org.apache.ambari:ambari-project
[INFO] from version 2.2.1.1.0 to 2.2.5.5
[INFO] Updating project org.apache.ambari:ambari-web
[INFO] from version 2.2.1.1.0 to 2.2.5.5
上面的例子中我单独修改了Ambari View的版本号。Ambari View的版本号继承自Ambari Project,Ambari Project的版本号继承自Ambari Main,此时对Ambari Main进行修改版本号的操作,Ambari Project的版本号会随之改变,而Ambari View的版本号将不会改变。
Ps:编译Ambari过程中被版本号的问题折磨了许久,通过baidu和Google也没找到有用的答案,最后通过粗略自学Maven、读Ambari各个模块的pom文件、反复编译实验,找到了一些来龙去脉。把实验过程中的一些心得记录到CSDN,希望能帮到进行Ambari开发的兄弟。由于内容是我自己做开发过程中的一些笔记,不代表权威,谬误之处请指正。
原文转载至: http://blog.csdn.net/royma_1990/article/details/51749981