Ambari源码编译版本号问题

Ambari子模块继承两套版本号

Ambari有很多个模块,最顶层的是Ambari Main(Ambari主模块)。大部分模块的版本号继承自Ambari Main,而Ambari Metrics及其下面的模块则未继承自Ambari Main,下面姑且分别称之为Ambari Main系和Ambari Metrics系。

Ambari Main系

Ambari > Ambari Project > Ambari Web/Server/Agent ……

Ambari Web/Server/Agent等模块都继承了Ambari Project模块的版本号,参见Ambari Server的pom.xml:

/ambari-branch-2.2.1/ambari-server/pom.xml

<parent>
    <groupId>org.apache.ambari</groupId>
    <artifactId>ambari-project</artifactId>
    <version>2.2.1.0.0</version>
    <relativePath>../ambari-project</relativePath>
</parent>

Ambari Project模块则又继承了Ambari Main的版本号,参见Ambari Project的pom.xml:

/ambari-branch-2.2.1/ambari-project/pom.xml

<parent>
    <groupId>org.apache.ambari</groupId>
    <artifactId>ambari</artifactId>
    <version>2.2.1.0.0</version>
</parent>

Ambari Metrics系

Ambari Metrics > Ambari Metrics Common/Collector/Monitor……

Ambari Metrics模块并未继承Ambari Main的版本号,Ambari Metrics的pom.xml中未配置<parent>标签:

/ambari-branch-2.2.1/ambari-metrics//pom.xml

Ambari Metrics Common/Collector/Monitor等模块则继承了Ambari Metrics模块的版本号,参见Ambari Metrics Common的pom.xml:

/ambari-branch-2.2.1/ambari-metrics/ambari-metrics-common/pom.xm

<parent>
    <artifactId>ambari-metrics</artifactId>
    <groupId>org.apache.ambari</groupId>
    <version>2.2.1.0.0</version>
</parent>

以我们下载的Ambari 2.2.1版本的源码为例,Ambari Main的原始版本号为2.2.1.0.0,Ambari Project/Web/Server/Agent/Metrics等模块的原始版本号也是2.2.1.0.0。
执行以下命令修改Ambari Main的版本号至2.2.1.1:

# cd /ambari/ambari-branch-2.2.1
# mvn versions:set -DnewVersion=2.2.1.1

然后查看各个模块的pom.xml发现,Ambari Project/Server/Agent等模块pom文件中版本号都已经更新为2.2.1.1,而Ambari Metrics模块的pom文件中版本号依然为2.2.1.0.0。

执行以下命令对Ambari Main进行编译:

# mvn -B install package rpm:rpm -DnewVersion=2.2.1.1
 -DskipTests -Dpython.ver="python >= 2.6" -Preplaceurl

编译结束后,可以发现
Ambari Web/Views等模块已经生成了版本号为2.2.1.1的rpm包:

/ambari-branch-2.2.1/ambari-web/target/rpm/ambari-web/RPMS/noarch/ambari-web-2.2.1.1-1.noarch.rpm

而Ambari Metrics模块生成的依然是版本号为2.2.1.0的rpm包:

/ambari-branch-2.2.1/ambari-metrics/target/rpm/ambari-metrics/RPMS/noarch/ambari-metrics-2.2.1.0-0.noarch.rpm

版本号格式规范

pom.xml文件中以正则表达式限定各模块的各种版本号格式。

parse-version

<execution>
    <id>regex-property</id>
    <goals>
    <goal>regex-property</goal>
    </goals>
    <configuration>
        <name>ambariVersion</name>
        <value>${project.version}</value>
        <regex>^([0-9]+)\.([0-9]+)\.([0-9]+)\.([0-9]+)(\.|-).*</regex>
        <replacement>$1.$2.$3.$4</replacement>
        <failIfNoMatch>false</failIfNoMatch>
    </configuration>
</execution>

parse-package-version

Ambari Main系

参见Ambari的pom.xml:

/ambari-branch-2.2.1/pom.xml

<execution>
  <id>parse-package-version</id>
  <goals>
        <goal>regex-property</goal>
  </goals>
  <configuration>
    <name>package-version</name>
    <value>${project.version}</value>
    <regex>^([0-9]+)\.([0-9]+)\.([0-9]+)\.([0-9]+)((\.|-).*)?</regex>
    <replacement>$1.$2.$3.$4</replacement>
    <failIfNoMatch>true</failIfNoMatch>
  </configuration>
</execution>
Ambari Metrics系

参见Ambari Metrics的pom.xml:

/ambari-branch-2.2.1/ambari-metrics//pom.xml

<execution>
  <id>parse-package-version</id>
  <goals>
    <goal>regex-property</goal>
  </goals>
  <configuration>
    <name>package-version</name>
    <value>${project.version}</value>
    <regex>^([0-9]+)\.([0-9]+)\.([0-9]+)\.([0-9]+)(\.|-).*</regex>
    <replacement>$1.$2.$3.$4</replacement>
    <failIfNoMatch>true</failIfNoMatch>
  </configuration>
</execution>

parse-package-release

Ambari Main系

参见Ambari的pom.xml:

/ambari-branch-2.2.1/pom.xml

<execution>
  <id>parse-package-release</id>
  <goals>
    <goal>regex-property</goal>
  </goals>
  <configuration>
    <name>package-release</name>
    <value>${project.version}</value>
    <regex>^([0-9]+)\.([0-9]+)\.([0-9]+)\.([0-9]+)((\.|-)(([0-9]+)|(SNAPSHOT)|(techwin)).*)?</regex>
    <replacement>$7</replacement>
    <failIfNoMatch>true</failIfNoMatch>
  </configuration>
</execution>
Ambari Metrics系

参见Ambari Metrics的pom.xml:

/ambari-branch-2.2.1/ambari-metrics//pom.xml

<execution>
  <id>parse-package-release</id>
  <goals>
    <goal>regex-property</goal>
  </goals>
  <configuration>
    <name>package-release</name>
    <value>${project.version}</value>
    <regex>
      ^([0-9]+)\.([0-9]+)\.([0-9]+)\.([0-9]+)(\.|-)((([a-zA-Z]+)?([0-9]+))|(SNAPSHOT)).*
    </regex>
    <replacement>$7</replacement>
    <failIfNoMatch>true</failIfNoMatch>
  </configuration>
</execution>

版本号格式验证

set -DnewVersion

执行以下命令修改源代码版本号时,会根据pom.xml中的
parse-version进行版本号检查。

# mvn versions:set -DnewVersion=x.x.x.x

但是由于该配置项中配置了
<failIfNoMatch>false</failIfNoMatch>
所以即便格式不符合正则表达式的规范,也不会报错。

install package

执行以下命令进行模块编译时,会先根据pom.xml中的
parse-package-version进行版本号检查,如果检查通过会再根据parse-package-release进行版本号检查,只有两次检查都通过才能成功编译。

# mvn -B install package rpm:rpm -DnewVersion=2.2.1 -DskipTests -Dpython.ver="python >= 2.6" -Preplaceurl

由于这两个配置项中配置了
<failIfNoMatch>true</failIfNoMatch>
所以当pom.xml文件中声明的版本号格式不符合正则表达式的规范时,编译会报错。

未通过parse-package-version检查的情况:

[ERROR] Failed to execute goal org.codehaus.mojo:build-helper-maven-plugin:1.8:regex-property (parse-package-version) on project ambari-views: No match to regex ‘^([0-9]+).([0-9]+).([0-9]+).([0-9]+)((.|-).*)?’ found in ‘2.2.1’. -> [Help 1]

通过parse-package-version检查,而未通过parse-package-release检查的情况:

[ERROR] Failed to execute goal org.codehaus.mojo:build-helper-maven-plugin:1.8:regex-property (parse-package-release) on project ambari-metrics: No match to regex ‘^([0-9]+).([0-9]+).([0-9]+).([0-9]+)(.|-)((([a-zA-Z]+)?([0-9]+))|(SNAPSHOT)).*’ found in ‘2.2.0.0.’. -> [Help 1]

总体来讲,Ambari Metrics系与Ambari Main系相比,前者对于版本号的规则稍微复杂一点。

版本号格式导致的ambari-metrics编译报错

[INFO] ------------------------------------------------------------------------
[INFO] Building ambari-metrics 2.2.1.1
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- build-helper-maven-plugin:1.8:parse-version (parse-version) @ ambari-metrics ---
[INFO]
[INFO] --- build-helper-maven-plugin:1.8:regex-property (regex-property) @ ambari-metrics ---
[INFO] No match to regex '^([0-9]+)\.([0-9]+)\.([0-9]+)\.([0-9]+)(\.|-).*' found in '2.2.1.1'. The initial value '2.2.1.1' is left as-is...
[INFO]
[INFO] --- build-helper-maven-plugin:1.8:regex-property (parse-package-version) @ ambari-metrics ---
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] ambari-metrics .................................... FAILURE [1.604s]
[INFO] Ambari Metrics Common ............................. SKIPPED
[INFO] Ambari Metrics Hadoop Sink ........................ SKIPPED
[INFO] Ambari Metrics Flume Sink ......................... SKIPPED
[INFO] Ambari Metrics Kafka Sink ......................... SKIPPED
[INFO] Ambari Metrics Storm Sink ......................... SKIPPED
[INFO] Ambari Metrics Collector .......................... SKIPPED
[INFO] Ambari Metrics Monitor ............................ SKIPPED
[INFO] Ambari Metrics Assembly ........................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2.994s
[INFO] Finished at: Wed Jun 22 14:19:26 CST 2016
[INFO] Final Memory: 6M/15M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:build-helper-maven-plugin:1.8:regex-property (parse-package-version) on project ambari-metrics: No match to regex '^([0-9]+)\.([0-9]+)\.([0-9]+)\.([0-9]+)(\.|-).*' found in '2.2.1.1'. -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

报错信息提示版本号2.2.1.1 不符合版本号格式要求:

^([0-9]+)\.([0-9]+)\.([0-9]+)\.([0-9]+)(\.|-).*

参照正则表达式设置正确的版本号格式,如2.2.1.1.0

版本号不一致导致的Ambari Server编译报错

Ambari Server的ambari-metrics-common依赖

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Ambari Main ....................................... SUCCESS [19.820s]
[INFO] Apache Ambari Project POM ......................... SUCCESS [0.835s]
[INFO] Ambari Web ........................................ SUCCESS [37.145s]
[INFO] Ambari Views ...................................... SUCCESS [7.536s]
[INFO] Ambari Admin View ................................. SUCCESS [5:02.761s]
[INFO] ambari-metrics .................................... SUCCESS [3.859s]
[INFO] Ambari Metrics Common ............................. SUCCESS [5.188s]
[INFO] Ambari Metrics Hadoop Sink ........................ SUCCESS [10.753s]
[INFO] Ambari Metrics Flume Sink ......................... SUCCESS [5.335s]
[INFO] Ambari Metrics Kafka Sink ......................... SUCCESS [5.929s]
[INFO] Ambari Metrics Storm Sink ......................... SUCCESS [17.246s]
[INFO] Ambari Metrics Collector .......................... SUCCESS [5:49.381s]
[INFO] Ambari Metrics Monitor ............................ SUCCESS [9.156s]
[INFO] Ambari Metrics Assembly ........................... SUCCESS [5:10.913s]
[INFO] Ambari Server ..................................... FAILURE [29.396s]
[INFO] Ambari Agent ...................................... SKIPPED
[INFO] Ambari Client ..................................... SKIPPED
[INFO] Ambari Python Client .............................. SKIPPED
[INFO] Ambari Groovy Client .............................. SKIPPED
[INFO] Ambari Shell ...................................... SKIPPED
[INFO] Ambari Python Shell ............................... SKIPPED
[INFO] Ambari Groovy Shell ............................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 18:39.688s
[INFO] Finished at: Wed Jun 22 09:57:27 CST 2016
[INFO] Final Memory: 153M/366M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project ambari-server: 
Could not resolve dependencies for project org.apache.ambari:ambari-server:jar:2.2.1.1: 
Could not find artifact org.apache.ambari:ambari-metrics-common:jar:2.2.1.1 in oss.sonatype.org (https://oss.sonatype.org/content/groups/staging) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :ambari-server

这次实验试图把Ambari编译到2.2.1.1版本,然而编译到Ambari Server的时候失败了,报错信息中提示Ambari Server依赖的ambari-metrics-common:jar:2.2.1.1无法找到。我们进入以下目录查看发现Ambari Metrics Common模块生成的jar包是2.2.1.0.0版本的,而Ambari Server依赖需要的是2.2.1.1版本的:

/ambari-branch-2.2.1/ambari-metrics/ambari-metrics-common/target/ambari-metrics-common-2.2.1.0.0.jar

导致该问题出现的原因,就是上面提到的Ambari Metrics模块未继承Ambari Main的版本号,导致 Ambari Metrics模块版本号与其他模块不一致。

参照官方对该版本不一致问题的解释:
https://cwiki.apache.org/confluence/display/AMBARI/Ambari+Development

RHEL/CentOS 6:
mvn versions:set -DnewVersion=${AMBARI_VERSION}

#Note: The ambari-metrics project is not wired up to the main ambari project. However there is a dependency on ambari-metrics-common to build the ambari-server RPM.
#Hence you also need to set ambari-metrics project version as well.
pushd ambari-metrics
mvn versions:set -DnewVersion=${AMBARI_VERSION}
popd

mvn -B clean install package rpm:rpm -DskipTests -Dpython.ver="python >= 2.6" -Preplaceurl

解决方案

上面官方的解释中也提到了解决方案,就是在编译前对Ambari Metrics模块再单独进行一次版本更改,使Ambari Metrics模块与Ambari其他模块的版本号保持一致。
然而个人以为官方文档中对版本号的解释有问题:

Note: Replace ${AMBARI_VERSION} with a 4-digit version you want the artifacts to be 
(e.g., -DnewVersion=1.6.1.1)

官方说明的版本号格式为4个数字,如1.6.1.1格式。这个格式对Ambari Main系的模块来说是没问题的,但是对Ambari Metrics系的模块来讲就是无效的格式,参考前面“版本号格式导致的ambari-metrics编译报错”。

稳妥的解决方案就是把Ambari Main系和Ambari Metrics系的版本号统一设置为5个数字格式,如2.2.1.1.0。

以我的项目为例:

# cd /ambari/ambari-branch-2.2.1
# mvn clean
修改Ambari主项目版本号:
# mvn versions:set -DnewVersion=2.2.1.1.0
修改Ambari Metrics版本号:
# pushd ambari-metrics
# mvn versions:set -DnewVersion=2.2.1.1.0
# popd
编译项目:
# cd /ambari/ambari-branch-2.2.1
# mvn -B install package rpm:rpm -DnewVersion=2.2.1.1.0 -DskipTests -Dpython.ver="python >= 2.6" -Preplaceurl

子模块版本号覆盖继承版本号

一般来说,子模块的版本号与继承的父模块版本号一致,当执行set -DnewVersion命令对父模块进行修改版本号的操作,会同时修改子模块的版本号。子模块pom.xml中,父模块和子模块的版本号会同步改变:

#  mvn versions:set -DnewVersion=2.2.5.5

[INFO] Building Ambari Main 2.2.1.1.0
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- versions-maven-plugin:2.2:set (default-cli) @ ambari ---
[INFO] Searching for local aggregator root...
[INFO] Local aggregation root: /ambari/ambari-branch-2.2.1
[INFO] Processing change of org.apache.ambari:ambari:2.2.1.1.0 -> 2.2.5.5
[INFO] Processing org.apache.ambari:ambari
[INFO] Updating project org.apache.ambari:ambari
[INFO] from version 2.2.1.1.0 to 2.2.5.5
[INFO]
[INFO] Processing org.apache.ambari:ambari-project
[INFO] Updating parent org.apache.ambari:ambari
[INFO] from version 2.2.1.1.0 to 2.2.5.5
[INFO] Updating project org.apache.ambari:ambari-project
[INFO] from version 2.2.1.1.0 to 2.2.5.5
[INFO]
[INFO] Processing org.apache.ambari.contrib.views:ambari-contrib-views
[INFO] Updating parent org.apache.ambari:ambari-project
[INFO] from version 2.2.1.1.0 to 2.2.5.5
[INFO] Updating project org.apache.ambari.contrib.views:ambari-contrib-views
[INFO] from version 2.2.1.1.0 to 2.2.5.5

参考上面的日志发现,由于Ambari Project的版本号继承自Ambari,此时对Ambari进行修改版本号的操作,Ambari Project的版本号会随之改变。此时对于Ambari Project的pom.xml文件有两步操作:

更改继承自父模块的版本号:

[INFO] Updating parent org.apache.ambari:ambari
[INFO] from version 2.2.1.1.0 to 2.2.5.5

更改后的父模块版本号:

<parent>
  <groupId>org.apache.ambari</groupId>
   <artifactId>ambari</artifactId>
   <version>2.2.5.5</version>
 </parent>

更改子模块本身的版本号:

[INFO] Updating project org.apache.ambari:ambari-project
[INFO] from version 2.2.1.1.0 to 2.2.5.5

更改后的子模块版本号:

<groupId>org.apache.ambari</groupId>
<artifactId>ambari-project</artifactId>
<version>2.2.5.5/version>

但是如果执行set -DnewVersion命令对子模块进行了单独的修改版本号,就会出现如下所示的子模块版本号与父模块版本号不一致的情况,如下所示我单独将Ambari View的版本号由2.2.1.0.0改为了2.2.2.0.0。

<parent>
    <groupId>org.apache.ambari</groupId>
    <artifactId>ambari-project</artifactId>
    <version>2.2.1.0.0</version>
    <relativePath>../ambari-project</relativePath>
</parent>
<modelVersion>4.0.0</modelVersion>
<groupId>org.apache.ambari</groupId>
<artifactId>ambari-views</artifactId>
<packaging>jar</packaging>
<name>Ambari Views</name>
<version>2.2.2.0.0</version>

此时再执行set -DnewVersion命令对父模块进行修改版本号的操作,将不会修改子模块的版本号。子模块pom.xml中,只有父模块版本号会发生改变:

[INFO] Processing org.apache.ambari:ambari-shell
[INFO] Updating parent org.apache.ambari:ambari-project
[INFO] from version 2.2.1.1.0 to 2.2.5.5
[INFO] Updating project org.apache.ambari:ambari-shell
[INFO] from version 2.2.1.1.0 to 2.2.5.5
[INFO]
[INFO] Processing org.apache.ambari:ambari-views
[INFO] Updating parent org.apache.ambari:ambari-project
[INFO] from version 2.2.1.1.0 to 2.2.5.5
[INFO]
[INFO] Processing org.apache.ambari:ambari-web
[INFO] Updating parent org.apache.ambari:ambari-project
[INFO] from version 2.2.1.1.0 to 2.2.5.5
[INFO] Updating project org.apache.ambari:ambari-web
[INFO] from version 2.2.1.1.0 to 2.2.5.5

上面的例子中我单独修改了Ambari View的版本号。Ambari View的版本号继承自Ambari Project,Ambari Project的版本号继承自Ambari Main,此时对Ambari Main进行修改版本号的操作,Ambari Project的版本号会随之改变,而Ambari View的版本号将不会改变。

Ps:编译Ambari过程中被版本号的问题折磨了许久,通过baidu和Google也没找到有用的答案,最后通过粗略自学Maven、读Ambari各个模块的pom文件、反复编译实验,找到了一些来龙去脉。把实验过程中的一些心得记录到CSDN,希望能帮到进行Ambari开发的兄弟。由于内容是我自己做开发过程中的一些笔记,不代表权威,谬误之处请指正。

原文转载至: http://blog.csdn.net/royma_1990/article/details/51749981

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值