黑猴子的家：Centos7 编译 Azkaban 3.59.0

1、官网地址

https://azkaban.github.io/

 

2、下载 Azkaban 3.59 源码

（1）首页 -> Downloads

 

（2）Releases

We use ‘Github Releases’ for creating version numbers. Release tags can be found here: https://github.com/azkaban/azkaban/releases

（3）GitHub -> azkaban3.59.0 Source code(tar.gz)

 

3、编译源码前，环境准备

（1）Maven

[root@hadoop102 software]# mvn -v
Apache Maven 3.5.0 (ff8f5e7444045639af65f6095c62210b5713f426; 2017-04-04T03:39:06+08:00)
Maven home: /opt/module/apache-maven-3.5.0
Java version: 1.8.0_144, vendor: Oracle Corporation
Java home: /opt/module/jdk1.8.0_144/jre
Default locale: zh_CN, platform encoding: UTF-8
OS name: "linux", version: "3.10.0-514.el7.x86_64", arch: "amd64", family: "unix"

（2）Gradle

下载地址
https://services.gradle.org/distributions/gradle-4.6-all.zip

[root@hadoop102 software]# gradle -version

------------------------------------------------------------
Gradle 4.6
------------------------------------------------------------

Build time:   2018-02-28 13:36:36 UTC
Revision:     8fa6ce7945b640e6168488e4417f9bb96e4ab46c

Groovy:       2.4.12
Ant:          Apache Ant(TM) version 1.9.9 compiled on February 2 2017
JVM:          1.8.0_144 (Oracle Corporation 25.144-b01)
OS:           Linux 3.10.0-514.el7.x86_64 amd64

（3）Ant

[root@hadoop102 software]# ant -v
Apache Ant(TM) version 1.9.9 compiled on February 2 2017
Trying the default build file: build.xml
Buildfile: build.xml does not exist!
Build failed

（4）Java

[root@hadoop102 software]# java -version
openjdk version "1.8.0_144"
OpenJDK Runtime Environment (build 1.8.0_144-b14)
OpenJDK 64-Bit Server VM (build 25.102-b14, mixed mode)

4、上传 Linux 解压

[root@hadoop102 software]# tar -xzvf azkaban-3.59.0.tar.gz -C ../module/

5、修改 azkaban 配置文件

防止从网上下载gradle-4.6-all.zip包

（1）进入配置文件目录，查看

[root@hadoop102 software]# cd ../module/azkaban-3.59.0/gradle/wrapper/
[root@hadoop102 wrapper]# pwd
/opt/module/azkaban-3.59.0/gradle/wrapper
[root@hadoop102 wrapper]# ll
总用量 60
-rw-rw-r-- 1 root root 54708 9月  25 02:58 gradle-wrapper.jar
-rw-rw-r-- 1 root root   779 9月  25 02:58 gradle-wrapper.properties

（2）修改前

[root@hadoop102 wrapper]# vim gradle-wrapper.properties

distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
zipStoreBase=GRADLE_USER_HOME
zipStorePath=wrapper/dists
distributionUrl=https\://services.gradle.org/distributions/gradle-4.6-all.zip

（3）修改后

distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
zipStoreBase=GRADLE_USER_HOME
zipStorePath=wrapper/dists
distributionUrl=gradle-4.6-all.zip

6、复制 gradle-4.6-all.zip 包

复制 gradle-4.6-all.zip 包 到 /opt/module/azkaban-3.59.0/gradle/wrapper 目录下

[root@hadoop102 software]# cp gradle-4.6-all.zip ../module/azkaban-3.59.0/gradle/wrapper/

7、开始编译源码

[root@hadoop102 software]# cd ../module/azkaban-3.59.0/
[root@hadoop102 wrapper]# pwd
/opt/module/azkaban-3.59.0/
[root@hadoop102 azkaban-3.59.0]# ./gradlew distTar

Download https://repo.maven.apache.org/maven2/commons-collections/commons-collections/3.2.2/commons-collections-3.2.2.jar

Download https://repo.maven.apache.org/maven2/commons-fileupload/commons-fileupload/1.2.1/commons-fileupload-1.2.1.jar

Download https://repo.maven.apache.org/maven2/com/google/code/gson/gson/2.8.1/gson-2.8.1.jar

Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpclient/4.5.3/httpclient-4.5.3.jar

Download https://repo.maven.apache.org/maven2/net/sf/jopt-simple/jopt-simple/4.3/jopt-simple-4.3.jar

Download https://repo.maven.apache.org/maven2/javax/mail/mail/1.4.5/mail-1.4.5.jar

Download https://repo.maven.apache.org/maven2/mysql/mysql-connector-java/5.1.28/mysql-connector-java-5.1.28.jar

Download https://repo.maven.apache.org/maven2/org/yaml/snakeyaml/1.18/snakeyaml-1.18.jar

Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcore/4.4.6/httpcore-4.4.6.jar

> Task :azkaban-web-server:npmSetup

/opt/module/azkaban-3.59.0/azkaban-web-server/.gradle/npm/npm-v5.6.0/bin/npm -> /opt/module/azkaban-3.59.0/azkaban-web-server/.gradle/npm/npm-v5.6.0/lib/node_modules/npm/bin/npm-cli.js

/opt/module/azkaban-3.59.0/azkaban-web-server/.gradle/npm/npm-v5.6.0/bin/npx -> /opt/module/azkaban-3.59.0/azkaban-web-server/.gradle/npm/npm-v5.6.0/lib/node_modules/npm/bin/npx-cli.js

+ npm@5.6.0

added 476 packages in 79.014s

Download https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.9/commons-codec-1.9.jar

<======-------> 48% EXECUTING [13m 11s]

> :azkaban-common:compileJava > Resolve files of :azkaban-common:compileClasspath > commons-codec-1.9.jar > 15 KB/257 KB downloaded

> :azkaban-web-server:npmSetup

> IDLE

8、编译成功

[root@hadoop102 azkaban-3.59.0]# ./gradlew distTar
Parallel execution with configuration on demand is an incubating feature.
Download https://repo.maven.apache.org/maven2/org/tachyonproject/tachyon/0.6.4/tachyon-0.6.4.jar
Download https://repo.maven.apache.org/maven2/org/apache/hive/hive-metastore/0.12.0/hive-metastore-0.12.0.jar

> Task :azkaban-web-server:npm_install 
added 39 packages in 4.179s

Download https://repo.maven.apache.org/maven2/com/h2database/h2/1.4.193/h2-1.4.193.jar

> Task :az-hadoop-jobtype-plugin:compileJava 
注: 某些输入文件使用或覆盖了已过时的 API。
注: 有关详细信息, 请使用 -Xlint:deprecation 重新编译。
注: /opt/module/azkaban-3.59.0/az-hadoop-jobtype-plugin/src/main/java/azkaban/jobtype/HadoopSecureSparkWrapper.java使用了未经检查或不安全的操作。
注: 有关详细信息, 请使用 -Xlint:unchecked 重新编译。


BUILD SUCCESSFUL in 1m 30s
54 actionable tasks: 9 executed, 45 up-to-date

9、编译成功后，jar包 位置

[root@hadoop102 azkaban-3.59.0]# find /opt/module/azkaban-3.59.0/ -name  *.tar.gz
/opt/module/azkaban-3.59.0/az-crypto/build/distributions/az-crypto-0.1.0-SNAPSHOT.tar.gz
/opt/module/azkaban-3.59.0/az-hadoop-jobtype-plugin/build/distributions/az-hadoop-jobtype-plugin-0.1.0-SNAPSHOT.tar.gz
/opt/module/azkaban-3.59.0/az-hdfs-viewer/build/distributions/az-hdfs-viewer-0.1.0-SNAPSHOT.tar.gz
/opt/module/azkaban-3.59.0/az-jobsummary/build/distributions/az-jobsummary-0.1.0-SNAPSHOT.tar.gz
/opt/module/azkaban-3.59.0/az-reportal/build/distributions/az-reportal-0.1.0-SNAPSHOT.tar.gz
/opt/module/azkaban-3.59.0/azkaban-db/build/distributions/azkaban-db-0.1.0-SNAPSHOT.tar.gz
/opt/module/azkaban-3.59.0/azkaban-exec-server/build/distributions/azkaban-exec-server-0.1.0-SNAPSHOT.tar.gz
/opt/module/azkaban-3.59.0/azkaban-hadoop-security-plugin/build/distributions/azkaban-hadoop-security-plugin-0.1.0-SNAPSHOT.tar.gz
/opt/module/azkaban-3.59.0/azkaban-solo-server/build/distributions/azkaban-solo-server-0.1.0-SNAPSHOT.tar.gz
/opt/module/azkaban-3.59.0/azkaban-web-server/build/distributions/azkaban-web-server-0.1.0-SNAPSHOT.tar.gz
/opt/module/azkaban-3.59.0/azkaban-web-server/.gradle/npm/npm-v5.6.0/lib/node_modules/npm/test/fixtures/github-com-BryanDonovan-dummy-npm-bar.git.tar.gz
/opt/module/azkaban-3.59.0/azkaban-web-server/.gradle/npm/npm-v5.6.0/lib/node_modules/npm/test/fixtures/github-com-BryanDonovan-dummy-npm-buzz.git.tar.gz
/opt/module/azkaban-3.59.0/azkaban-web-server/.gradle/npm/npm-v5.6.0/lib/node_modules/npm/test/fixtures/github-com-BryanDonovan-dummy-npm-foo.git.tar.gz
/opt/module/azkaban-3.59.0/azkaban-web-server/.gradle/npm/npm-v5.6.0/lib/node_modules/npm/test/fixtures/github-com-BryanDonovan-npm-git-test.git.tar.gz