azkaban 安装

准备工作

1） 创建相应的文件夹

mkdir /opt/azkaban-3.34

2） 编译环境
yum -y install make gcc-c++ cmake bison-devel ncurses-devel

构建azkaban

获取源代码

1. git clone https://github.com/azkaban/azkaban.git
   2.进入目录执行项目构建
   cd /opt/azkaban-3.34.x;
   ./gradlew build installDist
   执行完构建之后 源代码构建完成形成完成的项目结构此时
   maven下载太慢的话可以修改gradle仓库库的地址 ，修改文件build.gradle
   替换gradle maven仓库地址

allprojects {
    repositories {
        maven{ url 'http://maven.aliyun.com/nexus/content/groups/public'}
        mavenLocal()
    }
}

我们就可以进行下面的部署工作了

部署 Azkaban

部署mysql ，这里就不解释如何部署mysql了

直接使用之前安装的db进行安装
2. 在mysql 服务器创建 Azkaban 数据库
MYSQL> CREATE DATABASE azkaban;
3. 授权用户访问
MYSQL> CREATE USER ‘root’@’%’ IDENTIFIED BY ‘123456’;
MYSQL> GRANT ALL ON azkaban.* to ‘root’@’%’ WITH GRANT OPTION;
4. 设置mysql 包大小，并重启mysql
Vi /etc/my.cnf
max_allowed_packet=1024M
5. 执行sql 文件创建表结构
sql路径：/opt/azkaban-3.34.x.src/azkaban-db/build/sql/create-all-sql-0.1.0-SNAPSHOT.sql

安装azkaban-web-server

1.       将/opt/azkaban-3.34.x/azkaban-web-server/build/distributions/azkaban-web-server-0.1.0-SNAPSHOT.tar.gz解压缩，并拷贝到安装服务器
   

   cp /opt/azkaban-3.34.x/azkaban-web-server/build/distributions/azkaban-web-server-0.1.0-SNAPSHOT.tar.gz /opt/azkaban-3.34
   cd /opt/azkaban-3.34
   tar zxvf azkaban-web-server-0.1.0-SNAPSHOT.tar.gz
   mv azkaban-web-server-0.1.0-SNAPSHOT web-server
   cd web-server
   mkdir conf extlib plugins azkaban logs
   cp /opt/hadoop-2.8.2/share/hadoop/common/hadoop-common-2.8.2.jar /opt/azkaban-3.34/web-server/extlib/
   cp /opt/azkaban-3.34.x/azkaban-solo-server/build/resources/main/conf/* /opt/azkaban-3.34/web-server/conf
   cp /opt/azkaban-3.34.x/azkaban-solo-server/build/resources/main/log4j.properties /opt/azkaban-3.34/web-server/conf/
   cp /opt/azkaban-3.34.x/azkaban-web-server/src/main/resources/azkaban/* /opt/azkaban-3.34/web-server/azkaban -r
   cp /opt/mysql-connector-java-5.1.44-bin.jar /opt/azkaban-3.34/web-server/extlib/
   cd /opt/azkaban-3.34/web-server
   keytool -keystore keystore -alias jetty -genkey -keyalg RSA #密码 123456

azkaban.properties

	echo '' > /opt/azkaban-3.34/web-server/conf/azkaban.properties
	vi /opt/azkaban-3.34/web-server/conf/azkaban.properties

# Azkaban Personalization Settings
azkaban.name=Test
azkaban.label=My Local Azkaban
azkaban.color=#FF3601
azkaban.default.servlet.path=/index
web.resource.dir=web/
default.timezone.id=Asia/Shanghai

# Azkaban UserManager class
user.manager.class=azkaban.user.XmlUserManager
user.manager.xml.file=conf/azkaban-users.xml

# Loader for projects
executor.global.properties=conf/global.properties
azkaban.project.dir=projects

database.type=mysql
mysql.port=3306
mysql.host=host1
mysql.database=azkaban
mysql.user=root
mysql.password=123456
mysql.numconnections=100

# Velocity dev mode
velocity.dev.mode=false

# Azkaban Jetty server properties.
jetty.maxThreads=25
jetty.ssl.port=8443
# jetty.use.ssl=false
jetty.port=8081
jetty.keystore=keystore
jetty.password=123456
jetty.keypassword=123456
jetty.truststore=keystore
jetty.trustpassword=123456
# Azkaban Executor settings
executor.port=12321

# mail settings
mail.sender= 
mail.host=
job.failure.email=
job.success.email=

lockdown.create.projects=false

cache.directory=cache

# JMX stats
jetty.connector.stats=true
executor.connector.stats=true

用户设置

安装 executor

cp /opt/azkaban-3.34.x/azkaban-exec-server/build/distributions/azkaban-exec-server-0.1.0-SNAPSHOT.tar.gz /opt/azkaban-3.34
 cd /opt/azkaban-3.34
tar zxvf azkaban-exec-server-0.1.0-SNAPSHOT.tar.gz
mv azkaban-exec-server-0.1.0-SNAPSHOT executor
cd executor
mkdir -p extlib plugins/jobtypes conf
cp /opt/hadoop-2.8.2/share/hadoop/common/hadoop-common-2.8.2.jar /opt/azkaban-3.34/executor/extlib/
cp /opt/azkaban-3.34.x/azkaban-solo-server/build/resources/main/conf/* /opt/azkaban-3.34/executor/conf
cp /opt/azkaban-3.34.x/azkaban-solo-server/build/resources/main/log4j.properties /opt/azkaban-3.34/executor/conf/
cp /opt/azkaban-3.34.x/azkaban-solo-server/src/main/resources/commonprivate.properties /opt/azkaban-3.34/executor/plugins/jobtypes
cp /opt/mysql-connector-java-5.1.44-bin.jar /opt/azkaban-3.34/executor/extlib/

azkaban.properties

	echo '' >/opt/azkaban-3.34/executor/conf/azkaban.properties
	vi /opt/azkaban-3.34/executor/conf/azkaban.properties

# Azkaban
 default.timezone.id=Asia/Shanghai

 # Azkaban JobTypes Plugins
 azkaban.jobtype.plugin.dir=plugins/jobtypes

 # Loader for projects
 executor.global.properties=conf/global.properties
 azkaban.project.dir=projects

database.type=mysql
mysql.port=3306
mysql.host=host1
mysql.database=azkaban
mysql.user=root
mysql.password=123456
mysql.numconnections=100

# Azkaban Executor settings
executor.maxThreads=50
executor.port=12321
executor.flow.threads=30

# JMX stats
jetty.connector.stats=true
executor.connector.stats=true

# uncomment to enable inmemory stats for azkaban
#executor.metric.reports=true
#executor.metric.milisecinterval.default=60000

数据库中插入executor节点

azkaban数据库
insert into executors(host,port,active) values(“host1”,12321,1);

启动azkaban（阿兹卡班）服务

1） 先启动执行者服务，然后在启动web服务
cd /opt/azkaban-3.34/executor
bin/start-exec.sh
cd /opt/azkaban-3.34/web-server
bin/start-web.sh
9. 通过:https://ipxxxx:8443访问azkaban

错误解决

java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration

hadoop-common-x.jar引入