Azkaban学习之路

概述：

Azkaban是在LinkedIn上创建的批处理工作流作业调度程序，用于运行Hadoop作业。Azkaban通过作业依赖性解决订单，并提供易于使用的Web用户界面来维护和跟踪您的工作流程

特点：

1）提供可视化界面--web界面

2）工作流->jar包

3) 设置任务间的关系

4）权限设置

5）模块化

6）随时启停任务

7）课查看日志记录

• 兼容任何版本的Hadoop
• 易于使用的Web UI
• 简单的Web和http工作流上传
• 项目工作区
• 调度工作流程
• 模块化和可插入
• 身份验证和授权
• 跟踪用户操作
• 有关失败和成功的电子邮件提醒
• SLA警报和自动查杀
• 重试失败的工作

与Oozie对比：

azkaban是轻量级的调度工具

1）功能

两个任务流调度可以调度使用mapreduce java 脚本工作流任务，定时调度任务

2）使用

az直接传参

Oozie 直接传单 EL表达式

3）定时

az基于时间的执行任务

Oozie任务基于时间和数据

4）资源

az权限控制

Oozie无权限控制

安装部署

1）创建ssl配置

2）同步时间

生成时区文件

tzselect 生成 5 -》9-》1-》yes

集群同步时间

date -s 具体时间

下载镜像

https://github.com/azkaban/azkaban  azkaban-3.65.0.tar.gz

解压后目录介绍

azkaban-common 常用的工具类(基础架构) 
azkaban-db 与数据库相关的 
azkaban-exec-server 执行的server 
azkaban-hadoop-security-plugin       hadoop安全的插件(不用去管它) 
azkaban-solo-server 我们所要部署的模式 
azkaban-spi 存储一些接口相关的东西(异常处理这些) 
azkaban-web-server web-server(ui上的东西)

当前版本需要编译：

编译命令：

 ->cd azkaban-3.65.0

 

此时会下载https://services.gradle.org/distributions/gradle-4.6-all.zip 卡一段时间 ，

gradle-4.6-all.zip可提前下载好，放到/opt/azkaban-3.65.0_src/gradle/wrapper目录下，然后修改配置参数

进进入到该目录下/opt/azkaban-3.65.0_src/gradle/wrapper

gradle-wrapper.properties 的

distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
zipStoreBase=GRADLE_USER_HOME
zipStorePath=wrapper/dists
#distributionUrl=https\://services.gradle.org/distributions/gradle-4.6-all.zip
distributionUrl=gradle-4.6-all.zip

 

->./gradlew distTar 执行，这一步我执行了快一个小时，太慢了，并且同一个包下载超时了好几次，才下载完编译完成

编译后获取压缩包：

cd  /home/hadoop/apps/azkaban/azkaban-exec-server/build/distributions
cd /home/hadoop/apps/azkaban/azkaban-web-server/build/distributions
cd /home/hadoop/apps/azkaban/azkaban-db/build/distributions

cd /home/hadoop/apps/azkaban/azkaban-solo_server/build/distributions
将distributions下的azkaban-db-0.1.0-SNAPSHOT.tar.gz 和，azkaban-solo-server-0.1.0-SNAPSHOT.tar.gz，
azkaban-exec-server-0.1.0-SNAPSHOT.tar.gz azkaban-web-server-0.1.0-SNAPSHOT.tar.gz

分别解压到固定一个目录下

配置修改：

azkaban-web-server/conf/azkaban.properties和azkaban-solo-server/conf/azkaban.properties

# Azkaban Personalization Settings
azkaban.name=Hello World
azkaban.label=My Local Azkaban
azkaban.color=#FF3601
azkaban.default.servlet.path=/index
web.resource.dir=/opt/azkaban-3.65.0/azkaban-web-server/web/
default.timezone.id=Asia/Shanghai
# Azkaban UserManager class
user.manager.class=azkaban.user.XmlUserManager
user.manager.xml.file=/opt/azkaban-3.65.0/azkaban-web-server/conf/azkaban-users.xml
# Loader for projects
executor.global.properties=/opt/azkaban-3.65.0/azkaban-web-server/conf/global.properties
azkaban.project.dir=projects
# Velocity dev mode
velocity.dev.mode=false
# Azkaban Jetty server properties.
jetty.use.ssl=true
jetty.maxThreads=25
jetty.port=8081
jetty.ssl.port=8448
jetty.keystore=/opt/keystore/keystore
jetty.password=root9211
jetty.keypassword=root9211
jetty.truststore=/opt/keystore/keystore
jetty.trustpassword=root9211
# Where the Azkaban web server is located
azkaban.webserver.url=http://192.168.8.128:8081
# mail settings
mail.sender=
mail.host=
# User facing web server configurations used to construct the user facing server URLs. They are useful when there is a reverse proxy between Azkaban web servers and users.
# enduser -> myazkabanhost:443 -> proxy -> localhost:8081
# when this parameters set then these parameters are used to generate email links.
# if these parameters are not set then jetty.hostname, and jetty.port(if ssl configured jetty.ssl.port) are used.
# azkaban.webserver.external_hostname=myazkabanhost.com
# azkaban.webserver.external_ssl_port=443
# azkaban.webserver.external_port=8081
job.failure.email=
job.success.email=
lockdown.create.projects=false
cache.directory=cache
# JMX stats
jetty.connector.stats=true
executor.connector.stats=true
#Multiple Executor
azkaban.use.multiple.executors=true
# Azkaban plugin settings
azkaban.jobtype.plugin.dir=/opt/azkaban-3.65.0/azkaban-exec-server/plugins/jobtypes
# Azkaban mysql settings by default. Users should configure their own username and password.
database.type=mysql
mysql.port=3306
mysql.host=192.168.8.128
mysql.database=azkaban
mysql.user=root
mysql.password=root9211
mysql.numconnections=100
# Azkaban Executor settings
executor.maxThreads=50
executor.flow.threads=30
executor.port=12321

 

 

 

 

生成秘钥：

keytool -keystore keystore -alias jetty -genkey -keyalg RSA

 keytool -keystore keystore -alias jetty -genkey -keyalg RSA
Enter keystore password:  
Re-enter new password: 
What is your first and last name?
  [Unknown]:  
What is the name of your organizational unit?
  [Unknown]:  
What is the name of your organization?
  [Unknown]:  
What is the name of your City or Locality?
  [Unknown]:  shanghai
What is the name of your State or Province?
  [Unknown]:  shanghai
What is the two-letter country code for this unit?
  [Unknown]:  China
Is CN=Unknown, OU=Unknown, O=Unknown, L=shanghai, ST=shanghai, C=China correct?
  [no]:  y

Enter key password for <jetty>
        (RETURN if same as keystore password):  
Re-enter new password: 
They don't match. Try again
Enter key password for <jetty>
        (RETURN if same as keystore password):  
Re-enter new password: 

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore keystore -destkeystore keystore -deststoretype pkcs12".

生成keystore，将其移动到server文件夹下

时间同步设置

date -s '2019-06-02 22:34:50'

hwclock -w

tzselect 5-》9-》1-》Asia/shanghai

修改配置参数；

azkaban-web-server/conf/azkaban-web-server/bin

启动报错：

-》./start-exec.sh 

Error injecting constructor, java.io.FileNotFoundException: conf/global.properties (No such file or directory)
  at azkaban.execapp.FlowRunnerManager.<init>(FlowRunnerManager.java:148)
  at azkaban.execapp.FlowRunnerManager.class(FlowRunnerManager.java:93) 

解决方案：

executor.global.properties=../conf/global.properties 按照自己的进行修改

2018/12/26 22:18:50.219 +0800 INFO [AzkabanExecutorServer] [Azkaban] Registering MBeans...
2018/12/26 22:18:50.234 +0800 INFO [AzkabanExecutorServer] [Azkaban] Bean azkaban.jmx.JmxJettyServer registered.
2018/12/26 22:18:50.237 +0800 INFO [AzkabanExecutorServer] [Azkaban] Bean azkaban.execapp.jmx.JmxFlowRunnerManager registered.
2018/12/26 22:18:50.238 +0800 INFO [AzkabanExecutorServer] [Azkaban] Bean azkaban.execapp.jmx.JmxJobMBeanManager registered.
2018/12/26 22:18:50.243 +0800 INFO [AzkabanExecutorServer] [Azkaban] Bean azkaban.execapp.jmx.JmxJobCallback registered.
2018/12/26 22:18:50.244 +0800 INFO [AzkabanExecutorServer] [Azkaban] No value for property: jmx.attribute.processor.class was found
2018/12/26 22:18:50.533 +0800 ERROR [MySQLDataSource] [Azkaban] Failed to find write-enabled DB connection. Wait 15 seconds and retry. No.Attempt = 1
java.sql.SQLException: Cannot create PoolableConnectionFactory (Access denied for user 'root'@'Master' (using password: YES))
        at org.apache.commons.dbcp2.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:2294)

解决方案：

MySql中user表中主机配置 配置只要是root用户+密码，在任何主机上都能登录MySQL数据库。

1）进入mysql [root@hadoop102 mysql-libs]# mysql -uroot -p000000

2）显示数据库 mysql>show databases;

3）使用mysql数据库 mysql>use mysql;

4）展示mysql数据库中的所有表 mysql>show tables;

5）展示user表的结构     mysql>desc user;

6）查询user表 mysql>select User, Host, Password from user;

7）修改user表，把Host表内容修改为%     mysql>update user set host='%' where host='localhost';

8）删除root用户的其他host mysql>delete from user where Host='hadoop102 '; mysql>delete from user where Host='127.0.0.1'; mysql>delete from user where Host='::1';

9）刷新     mysql>flush privileges;

10）退出 mysql> quit;

2018/12/27 19:42:41.012 +0800 INFO [JobCallbackManager] [Azkaban] azkabanHostName 192.168.8.128:8081
2018/12/27 19:42:41.012 +0800 INFO [AzkabanExecutorServer] [Azkaban] Registering MBeans...
2018/12/27 19:42:41.034 +0800 INFO [AzkabanExecutorServer] [Azkaban] Bean azkaban.jmx.JmxJettyServer registered.
2018/12/27 19:42:41.036 +0800 INFO [AzkabanExecutorServer] [Azkaban] Bean azkaban.execapp.jmx.JmxFlowRunnerManager registered.
2018/12/27 19:42:41.037 +0800 INFO [AzkabanExecutorServer] [Azkaban] Bean azkaban.execapp.jmx.JmxJobMBeanManager registered.
2018/12/27 19:42:41.038 +0800 INFO [AzkabanExecutorServer] [Azkaban] Bean azkaban.execapp.jmx.JmxJobCallback registered.
2018/12/27 19:42:41.038 +0800 INFO [AzkabanExecutorServer] [Azkaban] No value for property: jmx.attribute.processor.class was found
2018/12/27 19:42:41.593 +0800 INFO [AzkabanExecutorServer] [Azkaban] This executor wasn't found in the DB. Adding self.
2018/12/27 19:42:41.604 +0800 INFO [AzkabanExecutorServer] [Azkaban] Started Executor Server on Master:41490
2018/12/27 19:42:41.614 +0800 INFO [AzkabanExecutorServer] [Azkaban] Setting timezone to Asia/Shangha

解决方案：

info可忽略

 

 ERROR [PluginCheckerAndActionsLoader] [Azkaban] plugin path plugins/triggers doesn't exist!
2018/12/27 19:53:03.440 +0800 INFO [AzkabanWebServer] [Azkaban] Setting timezone to Asia/Shanghai
2018/12/27 19:53:03.440 +0800 INFO [AzkabanWebServer] [Azkaban] Registering MBeans...
2018/12/27 19:53:03.447 +0800 INFO [AzkabanWebServer] [Azkaban] Bean azkaban.jmx.JmxJettyServer registered.
2018/12/27 19:53:03.448 +0800 INFO [AzkabanWebServer] [Azkaban] Bean azkaban.jmx.JmxTriggerManager registered.
2018/12/27 19:53:03.449 +0800 INFO [AzkabanWebServer] [Azkaban] Bean azkaban.jmx.JmxExecutorManager registered.
2018/12/27 19:53:03.457 +0800 INFO [AzkabanWebServer] [Azkaban] Bean org.apache.log4j.jmx.HierarchyDynamicMBean registered.
2018/12/27 19:53:03.458 +0800 INFO [AzkabanWebServer] [Azkaban] ************* loginLoggerObjName is null, make sure there is a logger with name azkaban.webapp.servlet.LoginAbstractAzkabanServlet
2018/12/27 19:53:03.458 +0800 INFO [ExecutorManager] [Azkaban] Initializing executors from database.
2018/12/27 19:53:03.460 +0800 ERROR [ExecutorManager] [Azkaban] No active executors found
2018/12/27 19:53:03.460 +0800 ERROR [StdOutErrRedirect] [Azkaban] Exception in thread "main" 
2018/12/27 19:53:03.460 +0800 ERROR [StdOutErrRedirect] [Azkaban] azkaban.executor.ExecutorManagerException: No active executors found
2018/12/27 19:53:03.460 +0800 ERROR [StdOutErrRedirect] [Azkaban]       at azkaban.executor.ActiveExecutors.setupExecutors(ActiveExecutors.java:52)
2018/12/27 19:53:03.461 +0800 ERROR [StdOutErrRedirect] [Azkaban]       at azkaban.executor.ExecutorManager.setupExecutors(ExecutorManager.java:242)
2018/12/27 19:53:03.461 +0800 ERROR [StdOutErrRedirect] [Azkaban]       at azkaban.executor.ExecutorManager.initialize(ExecutorManager.java:167)
2018/12/27 19:53:03.461 +0800 ERROR [StdOutErrRedirect] [Azkaban]       at azkaban.executor.ExecutorManager.start(ExecutorManager.java:180)
2018/12/27 19:53:03.461 +0800 ERROR [StdOutErrRedirect] [Azkaban]       at azkaban.webapp.AzkabanWebServer.launch(AzkabanWebServer.java:235)
2018/12/27 19:53:03.461 +0800 ERROR [StdOutErrRedirect] [Azkaban]       at azkaban.webapp.AzkabanWebServer.main(AzkabanWebServer.java:226)

界面显示问题

输入账号密码后无法登陆，按钮不管用

解决方案：

web.resource.dir=/opt/azkaban-3.65.0/azkaban-web-server/web/ 添加绝对路径

写的command命令执行后报错

 [Azkaban] 192.168.8.1 admin "GET /flowtrigger ajax=fetchTrigger&projectId=2&flowId=command HTTP/1.1" browser
2018/12/28 21:56:22.533 +0800 ERROR [FlowTriggerScheduler] [Azkaban] unable to get scheduled flow triggers
java.lang.NullPointerException
        at azkaban.flowtrigger.quartz.FlowTriggerScheduler.getScheduledFlowTriggerJobs(FlowTriggerScheduler.java:132)
        at azkaban.webapp.servlet.FlowTriggerServlet.ajaxFetchTrigger(FlowTriggerServlet.java:63)
        at azkaban.webapp.servlet.FlowTriggerServlet.handleAJAXAction(FlowTriggerServlet.java:103)
        at azkaban.webapp.servlet.FlowTriggerServlet.handleGet(FlowTriggerServlet.java:53)
        at azkaban.webapp.servlet.LoginAbstractAzkabanServlet.doGet(LoginAbstractAzkabanServlet.java:123)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:668)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:770)
        at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
        at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:401)
        at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
        at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
        at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
        at org.mortbay.jetty.Server.handle(Server.java:326)
        at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
        at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
        at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
        at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
        at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
        at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
        at org.mortbay.jetty.security.SslSocketConnector$SslConnection.run(SslSocketConnector.java:713)
        at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
2018/12/28 21:56:22.539 +0800 INFO [LoginAbstractAzkabanServlet] [Azkaban] 192.168.8.1 admin "GET /manager project=command&ajax=fetchflowgraph&flow=command HTTP/1.1" browser
2018/12/28 21:56:24.099 +0800 INFO [LoginAbstractAzkabanServlet] [Azkaban] 192.168.8.1 admin "GET /manager project=command&flow=command&ajax=fetchFlowExecutions&start=0&length=16 HTTP/1.1" browser
2018/12/28 21:56:26.039 +0800 INFO [LoginAbstractAzkabanServlet] [Azkaban] 192.168.8.1 admin "GET /flowtriggerinstance project=command&flow=command&ajax=fetchTriggerInstances&start=0&length=16 HTTP/1.1" browser
2018/12/28 21:59:15.598 +0800 INFO [ExecutorManager] [Azkaban] Cleaning old logs from execution_logs
2018/12/28 21:59:15.598 +0800 INFO [ExecutorManager] [Azkaban] Cleaning old log files before 2018-10-05T21:59:15.598+08:00
2018/12/28 21:59:15.600 +0800 INFO [ExecutorManager] [Azkaban] Cleaned up 0 log entries.
2018/12/28 21:59:15.600 +0800 INFO [ExecutorManager] [Azkaban] log clean up time: 0 seconds.
2018/12/28 22:05:32.548 +0800 INFO [LoginAbstractAzkabanServlet] [Azkaban] 192.168.8.1 admin "GET /executor project=command&ajax=flowInfo&flow=command HTTP/1.1" browser
2018/12/28 22:05:32.562 +0800 INFO [LoginAbstractAzkabanServlet] [Azkaban] 192.168.8.1 admin "GET /executor project=command&ajax=fetchscheduledflowgraph&flow=command HTTP/1.1" browser
2018/12/28 22:05:46.113 +0800 INFO [LoginAbstractAzkabanServlet] [Azkaban] 192.168.8.1 admin "GET /manager project=command HTTP/1.1" browser
2018/12/28 22:05:48.824 +0800 INFO [LoginAbstractAzkabanServlet] [Azkaban] 192.168.8.1 admin "GET /manager project=command&ajax=fetchflowjobs&flow=command HTTP/1.1" browser
2018/12/28 22:05:56.655 +0800 INFO [LoginAbstractAzkabanServlet] [Azkaban] 192.168.8.1 admin "GET /executor project=command&ajax=fetchscheduledflowgraph&flow=command HTTP/1.1" browser
2018/12/28 22:05:56.656 +0800 INFO [LoginAbstractAzkabanServlet] [Azkaban] 192.168.8.1 admin "GET /executor project=command&ajax=flowInfo&flow=command HTTP/1.1" browser
2018/12/28 22:06:02.132 +0800 INFO [LoginAbstractAzkabanServlet] [Azkaban] 192.168.8.1 admin "GET /executor projectId=2&project=command&ajax=executeFlow&flow=command&disabled=%5B%5D&failureEmailsOverride=false&successEmailsOverride=false&failureAction=finishCurrent&failureEmails=&successEmails=&notifyFailureFirst=false&notifyFailureLast=false&concurrentOption=skip HTTP/1.1" browser
2018/12/28 22:06:02.133 +0800 INFO [ExecutorManager] [Azkaban] Submitting execution flow command by admin
2018/12/28 22:06:02.136 +0800 INFO [ExecutionFlowDao] [Azkaban] Flow given command given id 6
2018/12/28 22:06:02.188 +0800 ERROR [ExecutorManager] [Azkaban] Failed to update ExecutorInfo for executor : Master:45051 (id: 1), active=true
java.util.concurrent.ExecutionException: org.apache.http.conn.HttpHostConnectException: Connect to Master:45051 [Master/192.168.8.128] failed: Connection refused (Connection refused)
        at java.util.concurrent.FutureTask.report(FutureTask.java:122)
        at java.util.concurrent.FutureTask.get(FutureTask.java:206)
        at azkaban.executor.ExecutorManager.refreshExecutors(ExecutorManager.java:281)

不知道为何一直调用我的上面那个id的端口，后来active改为0 ，然后执行任务，执行成功

 select * from executors;                  
+----+--------+-------+--------+
| id | host   | port  | active |
+----+--------+-------+--------+
|  1 | Master | 45051 |      0 |
| 17 | Master | 12321 |      1 |
+----+--------+-------+--------+