Azkaban 调度

1.azkan 的Work Flow 调度

1.新建一个项目

2. 编写多个脚本执行

flow 脚本参数解释：

注意：脚本不能含有多余的空格和tab (\t)键

1)name：作业名称
2)type：作业类型（详细类型配置见第3章）
3)config：和作业类型相关的配置，也以KV值形式
4)dependsOn：作业依赖名称

实例：bigdata.flow 文件内容

nodes:
  - name: mysql_bidding_to_hdfs
    type: command
    config:
     command: /opt/shell/mysql_bidding_to_hdfs.sh all ${dt}
    
  - name: mysql_request_to_hdfs
    type: command
    config:
     command: /opt/shell/mysql_request_to_hdfs.sh all ${dt}
     
  - name: mysql_resource_to_hdfs
    type: command
    config: 
     command: /opt/shell/mysql_resource_to_hdfs.sh all ${dt}
     
  - name: mysql_shipment_to_hdfs
    type: command
    config: 
     command: /opt/shell/mysql_shipment_to_hdfs.sh all ${dt}
    
  - name: mysql_system_to_hdfs
    type: command
    config: 
     command: /opt/shell/mysql_system_to_hdfs.sh all ${dt}
    
  - name: mysql_trade_to_hdfs
    type: command
    config:
     command: /opt/shell/mysql_trade_to_hdfs.sh all ${dt}
    
  - name: hdfs_to_bidding_ods_db
    type: command
    dependsOn:
     - mysql_bidding_to_hdfs
    config:
     command: /opt/shell/hdfs_to_bidding_ods_db.sh all ${dt}
    
  - name: hdfs_to_request_ods_db
    type: command
    dependsOn: 
     - mysql_request_to_hdfs
    config:
     command: /opt/shell/hdfs_to_request_ods_db.sh all ${dt}
  - name: hdfs_to_shipment_ods_db
    type: command
    dependsOn: 
     - mysql_shipment_to_hdfs
    config:
     command: /opt/shell/hdfs_to_shipment_ods_db.sh all ${dt}
  - name: hdfs_to_source_ods_db
    type: command
    dependsOn: 
     - mysql_resource_to_hdfs
    config:
     command: /opt/shell/hdfs_to_source_ods_db.sh  all ${dt}
  - name: hdfs_to_system_ods_db
    type: command
    dependsOn: 
     - mysql_system_to_hdfs
    config:
     command: /opt/shell/hdfs_to_system_ods_db.sh all ${dt}
  - name: hdfs_to_trade_ods_db
    type: command
    dependsOn: 
     - mysql_trade_to_hdfs
    config:
     command: /opt/shell/hdfs_to_trade_ods_db.sh all ${dt}
 
  - name: dwd_to_dws
    type: command
    dependsOn: 
     - hdfs_to_bidding_ods_db
     - hdfs_to_request_ods_db
     - hdfs_to_shipment_ods_db
     - hdfs_to_source_ods_db
     - hdfs_to_system_ods_db
     - hdfs_to_trade_ods_db
    config:
     command: /opt/shell/dwd_to_dws.sh ${dt}
  - name: dws_to_ads
    type: command
    dependsOn: 
     - dwd_to_dws
    config:
     command: /opt/shell/dws_to_ads.sh ${dt}

3.azkaban.project 编写文件二：

azkaban-flow-version: 2.0

4. 两个文件打包为zip 格式，并且上传

5.有向五环图

6.执行：