Oozie Coordinator job 之定时任务

Oozie Coordinator job 之定时任务

使用 Coordinator job 可以执行定时任务和时间触发执行
需要注意的是 Oozie 默认使用的时区与中国时区不是一致的，需要进行一点修改

1.关于时区

a.修改 core-site.xml 文件(运行需要)需要清除编译文件，重启 tomcat 服务(不能是UTC+0800)

Oozie 安装及 examples app 的使用

<property>
    <name>oozie.processing.timezone</name>
    <value>GMT+0800</value>
</property>

b.修改 $OOZIE_HOME/oozie-server/webapps/oozie/oozie-console.js 文件(Web显示需要)，无需重启

function getTimeZone() {
Ext.state.Manager.setProvider(new Ext.state.CookieProvider());
return Ext.state.Manager.get("TimezoneId","GMT+0800");
}

2.拷贝 example 文件 cron-schedule

3.编辑 job.properties 文件(注意时间格式)

nameNode=hdfs://cen-ubuntu.cenzhongman.com:8020
jobTracker=localhost:8032
queueName=default
oozieAppsRoot=oozie-apps

oozie.coord.application.path=${nameNode}/user/cen/${oozieAppsRoot}/cron-schedule
start=2017-07-30T14:40+0800
end=2017-07-30T14:59+0800
workflowAppUri=${nameNode}/user/cen/${oozieAppsRoot}/cron-schedule

4.编辑 workflow.xml 文件(内容酌情添加，这里什么也不做)(修改了版本号)

<workflow-app xmlns="uri:oozie:workflow:0.5" name="no-op-wf">
    <start to="end"/>
    <end name="end"/>
</workflow-app>

5.编辑 coordinator.xml 文件(支持两种定时任务方式，下文详细说明)

<coordinator-app name="cron-coord" frequency="0/1 * * * *" start="${start}" end="${end}" timezone="GMT+0800"
                 xmlns="uri:oozie:coordinator:0.4">
        <action>
        <workflow>
            <app-path>${workflowAppUri}</app-path>
            <configuration>
                <property>
                    <name>jobTracker</name>
                    <value>${jobTracker}</value>
                </property>
                <property>
                    <name>nameNode</name>
                    <value>${nameNode}</value>
                </property>
                <property>
                    <name>queueName</name>
                    <value>${queueName}</value>
                </property>
            </configuration>
        </workflow>
    </action>
</coordinator-app>

注意事项

• 修改时区写法
• 修改版本号
• coordinator.xml 文件在本地文件系统中读取，HDFS 中无需修改不影响

6.上传文件至 HDFS 文件系统

7.执行任务

export OOZIE_URL=http://cen-ubuntu:11000/oozie/
bin/oozie job --config oozie-apps/cron-schedule/job.properties -run

关于定时方式

方式一：官方定义方式

EL Constant	Value	Example
${coord:minutes(int n)}	n	${coord:minutes(45)} --> 45
${coord:hours(int n)}	n * 60	${coord:hours(3)} --> 180
${coord:days(int n)}	variable	${coord:days(2)} --> minutes in 2 full days from the current date
${coord:months(int n)}	variable	${coord:months(1)} --> minutes in a 1 full month from the current date
${cron syntax}	variable	${0,10 15 * * 2-6} --> a job that runs every weekday at 3:00pm and 3:10pm UTC time

方式二：corntab方式

Crontab使用参考

Field name	Allowed Values	Allowed Special Characters
Minutes	0-59	, - * /
Hours	0-23	, - * /
Day-of-month	1-31	, - * ? / L W
Month	1-12 or JAN-DEC	, - * /
Day-of-Week	1-7 or SUN-SAT	, - * ? / L #

Example

Cron Expression	Meaning
10 9 * * *	Runs everyday at 9:10am
10,30,45 9 * * *	Runs everyday at 9:10am, 9:30am, and 9:45am
0 * 30 JAN 2-6	Runs at 0 minute of every hour on weekdays and 30th of January
0/20 9-17 * * 2-5	Runs every Mon, Tue, Wed, and Thurs at minutes 0, 20, 40 from 9am to 5pm
1 2 L-3 * *	Runs every third-to-last day of month at 2:01am
1 2 6W 3 ?	Runs on the nearest weekday to March, 6th every year at 2:01am
1 2 * 3 3#2	Runs every second Tuesday of March at 2:01am every year
0 10,13 * * MON-FRI	Runs every weekday at 10am and 1pm

注1：开启了检查频率，导致5分钟以内的频率运行失败

• 错误提示:Error: E1003 : E1003: Invalid coordinator application attributes, Coordinator job with frequency [2] minutes is faster than allowed maximum of 5 minutes (oozie.service.coord.check.maximum.frequency is set to true)

• 错误原因:开启了检查频率，导致5分钟以内的频率运行失败

• 解决：关闭频率检查功能 配置oozie-site.xml文件

    <property>
        <name>oozie.service.coord.check.maximum.frequency</name>
        <value>false</value>
    </property>

参考： 岑忠满的博客新站点 https://cenzm.xyz