quartz任务调度框架集群搭建
1. quartz 简介
Quartz 是开源任务调度框架中的翘首,它提供了强大任务调度机制,同时保持了使用的简单性。Quartz 允许开发人员灵活地定义触发器的调度时间表,并可以对触发器和任务进行关联映射。此外,Quartz提供了调度运行环境的持久化机制,可以保存并恢复调度现场,即使系统因故障关闭,任务调度现场数据并不会丢失。此外,Quartz还提供了组件式的侦听器、各种插件、线程池等功能。
2.quartz 核心概念
Quartz对任务调度的领域问题进行了高度的抽象,提出了调度器、任务和触发器这3个核心的概念,并在org.quartz通过接口和类对重要的这些核心概念进行描述:
调度器:负责调度作业和触发器;
触发器:设置作业执行的时间、参数、条件等;
作业:定时任务内容,被执行的程序;
以下是quartz1版本的接口定义,quartz2中部分类已变成接口。但概念不变
●Job:是一个接口,只有一个方法void execute(JobExecutionContext context),开发者实现该接口定义运行任务,JobExecutionContext类提供了调度上下文的各种信息。Job运行时的信息保存在 JobDataMap实例中;
●JobDetail:Quartz在每次执行Job时,都重新创建一个Job实例,所以它不直接接受一个Job的实例,相反它接收一个Job实现类,以便运行时通过newInstance()的反射机制实例化Job。因此需要通过一个类来描述Job的实现类及其它相关的静态信息,如Job名字、描述、关联监听器等信息,JobDetail承担了这一角色。
●Trigger:是一个类,描述触发Job执行的时间触发规则。主要有SimpleTrigger和 CronTrigger这两个子类。当仅需触发一次或者以固定时间间隔周期执行,SimpleTrigger是最适合的选择;而CronTrigger则可以通过Cron表达式定义出各种复杂时间规则的调度方案:如每早晨9:00执行,周一、周三、周五下午5:00执行等;
●Calendar:org.quartz.Calendar和java.util.Calendar不同,它是一些日历特定时间点的集合(可以简单地将org.quartz.Calendar看作java.util.Calendar的集合—— java.util.Calendar代表一个日历时间点,无特殊说明后面的Calendar即指org.quartz.Calendar)。一个 Trigger可以和多个Calendar关联,以便排除或包含某些时间点。
假设,我们安排每周星期一早上10:00执行任务,但是如果碰到法定的节日,任务则不执行,这时就需要在Trigger触发机制的基础上使用Calendar进行定点排除。针对不同时间段类型,Quartz在org.quartz.impl.calendar包下提供了若干 个Calendar的实现类,如AnnualCalendar、MonthlyCalendar、WeeklyCalendar分别针对每年、每月和每周 进行定义;
●Scheduler:代表一个Quartz的独立运行容器,Trigger和JobDetail可以注册到 Scheduler中,两者在Scheduler中拥有各自的组及名称,组及名称是Scheduler查找定位容器中某一对象的依据,Trigger的组及名称必须唯一,JobDetail的组和名称也必须唯一(但可以和Trigger的组和名称相同,因为它们是不同类型的)。Scheduler定义了多个接口方法,允许外部通过组及名称访问和控制容器中Trigger和JobDetail。
Scheduler可以将Trigger绑定到某一JobDetail中,这样当Trigger触发时,对应的Job就被执行。一个Job可以对应多个Trigger,但一个Trigger只能对应一个Job。可 以通过SchedulerFactory创建一个Scheduler实例。Scheduler拥有一个SchedulerContext,它类似于ServletContext,保存着Scheduler上下文信息,Job和Trigger都可以访问SchedulerContext内的信息。 SchedulerContext内部通过一个Map,以键值对的方式维护这些上下文数据,SchedulerContext为保存和获取数据提供了多个 put()和getXxx()的方法。可以通过Scheduler# getContext()获取对应的SchedulerContext实例;
●ThreadPool:Scheduler使用一个线程池作为任务运行的基础设施,任务通过共享线程池中的线程提高运行效率。
3.quartz与spring 整合
Spring为创建Quartz的Scheduler、Trigger和JobDetail提供了便利的FactoryBean类,以便能够在Spring容器中享受注入的好处。此外Spring还提供了一些便利工具类直接将Spring中的Bean包装成合法的任务。Spring进一步降低了使用Quartz的难度,能以更具Spring风格的方式使用Quartz。概括来说它提供了两方面的支持:
1)为Quartz的重要组件类提供更具Bean风格的扩展类;
2)提供创建Scheduler的BeanFactory类,方便在Spring环境下创建对应的组件对象,并结合Spring容器生命周期进行启动和停止的动作。
以下是集群的整合步骤: (项目用maven构建,quartz版本为2.2.1,spring版本为3.2.11.release)
(1) 首先配置pom文件:
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-core</artifactId>
<version>${spring.version}</version>
</dependency>
<!-- spring scheduler support -->
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-context-support</artifactId>
<version>${spring.version}</version>
</dependency>
<!-- log -->
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.15</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.6.1</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.6.1</version>
</dependency>
<!-- quartz -->
<dependency>
<groupId>org.quartz-scheduler</groupId>
<artifactId>quartz</artifactId>
<version>2.2.1</version>
</dependency>
<dependency>
<groupId>org.quartz-scheduler</groupId>
<artifactId>quartz-jobs</artifactId>
<version>2.2.1</version>
</dependency>
<dependency>
<groupId>com.oracle</groupId>
<artifactId>ojdbc6</artifactId>
<version>11.2.0.3.0</version>
</dependency>
<dependency>
<groupId>c3p0</groupId>
<artifactId>c3p0</artifactId>
<version>0.9.1.1</version>
</dependency>
(2) 配置spring数据源配置文件
<bean id="quartzOracleDataSource" class="com.mchange.v2.c3p0.ComboPooledDataSource" destroy-method="close">
<property name="driverClass" value="${数据库驱动类名}" />
<property name="jdbcUrl" value="${数据库连接URL}" />
<property name="user" value="${数据库用户 }" />
<property name="password" value="${数据库密码}" />
<property name="initialPoolSize" value="${连接池初始化大小}" />
<property name="minPoolSize" value="1" />
<property name="maxPoolSize" value="${连接池最大链接数}" />
<property name="acquireIncrement" value="当连接池中的连接耗尽的时候c3p0一次同时获取的连接数" />
<property name="maxIdleTime" value="${最大空闲时间,60秒内未使用则连接被丢弃。若为0则永不丢弃}" />
<property name="idleConnectionTestPeriod" value="检查所有连接池中的空闲连接的时间间隔(单位为秒)"/>
<property name="automaticTestTable" value="C3P0测试用的表"/>
</bean>
(3) 配置spring quartz配置文件
<bean name="quartzScheduler"
class="org.springframework.scheduling.quartz.SchedulerFactoryBean">
<!-- 指定quartz数据源,用于持久化quartz的job. quartz集群配置时,不可缺少 -->
<property name="dataSource">
<ref bean="quartzOracleDataSource" />
</property>
<!--applicationContextSchedulerContextKey: 是把spring上下 文以key/value的方式存放在了quartz的上下文中了,
可以用applicationContextSchedulerContextKey所定义的key得到对应的spring上下文-->
<property name="applicationContextSchedulerContextKey" value="springApplicationContextKey" />
<!--configLocation:用于指明quartz的配置文件的位置 -->
<property name="configLocation" value="classpath:quartz.properties" />
<property name="triggers">
<list>
<ref bean="trigger1"/>
</list>
</property>
</bean>
<!—corn 表达式触发器 -->
<bean id="trigger1"
class="org.springframework.scheduling.quartz.CronTriggerBean">
<property name="jobDetail" ref="jobDetail"/>
<property name="cronExpression" value="0 0/5 * ? * * *"/>
</bean>
<bean id="jobDetail"
class="org.springframework.scheduling.quartz.JobDetailFactoryBean">
<property name="jobClass">
<!-- 继承QuartzJobBean的类的引用 -->
<value>com.test.quartz.job.TestQuartzJobBean</value>
</property>
<property name="durability">
<!-- 是否存储到数据库 -->
<value>true</value>
</property>
<property name="jobDataAsMap">
<map>
<entry key="testJobKey">
<ref bean="testJobValue" />
</entry>
</map>
</property>
</bean>
(4) 配置quartz的配置文件
配置文件可参考quartz提供的example.
#==============================================================
#Configure Main Scheduler Properties
#==============================================================
org.quartz.scheduler.instanceName = CCMSQuartzScheduler
org.quartz.scheduler.instanceId = AUTO
org.quartz.scheduler.threadName = CCMS_Quartz_Scheduler
org.quartz.scheduler.makeSchedulerThreadDaemon = true
org.quartz.scheduler.skipUpdateCheck = true
#org.quartz.scheduler.xaTransacted = true
#==============================================================
#Configure ThreadPool
#==============================================================
org.quartz.threadPool.class = org.quartz.simpl.SimpleThreadPool
org.quartz.threadPool.threadCount = 10
org.quartz.threadPool.threadPriority = 9
org.quartz.threadPool.threadsInheritContextClassLoaderOfInitializingThread = true
org.quartz.threadPool.makeThreadsDaemons = true
#==============================================================
#Configure JobStore
#==============================================================
org.quartz.jobStore.misfireThreshold = 60000
org.quartz.jobStore.class = org.quartz.impl.jdbcjobstore.JobStoreTX
org.quartz.jobStore.driverDelegateClass=org.quartz.impl.jdbcjobstore.StdJDBCDelegate
org.quartz.jobStore.dontSetAutoCommitFalse = true
org.quartz.jobStore.tablePrefix = QRTZ_
org.quartz.jobStore.maxMisfiresToHandleAtATime=10
org.quartz.jobStore.isClustered = true
org.quartz.jobStore.clusterCheckinInterval = 20000
(5) 创建quartz 数据库
参见quartz 文件下的doc下的dbTables.
--
-- A hint submitted by a user: Oracle DB MUST be created as "shared" and the
-- job_queue_processes parameter must be greater than 2
-- However, these settings are pretty much standard after any
-- Oracle install, so most users need not worry about this.
--
-- Many other users (including the primary author of Quartz) have had success
-- runing in dedicated mode, so only consider the above as a hint ;-)
--
create user quartz identified by quartz;
grant connect to quartz;
grant resource to quartz;
commit;
delete from qrtz_fired_triggers;
delete from qrtz_simple_triggers;
delete from qrtz_simprop_triggers;
delete from qrtz_cron_triggers;
delete from qrtz_blob_triggers;
delete from qrtz_triggers;
delete from qrtz_job_details;
delete from qrtz_calendars;
delete from qrtz_paused_trigger_grps;
delete from qrtz_locks;
delete from qrtz_scheduler_state;
drop table qrtz_calendars;
drop table qrtz_fired_triggers;
drop table qrtz_blob_triggers;
drop table qrtz_cron_triggers;
drop table qrtz_simple_triggers;
drop table qrtz_simprop_triggers;
drop table qrtz_triggers;
drop table qrtz_job_details;
drop table qrtz_paused_trigger_grps;
drop table qrtz_locks;
drop table qrtz_scheduler_state;
CREATE TABLE qrtz_job_details
(
SCHED_NAME VARCHAR2(120) NOT NULL,
JOB_NAME VARCHAR2(200) NOT NULL,
JOB_GROUP VARCHAR2(200) NOT NULL,
DESCRIPTION VARCHAR2(250) NULL,
JOB_CLASS_NAME VARCHAR2(250) NOT NULL,
IS_DURABLE VARCHAR2(1) NOT NULL,
IS_NONCONCURRENT VARCHAR2(1) NOT NULL,
IS_UPDATE_DATA VARCHAR2(1) NOT NULL,
REQUESTS_RECOVERY VARCHAR2(1) NOT NULL,
JOB_DATA BLOB NULL,
CONSTRAINT QRTZ_JOB_DETAILS_PK PRIMARY KEY (SCHED_NAME,JOB_NAME,JOB_GROUP)
);
CREATE TABLE qrtz_triggers
(
SCHED_NAME VARCHAR2(120) NOT NULL,
TRIGGER_NAME VARCHAR2(200) NOT NULL,
TRIGGER_GROUP VARCHAR2(200) NOT NULL,
JOB_NAME VARCHAR2(200) NOT NULL,
JOB_GROUP VARCHAR2(200) NOT NULL,
DESCRIPTION VARCHAR2(250) NULL,
NEXT_FIRE_TIME NUMBER(13) NULL,
PREV_FIRE_TIME NUMBER(13) NULL,
PRIORITY NUMBER(13) NULL,
TRIGGER_STATE VARCHAR2(16) NOT NULL,
TRIGGER_TYPE VARCHAR2(8) NOT NULL,
START_TIME NUMBER(13) NOT NULL,
END_TIME NUMBER(13) NULL,
CALENDAR_NAME VARCHAR2(200) NULL,
MISFIRE_INSTR NUMBER(2) NULL,
JOB_DATA BLOB NULL,
CONSTRAINT QRTZ_TRIGGERS_PK PRIMARY KEY (SCHED_NAME,TRIGGER_NAME,TRIGGER_GROUP),
CONSTRAINT QRTZ_TRIGGER_TO_JOBS_FK FOREIGN KEY (SCHED_NAME,JOB_NAME,JOB_GROUP)
REFERENCES QRTZ_JOB_DETAILS(SCHED_NAME,JOB_NAME,JOB_GROUP)
);
CREATE TABLE qrtz_simple_triggers
(
SCHED_NAME VARCHAR2(120) NOT NULL,
TRIGGER_NAME VARCHAR2(200) NOT NULL,
TRIGGER_GROUP VARCHAR2(200) NOT NULL,
REPEAT_COUNT NUMBER(7) NOT NULL,
REPEAT_INTERVAL NUMBER(12) NOT NULL,
TIMES_TRIGGERED NUMBER(10) NOT NULL,
CONSTRAINT QRTZ_SIMPLE_TRIG_PK PRIMARY KEY (SCHED_NAME,TRIGGER_NAME,TRIGGER_GROUP),
CONSTRAINT QRTZ_SIMPLE_TRIG_TO_TRIG_FK FOREIGN KEY (SCHED_NAME,TRIGGER_NAME,TRIGGER_GROUP)
REFERENCES QRTZ_TRIGGERS(SCHED_NAME,TRIGGER_NAME,TRIGGER_GROUP)
);
CREATE TABLE qrtz_cron_triggers
(
SCHED_NAME VARCHAR2(120) NOT NULL,
TRIGGER_NAME VARCHAR2(200) NOT NULL,
TRIGGER_GROUP VARCHAR2(200) NOT NULL,
CRON_EXPRESSION VARCHAR2(120) NOT NULL,
TIME_ZONE_ID VARCHAR2(80),
CONSTRAINT QRTZ_CRON_TRIG_PK PRIMARY KEY (SCHED_NAME,TRIGGER_NAME,TRIGGER_GROUP),
CONSTRAINT QRTZ_CRON_TRIG_TO_TRIG_FK FOREIGN KEY (SCHED_NAME,TRIGGER_NAME,TRIGGER_GROUP)
REFERENCES QRTZ_TRIGGERS(SCHED_NAME,TRIGGER_NAME,TRIGGER_GROUP)
);
CREATE TABLE qrtz_simprop_triggers
(
SCHED_NAME VARCHAR2(120) NOT NULL,
TRIGGER_NAME VARCHAR2(200) NOT NULL,
TRIGGER_GROUP VARCHAR2(200) NOT NULL,
STR_PROP_1 VARCHAR2(512) NULL,
STR_PROP_2 VARCHAR2(512) NULL,
STR_PROP_3 VARCHAR2(512) NULL,
INT_PROP_1 NUMBER(10) NULL,
INT_PROP_2 NUMBER(10) NULL,
LONG_PROP_1 NUMBER(13) NULL,
LONG_PROP_2 NUMBER(13) NULL,
DEC_PROP_1 NUMERIC(13,4) NULL,
DEC_PROP_2 NUMERIC(13,4) NULL,
BOOL_PROP_1 VARCHAR2(1) NULL,
BOOL_PROP_2 VARCHAR2(1) NULL,
CONSTRAINT QRTZ_SIMPROP_TRIG_PK PRIMARY KEY (SCHED_NAME,TRIGGER_NAME,TRIGGER_GROUP),
CONSTRAINT QRTZ_SIMPROP_TRIG_TO_TRIG_FK FOREIGN KEY (SCHED_NAME,TRIGGER_NAME,TRIGGER_GROUP)
REFERENCES QRTZ_TRIGGERS(SCHED_NAME,TRIGGER_NAME,TRIGGER_GROUP)
);
CREATE TABLE qrtz_blob_triggers
(
SCHED_NAME VARCHAR2(120) NOT NULL,
TRIGGER_NAME VARCHAR2(200) NOT NULL,
TRIGGER_GROUP VARCHAR2(200) NOT NULL,
BLOB_DATA BLOB NULL,
CONSTRAINT QRTZ_BLOB_TRIG_PK PRIMARY KEY (SCHED_NAME,TRIGGER_NAME,TRIGGER_GROUP),
CONSTRAINT QRTZ_BLOB_TRIG_TO_TRIG_FK FOREIGN KEY (SCHED_NAME,TRIGGER_NAME,TRIGGER_GROUP)
REFERENCES QRTZ_TRIGGERS(SCHED_NAME,TRIGGER_NAME,TRIGGER_GROUP)
);
CREATE TABLE qrtz_calendars
(
SCHED_NAME VARCHAR2(120) NOT NULL,
CALENDAR_NAME VARCHAR2(200) NOT NULL,
CALENDAR BLOB NOT NULL,
CONSTRAINT QRTZ_CALENDARS_PK PRIMARY KEY (SCHED_NAME,CALENDAR_NAME)
);
CREATE TABLE qrtz_paused_trigger_grps
(
SCHED_NAME VARCHAR2(120) NOT NULL,
TRIGGER_GROUP VARCHAR2(200) NOT NULL,
CONSTRAINT QRTZ_PAUSED_TRIG_GRPS_PK PRIMARY KEY (SCHED_NAME,TRIGGER_GROUP)
);
CREATE TABLE qrtz_fired_triggers
(
SCHED_NAME VARCHAR2(120) NOT NULL,
ENTRY_ID VARCHAR2(95) NOT NULL,
TRIGGER_NAME VARCHAR2(200) NOT NULL,
TRIGGER_GROUP VARCHAR2(200) NOT NULL,
INSTANCE_NAME VARCHAR2(200) NOT NULL,
FIRED_TIME NUMBER(13) NOT NULL,
SCHED_TIME NUMBER(13) NOT NULL,
PRIORITY NUMBER(13) NOT NULL,
STATE VARCHAR2(16) NOT NULL,
JOB_NAME VARCHAR2(200) NULL,
JOB_GROUP VARCHAR2(200) NULL,
IS_NONCONCURRENT VARCHAR2(1) NULL,
REQUESTS_RECOVERY VARCHAR2(1) NULL,
CONSTRAINT QRTZ_FIRED_TRIGGER_PK PRIMARY KEY (SCHED_NAME,ENTRY_ID)
);
CREATE TABLE qrtz_scheduler_state
(
SCHED_NAME VARCHAR2(120) NOT NULL,
INSTANCE_NAME VARCHAR2(200) NOT NULL,
LAST_CHECKIN_TIME NUMBER(13) NOT NULL,
CHECKIN_INTERVAL NUMBER(13) NOT NULL,
CONSTRAINT QRTZ_SCHEDULER_STATE_PK PRIMARY KEY (SCHED_NAME,INSTANCE_NAME)
);
CREATE TABLE qrtz_locks
(
SCHED_NAME VARCHAR2(120) NOT NULL,
LOCK_NAME VARCHAR2(40) NOT NULL,
CONSTRAINT QRTZ_LOCKS_PK PRIMARY KEY (SCHED_NAME,LOCK_NAME)
);
create index idx_qrtz_j_req_recovery on qrtz_job_details(SCHED_NAME,REQUESTS_RECOVERY);
create index idx_qrtz_j_grp on qrtz_job_details(SCHED_NAME,JOB_GROUP);
create index idx_qrtz_t_j on qrtz_triggers(SCHED_NAME,JOB_NAME,JOB_GROUP);
create index idx_qrtz_t_jg on qrtz_triggers(SCHED_NAME,JOB_GROUP);
create index idx_qrtz_t_c on qrtz_triggers(SCHED_NAME,CALENDAR_NAME);
create index idx_qrtz_t_g on qrtz_triggers(SCHED_NAME,TRIGGER_GROUP);
create index idx_qrtz_t_state on qrtz_triggers(SCHED_NAME,TRIGGER_STATE);
create index idx_qrtz_t_n_state on qrtz_triggers(SCHED_NAME,TRIGGER_NAME,TRIGGER_GROUP,TRIGGER_STATE);
create index idx_qrtz_t_n_g_state on qrtz_triggers(SCHED_NAME,TRIGGER_GROUP,TRIGGER_STATE);
create index idx_qrtz_t_next_fire_time on qrtz_triggers(SCHED_NAME,NEXT_FIRE_TIME);
create index idx_qrtz_t_nft_st on qrtz_triggers(SCHED_NAME,TRIGGER_STATE,NEXT_FIRE_TIME);
create index idx_qrtz_t_nft_misfire on qrtz_triggers(SCHED_NAME,MISFIRE_INSTR,NEXT_FIRE_TIME);
create index idx_qrtz_t_nft_st_misfire on qrtz_triggers(SCHED_NAME,MISFIRE_INSTR,NEXT_FIRE_TIME,TRIGGER_STATE);
create index idx_qrtz_t_nft_st_misfire_grp on qrtz_triggers(SCHED_NAME,MISFIRE_INSTR,NEXT_FIRE_TIME,TRIGGER_GROUP,TRIGGER_STATE);
create index idx_qrtz_ft_trig_inst_name on qrtz_fired_triggers(SCHED_NAME,INSTANCE_NAME);
create index idx_qrtz_ft_inst_job_req_rcvry on qrtz_fired_triggers(SCHED_NAME,INSTANCE_NAME,REQUESTS_RECOVERY);
create index idx_qrtz_ft_j_g on qrtz_fired_triggers(SCHED_NAME,JOB_NAME,JOB_GROUP);
create index idx_qrtz_ft_jg on qrtz_fired_triggers(SCHED_NAME,JOB_GROUP);
create index idx_qrtz_ft_t_g on qrtz_fired_triggers(SCHED_NAME,TRIGGER_NAME,TRIGGER_GROUP);
create index idx_qrtz_ft_tg on qrtz_fired_triggers(SCHED_NAME,TRIGGER_GROUP);
(6) 创建QuartzJobBean的继承类.
public class TestQuartzJobBean extends QuartzJobBean {
@Override
protected void executeInternal(JobExecutionContext jobexecutioncontext)
throws JobExecutionException {
System.out.println("hello quartz!");
}
}
创建测试类
public class QuartzAppTest {
public static void main(String[] args) {
ClassPathXmlApplicationContext cls =
new ClassPathXmlApplicationContext("classpath:spring-all.xml");
try {
Scheduler sch = (Scheduler) cls.getBean("quartzScheduler");
System.out.println("SchedulerName:" + sch.getSchedulerName());
} catch (SchedulerException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
备注:
1. spring 与quartz 整合时的版本问题;
spring3.1之前只支持quartz1,spring3.1 之后开始支持quartz2. 由于quartz2中很多接口已经从quartz1中的类,抽象成了接口.因此,spring中继承quartz的类的支持bean将无法在quartz2中使用.如遇到此类错误,请找到接口累的factoryBean即可.
例如:
在JobDetailBean 继承JobDetail类,而在quartz中已经变成了接口,则需要使用JobDetailFactoryBean去生成JobDetail 实例.
2. 关于quartz实例化到数据库中的问题
(1)Spring提供org.springframework.scheduling.quartz.MethodInvokingJobDetailFactoryBean实例化JobDetail对象,但此接口无法序列化到数据库.如需集群操作,请勿使用.
(2) jobDetail 需要设置 Durability属性.此属性默认为false.
non-durable的任务当不再与任何active trigger关联的时候将会从scheduler中被自动删除.因此需要持久化的jobDetail则需要设置成true.