Python 定时任务框架的 Apscheduler 案例共享

Python 定时任务框架的 Apscheduler 案例共享

介绍

现在您已经谈到了 Celery 的定时任务，请分享另一个框架 Apscheduler.Apscheduler 的全名是 Advanced Python Scheduler.It 是一个轻量级的 Python 定时任务调度框架。它还支持调度任务的异步执行和后台执行。我的小建议是将APScheduler用于一般项目，因为不需要像Celery那样启动worker，单独击败进程，并且API很简单。

- 需求背景

在前端时间十一公司业务爆炸的情况下，订单也是爆炸性的。为了在钉组中按时播报报关码的业务数据，此时需要简洁快速的解决方案。python用了不到半个小时，就偷偷写了一个不到30行的脚本（包括调试），以满足领导者的需求。

介绍

Apscheduler的官方文档可以查阅：Https：//apscheduler.readthedocs.io/en/latest/modules/triggers/cron.html#module-Apscheduler.triggers.cronOr：https：//apscheduler.readthedocs.io/en/latest/userguide.html#

- Python Timing Task Framework APScheduler，Advanced Python Scheduler （APScheduler） 是一个轻量级但功能强大的进程内任务调度器，可在指定的时间规则（时间规则：指定的日期时间、固定的时间间隔和 Linux 系统上类似 Crontab 的方式）内执行指定的作业;并且可以持久化框架，以确保在项目重新启动或崩溃恢复后可以恢复以前的作业继承继续运行。

特征

1. 独立于Linux的crontab时序

2.可以动态添加新的定时器任务，例如下单后30分钟内必须支付的定时器任务，否则取消订单将允许您使用此工具（每隔下一个订单添加此订单的定时器任务）

3. 持久保留添加的定时任务

四个组件

触发器：触发器包含按日期、间隔或 cronjob 表达式描述何时触发任务的计划逻辑。每个作业都有自己的触发器，除了初始配置外，这些触发器完全无状态。

作业存储：作业存储指定作业的存储位置。作业默认存储在内存中，作业也可以保存在各种数据库中。作业在存储在数据库中时进行序列化，在重新加载时进行反序列化。作业内存充当保存、加载、更新和查找作业的中介。作业存储不能在调度程序之间共享。

执行程序：执行程序将指定的作业（调用函数）提交到线程或进程池运行，并通知调度程序在任务完成时触发事件。

计划程序：任务计划程序，它是一个控制角色，用于配置作业存储、执行程序和触发器以添加、修改和删除任务。调度程序协调触发器、作业内存和执行程序的操作。通常只有一个调度程序在应用程序中运行。开发人员通常不需要直接处理作业内存、执行器触发器。配置作业内存和执行程序是通过调度程序完成的。

重要组件说明

触发器 - APScheduler 当前支持触发器：

DateTrigger IntervalTrigger CronTrigger

日期触发器：在指定的日期和时间执行一次

间隔触发器：定时执行，支持每秒、每分钟、每小时、每天、每周

CronTrigger：类似于Linux的Crontab Timer Task

DateTrigger 和 IntervalTrigger 易于理解且易于使用，重点是 CronTrigger 触发器。

CronTrigger 触发器的参数选项如下所示：

可用于 CronTrigger 的表达式：

执行程序 - APS调度程序当前支持的执行程序：

AsyncIOExecutor GeventExecutor ThreadPoolExecutor ProcessPoolExecutor TornadoExecutor TwistedExecutor

作业存储 - APS调度器当前支持的作业存储：

MemoryJobStore MongoDBJobStore RedisJobStore RethinkDBJobStore SQLAlchemyJobStore ZooKeeperJobStore

调度程序 - APS调度器当前支持的调度程序：

AsyncIOScheduler BackgroundScheduler --Non-blocking mode BlockingScheduler --Blocking mode GeventScheduler QtScheduler TornadoScheduler TwistedScheduler

作业作业 作业是 APS调度器的最小执行单位。创建作业时，需要指定要执行的函数、函数中所需的参数以及作业执行的一些设置。

id: Uniqueness of the specified job ID name: Specify the name of the job trigger: apscheduler Defined triggers for determining Job Execution time, according to settings trigger Rule, calculate the next time this is executed job Of Time, will execute when satisfied executor: apscheduler Defined executors, job Set the name of the executor at creation time, from string to scheduler Get to execute this job Executor, Execute job Specified function max_instances: Execute this job Maximum number of instances, executor implement job Under job Of id To calculate the number of executions, based on the maximum number of instances set Determine whether executable next_run_time: Job Next execution time, create Job You can specify a time when[datetime],If not specified, the default is based on trigger Get Touch Time of issue misfire_grace_time: Job Delayed execution time of, for example Job Plan execution time is 21:00:00，But due to service restart or other reasons 21:00:31 Execute if this is set key For 40,Then the job Will continue or this will be discarded job coalesce: Job Whether or not to merge execution is a bool Value.for example scheduler Stop 20 s After restarting, and job Trigger set to 5 s implement Once, so this job Four execution times are missed, if set to Yes, they will be merged into one execution, otherwise they will be executed one by one func: Job Functions executed args: Job Location parameters required to execute functions kwargs: Job Keyword parameters required to execute a function

创建步骤

有四个基本步骤： 创建dispatcher_adding调度任务/触发器（满足标准）执行

# 1. Create a scheduler # Background execution of this program will not block scheduler = BackgroundScheduler() # 2. Add Schedule Tasks # 3. Trigger triggers='interval' # Execute every 20 seconds scheduler.add_job(main, 'interval', seconds=20) # 4. Conditional Executors scheduler.start()

触发器使用三种方案

日期 - 定期计划（作业在特定日期执行时仅执行一次）

from apscheduler.schedulers.background import BackgroundScheduler, BlockingScheduler sched = BlockingScheduler() def my_job(): print(1) # The job will be executed on November 6th, 2009 sched.add_job(my_job, 'date', run_date=date(2009, 11, 6), args=['text']) # The job will be executed on November 6th, 2009 at 16:30:05 sched.add_job(my_job, 'date', run_date=datetime(2009, 11, 6, 16, 30, 5), args=['text'])

间隔 - 间隔 计划（执行频率）

from datetime import datetime import os from apscheduler.schedulers.blocking import BlockingScheduler def tick(): print('Tick! The time is: %s' % datetime.now()) if __name__ == '__main__': scheduler = BlockingScheduler() # sep1 executes every 3 seconds scheduler.add_job(tick, 'interval', seconds=3) # sep2 indicates a task is performed every 3 days at 17:19:07 seconds scheduler.add_job(tick, 'interval', days=3, hours=17, minutes=19, seconds=7) # Execute every 20 seconds scheduler.add_job(tick, 'interval', seconds=61) print('Press Ctrl+{0} to exit'.format('Break' if os.name == 'nt' else 'C ')) try: scheduler.start() except (KeyboardInterrupt, SystemExit): pass

cron - 在给定时间（在指定时间段）执行：

from datetime import datetime import os from apscheduler.schedulers.blocking import BlockingScheduler def tick(): print('Tick! The time is: %s' % datetime.now()) if __name__ == '__main__': scheduler = BlockingScheduler() # Represents a task performed at 19:23 a day scheduler.add_job(tick, 'cron', hour=19,minute=23) # Run at 8:00 a day scheduler.add_job(tick, 'cron', day_of_week='0-6', hour=8, minute=00, second=00) # Every day at 0, 1 and 8 o'clock scheduler.add_job(tick,'cron',month='*', day='*', hour='0,1,8',minute='00') # Indicates that the program will be executed at 1719:07 seconds on March 22, 2017 scheduler.add_job(tick, 'cron', year=2017, month=3, day=22, hour=17, minute=19, second=7) # Indicates that the task will execute the program on the third Friday in June, 7, 8, 11, and December at 00:00, 01:00, 02:00, 03:00 scheduler.add_job(tick, 'cron', month='6-8,11-12', day='3rd fri', hour='0-3') # Represents from Monday to Friday at 5:30 (AM) until 2014-05-30 00:00:00 scheduler.add_job(tick, 'cron', day_of_week='mon-fri', hour=5, minute=30, end_date='2014-05-30') # Represents that the program is executed every 5 seconds, equivalent to seconds = 5 in interval interval scheduling scheduler.add_job(tick, 'cron', second='*/5') print('Press Ctrl+{0} to exit'.format('Break' if os.name == 'nt' else 'C ')) try: scheduler.start() except (KeyboardInterrupt, SystemExit): pass

　　

- 完整演示

#!/usr/bin/env python # -*- coding: utf-8 -*- ''' # @Time : 2020/11/17 0017 22:16 # @Author : liudinglong # @File : send_demo.py # @Description: # @Question: ''' from datetime import datetime import json import urllib.request import pymysql as pms from apscheduler.schedulers.background import BackgroundScheduler,BlockingScheduler import os # The following modules are used to close ssl validation under Mac import ssl ''' ---------------------------------------------- # The following support libraries need to be installed under the CMD command: # pip install apscheduler # pip install pymysql ---------------------------------------------- ''' # Turn off ssl authentication on Mac and Linux or error will occur ssl._create_default_https_context = ssl._create_unverified_context # Your nail robot url global myurl my_url = " https://oapi.dingtalk.com/robot/send?access_token=XXXXXXXXXXXX" def send_request(url, datas): # Incoming url and content send request # Build a request header header = { "Content-Type": "application/json", "Charset": "UTF-8" } sendData = json.dumps(datas) # Converting dictionary-type data into json format sendDatas = sendData.encode("utf-8") # Request for python3 requires data to be of byte type # Send Request request = urllib.request.Request(url=url, data=sendDatas, headers=header) # Build the data sent back by the request into a file format opener = urllib.request.urlopen(request) # Print returned results print(opener.read()) def get_mysqldatas(sql): # A function that passes in sql to export data, instance MySQL requires pymysql library to be installed first, cmd window command: pip install pymysql # Connect to the database conn = pms.connect(host='server address', user='User name', passwd='Password', database='data base', port=3306, charset="utf8") # Create a cursor object using the cursor() method cur = conn.cursor() # Execute SQL using execute() method cur.execute(sql) # Get the data you need datas = cur.fetchall() # Close Connection cur.close() # Return required data return datas def get_ddmodel_datas(type): # Return pin model data, 1:text; 2:markdown everyone; 3:markdown with picture, @recipient; 4:link type if type == 1: my_data = { "msgtype": "text", "text": { "content": " " }, "at": { "atMobiles": [ "188XXXXXXX" ], "isAtAll": False } } elif type == 2: my_data = { "msgtype": "markdown", "markdown": {"title": " ", "text": " " }, "at": { "isAtAll": True } } elif type == 3: my_data = { "msgtype": "markdown", "markdown": {"title": " ", "text": " " }, "at": { "atMobiles": [ "188XXXXXXXX" ], "isAtAll": False } } elif type == 4: my_data = { "msgtype": "link", "link": { "text": " ", "title": " ", "picUrl": "", "messageUrl": " " } } return my_data def main(): print('Main! The time is: %s' % datetime.now()) # Design request content links in pinned data format Https://open-doc.dingtalk.com/docs/doc.htm?Spm=a219a.7629140.0.0.p7hJKp&treeId=257&articleId=105735&docType=1 # Call the pin robot global variable myurl global myurl # 1.Text type mass messaging # Merge headings and data My_content = "hello, @188XXXXXXXX This is a test message" my_data = get_ddmodel_datas(1) # Write text into request format my_data["text"]["content"] = My_content send_request(my_url, my_data) # 2.Markdown type mass messaging (MySQL query results sent) # Get sql data sql = "SELECT branch_no,count(*) from wzy_customer_user group by branch_no order by branch_no" my_mydata = get_mysqldatas(sql) str1 = '\t\n\r' seq = [] for i in range(len(my_mydata)): seq.append(str(my_mydata[i])) data = str1.join(seq) data = data.replace('\'', '') data = data.replace('(', '') data = data.replace(')', '') data = data.replace(',', '\t') print(data) Mytitle = "#### XXX Report\r\n Unitst Quantitytn\r%s" my_Mytitle = Mytitle.join('\t\n') % data my_data = get_ddmodel_datas(2) my_data["markdown"]["title"] = "XXXX Notice" my_data["markdown"]["text"] = my_Mytitle send_request(my_url, my_data) # 3.Markdown (with picture@object) my_data = get_ddmodel_datas(3) my_data["markdown"]["title"] = "System Alert" my_data["markdown"][ "text"] = "#### System Alert Content \n > @188XXXXXXXX \n\n > \n > ####\#\# Distribution at 20:00 [Details]( http://www.baidu.cn/)" send_request(my_url, my_data) # Font color:<font color='#FF4500'size=2>%s</font>double\nn means line break # 4.Link type mass messaging my_data = get_ddmodel_datas(4) my_data["link"]["text"] = "Group robots are an advanced extension of the nail swarm.Group robots can aggregate information from third-party services into group chats to achieve automated information synchronization. " my_data["link"]["title"] = "Custom Robot Protocol" my_data["link"][ "messageUrl"] = " https://open-doc.dingtalk.com/docs/doc.htm?spm=a219a.7629140.0.0.Rqyvqo&treeId=257&articleId=105735&docType=1" send_request(my_url, my_data) if __name__ == "__main__": # To perform tasks on a regular basis, you need to first install the apscheduler library, cmd window command: pip install apscheduler # Execute with script # 1. Create a scheduler # scheduler = BlockingScheduler() --Blocking method # Background execution of this program will not block scheduler = BackgroundScheduler() # 2. Add Schedule Tasks # 3. Trigger triggers='interval' # Execute every 20 seconds scheduler.add_job(main, 'interval', seconds=20) ''' ***Timed Execution Example*** #Execute once at a fixed time #sched.add_job(main, 'cron', year=2018, month=9, day=28, hour=15, minute=40, second=30) #Indicates that the program will be executed at 1719:07 seconds on March 22, 2017 scheduler.add_job(my_job, 'cron', year=2017,month = 03,day = 22,hour = 17,minute = 19,second = 07) #Indicates that the task will execute the program on the third Friday in June, 7, 8, 11, and December at 00:00, 01:00, 02:00, 03:00 scheduler.add_job(my_job, 'cron', month='6-8,11-12', day='3rd fri', hour='0-3') #Represents from Monday to Friday at 5:30 (AM) until 2014-05-30 00:00:00 scheduler.add_job(my_job(), 'cron', day_of_week='mon-fri', hour=5, minute=30,end_date='2014-05-30') #Represents that the program is executed every 5 seconds, equivalent to seconds = 5 in interval interval scheduling scheduler.add_job(my_job, 'cron',second = '*/5') ''' # 4. Conditional Executors scheduler.start() print('Press Ctrl+{0} to exit'.format('Break' if os.name == 'nt' else 'C')) try: # Other tasks are performed by separate threads while True: pass # time.sleep(60) # print('The process is executing!') except (KeyboardInterrupt, SystemExit): # Terminate Task scheduler.shutdown() print('Exit The Job!')

使用案例 - 订书钉集团及时广播消息

1.在引脚组助手中，自定义一个机器人，如图所示：

代码设置为每 10 秒发送一次，如下所示：

scheduler.add_job(main,'interval',seconds=10)

运行结果：

截图如下：

脚本部署

定时任务的脚本需要保留一段时间，因此如果使用 IDE 运行它，它肯定不起作用，因此您可以在服务器上获取它。

脚本上传到服务器，然后是相关库，然后在 Linux 上按如下方式启动：

linux Command Run py Script: nohup python -u test.py > out.log 2>&1 &

　　

日志：

　　

这里要注意的重要一点是使用 -u 作为参数的含义：

python The output of the is buffered, resulting in out.log You can't see the output right away. -u Parameter, so that python Buffering is not enabled.

Nohup意味着没有挂断电话。此命令的一般形式是：nohup。/test &

末尾添加 &是指在后台运行而不终止程序，因为终端关闭或断开连接。

这将启动 py 服务。

总结

这是对定时任务框架Apscheduler的简单使用，当你在工作中遇到其他需求时，你可以进一步理解学习是为了解决问题，以便更好的工作。同时，欢迎小伙伴交流交流他们的测试思路和工作方法。