基于DRF apscheduler的定时任务处理

鉴于restframework良好自带网页的功能,使用django-admin 创建一个新的app,安装restframework模块,为了让我们的django-admin后台网页好看点,采用流行的simpleui,这个模块Element-UI + Vue 加持,让古老的django admin 焕然一新。
二、安装simpleui模板
pip install simpleui
创建静态文件目录
在settings文件中配置好STATIC_ROOT= os.path.join(BASE_DIR,‘static’)
不然在python manage.py collectstatic会报错
再加上SIMPLEUI_HOME_INFO = False
在这里插入图片描述
在INSTALLED_APPS = [
‘simpleui’,
‘django.contrib.admin’,
‘django.contrib.auth’,]
注意simpleui一定要加在第一行,不然会被admin覆盖的,安装的效果失效
安装好之后django-admin的主页就变成下面的了,很好看
在这里插入图片描述
在这里插入图片描述
这界面一搞是不是逼格直接就上去了,很好看
创建一个项目django-admin startapp apsched 项目
1.安装django-apscheduler
pip install django-apscheduler
2.注册django-apscheduler
在setting.py文件中注册django-apscheduler为APP
INSTALLED_APPS = [

‘django_apscheduler’, # 定时任务
3,数据库迁移
python manage.py makemigrations
python manage.py migrate
到此可以看到MySQL数据已经建了两个表
在这里插入图片描述
这两个表已经建好了,推测出来使用了django-apscheduler这个模块安装之后会有model建立好,实际也的确如此
查看apsched app中的serializers.py

from django_apscheduler.models import DjangoJob
from rest_framework.serializers import ModelSerializer
class DjangoJobSerializer(ModelSerializer):
    class Meta:
        fields = '__all__'
        model = DjangoJob

查看apsched app中的views.py

from django.shortcuts import render

# Create your views here.
from apscheduler.schedulers.background import BackgroundScheduler
from django_apscheduler.jobstores import DjangoJobStore,register_job
from rest_framework.viewsets import ModelViewSet
from django.http import  JsonResponse
import datetime,time
from rest_framework.decorators import action
from .serializers import DjangoJobSerializer
from django_apscheduler.models import DjangoJob
import threading,os
def my_job():
    print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S'),threading.current_thread().getName(),threading.get_ident(),os.getpid())
    print("============开始休息10S=====================")
    time.sleep(10)
    print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S'),threading.current_thread().getName(),threading.get_ident(),os.getpid())
    print("============继续休息10S=====================")
    time.sleep(10)
    print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S'),threading.current_thread().getName(),threading.get_ident(),os.getpid())
    print("============继续休息10S=====================")
    time.sleep(10)
    print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S'),threading.current_thread().getName(),threading.get_ident(),os.getpid())
    print("================休息结束=====================")

from apscheduler.executors.pool import ThreadPoolExecutor,ProcessPoolExecutor

EXECUTORS = {
    'default': ThreadPoolExecutor(2),
    # 'default': ProcessPoolExecutor(5)
}
JOB_DEFAULTS = {
    'coalesce': False,
    'max_instances': 1,
    'misfire_grace_time': None
}
# 实例化调度器
scheduler=BackgroundScheduler(job_defaults=JOB_DEFAULTS,executors=EXECUTORS,timezone='Asia/Shanghai')
# 调度器使用默认的DjangoJobStore()
scheduler.add_jobstore(DjangoJobStore(), 'default')
class DjangoJobViewSet(ModelViewSet):
    permission_classes = []
    authentication_classes = []
    queryset = DjangoJob.objects.all()
    serializer_class = DjangoJobSerializer
    def create(self,request):
        print(dir(request))
        try:
            # id = request.data.get('id')
            trigger_type = request.data.get('trigger_type')
            if trigger_type == "date":
                run_time = request.data.get('run_time')
                job=scheduler.add_job(func=my_job,
                                        trigger=trigger_type,
                                        next_run_time=run_time,
                                        replace_existing=True,
                                        coalesce=False)
                print(job,type(job),job.__getstate__().get('id'))
                print("添加一次性任务成功---[ %s ] " % job.__getstate__().get('id'))
            elif trigger_type == 'interval':
                print(request.data)
                seconds = request.data.get('interval_time')
                seconds = int(seconds)
                print('seconds value is',seconds)
                if seconds <= 0:
                    raise TypeError('请输入大于0的时间间隔!')
                job=scheduler.add_job(func=my_job,
                                  trigger=trigger_type,
                                  seconds=seconds,
                                  replace_existing=True,
                                  coalesce=False)
                print("添加一次性任务成功---[ %s ] " % job.__getstate__().get('id'))
            elif trigger_type == "cron":
                print(type(eval(request.data.get("run_time"))))
                print(eval(request.data.get("run_time"))["day_of_week"])
                day_of_week = eval(request.data.get("run_time"))["day_of_week"]
                hour = eval(request.data.get("run_time"))["hour"]
                minute =eval(request.data.get("run_time"))["minute"]
                second = eval(request.data.get("run_time"))["second"]
                job=scheduler.add_job(func=my_job, trigger=trigger_type, day_of_week=day_of_week,
                                  hour=hour, minute=minute,
                                  second=second, replace_existing=True)
                print("添加周期执行任务成功任务成功---[ %s ] " % job.__getstate__().get('id'))
            return JsonResponse({'msg':'新增任务成功'})
        except Exception as e:
            return JsonResponse({'msg':f'新增任务失败error:{e}'})
    @action(methods=['POST','GET'],detail=True)
    def pause(self,request,*args,**kwargs):
        print(args,kwargs)
        response = {'status': False}
        try:
            target_id=kwargs['pk']
            scheduler.pause_job(target_id)
            response['status'] = True
            response['msg'] = "job[%s] pause success!" %(target_id)
        except Exception as e:
            response['msg'] = str(e)
        return JsonResponse(response)
    @action(methods=['POST','GET'],detail=True)
    def resume(self,request,pk):
        response = {'status': False}
        try:
            scheduler.resume_job(pk)
            response['status'] = True
            response['msg'] = "job[%s] resume success!" % pk
        except Exception as e:
            response['msg'] = str(e)
        return JsonResponse(response)

    def update(self, request, *args,**kwargs):
        pk=kwargs['pk']
        print('put方法pk值是:',pk)
        try:
            trigger_type = request.data.get('trigger_type')
            if trigger_type == "date":
                run_time = request.data.get('run_time')
                job = scheduler.modify_job(pk,jobstore=None, func=my_job,
                                           trigger=trigger_type,
                                           next_run_time=run_time,
                                           replace_existing=True,
                                           coalesce=False)
                print(job, type(job), job.__getstate__().get('id'))
                print("修改一次性任务成功---[ %s ] " % job.__getstate__().get('id'))
            elif trigger_type == 'interval':
                print(request.data)
                seconds = request.data.get('interval_time')
                seconds = int(seconds)
                print('seconds value is', seconds)
                if seconds <= 0:
                    raise TypeError('请输入大于0的时间间隔!')
                job = scheduler.modify_job(pk,jobstore=None, func=my_job,
                                           trigger=trigger_type,
                                           seconds=seconds,
                                           replace_existing=True,
                                           coalesce=False)
                print("修改一次性任务成功---[ %s ] " % job.__getstate__().get('id'))
            elif trigger_type == "cron":
                print(type(eval(request.data.get("run_time"))))
                print(eval(request.data.get("run_time"))["day_of_week"])
                day_of_week = eval(request.data.get("run_time"))["day_of_week"]
                hour = eval(request.data.get("run_time"))["hour"]
                minute = eval(request.data.get("run_time"))["minute"]
                second = eval(request.data.get("run_time"))["second"]
                print(hour,minute,second)
                temp_dict=dict(day_of_week=day_of_week,
                                           hour=hour, minute=minute,
                                           second=second)
                job=scheduler.reschedule_job(pk,trigger='cron',**temp_dict)
                print("修改执行任务成功任务成功---[ %s ] " % job.__getstate__().get('id'))
            return  JsonResponse(dict(msg=f'修改单个任务成功{pk}'))
        except Exception as e:
            return  JsonResponse(dict(msg=f'修改单个任务失败{pk}报错:{e}'))

# 调度器开始运行
scheduler.start()

上面这个脚本是关键脚本,使用了restframework的action功能
permission_classes = []
authentication_classes = []
上面的框架项目因为还有别的app采用了验证,在restframework的权限认证中也已经settings中设置,所以为了方便测试上面两个权限要设置为空值,不然会提示权限不够
urls.py的脚本

from django.contrib import admin
from django.urls import path,re_path
from django.conf.urls import include,url
from rest_framework import routers
router=routers.DefaultRouter()
from .views import  DjangoJobViewSet
router.register('apsjob',DjangoJobViewSet)
urlpatterns = [
    path('',include(router.urls)),
]

上面的views.py脚本中有interval,cron,date(指定日期)几种定时任务
把项目跑起来

PS F:\pyprogram3\restful_manu> python .\manage.py runserver
Watching for file changes with StatReloader
Performing system checks...

System check identified 7 issues (0 silenced).
June 08, 2022 - 10:51:43
Django version 3.2.13, using settings 'restful_manu.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CTRL-BREAK.

由于有些变量需要前端传入进去,我偷懒当然前端也比较麻烦就没做,采用了postman模拟前端传入变量,下面验证三种闯入
第一种cron运行
在这里插入图片描述

在这里插入图片描述

可以看出来的确已经创建四个cron定时任务,我们代码里面加上了线程ID,线程名称和进程ID,在EXECUTORS设置了2个线程池,同一时刻只允许job运行一次,max_instances:1这个就是对这限制
后端也可以看出执行成功了
在这里插入图片描述
可以看出来django运行定时任务的时候都是单进程多线程,上面我设置了4个定时任务,都是同一时刻执行,2个线程分配出来,先2个运行结束了后面在运行两个
如果是5个定时任务和我们上面的结果也是一致的

2022-06-08 15:10:00 ThreadPoolExecutor-1_0 8424 16936
============开始休息10S=====================
2022-06-08 15:10:00 ThreadPoolExecutor-1_1 12196 16936
============开始休息10S=====================
2022-06-08 15:10:10 ThreadPoolExecutor-1_0 8424 16936
============继续休息10S=====================
2022-06-08 15:10:10 ThreadPoolExecutor-1_1 12196 16936
============继续休息10S=====================
2022-06-08 15:10:20 ThreadPoolExecutor-1_0 8424 16936
============继续休息10S=====================
2022-06-08 15:10:20 ThreadPoolExecutor-1_1 12196 16936
============继续休息10S=====================
2022-06-08 15:10:30 ThreadPoolExecutor-1_0 8424 16936
================休息结束=====================
2022-06-08 15:10:30 ThreadPoolExecutor-1_1 12196 16936
================休息结束=====================
2022-06-08 15:10:30 ThreadPoolExecutor-1_0 8424 16936
============开始休息10S=====================
2022-06-08 15:10:30 ThreadPoolExecutor-1_1 12196 16936
============开始休息10S=====================
2022-06-08 15:10:40 ThreadPoolExecutor-1_0 8424 16936
============继续休息10S=====================
2022-06-08 15:10:40 ThreadPoolExecutor-1_1 12196 16936
============继续休息10S=====================
2022-06-08 15:10:50 ThreadPoolExecutor-1_0 8424 16936
============继续休息10S=====================
2022-06-08 15:10:50 ThreadPoolExecutor-1_1 12196 16936
============继续休息10S=====================
2022-06-08 15:11:00 ThreadPoolExecutor-1_0 8424 16936
================休息结束=====================
2022-06-08 15:11:00 ThreadPoolExecutor-1_0 8424 16936
============开始休息10S=====================
2022-06-08 15:11:00 ThreadPoolExecutor-1_1 12196 16936
================休息结束=====================
2022-06-08 15:11:10 ThreadPoolExecutor-1_0 8424 16936
============继续休息10S=====================
2022-06-08 15:11:20 ThreadPoolExecutor-1_0 8424 16936
============继续休息10S=====================
2022-06-08 15:11:30 ThreadPoolExecutor-1_0 8424 16936
================休息结束=====================

一个进程分配两个线程,一个线程运行结束了继续下一个,类似于thread.start() thread.join()这种模式
通过restframework网页可以看出action的内容都已经体现
在这里插入图片描述
这里面的delete,pause,resume功能都能成功实现
MAX_INSTANCES代表的是可以同时运行的实例数目,也就是如果job A在11:00运行了,每10分钟运行一次,但是job A在11:10还没结束,那么11:10时候的jobA就会停下来
第二种interval运行
在这里插入图片描述

在这里插入图片描述
最后一种date运行
在这里插入图片描述
可以观察也是运行成功了,只是这种定时任务也就运行一次很短暂,在运行的时候定时任务已经不存在了,就像某些动物一样开始到结束昙花一现
需要注意的如果修改scheduler的运行时间参数直接通过数据库修改是不起作用的,我试验过,但是不起作用
需要是固定的函数来修改,我使用modify_job修改过,代码

scheduler.modify_job('61cf7ed98e974522a61e69d4e5e78fe6',jobstore=None,func=my_job,trigger='cron', day_of_week=2,hour=14, minute=50,second=00, replace_existing=True)

但是提示我Expected a trigger instance, got str instead’,具体原因看了下源码这样修改我modify还是没成功但是rescheduler_job成功修改了,上面有代码

tmp_dict = {"seconds":10}
        temp_trigger = 	scheduler._create_trigger(trigger='interval',trigger_args=tmp_dict)
        result = scheduler.modify_job(pk,trigger=temp_trigger)

        rescheduler_job也是最终使用modify_job修改

        trigger = self._create_trigger(trigger, trigger_args)
        now = datetime.now(self.timezone)
        next_run_time = trigger.get_next_fire_time(None, now)
        return self.modify_job(job_id, jobstore, trigger=trigger, next_run_time=next_run_time)

django-apscheduler不能开启多进程模式会报错,开启方式

EXECUTORS = {
   # 'default': ThreadPoolExecutor(2),
     'default': ProcessPoolExecutor(5)
}

报错内容

django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.
Error running job 4de15bdff2484c1ea5c303fc755fe021
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.

定时任务cron相关参数说明

year (int|str)4-digit year
month (int|str) – month (1-12)
day (int|str) – day of the (1-31)
week (int|str) – ISO week (1-53)
day_of_week (int|str) – number or name of weekday (0-6 or mon,tue,wed,thu,fri,sat,sun)
hour (int|str) – hour (0-23)
minute (int|str) – minute (0-59)
second (int|str) – second (0-59)
start_date (datetime|str) – earliest possible date/time to trigger on (inclusive)
end_date (datetime|str) – latest possible date/time to trigger on (inclusive)
timezone (datetime.tzinfo|str) – time zone to use for the date/time calculations (defaults to scheduler timezone)

本文的api接口create,update函数都是自己手动改写的,restframework调用的时候网页修改会失效

  • 0
    点赞
  • 2
    收藏
    觉得还不错? 一键收藏
  • 1
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论 1
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值