python定时任务contrib_Django + celery + beat 执行定时任务

开发环境

python                         3.6.8

django                         1.11

celery                           4.3.0

django-celery-results  1.1.2

django-celery-beat     1.5.0

安装 django + celery + results

安装 django-celery-beat

pip install django-celery-beat

配置 settings.py

INSTALLED_APPS = [

# ...

'django_celery_results', # 查看 celery 执行结果

'django_celery_beat', # pip install django-celery-beat

]

生成 Django-celery-beat 关联表

python manage.py migrate

(joyoo) yinzhuoqundeMacBook-Pro:joyoo yinzhuoqun$ python manage.py migrate

raven.contrib.django.client.DjangoClient: 2019-12-16 23:01:02,792 /Users/yinzhuoqun/.pyenv/joyoo/lib/python3.6/site-packages/raven/base.py [line:213] INFO Raven is not configured (logging is disabled). Please see the documentation for more information.

Operations to perform:

Apply all migrations: admin, auth, blog, captcha, contenttypes, django_celery_beat, django_celery_results, logger, photo, sessions, sites, user, users

Running migrations:

Applying django_celery_beat.0001_initial... OK

Applying django_celery_beat.0002_auto_20161118_0346... OK

Applying django_celery_beat.0003_auto_20161209_0049... OK

Applying django_celery_beat.0004_auto_20170221_0000... OK

Applying django_celery_beat.0005_add_solarschedule_events_choices... OK

Applying django_celery_beat.0006_auto_20180322_0932... OK

Applying django_celery_beat.0007_auto_20180521_0826... OK

Applying django_celery_beat.0008_auto_20180914_1922... OK

Applying django_celery_beat.0006_auto_20180210_1226... OK

Applying django_celery_beat.0006_periodictask_priority... OK

Applying django_celery_beat.0009_periodictask_headers... OK

Applying django_celery_beat.0010_auto_20190429_0326... OK

Applying django_celery_beat.0011_auto_20190508_0153... OK

app 目录 tasks.py 添加 task

#!/usr/bin/env python3

# -*- coding: utf-8 -*-

"""

@author: yinzhuoqun

@site: http://zhuoqun.info/

@email: yin@zhuoqun.info

@time: 2019/12/15 12:34 AM

"""

import json

import requests

import datetime

from celery import task

from django.core.cache import cache

from django.core.mail import send_mail

from django.utils import timezone

from .models import TodayViewPage

# from django.core.mail import EmailMultiAlternatives

@task

def task_send_dd_text(url, msg, atMoblies, atAll="flase"):

"""

发送钉钉提醒

:param url:

:param msg:

:param atMoblies:

:param atAll:

:return:

"""

body = {

"msgtype": "text",

"text": {

"content": msg

},

"at": {

"atMobiles": atMoblies,

"isAtAll": atAll

}

}

headers = {'content-type': 'application/json',

'User-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:22.0) Gecko/20100101 Firefox/22.0'}

r = requests.post(url, headers=headers, data=json.dumps(body))

# print(r.text)

return r.text

@task

def task_send_mail(*args, **kwargs):

"""

django 的 发送邮件，支持 html，html_message="html 内容"

:param args:

:param kwargs:

:return:

"""

return send_mail(*args, **kwargs)

@task

def task_save_view_page():

"""

保存前一日访问量

:return:

"""

today = datetime.date.today()

today_page_view = cache.get("today_page_view_%s" % (today.day - 1))

if today_page_view:

yesterday = datetime.date.today() + datetime.timedelta(days=-1)

page_view_query = TodayViewPage.objects.filter(today=yesterday)

if page_view_query.exists():

page_view_obj = page_view_query.first()

page_view_obj.view_page = today_page_view

page_view_obj.save()

else:

page_view_obj = TodayViewPage(

today=yesterday,

view_page=today_page_view

)

page_view_obj.save()

return "view page save success"

return "view page save failed"

管理后台配置 Periodic tasks

启动 celery worker

# 项目根目录终端执行(joyoo 项目名称)

celery -A joyoo worker -l info

# 守护进程

/root/.virtualenvs/blog/bin/celery multi start w1 -A joyoo -l info --logfile=./celerylog.log

启动 celery beat

# 项目根目录终端执行(joyoo 项目名称)

celery -A joyoo beat -l info --scheduler django_celery_beat.schedulers:DatabaseScheduler

celery beat 日志

(joyoo) yinzhuoqundeMacBook-Pro:joyoo yinzhuoqun$ celery -A joyoo beat -l info --scheduler django_celery_beat.schedulers:DatabaseScheduler

celery beat v4.3.0 (rhubarb) is starting.

raven.contrib.django.client.DjangoClient: 2019-12-17 00:18:36,990 /Users/yinzhuoqun/.pyenv/joyoo/lib/python3.6/site-packages/raven/base.py [line:213] INFO Raven is not configured (logging is disabled). Please see the documentation for more information.

__ - ... __ - _

LocalTime -> 2019-12-17 00:18:38

Configuration ->

. broker -> redis://127.0.0.1:6379/0

. loader -> celery.loaders.app.AppLoader

. scheduler -> django_celery_beat.schedulers.DatabaseScheduler

. logfile -> [stderr]@%INFO

. maxinterval -> 5.00 seconds (5s)

[2019-12-17 00:18:38,076: INFO/MainProcess] beat: Starting...

[2019-12-17 00:18:38,076: INFO/MainProcess] Writing entries...

[2019-12-17 00:18:43,105: INFO/MainProcess] Writing entries...

[2019-12-17 00:19:23,155: INFO/MainProcess] DatabaseScheduler: Schedule changed.

[2019-12-17 00:19:23,155: INFO/MainProcess] Writing entries...

[2019-12-17 00:20:00,008: INFO/MainProcess] Scheduler: Sending due task celery.save_view_page (blog.tasks.task_save_view_page)

[2019-12-17 00:21:45,165: INFO/MainProcess] Writing entries...

[2019-12-17 00:24:45,410: INFO/MainProcess] Writing entries...

[2019-12-17 00:27:45,649: INFO/MainProcess] Writing entries...

[2019-12-17 00:30:45,878: INFO/MainProcess] Writing entries...

[2019-12-17 00:32:05,981: INFO/MainProcess] DatabaseScheduler: Schedule changed.

[2019-12-17 00:32:05,981: INFO/MainProcess] Writing entries...

[2019-12-17 00:33:00,001: INFO/MainProcess] Scheduler: Sending due task celery.save_view_page (blog.tasks.task_save_view_page)

[2019-12-17 00:33:50,066: INFO/MainProcess] Writing entries...

celery worker 日志

(joyoo) yinzhuoqundeMacBook-Pro:joyoo yinzhuoqun$ celery -A joyoo worker -l info

raven.contrib.django.client.DjangoClient: 2019-12-17 00:18:48,673 /Users/yinzhuoqun/.pyenv/joyoo/lib/python3.6/site-packages/raven/base.py [line:213] INFO Raven is not configured (logging is disabled). Please see the documentation for more information.

-------------- celery@yinzhuoqundeMacBook-Pro.local v4.3.0 (rhubarb)

---- **** -----

--- * *** * -- Darwin-18.6.0-x86_64-i386-64bit 2019-12-17 00:18:49

-- * - **** ---

- ** ---------- [config]

- ** ---------- .> app: joyoo:0x10bb887f0

- ** ---------- .> transport: redis://127.0.0.1:6379/0

- ** ---------- .> results:

- *** --- * --- .> concurrency: 12 (prefork)

-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)

--- ***** -----

-------------- [queues]

.> celery exchange=celery(direct) key=celery

[tasks]

. blog.tasks.task_save_view_page

. blog.tasks.task_send_dd_text

. blog.tasks.task_send_mail

[2019-12-17 00:18:50,413: INFO/MainProcess] Connected to redis://127.0.0.1:6379/0

[2019-12-17 00:18:50,423: INFO/MainProcess] mingle: searching for neighbors

[2019-12-17 00:18:51,447: INFO/MainProcess] mingle: all alone

[2019-12-17 00:18:51,462: WARNING/MainProcess] /Users/yinzhuoqun/.pyenv/joyoo/lib/python3.6/site-packages/celery/fixups/django.py:202: UserWarning: Using settings.DEBUG leads to a memory leak, never use this setting in production environments!

warnings.warn('Using settings.DEBUG leads to a memory leak, never '

[2019-12-17 00:18:51,463: INFO/MainProcess] celery@yinzhuoqundeMacBook-Pro.local ready.

[2019-12-17 00:20:00,030: INFO/MainProcess] Received task: blog.tasks.task_save_view_page[b375473b-5887-4cb4-8ddd-2d8cfdec79cc]

[2019-12-17 00:20:00,070: INFO/ForkPoolWorker-8] Task blog.tasks.task_save_view_page[b375473b-5887-4cb4-8ddd-2d8cfdec79cc] succeeded in 0.037283750003552996s: '访问量保存成功'

[2019-12-17 00:33:00,003: INFO/MainProcess] Received task: blog.tasks.task_save_view_page[beb2a503-a116-4b8c-855a-0b645aa4079b]

[2019-12-17 00:33:00,047: INFO/ForkPoolWorker-9] Task blog.tasks.task_save_view_page[beb2a503-a116-4b8c-855a-0b645aa4079b] succeeded in 0.04218383401166648s: '访问量保存成功'