celery在django的USE_TZ为False时不停的执行任务调度

说明:最新版本django-celery-beat(1.5.0以上)已经修复此bug;

环境

Django==2.0.13
# 使用celery实现计划任务!!
celery==4.3.0rc1
redis==3.0.1
# django-celery-beat==1.4.0
# 修复TZ不为默认时,start time设置引起错误;后续官方发布新版本直接使用官方版本。
git+https://gitee.com/enlangs/django-celery-beat.git
django-celery-results==1.0.4

现象

不停的执行Scheduler: Sending due task

[2019-02-26 15:40:03,798: INFO/MainProcess] Scheduler: Sending due task Test Task V2
[2019-02-26 15:40:03,799: DEBUG/MainProcess] Test Task V2 sent. id->89e18c07-f83f-445d-8196-41e6d4bb044a
[2019-02-26 15:40:03,829: INFO/MainProcess] Scheduler: Sending due task Test Task V2
[2019-02-26 15:40:03,830: DEBUG/MainProcess] Test Task V2 sent. id->3b6833be-231f-46c2-a5d8-c9f9617657b4
[2019-02-26 15:40:03,859: INFO/MainProcess] Scheduler: Sending due task Test Task V2
[2019-02-26 15:40:03,861: DEBUG/MainProcess] Test Task V2 sent. id->367b3f23-95f2-428c-9d9f-1316171873e8
[2019-02-26 15:40:03,891: INFO/MainProcess] Scheduler: Sending due task Test Task V2
[2019-02-26 15:40:03,892: DEBUG/MainProcess] Test Task V2 sent. id->d36e725b-721d-4394-b9e4-eee5c2dfff23
[2019-02-26 15:40:03,921: INFO/MainProcess] Scheduler: Sending due task Test Task V2
[2019-02-26 15:40:03,923: DEBUG/MainProcess] Test Task V2 sent. id->34d28f8c-557a-48a7-8a96-5d9bc45c2c49
[2019-02-26 15:40:03,953: INFO/MainProcess] Scheduler: Sending due task Test Task V2
[2019-02-26 15:40:03,954: DEBUG/MainProcess] Test Task V2 sent. id->2f620417-8635-430d-81d7-22074d0de200
[2019-02-26 15:40:03,985: INFO/MainProcess] Scheduler: Sending due task Test Task V2
[2019-02-26 15:40:03,986: DEBUG/MainProcess] Test Task V2 sent. id->75d2ba9e-f1ea-4dd4-b7cc-614fbb7ef0cd
[2019-02-26 15:40:04,016: INFO/MainProcess] Scheduler: Sending due task Test Task V2
[2019-02-26 15:40:04,017: DEBUG/MainProcess] Test Task V2 sent. id->300a9043-aeae-4c4f-aef1-9e3813c899a1
[2019-02-26 15:40:04,048: INFO/MainProcess] Scheduler: Sending due task Test Task V2
[2019-02-26 15:40:04,049: DEBUG/MainProcess] Test Task V2 sent. id->56fa9b3a-face-498f-9c5b-8eab0b2bf44c
[2019-02-26 15:40:04,079: INFO/MainProcess] Scheduler: Sending due task Test Task V2
[2019-02-26 15:40:04,081: DEBUG/MainProcess] Test Task V2 sent. id->a6e709f1-9700-4aca-a73e-a4c5c00b5d21
[2019-02-26 15:40:04,110: INFO/MainProcess] Scheduler: Sending due task Test Task V2
[2019-02-26 15:40:04,111: DEBUG/MainProcess] Test Task V2 sent. id->ab8e990d-a325-41cc-aed3-fac7b25edd1f
[2019-02-26 15:40:04,142: INFO/MainProcess] Scheduler: Sending due task Test Task V2
[2019-02-26 15:40:04,143: DEBUG/MainProcess] Test Task V2 sent. id->87a69f6d-caee-4fc1-be82-e26219130213
[2019-02-26 15:40:04,174: INFO/MainProcess] Scheduler: Sending due task Test Task V2
[2019-02-26 15:40:04,175: DEBUG/MainProcess] Test Task V2 sent. id->0094f01e-b3df-42de-b806-5638ed524de8
[2019-02-26 15:40:04,205: INFO/MainProcess] Scheduler: Sending due task Test Task V2
[2019-02-26 15:40:04,207: DEBUG/MainProcess] Test Task V2 sent. id->87f3252a-5117-4b4c-a54d-ee392c00866c
[2019-02-26 15:40:04,237: INFO/MainProcess] Scheduler: Sending due task Test Task V2
[2019-02-26 15:40:04,238: DEBUG/MainProcess] Test Task V2 sent. id->224aed9d-5284-4fef-b49b-61292d639aec
[2019-02-26 15:40:04,268: INFO/MainProcess] Scheduler: Sending due task Test Task V2
[2019-02-26 15:40:04,269: DEBUG/MainProcess] Test Task V2 sent. id->984f49cf-3e34-495f-8a3c-bc4df2981501
[2019-02-26 15:40:04,300: INFO/MainProcess] Scheduler: Sending due task Test Task V2
[2019-02-26 15:40:04,301: DEBUG/MainProcess] Test Task V2 sent. id->ea2416bd-95d1-43e5-be75-ab3ccf891647
[2019-02-26 15:40:04,331: INFO/MainProcess] Scheduler: Sending due task Test Task V2
[2019-02-26 15:40:04,332: DEBUG/MainProcess] Test Task V2 sent. id->a551dde6-2cfa-426b-b9c4-a6a62c9a23da
[2019-02-26 15:40:04,362: INFO/MainProcess] Scheduler: Sending due task Test Task V2
[2019-02-26 15:40:04,364: DEBUG/MainProcess] Test Task V2 sent. id->36cc7207-eb4c-47bc-b1cb-f194e56225ca

解决方案

setting文件中添加DJANGO_CELERY_BEAT_TZ_AWARE

# celery 配置
CELERY_RESULT_BACKEND = 'django-db'
CELERY_ACCEPT_CONTENT = ['application/json']
CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'
CELERY_ENABLE_UTC = False
CELERY_TIMEZONE = 'Asia/Shanghai'
CELERYD_PREFETCH_MULTIPLIER = 2  # celery worker 每次去redis取任务的数量
CELERYD_MAX_TASKS_PER_CHILD = 40  # 每个worker执行了多少任务就会死掉

# 官方用来修复CELERY_ENABLE_UTC=False and USE_TZ = False 时时间比较错误的问题;
# 详情见:https://github.com/celery/django-celery-beat/pull/216/files
DJANGO_CELERY_BEAT_TZ_AWARE = False

# 使用django_celery_beat插件用来动态配置任务!
CELERY_BEAT_SCHEDULER = 'django_celery_beat.schedulers:DatabaseScheduler'

# celery结果禁止编辑
DJANGO_CELERY_RESULTS_TASK_ID_MAX_LENGTH = 191
DJANGO_CELERY_RESULTS = {
    'ALLOW_EDITS': False
}

celery.py文件中添加:

from logging import Logger

import requests
from celery import Celery
from celery.utils.log import get_task_logger
from django.utils import timezone

DJANGO_SETTINGS_MODULE = os.environ.get('DJANGO_SETTINGS_MODULE')

os.environ.setdefault('DJANGO_SETTINGS_MODULE',
                      DJANGO_SETTINGS_MODULE if DJANGO_SETTINGS_MODULE else 'test.settings')

app = Celery('cgadmin')
# IMPORTANT: fix django-celery-beat with `USE_TZ = False`, interval tasks be called continuously
app.now = timezone.now

app.config_from_object('django.conf:settings', namespace='CELERY')
app.autodiscover_tasks()

logger: Logger = get_task_logger(__name__)

参考:

https://github.com/celery/django-celery-beat/issues/109
https://github.com/celery/django-celery-beat/pull/216

  • 1
    点赞
  • 4
    收藏
    觉得还不错? 一键收藏
  • 2
    评论
评论 2
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值