django3 使用celery+redis实现异步任务

准备工作

1. 安装：

django==3.0.5
celery==4.4.2
eventlet==0.25.2
redis==3.3.11

2. 修改settings.py文件

# celery 设置
# celery中间人 redis://redis服务所在的ip地址:端口/数据库号
BROKER_URL = 'redis://127.0.0.1:6379/1'
# celery结果返回，可用于跟踪结果
CELERY_RESULT_BACKEND = 'redis://127.0.0.1:6379/1'

# celery内容等消息的格式设置
CELERY_ACCEPT_CONTENT = ['application/json', ]
CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'

# celery时区设置，使用settings中TIME_ZONE同样的时区
CELERY_TIMEZONE = TIME_ZONE

3. 在settings.py同级目录创建celery.py文件

内容如下

# coding:utf-8
from __future__ import absolute_import, unicode_literals
from celery import Celery
from django.conf import settings
import os


# 获取当前文件夹名，即为该Django的项目名
project_name = os.path.split(os.path.abspath('.'))[-1]
project_settings = '%s.settings' % project_name

# 设置环境变量
os.environ.setdefault('DJANGO_SETTINGS_MODULE', project_settings)

# 实例化Celery,网上很多教程这里都是没有设置broker造成启动失败
app = Celery('LED',broker=settings.BROKER_URL,backend=settings.CELERY_RESULT_BACKEND)

app.conf['imports'] = ['app.v1.BasicMethod.tasks', ] # tasks文件的位置，不写这个会报KeyError

# 使用django的settings文件配置celery
app.config_from_object('django.conf:settings', namespace='CELERY')

# Celery加载所有注册的应用
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)

@app.task(bind=True)
def debug_task(self):

    print('Request: {0!r}'.format(self.request))

在 当前目录的__init__.py文件中写入

from __future__ import absolute_import

# This will make sure the app is always imported when
# Django starts so that shared_task will use this app.
from .celery import app as celery_app

__all__ = ('celery_app',)

4. 创建tasks.py文件

内容如下

# coding:utf-8
from LED import celery_app #导入celery的app
from time import sleep


@celery_app.task
def helloWorld():
    print('hello world')
    sleep(10)
    print('hello world')
    return 'hello world ready'

5. 启用celery

在manage.py同级下运行以下命令

celery -A LED worker -l info 

windows下报错ValueError: not enough values to unpack (expected 3, got 0)就使用
celery -A LED worker -l info -P eventlet

如果使用时在windows平台下报错django.db.utils.DatabaseError: DatabaseWrapper objects created in a thread can only be used in that same thread的话，请修改celery源码

找到源码中这个方法

def _patch_eventlet():
    import eventlet
    import eventlet.debug

    eventlet.monkey_patch()
    blockdetect = float(os.environ.get('EVENTLET_NOBLOCK', 0))
    if blockdetect:
        eventlet.debug.hub_blocking_detection(blockdetect, blockdetect)

修改为

def _patch_eventlet():
    import eventlet
    import eventlet.debug

    eventlet.monkey_patch(thread=False) # 修改部分
    blockdetect = float(os.environ.get('EVENTLET_NOBLOCK', 0))
    if blockdetect:
        eventlet.debug.hub_blocking_detection(blockdetect, blockdetect)

该方法参考至https://blog.csdn.net/u014007037/article/details/86645862

感谢作者！

6. 调用tasks异步任务方法

from tasks import helloWorld

helloWorld.delay()