flask-celery异步集成

Celery是什么？

Celery是个异步分布式任务队列。
通过Celery在后台跑任务并不像用线程那么的简单，但是用Celery的话，能够使应用有较好的可扩展性，因为Celery是个分布式架构。下面介绍Celery的三个核心组件。

1. 生产者(Celery client)。生产者(Celery client)发送消息。在Flask上工作时，生产者(Celery client)在Flask应用内运行。
2. 消费者(Celery workers)。消费者用于处理后台任务。消费者(Celery client)可以是本地的也可以是远程的。我们可以在运行Flask的server上运行一个单一的消费者(Celery workers)，当业务量上涨之后再去添加更多消费者(Celery workers)。
3. 消息传递者(message broker)。生产者(Celery client)和消费者(Celery workers)的信息的交互使用的是消息队列(message queue)。Celery支持若干方式的消息队列，其中最常用的是RabbitMQ和Redis.

话不多说上代码先。

版本要球：

Flask==1.0.2
celery==3.1.21
redis==2.10.5

代码：

新建celery_app模块与app同级别 如图：

celery_app模块下有两个文件 tasks.py __init__.py：

tasks.py

#! /usr/bin/env python
# -*- coding:utf-8 -*-
"""
__title__ = ''
__author__ = 'BY'
__mtime__ = '2018/11/30'
"""
from app.models.article import ArticleSource
from app.user.services.services_article_Info import ServicesArticleInfo
from . import celery
import os


@celery.task
def a_task():
    """
    :return:
    """
    from app import create_app, db
    app = create_app( os.getenv( 'development' ) or 'default' )
    article_source = ArticleSource( name="116" )
    db.session.add( article_source )
    db.session.commit()
    db.session.close()
    return True


@celery.task
def create_article_task(crawl_id, operator_id, user_id ):
    """
    :param crawl_id:
    :param operator_id:
    :param user_id:
    :return:
    """
    from app import create_app, db
    from app.article.services.article_service import ArticleService
    app = create_app( os.getenv( 'development' ) or 'default' )
    article_service = ArticleService()
    article_service.create_article_by_crawl(crawl_id=crawl_id, operator_id=operator_id, user_id=user_id)
    return True


@celery.task
def delete_article_task(creator_id_list):
    """
    :param creator_id_list:
    :return:
    """
    from app import create_app, db
    app = create_app(os.getenv( 'development' ) or 'default' )
    services_article_info = ServicesArticleInfo()
    services_article_info.delete_article(creator_id_list)
    return True



@celery.task
def delete_comments_task(commenter_id_list):
    """
    :param commenter_id_list:

    :return:
    """
    from app import create_app, db
    app = create_app(os.getenv( 'development' ) or 'default' )
    services_article_info = ServicesArticleInfo()
    services_article_info.delete_comments(commenter_id_list)
    return True

__init__.py:

#! /usr/bin/env python
# -*- coding:utf-8 -*-
"""
__title__ = ''
__author__ = 'BY'
__mtime__ = '2018/11/30'
"""
from celery import Celery, platforms
from setting import redis_url

platforms.C_FORCE_ROOT = True

celery = Celery( 'celery_app',
                 broker=redis_url,
                 backend=redis_url,
                 include=['celery_app.tasks', "celery_app.user_tasks"],
                 C_FORCE_ROOT=True,

                 )

运行celery:

celery worker -A celery_app.celery --loglevel=info