1 RabbitMQ
-
介绍
- 队列消息是消息在传输过程中保存消息的容器
- 现主流消息队列：RabbitMQ、ActiveMQ、Kafka等
- RabbitMQ与ActiveMQ比较
- 系统吞吐量：RabbitMQ好于ActiveMQ
- 持久化消息：都支持
- 高并发和可靠性：RabbitMQ好于ActiveMQ
- RabbitMQ与Kafka
- 系统吞吐量：RabbitMQ弱于Kafka
- 可靠性和稳定性：RabbitMQ好于Kafka
- 设计初衷：Kafka用来处理日志
- RabbitMQ与ActiveMQ比较
-
安装RabbitMQ(centos7)
-
先安装erlang
# 在/etc/yum.repos.d/目录中添加rabbitmq_erlang.repo文件 # sudo vi /etc/yum.repos.d/rabbitmq_erlang.repo # 在文件中添加 # In /etc/yum.repos.d/rabbitmq_erlang.repo [rabbitmq-erlang] name=rabbitmq-erlang baseurl=https://dl.bintray.com/rabbitmq-erlang/rpm/erlang/21/el/7 gpgcheck=1 gpgkey=https://dl.bintray.com/rabbitmq/Keys/rabbitmq-release-signing-key.asc repo_gpgcheck=0 enabled=1 # 执行安装(测试时安装1分钟，实际安装，需要换源) sudo yum install erlang -y -
安装RabbitMQ
rpm --import https://github.com/rabbitmq/signing-keys/releases/download/2.0/rabbitmq-release-signing-key.asc # 在/etc/yum.repos.d/目录中添加rabbitmq_server.repo文件 vi /etc/yum.repos.d/rabbitmq_server.repo # 在文件中添加 # In /etc/yum.repos.d/rabbitmq_server.repo [bintray-rabbitmq-server] name=bintray-rabbitmq-rpm baseurl=https://dl.bintray.com/rabbitmq/rpm/rabbitmq-server/v3.7.x/el/7/ gpgcheck=0 repo_gpgcheck=0 enabled=1 # 执行安装(实际安装1分钟) sudo yum install rabbitmq-server -y sudo systemctl start rabbitmq-server.service
-
2 Celery
-
安装
pip install -U Celery -
一把辛酸泪的配置
试验了好久，才成功，要异步调用发送短信的方式必须得先加载django环境。。试了半天，一把辛酸泪，好了不多说，看怎么配置的吧
-
首先在manage.py同级目录下创建一个celery_tasks文件夹
-
在celery_tasks中创建tasks文件夹、config.py、main.py、test.py、
__init__.py -
在tasks文件夹中创建
__init__.py、sms.py -
接下来就是各个文件的内容了
main.py(最关键，最头疼研究半天的地方)
import os import sys BASE_DIR = os.path.dirname(os.path.dirname(__file__)) sys.path.insert(0, BASE_DIR) print(BASE_DIR) from celery import Celery # 工作看板，发布任务和接收处理任务，异步执行的信息 # 注：settings.dev是我自己的配置文件目录，若不同，改变一下即可 os.environ.setdefault("DJANGO_SETTINGS_MODULE", "meiduo_mall.settings.dev") # 创建实例 celery_app = Celery('meiduo_celery') # 加载celery配置 celery_app.config_from_object('celery_tasks.config') # 自动注册celery任务 celery_app.autodiscover_tasks(['tasks.sms'])config.py
# 指定消息队列的位置 broker_url= 'amqp://guest:guest@localhost:5672'sms.py
from main import celery_app from verifications.lib.yuntongxun.sms import CCP @celery_app.task() def send_sms(mobile, sms_code, t, template_id): try: ccp = CCP() # ccp.send_template_sms(mobile, [sms_code, 5], 1) res = ccp.send_template_sms(mobile, [sms_code, t], template_id) except Exception as e: res = -1至此，配置算是完工了，虽然代码量不大，但调起来够头疼的
-
-
终端运行一下
在celery_tasks目录下即可
celery -A main worker -l info若看到如下结果说明成功了一半
-------------- celery@study v4.3.0 (rhubarb) ---- **** ----- --- * *** * -- Linux-3.10.0-514.26.2.el7.x86_64-x86_64-with-centos-7.3.1611-Core 2019-12-04 02:39:28 -- * - **** --- - ** ---------- [config] - ** ---------- .> app: meiduo_celery:0x7f84f9bcb400 - ** ---------- .> transport: amqp://guest:**@localhost:5672// - ** ---------- .> results: disabled:// - *** --- * --- .> concurrency: 1 (prefork) -- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker) --- ***** ----- -------------- [queues] .> celery exchange=celery(direct) key=celery [tasks] . tasks.sms.send_sms -
测试
test.py
from tasks.sms import send_sms ret = send_sms.delay('你的手机号', 8888, 5, 1)成功收到短信就算成功了
3 改造注册中的发送短信
-
view的改造
from celery_tasks.tasks.sms import send_sms send_sms.delay(mobile, sms_code, 5, 1)本来以为终于完工了，结果又出了个让我有些疑惑的问题
如下celery运行的报错
[2019-12-04 03:03:18,395: ERROR/MainProcess] Received unregistered task of type 'celery_tasks.tasks.sms.send_sms'. The message has been ignored and discarded. Did you remember to import the module containing this task? Or maybe you're using relative imports? Please see http://docs.celeryq.org/en/latest/internals/protocol.html for more information. The full contents of the message body was: '[["我的手机号", "067287", 5, 1], {}, {"callbacks": null, "errbacks": null, "chain": null, "chord": null}]' (106b) Traceback (most recent call last): File "/home/python/.virtualenvs/meiduo/lib/python3.6/site-packages/celery/worker/consumer/consumer.py", line 559, in on_task_received strategy = strategies[type_] KeyError: 'celery_tasks.tasks.sms.send_sms'这一下又懵逼了。。连续尝试了三回，抱着死马当活马医的想法，终止Celery，重启了一下
-------------- celery@study v4.3.0 (rhubarb) ---- **** ----- --- * *** * -- Linux-3.10.0-514.26.2.el7.x86_64-x86_64-with-centos-7.3.1611-Core 2019-12-04 03:07:04 -- * - **** --- - ** ---------- [config] - ** ---------- .> app: meiduo_celery:0x7f400a8063c8 - ** ---------- .> transport: amqp://guest:**@localhost:5672// - ** ---------- .> results: disabled:// - *** --- * --- .> concurrency: 1 (prefork) -- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker) --- ***** ----- -------------- [queues] .> celery exchange=celery(direct) key=celery [tasks] . celery_tasks.tasks.sms.send_sms . tasks.sms.send_smstasks里的任务变成两条了？也能执行异步发送短信的任务了。。。不明所以，但感觉这样怪怪的，决定注释掉一行代码
import os import sys BASE_DIR = os.path.dirname(os.path.dirname(__file__)) sys.path.insert(0, BASE_DIR) print(BASE_DIR) from celery import Celery # 工作看板，发布任务和接收处理任务，异步执行的信息 os.environ.setdefault("DJANGO_SETTINGS_MODULE", "meiduo_mall.settings.dev") # 创建实例 celery_app = Celery('meiduo_celery') # 加载celery配置 celery_app.config_from_object('celery_tasks.config') # 自动注册celery任务 # celery_app.autodiscover_tasks(['tasks.sms'])试验了一下，能成功
-------------- celery@study v4.3.0 (rhubarb) ---- **** ----- --- * *** * -- Linux-3.10.0-514.26.2.el7.x86_64-x86_64-with-centos-7.3.1611-Core 2019-12-04 03:21:02 -- * - **** --- - ** ---------- [config] - ** ---------- .> app: meiduo_celery:0x7f199ca91358 - ** ---------- .> transport: amqp://guest:**@localhost:5672// - ** ---------- .> results: disabled:// - *** --- * --- .> concurrency: 1 (prefork) -- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker) --- ***** ----- -------------- [queues] .> celery exchange=celery(direct) key=celery [tasks] . celery_tasks.tasks.sms.send_sms# 运行成功 [2019-12-04 03:21:47,918: INFO/ForkPoolWorker-1] Task celery_tasks.tasks.sms.send_sms[1a9c08d0-16ca-4242-a860-a17d590cad9c] succeeded in 0.09093303099507466s: None哎，至此，蛋疼的异步处理总算完成了
364
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
