为什么要使用Rabbitmq

It all depends on what you want to use it for.

RabbitMQ is the recommended solution, it is widely deployed, tested and supports all features. With optimized configuration you can process up to 15000 persistent tasks/s with a single process. RabbitMQ also comes with management and monitoring solutions (full disclosure: I work with the RabbitMQ team at VMware, and I'm the Celery project lead).

Redis is also a popular option, it supports almost all features, but it is not very durable in its default configuration, if the Redis server is abruptly terminated you may at worst lose minutes of data (http://redis.io/topics/persistence). Even with append-only mode there is a chance of data loss if the celeryd instance is abruptly killed: this is because Redis does not support message acknowledgements. We are working on a patch to emulate this, but that will impact performance quite a lot (in the mean-time you should set CELERYD_PREFETCH_MULTIPLIER=1 to minimize the impact of such a data loss scenario).

The database backends were originally made for testing purposes only, but they are used in production nonetheless. The database backends also does not support events and broadcast, so the Celery monitor and remote control commands (celeryctl) will not work. It may work well enough for small deployments, but if you ever have the need to add more celeryd instances you should pick another option. Sadly, databases are not usually suited for this kind of work (though postgres 9 comes with new features that makes it "more suitable", that we may eventually support).

MongoDB I have not had much experience with myself, but I know the Celery backend is being used and is actively maintained (it just recently gained support for monitoring and remote control support). I imagine, though, that it has the same durability problems as mention above with Redis.

In addition there is support for using Amazon SQS, Beanstalk, CouchDB. These backends do not support monitoring and remote control commands.

In short; If not losing tasks is crucial, you should be using RabbitMQ. The database backend is OK with low task volumes and one or maybe two celeryd instances.

转载于:https://www.cnblogs.com/skyendless/archive/2012/03/06/2381851.html