python中的异步Rabbitmq客户端

RabbitMQ is high performance of message-queueing software writing in Erlang. It is used to dispatch messages it receives. The client connecting and sending messages to it is called producer and that waiting for message is called consumer. The basic skeleton of RabbitMQ is as follow:

RabbitMQ是用Erlang编写的高性能的消息队列软件。 它用于调度收到的消息。 连接并向其发送消息的客户端称为生产者，而等待消息的客户端称为使用者。 RabbitMQ的基本框架如下：

RabbitMQ is widely used in a large system which helps to load balances. Many time-wasting processes will bring out worse user experiences. For example, if you purchase online on credit card, what does happen when in backend? Your order information will be split into various messages and those will be sent to RabbitMQ or any other message queue system. The repository client will consume this message and prepares the goods for you. The payment client will consume the message and email the bill to you. As you can see, all the works are done by different worker client. As you can see, you don’t have to wait the above jobs and all you to do is “one-click”.

RabbitMQ被广泛用于有助于负载平衡的大型系统中。 许多浪费时间的过程会带来更糟的用户体验。 例如，如果您使用信用卡在线购买，在后端时会发生什么？ 您的订单信息将被分为各种消息，并将这些消息发送到RabbitMQ或任何其他消息队列系统。 资源库客户端将使用此消息并为您准备商品。 付款客户将使用该消息，并将账单通过电子邮件发送给您。 如您所见，所有工作都是由不同的工作人员客户端完成的。 如您所见，您不必等待上述工作，而您要做的就是“一键式”。

If thousands of client send messages to RabbitMQ simultaneously, what would like to occur?

如果成千上万的客户端同时向RabbitMQ发送消息，将会发生什么？

Traditional RabbitMQ’s architecture is the publishers , consumers and RabbitMQ server are distributed among different hosts. publishers and consumers connect to server with TCP connection. As we all known, IO will do harm to system performance because it will invoke system call. Multithreads solution seems to be nice but it also brings out disasters if you take C10K problem into consideration.

传统RabbitMQ的体系结构是发布者，使用者和RabbitMQ服务器分布在不同的主机之间。 发布者和使用者使用TCP连接连接到服务器。 众所周知，IO将损害系统性能，因为它将调用系统调用。 多线程解决方案似乎不错，但如果考虑C10K问题，它也会带来灾难。

Is any strategies that we can take? The answer is yes, using asynchronous no-blocking framework.

我们可以采取任何策略吗？ 答案是肯定的，使用异步无阻塞框架。

2 Pika RabbitMQ驱动程序 (2 Pika RabbitMQ Driver)

Pika is RabbitMQ driver for Python.

Pika是Python的RabbitMQ驱动程序。

2.1 Synchronous Producer

2.1同步生产者

Let’s take the first glancing at synchronous producer.

让我们首先看一下同步生产者。

How can we use this publisher client?

我们如何使用此发布者客户端？

With SyncAMQProducer("127.0.0.1", "exchange_name") as p:
    p.publish("topic.one", "message to send")

When we create a instance of SyncRabbitMQProducer, it establishes a TCP connection with RabbitMQ server with BlockingConnection which means it will block the process. When invokes publish method, it will check whether channel has been built and then publish messages.

当我们创建SyncRabbitMQProducer的实例时，它使用BlockingConnection与RabbitMQ服务器建立TCP连接，这意味着它将阻止该进程。 当调用publish方法时，它将检查通道是否已建立，然后发布消息。

2.2 Drawbacks

2.2缺点

The handshake process for a RabbitMQ connection is actually quite involved and requires at least 7 TCP packets (more if TLS is used). Each publisher establishes a connection, creates a new channel, sends messages and closes channel and connection. Most of time is wasted in system IO. How can we solve this problem? The answer is TornadoConnection which is provided by pika library. You can using tornado IO loop to achieve asynchronous connection.

RabbitMQ连接的握手过程实际上涉及很多，并且至少需要7个TCP数据包(如果使用TLS，则更多)。 每个发布者都建立一个连接，创建一个新频道，发送消息并关闭频道和连接。 大部分时间都浪费在系统IO上。 我们如何解决这个问题？ 答案是pika库提供的TornadoConnection 。 您可以使用龙卷风IO循环来实现异步连接。

3异步客户端 (3 Asynchronous Client)

• Each RabbitMQ client has two connections, one for publishing message and the other for consuming message.
  每个RabbitMQ客户端都有两个连接，一个用于发布消息，另一个用于使用消息。
• Creating a new channel as long as it publish message.
  只要发布消息，就创建一个新通道。
• Aside from publisher and consumer, it have to plays a role of RPC.
  除了发布者和使用者之外，它还必须扮演RPC的角色。
• The client must to be robust which means you have to handler errors as much as possible.
  客户端必须健壮，这意味着您必须尽可能多地处理错误。

3.1 Asynchronous Connection

3.1异步连接

Here in line 28, we create new instance of TornadoConnection but its initialization is asynchronous, it will yield out by IO loop. Once connection is open, it will callback open_callback in line 15 . We set connection into Future as result. In line 2, the connect method wait those two connection to establish before it return. But we should be aware of connect method is also asynchronous.

在第28行，我们创建了TornadoConnection的新实例，但是它的初始化是异步的，它将通过IO循环产生。 连接打开后，它将在第15行中回调open_callback 。 结果将连接设置为Future 。 在第2行中， connect方法在返回之前等待这两个连接建立。 但是我们应该知道connect方法也是异步的。

3.2 Create new Channel

3.2建立新频道

We have to admit the fact that RabbitMQ connections are a lot more resource heavy than channel. Connection should be long live, and channels can be open and closed frequently. So our strategy is that we create a new channel for each publish and consume. After publishing message, the channel will be useless so we choose to close it as soon as possible.

我们必须承认，RabbitMQ连接比通道资源占用更多的事实。 连接应该长寿，并且通道可以经常打开和关闭。 因此，我们的策略是为每个发布和使用创建一个新渠道。 发布消息后，该频道将无用，因此我们选择尽快关闭它。

I guess you have known how the code works. It does not create a new channel as channel() method called in line 16. It will invoke open_callback in line 11 once the channel is open.

我想您已经知道代码的工作原理。 它不会创建第16行中称为channel()方法的新通道。 通道打开后，它将在第11行中调用open_callback 。

3.3 Exchange, Queue and Routing Key

3.3交换，队列和路由密钥

The exchange, queue and routing key are basic terminologies in RabbitMQ. All of them can be declared or bound in a synchronous way.

交换，队列和路由键是RabbitMQ中的基本术语。 所有这些都可以以同步方式声明或绑定。

Those three methods share the same idea with previous asynchronouse code.

这三种方法与以前的异步代码具有相同的想法。

3.3 Publish

3.3发布

All the codes are self-explanatory.

所有代码都是不言自明的。

3.4 Receive

3.4接收

As we mentioned above, we have to take RPC into consideration. So after consuming the message by passing handler, we check the message properties whether it was not None. If it was not none, we create a channel to publish result into RabbitMQ. Here is the code:

如上所述，我们必须考虑RPC。 因此，在通过传递处理程序使用了消息之后，我们检查消息属性是否不是None 。 如果不是全部，我们将创建一个将结果发布到RabbitMQ的渠道。 这是代码：

Different from publish, receive declares exchange and queue and binds routing key ahead.

与发布不同，接收声明交换和排队并预先绑定路由密钥。

3.5 RPC

3.5 RPC

RabbitMQ helps us to implement a micro Remote Process Call (RPC) framework. We send message to the queue and consumer receive and handle it. After then sends result back into RabbitMQ. Finally, we complete a RPC procedure.

RabbitMQ帮助我们实现了微型远程进程调用(RPC)框架。 我们将消息发送到队列，然后消费者接收并处理它。 之后，将结果发送回RabbitMQ。 最后，我们完成一个RPC过程。

To avoid declaring exchange and queue repeat, we take exchange as key to prevent it. And Creating callback queue randomly by RabbitMQ. Before we send message to RabbitMQ, we start a consume wait result from `server` in callback queue. In _call method, we using uuid to generate a unique key for this call and a Future to wait result asynchronous. Apart from basic feature, the rpc provider timeout to prevent unexpected condition from server.

为了避免声明交换和队列重复，我们将交换作为防止它的关键。 并通过RabbitMQ随机创建回调队列。 在将消息发送到RabbitMQ之前，我们从回调队列中的`server`开始消耗等待结果。 在_call方法中，我们使用uuid生成此调用的唯一键，并使用Future异步生成结果。 除了基本功能外， rpc提供程序超时还可以防止server发生意外情况。

翻译自: https://medium.com/@fenga/an-asynchronous-rabbitmq-client-in-python-274a310858a1