1.Introduction
RabbitMQ is a message broker: it accepts and forwards messages. You can think about it as a post office: when you put the mail that you want posting in a post box, you can be sure that Mr. or Ms. Mailperson will eventually deliver the mail to your recipient. In this analogy, RabbitMQ is a post box, a post office and a postman.
- RabbitMQ是一个消息中间件，它接受和转发消息，可以看做是一个邮局
The major difference between RabbitMQ and the post office is that it doesn’t deal with paper, instead it accepts, stores and forwards binary blobs of data ‒ messages.
- 和邮局不同的是，RabbitMQ存储和转发的是二进制的数据消息
1. jargon 术语
Producing means nothing more than sending. A program that sends messages is a producer
- 消息生产/发送
A queue is the name for a post box which lives inside RabbitMQ. Although messages flow through RabbitMQ and your applications, they can only be stored inside a queue. A queue is only bound by the host’s memory & disk limits, it’s essentially a large message buffer. Many producers can send messages that go to one queue, and many consumers can try to receive data from one queue. This is how we represent a queue:
- 队列，消息存储在队列中，队列仅受内存和硬盘的限制，本质上是一个大的消息缓冲区。多个生产者可以发送到一个队列中，多个消费者可以一个队列中接收数据消息。
Consuming has a similar meaning to receiving. A consumer is a program that mostly waits to receive messages:
- 消费者，等待消费消息
2.RabbitMQ的几种模式
1.Simple
public class Send {
private final static String QUEUE_NAME = "hello";
public static void main(String[] args) {
ConnectionFactory factory = new ConnectionFactory();
//>> TODO 连接本地 RabbitMQ.
factory.setHost(ConfigUtils.host);
//>> TODO 创建Channel，完成工作的大部分api都驻留在此通道.
try (Connection connection = factory.newConnection();
Channel channel = connection.createChannel()) {
//>> TODO 定义消息队列 声明消息队列是幂等的，只会在不存在的时候创建.
channel.queueDeclare(QUEUE_NAME, false, false, false, null);
String message = "Hello World!";
//>> TODO 发布消息到队列中 消息内容是一个字节数组.
channel.basicPublish("", QUEUE_NAME, null, message.getBytes());
System.out.println(" [x] Sent '" + message + "'");
} catch (TimeoutException | IOException e) {
e.printStackTrace();
}
}
}
public class Recv {
private final static String QUEUE_NAME = "hello";
public static void main(String[] argv) throws Exception {
ConnectionFactory factory = new ConnectionFactory();
factory.setHost(ConfigUtils.host);
Connection connection = factory.newConnection();
Channel channel = connection.createChannel();
channel.queueDeclare(QUEUE_NAME, false, false, false, null);
System.out.println(" [*] Waiting for messages. To exit press CTRL+C");
//>> TODO 传递回调 缓冲推送的消息 .
DeliverCallback deliverCallback = (consumerTag, delivery) -> {
String message = new String(delivery.getBody(), StandardCharsets.UTF_8);
System.out.println(" [x] Received '" + message + "'");
};
channel.basicConsume(QUEUE_NAME, true, deliverCallback, consumerTag -> {
});
}
}
2.Work Queues
In this one we’ll create a Work Queue that will be used to distribute time-consuming tasks among multiple workers.
- 工作队列用于处理多工作者间分配耗时任务
The main idea behind Work Queues (aka: Task Queues) is to avoid doing a resource-intensive task immediately and having to wait for it to complete. Instead we schedule the task to be done later. We encapsulate a task as a message and send it to a queue. A worker process running in the background will pop the tasks and eventually execute the job. When you run many workers the tasks will be shared between them.
This concept is especially useful in web applications where it’s impossible to handle a complex task during a short HTTP request window.
-
工作队列（任务队列）主要思想是避免立即执行资源紧密型的任务，并且一直等待它执行完成。
-
这个概念在web应用中特别有用，尤其是在短http请求中处理复杂任务
1.Round-robin dispatching（轮询分发）
By default, RabbitMQ will send each message to the next consumer, in sequence. On average every consumer will get the same number of messages. This way of distributing messages is called round-robin. Try this out with three or more workers
- 默认的，RabbitMQ会按照顺序发送每条消息到下一个消费者，消费者会均分这些消息，这种分发消息的方式为轮询。
2.Message acknowledgment（消息确认）
public class Recv {
private final static String QUEUE_NAME = "hello";
public static void main(String[] argv) throws Exception {
ConnectionFactory factory = new ConnectionFactory();
factory.setHost(ConfigUtils.host);
Connection connection = factory.newConnection();
Channel channel = connection.createChannel();
channel.queueDeclare(QUEUE_NAME, false, false, false, null);
System.out.println(" [*] Waiting for messages. To exit press CTRL+C");
//>> TODO 传递回调 缓冲推送的消息 消息收到的回调方法.
DeliverCallback deliverCallback = (consumerTag, delivery) -> {
String message = new String(delivery.getBody(), StandardCharsets.UTF_8);
handlerTask(message);
System.out.println(" [x] Received '" + message + "'");
};
channel.basicConsume(QUEUE_NAME, true, deliverCallback, consumerTag -> {
});
}
//任务处理，模拟耗时
public static void handlerTask(String message) {
if (message.startsWith("a")) {
try {
Thread.sleep(1000);
System.out.println("handling [" + message + "]...");
} catch (InterruptedException e) {
e.printStackTrace();
}finally {
System.out.println("handler [" + message + "] done");
}
}
}
}
-
当前的代码，在RabbitMQ分发消息到consumer的时候，将会标记为删除，这种情况下，如何kill了正在处理的的worker，那么消息将会出现丢失。
-
我们希望不丢掉任何任务，如何某个worker挂掉了，那么希望任务被别的worker进行处理。
In order to make sure a message is never lost, RabbitMQ supports message acknowledgments. An acknowledgement is sent back by the consumer to tell RabbitMQ that a particular message has been received, processed and that RabbitMQ is free to delete it.
- 为了确保消息不丢失，RabbitMQ支持消息确认机制。这种机制是当发送到consumer消息后，告知RabbitMQ特定的消息已经收到，RabbitMQ可以随意删除它。
If a consumer dies (its channel is closed, connection is closed, or TCP connection is lost) without sending an ack, RabbitMQ will understand that a message wasn’t processed fully and will re-queue it. If there are other consumers online at the same time, it will then quickly redeliver it to another consumer. That way you can be sure that no message is lost, even if the workers occasionally die.
- 如果consumer挂了，没有返回确认信息，RabbitMQ会重新入队消息，将消息发送个其他的消费者，这种方式确保消息不会丢失。
Manual message acknowledgments are turned on by default. In previous examples we explicitly turned them off via the autoAck=true flag. It’s time to set this flag to false and send a proper acknowledgment from the worker, once we’re done with a task
- 默认情况下手动确认消息是打开的，我们通过设置autoAck=true 关闭了它。
3.Message durability（消息持久化）
- 设置队列持久化
boolean durable = true;
channel.queueDeclare("hello", durable, false, false, null);
-
RabbitMQ不允许使用不同的参数定义同一个队列，队列定义需要producer和consumer保持一致
-
设置消息持久化
channel.basicPublish("", "task_queue",
//消息持久化配置
MessageProperties.PERSISTENT_TEXT_PLAIN,
message.getBytes());
这种方式扔不能保证消息能够持久化到硬盘，如果需要更强的保证机制，使用publisher confirms。
4.Fair dispatch（公平分发）
- 假如有两个worker，RabbitMQ可能会出现将奇数消息（很重）发送给一个worker，另外一个都是轻量的消息，消息分发是不公平的。
This happens because RabbitMQ just dispatches a message when the message enters the queue. It doesn’t look at the number of unacknowledged messages for a consumer. It just blindly dispatches every n-th message to the n-th consumer.
- RabbitMQ分发消息是按照入队的消息发送的，并不会查看consumer未确认的消息，只是盲目的将第n个消息发送到第n个消费者。
In order to defeat that we can use the basicQos method with the prefetchCount = 1 setting. This tells RabbitMQ not to give more than one message to a worker at a time. Or, in other words, don’t dispatch a new message to a worker until it has processed and acknowledged the previous one. Instead, it will dispatch it to the next worker that is not still busy.
- 通过设置basicQos，告诉RabbitMQ不要发送超过一个消息到worker，或者，不要分发新的消息到worker中直到它处理并且确认了上一个
3.Publish/Subscribe
In the previous tutorial we created a work queue. The assumption behind a work queue is that each task is delivered to exactly one worker. In this part we’ll do something completely different – we’ll deliver a message to multiple consumers. This pattern is known as “publish/subscribe”
- work queue 是将task分到一个worker中。发布一个消息到多个consumer，这种模式成为发布订阅。
1.Exchanges
The core idea in the messaging model in RabbitMQ is that the producer never sends any messages directly to a queue. Actually, quite often the producer doesn’t even know if a message will be delivered to any queue at all
- RabbitMQ消息模型的核心思想是，生产者不会发送任何消息直接到队列中。实际上，生产者不知道是否消息会被传递到任何队列
Instead, the producer can only send messages to an exchange. An exchange is a very simple thing. On one side it receives messages from producers and the other side it pushes them to queues. The exchange must know exactly what to do with a message it receives. Should it be appended to a particular queue? Should it be appended to many queues? Or should it get discarded. The rules for that are defined by the exchange type.
- 生产者发送消息到exchange中，一方面它接受生产者的消息，另一方面发送到队列中。exchange必须知道它收到消息后如何处理。是追加到特定队列？所有队列？或者被丢弃？规则是被exchange type定义的。
There are a few exchange types available: direct, topic, headers and fanout. We’ll focus on the last one – the fanout. Let’s create an exchange of this type, and call it logs:
-
exchange类型可以选择的有 direct、topic、headers、fanout
-
发送到执行exchange （logs）
channel.basicPublish( "logs", "", null, message.getBytes());
2.Temporary queues（临时队列）
In the Java client, when we supply no parameters to queueDeclare() we create a non-durable, exclusive, autodelete queue with a generated name
- 当我们没有设置参数时，会创建一个非持久的，排他，自动删除的队列，with生成的名称（random name）。
String queueName = channel.queueDeclare().getQueue();
3.Bindings（绑定）
We’ve already created a fanout exchange and a queue. Now we need to tell the exchange to send messages to our queue. That relationship between exchange and a queue is called a binding
- 我们已经创建一个fanout 交换和队列，现在需要告诉exchange发送消息到队列中。这种在exchange和queue之间的关系成为binding。
channel.queueBind(queueName, "logs", "");
4.Putting it all together
The producer program, which emits log messages, doesn’t look much different from the previous tutorial. The most important change is that we now want to publish messages to our logs exchange instead of the nameless one. We need to supply a routingKey when sending, but its value is ignored for fanout exchanges
- 和之前最大的不同是，生产者发送消息到exchange中，而不是匿名的exchange，我们需要在发送的时候指定一个routingKey，但是这个属性在 fanout exchange会被忽略。
4.Routing
In the previous tutorial we built a simple logging system. We were able to broadcast log messages to many receivers.
In this tutorial we’re going to add a feature to it - we’re going to make it possible to subscribe only to a subset of the messages. For example, we will be able to direct only critical error messages to the log file (to save disk space), while still being able to print all of the log messages on the console
- 之前的教程，构建了一个简单的日志系统，我们可以广播日志到需要接受者。
- 我们将只订阅消息的一个子集。example，将错误的日志打到磁盘，所有的日志打印到控制台。
1.Bindings（绑定）
- 之前的绑定方式，binding是exchange和queue的关系
channel.queueBind(queueName, EXCHANGE_NAME, "");
- Bindings可以设置一个额外的属性routingKey
channel.queueBind(queueName, EXCHANGE_NAME, "black");
The meaning of a binding key depends on the exchange type. The fanout exchanges, which we used previously, simply ignored its value.
- binding key依赖exchange type ，fanout会忽略binding key
2.Direct exchange
We were using a fanout exchange, which doesn’t give us much flexibility - it’s only capable of mindless broadcasting.
We will use a direct exchange instead. The routing algorithm behind a direct exchange is simple - a message goes to the queues whose binding key exactly matches the routing key of the message.
- fanout只能盲目广播
- 使用direct exchange替代。消息发送到 binding key和routing key匹配的队列
In this setup, we can see the direct exchange X with two queues bound to it. The first queue is bound with binding key orange, and the second has two bindings, one with binding key black and the other one with green.
- 在这个设置中，可以看到direct exchange X 绑定了两个队列，第一个队列绑定了orange，第二个有两个绑定，一个binding key是black一个是green。
In such a setup a message published to the exchange with a routing key orange will be routed to queue Q1. Messages with a routing key of black or green will go to Q2. All other messages will be discarded
- 在这样的设置中，一个使用routing key orange 发布到exchange中，被路由到Q1，使用routing key black和green的发送到Q2，其他的消息被丢弃。
3.Multiple bindings
It is perfectly legal to bind multiple queues with the same binding key. In our example we could add a binding between X and Q1 with binding key black. In that case, the direct exchange will behave like fanout and will broadcast the message to all the matching queues. A message with routing key black will be delivered to both Q1 and Q2.
- 使用相同的binding key（应该是routing key）绑定不同的队列是合法的。
4.Emitting logs
- 发送日志，使用direct替代fanout
channel.basicPublish(EXCHANGE_NAME, severity, null, message.getBytes());
To simplify things we will assume that ‘severity’ can be one of ‘info’, ‘warning’, ‘error’.
5.Subscribing
String queueName = channel.queueDeclare().getQueue();
for(String severity : argv){
channel.queueBind(queueName, EXCHANGE_NAME, severity);
}
- 接收消息可以定义不同的绑定
6.Putting it all together
- 一个队列仅绑定error级别日志
- 另外一个绑定所有类型的
5.Topic
In the previous tutorial we improved our logging system. Instead of using a fanout exchange only capable of dummy broadcasting, we used a direct one, and gained a possibility of selectively receiving the logs.
- 之前提升了日志系统，使用direct替换了fanout，可以进行选择性的日志接收
1.Topic exchange
Messages sent to a topic exchange can’t have an arbitrary routing_key - it must be a list of words, delimited by dots. The words can be anything, but usually they specify some features connected to the message. A few valid routing key examples: “stock.usd.nyse”, “nyse.vmw”, “quick.orange.rabbit”. There can be as many words in the routing key as you like, up to the limit of 255 bytes.
- 消息发送到topic 交换不能有任意的routing key，它必须是由 . 分隔的单词列表。最大长度是255 bytes
The binding key must also be in the same form. The logic behind the topic exchange is similar to a direct one - a message sent with a particular routing key will be delivered to all the queues that are bound with a matching binding key. However there are two important special cases for binding keys:
- binding key 也必须是相同的形式。使用特定routing key的将会被发送到所有匹配binding key的队列。有一下两种匹配模式：
- * (star) can substitute for exactly one word.（恰好一个）
- # (hash) can substitute for zero or more words.（0个或者多个）
- 匹配不到这两种模式的消息将被丢弃
When a queue is bound with “#” (hash) binding key - it will receive all the messages, regardless of the routing key - like in fanout exchange.
When special characters, “*” (star) and “#” (hash), aren’t used in bindings, the topic exchange will behave just like a direct one.
- 如果绑定 # 那么会接收所有消息，类似fanout
- 如果没有使用特定字符 * 或者 #，那么topic的行为会类似direct
6.Rpc（Remote procedure call）远程程序调用
But what if we need to run a function on a remote computer and wait for the result? Well, that’s a different story. This pattern is commonly known as Remote Procedure Call or RPC.
- 如果我们需要跑一个函数在远程计算机，并且等待结果，这种模式称为RPC。
In this tutorial we’re going to use RabbitMQ to build an RPC system: a client and a scalable RPC server.
- 这个教程中使用RabbitMQ构建一个RPC系统，一个客户端和一个可伸缩的RPC 服务器
1.Client interface
- 调用call方法 返回斐波那契数列结果
FibonacciRpcClient fibonacciRpc = new FibonacciRpcClient();
String result = fibonacciRpc.call("4");
System.out.println( "fib(4) is " + result);
2.Callback queue
In general doing RPC over RabbitMQ is easy. A client sends a request message and a server replies with a response message. In order to receive a response we need to send a ‘callback’ queue address with the request. We can use the default queue (which is exclusive in the Java client). Let’s try it:
- 客户端发送请求消息，服务器用响应消息进行响应。为了接收响应，我们需要发送一个“回调”队列地址与请求。我们可以使用默认队列
callbackQueueName = channel.queueDeclare().getQueue();
BasicProperties props = new BasicProperties
.Builder()
.replyTo(callbackQueueName)
.build();
channel.basicPublish("", "rpc_queue", props, message.getBytes());
// ... then code to read a response message from the callback_queue ...
3.Message properties
The AMQP 0-9-1 protocol predefines a set of 14 properties that go with a message. Most of the properties are rarely used, with the exception of the following:
- deliveryMode: Marks a message as persistent (with a value of 2) or transient (any other value). You may remember this property from the second tutorial.（消息持久化）
- contentType: Used to describe the mime-type of the encoding. For example for the often used JSON encoding it is a good practice to set this property to: application/json.）（json请求类型）
- replyTo: Commonly used to name a callback queue.（callback 队列）
- correlationId: Useful to correlate RPC responses with requests. （关联rpc 请求和响应）
4.Correlation Id
In the method presented above we suggest creating a callback queue for every RPC request. That’s pretty inefficient, but fortunately there is a better way - let’s create a single callback queue per client.
- 上面的方法为每个RPC请求都创建一个回调队列，是非常低效的，有一种更好的方式，可以为每个客户端创建一个回调队列
That raises a new issue, having received a response in that queue it’s not clear to which request the response belongs. That’s when the correlationId property is used. We’re going to set it to a unique value for every request. Later, when we receive a message in the callback queue we’ll look at this property, and based on that we’ll be able to match a response with a request. If we see an unknown correlationId value, we may safely discard the message - it doesn’t belong to our requests.
- 这回有一个新的问题，收到响应后不知道是哪个请求的结果，因此使用correlationId，为每个请求设置一个唯一值，通过这个属性，匹配请求和响应的结果，如果有一个未知的correlationId，则丢弃消息，它不属于这个请求。
Our RPC will work like this:
- For an RPC request, the Client sends a message with two properties: replyTo, which is set to an anonymous exclusive queue created just for the request, and correlationId, which is set to a unique value for every request.（对于RPC请求，客户端发送一个消息携带两个属性，replyTo，是为请求创建的一个排他队列，correlationId，是请求的唯一值）
- The request is sent to an rpc_queue queue.（请求发送到rpc队列）
- The RPC worker (aka: server) is waiting for requests on that queue. When a request appears, it does the job and sends a message with the result back to the Client, using the queue from the replyTo field.（RPC worker或者说是server 等待请求，当请求到达，它处理任务并且发送消息的结果到客户端，使用replyTo字段中的队列。）
- The client waits for data on the reply queue. When a message appears, it checks the correlationId property. If it matches the value from the request it returns the response to the application.（客户端等待应答队列上的数据，当消息到达，它检查correlationId属性，如果匹配请求发送的id，那么将响应结果返回）
5.代码设计
The server code is rather straightforward:（server端）
- As usual we start by establishing the connection, channel and declaring the queue.（建立连接，channel，定义队列）
- We might want to run more than one server process. In order to spread the load equally over multiple servers we need to set the prefetchCount setting in channel.basicQos.（设置channel.basicQos，平均服务端的负载）
- We use basicConsume to access the queue, where we provide a callback in the form of an object (DeliverCallback) that will do the work and send the response back.（使用basicConsume访问队列，使用DeliverCallback处理任务，发送响应 ）
The client code is slightly more involved: （client端代码）
- We establish a connection and channel.（建立连接和channel）
- Our call method makes the actual RPC request.（调用rpc请求方法）
- Here, we first generate a unique correlationId number and save it - our consumer callback will use this value to match the appropriate response.（生成correlationId，我们的消费者回调会使用这个值匹配响应）
- Then, we create a dedicated exclusive queue for the reply and subscribe to it.（创建一个专用的应答排他队列）
- Next, we publish the request message, with two properties: replyTo and correlationId.（发送请求信息，并携带replayTo和correlationId）
- At this point we can sit back and wait until the proper response arrives.（等待响应）
- Since our consumer delivery handling is happening in a separate thread, we’re going to need something to suspend the main thread before the response arrives. Usage of BlockingQueue is one possible solutions to do so. Here we are creating ArrayBlockingQueue with capacity set to 1 as we need to wait for only one response.（需要在响应到达之前挂起主线程，使用BlockingQueue）
- The consumer is doing a very simple job, for every consumed response message it checks if the correlationId is the one we’re looking for. If so, it puts the response to BlockingQueue.（消费者判断correlationId是不是我们要的，是则放入到BlockingQueue中）
- At the same time main thread is waiting for response to take it from BlockingQueue.（主线程正在等待BlockingQueue的响应）
- Finally we return the response back to the user.（返回响应给用户）
7.Publisher Confirms
Publisher confirms are a RabbitMQ extension to implement reliable publishing. When publisher confirms are enabled on a channel, messages the client publishes are confirmed asynchronously by the broker, meaning they have been taken care of on the server side
- 发布者确认是RabbitMQ扩展来实现可靠的发布。当发送者机制在channel启用后，客户端发送的消息将由borker异步确认，意味着他们已经在服务端得到处理。
1.Enabling Publisher Confirms on a Channel（启用发布确认通道）
Publisher confirms are a RabbitMQ extension to the AMQP 0.9.1 protocol, so they are not enabled by default. Publisher confirms are enabled at the channel level with the confirmSelect method:
Channel channel = connection.createChannel();
channel.confirmSelect();
2.Strategy #1: Publishing Messages Individually （单独发送）
Let’s start with the simplest approach to publishing with confirms, that is, publishing a message and waiting synchronously for its confirmation:
- 发布消息并同步等待确认
while (thereAreMessagesToPublish()) {
byte[] body = ...;
BasicProperties properties = ...;
channel.basicPublish(exchange, queue, properties, body);
// uses a 5 second timeout
channel.waitForConfirmsOrDie(5_000);
}
In the previous example we publish a message as usual and wait for its confirmation with the Channel#waitForConfirmsOrDie(long) method. The method returns as soon as the message has been confirmed. If the message is not confirmed within the timeout or if it is nack-ed (meaning the broker could not take care of it for some reason), the method will throw an exception. The handling of the exception usually consists in logging an error message and/or retrying to send the message.
- 在这个例子中发送一个消息等待Channel#waitForConfirmsOrDie(long) 方法进行消息确认，消息一经确认，则立刻返回，如果消息在超时时间内没有返回，将会抛出异常。消息的异常处理通常包括记录异常信息，或者消息发送重试。
This technique is very straightforward but also has a major drawback: it significantly slows down publishing, as the confirmation of a message blocks the publishing of all subsequent messages. This approach is not going to deliver throughput of more than a few hundreds of published messages per second. Nevertheless, this can be good enough for some applications
- 这种方式非常简单，但是也有一个缺点，显著地降低了消息发布的速度。因为消息的确认会阻塞后续消息的发布。这种方式的吞吐不会超过每秒几百条
560
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
