rabbitMQ Work Queue

目录

1、work queue（工作队列）  

2、async task（异步任务）

3、Round-robin dispatching（轮询分发）

4、Message acknowledgment（消息确认）

5、Message durability（消息持久化）

6、Fair dispatch（公平分发）

---

1、work queue（工作队列）  

        The main idea behind a work queue (task queue) is to avoid executing a resource-intensive task immediately and having to wait for it to complete. Instead, we schedule tasks for later. We encapsulate the task as a message and send it to the queue. The worker process running in the background will pop up the task and finally execute the job. When you run multiple workers, tasks are shared between them.

        This concept is particularly useful in web applications, where it is impossible to handle complex tasks in short HTTP request Windows.

2、async task（异步任务）

        Now we will send a string representing a complex task. We don't have real world tasks, like resizing an image or rendering a pdf file, so let's pretend we're busy - by using the Thread.sleep() function. We'll take the number of points in the string as its complexity; Each point will account for one second of "work." For example, Hello... It'll only take three seconds.

• 生产者

    public static void main(String[] args) throws Exception {

        // 发送的消息中带有 '.' , 一个.意味着消息处理的时间需要1s
        String msg = Arrays.toString(args);

        // 建立连接
        ConnectionFactory connectionFactory = new ConnectionFactory();
        connectionFactory.setHost("localhost");
        // 创建channel
        try (Connection connection = connectionFactory.newConnection(); Channel channel = connection.createChannel()) {
            // 声明队列
            channel.queueDeclare("NewTask", false, false, false, null);
            // 发送消息
            channel.basicPublish("", "NewTask", null, msg.getBytes());
            log.info("send message: {}", msg);
        }
    }

• 消费者

public static void main(String[] args) throws Exception {
        ConnectionFactory connectionFactory = new ConnectionFactory();
        connectionFactory.setHost("localhost");

        Connection connection = connectionFactory.newConnection();
        Channel channel = connection.createChannel();
        channel.queueDeclare("NewTask", false, false, false, null);

        log.info(" [*] Waiting for messages. To exit press CTRL+C");

        // 回调函数,打印消息内容
        DeliverCallback deliverCallback = new DeliverCallback() {
            @Override
            public void handle(String consumerTag, Delivery delivery) throws IOException {
                String message = new String(delivery.getBody(), "UTF-8");
                log.info("before [x] Received '" + message + "'");
                // 处理任务耗时
                try {
                    doWork(message);
                } catch (InterruptedException e) {
                    throw new RuntimeException(e);
                }
                log.info("after [x] Received '" + message + "'");
            }
        };
        // 开启消费者监听
        channel.basicConsume("NewTask", true, deliverCallback, consumerTag ->{});
    }

• 模拟任务执行耗时

   /**
     * 工作任务执行耗时
     * @param message
     * @throws InterruptedException
     */
    public static void doWork(String message) throws InterruptedException {
        for (char ch: message.toCharArray()) {
            if (ch == '.') Thread.sleep(1000);
        }
    }

 结果如下：

3、Round-robin dispatching（轮询分发）

使用任务队列的优点之一是能够轻松地并行化工作。如果我们积压了大量的工作，我们可以增加更多的工作人员，这样就很容易扩大规模。

一个生产者，两个消费者，结果发现rabbitMQ会根据发送消息的次数来做轮询

4、Message acknowledgment（消息确认）

        In order to make sure a message is never lost, RabbitMQ supports message acknowledgments. An acknowledgement is sent back by the consumer to tell RabbitMQ that a particular message has been received, processed and that RabbitMQ is free to delete it.

        If a consumer dies (its channel is closed, connection is closed, or TCP connection is lost) without sending an ack, RabbitMQ will understand that a message wasn't processed fully and will re-queue it. If there are other consumers online at the same time, it will then quickly redeliver it to another consumer. That way you can be sure that no message is lost, even if the workers occasionally die.

        A timeout (30 minutes by default) is enforced on consumer delivery acknowledgement. This helps detect buggy (stuck) consumers that never acknowledge deliveries. You can increase this timeout as described in Delivery Acknowledgement Timeout.

channel.basicQos(1); // 公平分配

DeliverCallback deliverCallback = (consumerTag, delivery) -> {
  String message = new String(delivery.getBody(), "UTF-8");

  System.out.println(" [x] Received '" + message + "'");
  try {
    doWork(message);
  } finally {
    System.out.println(" [x] Done");
    // 手动提交ack
    channel.basicAck(delivery.getEnvelope().getDeliveryTag(), false);
  }
};
// 关闭自动确认qck
boolean autoAck = false;
channel.basicConsume(TASK_QUEUE_NAME, autoAck, deliverCallback, consumerTag -> { });

Using this code, you can ensure that even if you terminate a worker using CTRL+C while it was processing a message, nothing is lost. Soon after the worker terminates, all unacknowledged messages are redelivered

5、Message durability（消息持久化）

        We have learned how to make sure that even if the consumer dies, the task isn't lost. But our tasks will still be lost if RabbitMQ server stops.

        When RabbitMQ quits or crashes it will forget the queues and messages unless you tell it not to. Two things are required to make sure that messages aren't lost: we need to mark both the queue and messages as durable.

主要针对于rabbitMQ服务器

• 开启队列持久化（注意：RabbitMQ不允许你用不同的参数重新定义一个已经存在的队列，任何试图这样做的程序都会返回一个错误。）

• 开启消息持久化

6、Fair dispatch（公平分发）

         For example in a situation with two workers, when all odd messages are heavy and even messages are light, one worker will be constantly busy and the other one will do hardly any work. Well, RabbitMQ doesn't know anything about that and will still dispatch messages evenly. 

        This happens because RabbitMQ just dispatches a message when the message enters the queue. It doesn't look at the number of unacknowledged messages for a consumer. It just blindly dispatches every n-th message to the n-th consumer.

        rabbitMQ只关注队列中的消息数量，不关心未确认ack的消息数量