RabbitMQ（三）：工作队列

最新推荐文章于 2024-04-07 14:12:25 发布

saytime

最新推荐文章于 2024-04-07 14:12:25 发布

阅读量851

点赞数 1

分类专栏： RabbitMQ RabbitMQ 文章标签： RabbitMQ

本文链接：https://blog.csdn.net/saytime/article/details/80541323

版权

RabbitMQ 同时被 2 个专栏收录

9 篇文章 2 订阅

订阅专栏

RabbitMQ

9 篇文章 11 订阅

订阅专栏

一、工作队列

官方文档：http://www.rabbitmq.com/tutorials/tutorial-two-java.html

简单队列不足：不支持多个消费者

即一个生产者可以对应多个消费者同时消费，相比简单队列支持多消费者。因为实际工作中，生产者服务一般都是很简单的业务逻辑处理之后就发送到队列，消费者接收到队列的消息之后，进行复杂的业务逻辑处理，所以一般都是多个消费者进行处理。如果是一个消费者进行处理，那么队列会积压很多消息。

工作队列分两种情况：

轮询分发

不管消费者处理速度性能快慢，每个消费者都是按顺序分别每个拿一个的原则，比如3个消费者, 消费者1拿1个，然后消费者2拿一个，然后消费者3拿一个，然后消费者1开始拿，即使中间有消费者已经处理完了，也必须等待其他消费者都拿完一个才能消费到。
公平分发

根据消费者处理性能，性能好的消费的数据量多，性能差的消费的数据量少。

如上所示，如果配置有用户名密码以及vhost，则配置即可。

二、轮询分发

一个生产者，两个消费者，其中消费者处理只要1s，消费者2 处理需要2s

连接RabbitMQ工具类

package cn.saytime.rabbitmq.util;

import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;

import java.io.IOException;
import java.util.concurrent.TimeoutException;

/**
 * RabbitMQ连接工具类
 */
public class ConnectionUtil {

    private static final String host = "192.168.239.128";
    private static final int port = 5672;

    /**
     * 获取RabbitMQ Connection连接
     * @return
     * @throws IOException
     * @throws TimeoutException
     */
    public static Connection getConnection() throws IOException, TimeoutException {
        ConnectionFactory connectionFactory = new ConnectionFactory();
        connectionFactory.setHost(host);
        connectionFactory.setPort(port);

//        connectionFactory.setUsername("test");
//        connectionFactory.setPassword("123456");
//        connectionFactory.setVirtualHost("/vhost_test");

        return connectionFactory.newConnection();
    }
}

生产者

package cn.saytime.rabbitmq.work;

import cn.saytime.rabbitmq.util.ConnectionUtil;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;

import java.io.IOException;
import java.util.concurrent.TimeoutException;

/**
 * 生产者
 */
public class Send {

    private static final String QUEUE_NAME = "test_work_queue";

    public static void main(String[] args) throws IOException, TimeoutException {
        // 获取连接
        Connection connection = ConnectionUtil.getConnection();
        // 从连接开一个通道
        Channel channel = connection.createChannel();
        // 申明这个通道连接的队列
        channel.queueDeclare(QUEUE_NAME, false, false, false, null);

        for (int i = 0; i < 20; i++) {
            String message = "Hello RabbitMQ " + i;
            // 发送消息
            channel.basicPublish("", QUEUE_NAME, null, message.getBytes());
            System.out.println(" [x] Sent '" + message + "'");
        }

        // 关闭通道和连接
        channel.close();
        connection.close();
    }

}

消费者1

package cn.saytime.rabbitmq.work;

import cn.saytime.rabbitmq.util.ConnectionUtil;
import com.rabbitmq.client.*;

import java.io.IOException;
import java.util.concurrent.TimeoutException;

/**
 * 消费者
 */
public class Recv {

    private static final String QUEUE_NAME = "test_work_queue";

    public static void main(String[] args) throws IOException, TimeoutException {

        // 获取连接
        Connection connection = ConnectionUtil.getConnection();

        // 打开通道
        Channel channel = connection.createChannel();

        // 申明要消费的队列
        channel.queueDeclare(QUEUE_NAME, false, false, false, null);

        // 创建一个回调的消费者处理类
        Consumer consumer = new DefaultConsumer(channel) {
            @Override
            public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException {
                // 接收到的消息
                String message = new String(body);
                System.out.println(" [1] Received '" + message + "'");

                try {
                    Thread.sleep(1000);
                } catch (InterruptedException e) {
                    e.printStackTrace();
                } finally {
                    System.out.println(" [1] done ");
                }
            }
        };

        // 消费消息
        channel.basicConsume(QUEUE_NAME, true, consumer);

    }

}

消费者2

package cn.saytime.rabbitmq.work;

import cn.saytime.rabbitmq.util.ConnectionUtil;
import com.rabbitmq.client.*;

import java.io.IOException;
import java.util.concurrent.TimeoutException;

/**
 * 消费者
 */
public class Recv2 {

    private static final String QUEUE_NAME = "test_work_queue";

    public static void main(String[] args) throws IOException, TimeoutException {

        // 获取连接
        Connection connection = ConnectionUtil.getConnection();

        // 打开通道
        Channel channel = connection.createChannel();

        // 申明要消费的队列
        channel.queueDeclare(QUEUE_NAME, false, false, false, null);

        // 创建一个回调的消费者处理类
        Consumer consumer = new DefaultConsumer(channel) {
            @Override
            public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException {
                // 接收到的消息
                String message = new String(body);
                System.out.println(" [2] Received '" + message + "'");

                try {
                    Thread.sleep(2000);
                } catch (InterruptedException e) {
                    e.printStackTrace();
                } finally {
                    System.out.println(" [2] done ");
                }
            }
        };

        // 消费消息
        channel.basicConsume(QUEUE_NAME, true, consumer);

    }

}

测试

先启动两个消费者Recv、Recv2
再启动生产者Send

生产者控制台：

 [x] Sent 'Hello RabbitMQ 0'
 [x] Sent 'Hello RabbitMQ 1'
 [x] Sent 'Hello RabbitMQ 2'
 [x] Sent 'Hello RabbitMQ 3'
 [x] Sent 'Hello RabbitMQ 4'
 [x] Sent 'Hello RabbitMQ 5'
 [x] Sent 'Hello RabbitMQ 6'
 [x] Sent 'Hello RabbitMQ 7'
 [x] Sent 'Hello RabbitMQ 8'
 [x] Sent 'Hello RabbitMQ 9'
 [x] Sent 'Hello RabbitMQ 10'
 [x] Sent 'Hello RabbitMQ 11'
 [x] Sent 'Hello RabbitMQ 12'
 [x] Sent 'Hello RabbitMQ 13'
 [x] Sent 'Hello RabbitMQ 14'
 [x] Sent 'Hello RabbitMQ 15'
 [x] Sent 'Hello RabbitMQ 16'
 [x] Sent 'Hello RabbitMQ 17'
 [x] Sent 'Hello RabbitMQ 18'
 [x] Sent 'Hello RabbitMQ 19'

Process finished with exit code 0

消费者1：

 [1] Received 'Hello RabbitMQ 0'
 [1] done 
 [1] Received 'Hello RabbitMQ 2'
 [1] done 
 [1] Received 'Hello RabbitMQ 4'
 [1] done 
 [1] Received 'Hello RabbitMQ 6'
 [1] done 
 [1] Received 'Hello RabbitMQ 8'
 [1] done 
 [1] Received 'Hello RabbitMQ 10'
 [1] done 
 [1] Received 'Hello RabbitMQ 12'
 [1] done 
 [1] Received 'Hello RabbitMQ 14'
 [1] done 
 [1] Received 'Hello RabbitMQ 16'
 [1] done 
 [1] Received 'Hello RabbitMQ 18'
 [1] done

消费者2：

 [2] Received 'Hello RabbitMQ 1'
 [2] done 
 [2] Received 'Hello RabbitMQ 3'
 [2] done 
 [2] Received 'Hello RabbitMQ 5'
 [2] done 
 [2] Received 'Hello RabbitMQ 7'
 [2] done 
 [2] Received 'Hello RabbitMQ 9'
 [2] done 
 [2] Received 'Hello RabbitMQ 11'
 [2] done 
 [2] Received 'Hello RabbitMQ 13'
 [2] done 
 [2] Received 'Hello RabbitMQ 15'
 [2] done 
 [2] Received 'Hello RabbitMQ 17'
 [2] done 
 [2] Received 'Hello RabbitMQ 19'
 [2] done

可以发现消费者1 消费的数字全是整数，消费者2消费的全是奇数。

那么如果我事先启动三个消费者了，那么结果如何了？

因为20/3 除不了，如果消费者1,2,3按顺序启动，那么消费者1, 2会消费7条数据，消费者3消费6条，其中

消费者1 消费 0, 3, 6, 9, 12, 15, 18
消费者2 消费 1, 4, 7, 10, 13, 16, 19
消费者3 消费 2, 5, 8, 11, 14, 17

注意点：

如果生产者一次性发送完消息之后，再依次启动消费者1, 2, 3，之后只有消费者1 能消费到数据，消费者都启动之后，再生产的消息就会轮询分发到消费者1, 2, 3

三、公平分发

官方示例图：

因为生产者发送消息到队列之后，队列不知道消费者有没有处理完，所以多个消费者同时订阅同一个Queue中的消息，Queue中的消息会被平摊给多个消费者。为了实现公平分发，我们需要告诉队列，每次发一个给我，然后我反馈给你我有没有处理完，处理完了你再发一条给我。

在默认轮询分发的基础上，要实现公平分发，需要两点：

限制发送给同一个消费者不得超过1条消息，在这个消费者确认消息之前，不会发送下一条消息给这个消费者

int prefetchCount = 1;
channel.basicQos(prefetchCount);

默认自动应答改成手动应答


关闭自动应答
boolean autoAck = false;
channel.basicConsume(QUEUE_NAME, autoAck, consumer);

手动应答
channel.basicAck(envelope.getDeliveryTag(), false);

DeliveryTag 用来标识信道中投递的消息， RabbitMQ 推送消息给 Consumer 时，会附带一个 Delivery Tag，以便 Consumer 可以在消息确认时告诉 RabbitMQ 到底是哪条消息被确认了。

生产者

package cn.saytime.rabbitmq.workfair;

import cn.saytime.rabbitmq.util.ConnectionUtil;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;

import java.io.IOException;
import java.util.concurrent.TimeoutException;

/**
 * 生产者
 */
public class Send {

    private static final String QUEUE_NAME = "test_workfair_queue";

    public static void main(String[] args) throws IOException, TimeoutException {
        // 获取连接
        Connection connection = ConnectionUtil.getConnection();
        // 从连接开一个通道
        Channel channel = connection.createChannel();
        // 申明这个通道连接的队列
        channel.queueDeclare(QUEUE_NAME, false, false, false, null);

        channel.basicQos(1);

        for (int i = 0; i < 20; i++) {
            String message = "Hello RabbitMQ " + i;
            // 发送消息
            channel.basicPublish("", QUEUE_NAME, null, message.getBytes());
            System.out.println(" [x] Sent '" + message + "'");
            try {
                Thread.sleep(i*100);
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        }

        // 关闭通道和连接
        channel.close();
        connection.close();
    }

}

消费者1

package cn.saytime.rabbitmq.workfair;

import cn.saytime.rabbitmq.util.ConnectionUtil;
import com.rabbitmq.client.*;

import java.io.IOException;
import java.util.concurrent.TimeoutException;

/**
 * 消费者
 */
public class Recv {

    private static final String QUEUE_NAME = "test_workfair_queue";

    public static void main(String[] args) throws IOException, TimeoutException {

        // 获取连接
        Connection connection = ConnectionUtil.getConnection();

        // 打开通道
        final Channel channel = connection.createChannel();

        // 申明要消费的队列
        channel.queueDeclare(QUEUE_NAME, false, false, false, null);

        // 这样RabbitMQ就会使得每个Consumer在同一个时间点最多处理一个Message。换句话说，在接收到该Consumer的ack前，他它不会将新的Message分发给它。
        channel.basicQos(1);

        // 创建一个回调的消费者处理类
        Consumer consumer = new DefaultConsumer(channel) {
            @Override
            public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException {
                // 接收到的消息
                String message = new String(body);
                System.out.println(" [1] Received '" + message + "'");

                try {
                    Thread.sleep(1000);
                } catch (InterruptedException e) {
                    e.printStackTrace();
                } finally {
                    System.out.println(" [1] done ");
                    channel.basicAck(envelope.getDeliveryTag(), false);
                }
            }
        };

        // 消费消息
        channel.basicConsume(QUEUE_NAME, false, consumer);

    }

}

消费者2

package cn.saytime.rabbitmq.workfair;

import cn.saytime.rabbitmq.util.ConnectionUtil;
import com.rabbitmq.client.*;

import java.io.IOException;
import java.util.concurrent.TimeoutException;

/**
 * 消费者
 */
public class Recv2 {

    private static final String QUEUE_NAME = "test_workfair_queue";

    public static void main(String[] args) throws IOException, TimeoutException {

        // 获取连接
        Connection connection = ConnectionUtil.getConnection();

        // 打开通道
        Channel channel = connection.createChannel();

        // 申明要消费的队列
        channel.queueDeclare(QUEUE_NAME, false, false, false, null);

        // 这样RabbitMQ就会使得每个Consumer在同一个时间点最多处理一个Message。换句话说，在接收到该Consumer的ack前，他它不会将新的Message分发给它。
        channel.basicQos(1);

        // 创建一个回调的消费者处理类
        Consumer consumer = new DefaultConsumer(channel) {
            @Override
            public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException {
                // 接收到的消息
                String message = new String(body);
                System.out.println(" [2] Received '" + message + "'");

                try {
                    Thread.sleep(2000);
                } catch (InterruptedException e) {
                    e.printStackTrace();
                } finally {
                    System.out.println(" [2] done ");
                    channel.basicAck(envelope.getDeliveryTag(), false);
                }
            }
        };

        // 消费消息
        channel.basicConsume(QUEUE_NAME, false, consumer);

    }

}

测试

消费者1:

 [1] Received 'Hello RabbitMQ 0'
 [1] done 
 [1] Received 'Hello RabbitMQ 2'
 [1] done 
 [1] Received 'Hello RabbitMQ 4'
 [1] done 
 [1] Received 'Hello RabbitMQ 5'
 [1] done 
 [1] Received 'Hello RabbitMQ 7'
 [1] done 
 [1] Received 'Hello RabbitMQ 8'
 [1] done 
 [1] Received 'Hello RabbitMQ 10'
 [1] done 
 [1] Received 'Hello RabbitMQ 11'
 [1] done 
 [1] Received 'Hello RabbitMQ 13'
 [1] done 
 [1] Received 'Hello RabbitMQ 14'
 [1] done 
 [1] Received 'Hello RabbitMQ 15'
 [1] done 
 [1] Received 'Hello RabbitMQ 17'
 [1] done 
 [1] Received 'Hello RabbitMQ 19'
 [1] done

消费者2:

 [2] Received 'Hello RabbitMQ 1'
 [2] done 
 [2] Received 'Hello RabbitMQ 3'
 [2] done 
 [2] Received 'Hello RabbitMQ 6'
 [2] done 
 [2] Received 'Hello RabbitMQ 9'
 [2] done 
 [2] Received 'Hello RabbitMQ 12'
 [2] done 
 [2] Received 'Hello RabbitMQ 16'
 [2] done 
 [2] Received 'Hello RabbitMQ 18'
 [2] done

显然消费者1处理速度只要1s，所以消费的记录数要比消费者2要多很多。表示确实是公平分发。

注意点：

当关闭自动应答autoAck=false之后，在消费者处理消费数据之后一定要对消息进行手动反馈处理，可以是basicAck，也可以是basicNack， basicReject


BasicReject一次只能拒绝接收一个消息，而BasicNack方法可以支持一次0个或多个消息的拒收，并且也可以设置是否requeue。


// 拒绝当前消息，并使这条消息重新返回到队列中
channel.basicNack(envelope.getDeliveryTag(), false, true);
相当于
channel.basicReject(envelope.getDeliveryTag(), true);