rabbitMQ实战应用
-
异步任务处理:在Java项目中,可以将一些耗时的任务(如文件处理、邮件发送等)放入消息队列中,由后台的消费者异步地处理这些任务,提高系统的吞吐量和响应速度。
// 生产者 public class TaskProducer { private final static String QUEUE_NAME = "task_queue"; public static void main(String[] argv) throws Exception { ConnectionFactory factory = new ConnectionFactory(); factory.setHost("localhost"); try (Connection connection = factory.newConnection(); Channel channel = connection.createChannel()) { channel.queueDeclare(QUEUE_NAME, true, false, false, null); String message = "hello"; channel.basicPublish("", QUEUE_NAME, MessageProperties.PERSISTENT_TEXT_PLAIN, message.getBytes("UTF-8")); System.out.println(" [x] Sent '" + message + "'"); } } } // 消费者 public class TaskConsumer { private final static String QUEUE_NAME = "task_queue"; public static void main(String[] argv) throws Exception { ConnectionFactory factory = new ConnectionFactory(); factory.setHost("localhost"); try (Connection connection = factory.newConnection(); Channel channel = connection.createChannel()) { channel.queueDeclare(QUEUE_NAME, true, false, false, null); System.out.println(" [*] Waiting for messages. To exit press CTRL+C"); DeliverCallback deliverCallback = (consumerTag, delivery) -> { String message = new String(delivery.getBody(), "UTF-8"); System.out.println(" [x] Received '" + message + "'"); try { doWork(message); } finally { System.out.println(" [x] Done"); channel.basicAck(delivery.getEnvelope().getDeliveryTag(), false); } }; channel.basicConsume(QUEUE_NAME, false, deliverCallback, consumerTag -> { }); } } private static void doWork(String task) { try { Thread.sleep(1000); // 模拟耗时任务 } catch (InterruptedException _ignored) { Thread.currentThread().interrupt(); } } }
-
事件驱动架构:通过RabbitMQ实现事件驱动架构,不同模块之间通过消息进行通信,从而实现松耦合、高内聚的系统架构。
// 生产者 public class EventProducer { private final static String EXCHANGE_NAME = "events"; public static void main(String[] argv) throws Exception { ConnectionFactory factory = new ConnectionFactory(); factory.setHost("localhost"); try (Connection connection = factory.newConnection(); Channel channel = connection.createChannel()) { channel.exchangeDeclare(EXCHANGE_NAME, BuiltinExchangeType.FANOUT); String message = "event message"; channel.basicPublish(EXCHANGE_NAME, "", null, message.getBytes("UTF-8")); System.out.println(" [x] Sent '" + message + "'"); } } } // 消费者 public class EventConsumer { private final static String EXCHANGE_NAME = "events"; public static void main(String[] argv) throws Exception { ConnectionFactory factory = new ConnectionFactory(); factory.setHost("localhost"); try (Connection connection = factory.newConnection(); Channel channel = connection.createChannel()) { channel.exchangeDeclare(EXCHANGE_NAME, BuiltinExchangeType.FANOUT); String queueName = channel.queueDeclare().getQueue(); channel.queueBind(queueName, EXCHANGE_NAME, ""); System.out.println(" [*] Waiting for events. To exit press CTRL+C"); DeliverCallback deliverCallback = (consumerTag, delivery) -> { String message = new String(delivery.getBody(), "UTF-8"); System.out.println(" [x] Received '" + message + "'"); }; channel.basicConsume(queueName, true, deliverCallback, consumerTag -> { }); } } }
-
分布式系统协调:在分布式系统中,RabbitMQ可以用作协调服务,实现分布式锁、分布式事务等功能,确保系统的一致性和可靠性。
import com.rabbitmq.client.*; import java.io.IOException; import java.util.concurrent.TimeoutException; public class Coordinator { private final static String TASK_QUEUE_NAME = "task_queue"; public static void main(String[] args) throws IOException, TimeoutException { ConnectionFactory factory = new ConnectionFactory(); factory.setHost("localhost"); try (Connection connection = factory.newConnection(); Channel channel = connection.createChannel()) { channel.queueDeclare(TASK_QUEUE_NAME, true, false, false, null); System.out.println(" [*] Waiting for messages. To exit press CTRL+C"); // 设置同时最大接收未确认消息数量 int prefetchCount = 1; channel.basicQos(prefetchCount); DeliverCallback deliverCallback = (consumerTag, delivery) -> { String message = new String(delivery.getBody(), "UTF-8"); System.out.println(" [x] Received '" + message + "'"); try { // 模拟处理任务 doTask(message); } finally { System.out.println(" [x] Done"); // 手动发送消息确认 channel.basicAck(delivery.getEnvelope().getDeliveryTag(), false); } }; // 监听队列并消费消息 boolean autoAck = false; // 关闭自动确认模式 channel.basicConsume(TASK_QUEUE_NAME, autoAck, deliverCallback, consumerTag -> { }); } } private static void doTask(String task) { // 模拟任务处理 try { Thread.sleep(1000); // 模拟耗时任务 } catch (InterruptedException e) { Thread.currentThread().interrupt(); } } }
在这个示例中,
Coordinator
类代表了一个分布式系统的协调者,它监听名为task_queue
的队列,并且当收到消息时会模拟处理任务。其他系统的组件可以向这个队列发送消息,以请求协调和执行任务。 -
日志收集与分析:将系统产生的日志通过消息队列发送到日志收集系统中进行集中管理和分析,便于监控系统运行状态和故障排查。
要使用RabbitMQ实现日志收集与分析,可以设置一个日志发布者,将日志消息发送到RabbitMQ队列,然后有一个或多个日志消费者从队列中接收日志消息并进行相应的分析和处理。下面是一个简单的Java代码示例:
import com.rabbitmq.client.*; import java.io.IOException; import java.util.concurrent.TimeoutException; public class LogProducer { private final static String EXCHANGE_NAME = "logs"; public static void main(String[] args) throws IOException, TimeoutException { ConnectionFactory factory = new ConnectionFactory(); factory.setHost("localhost"); try (Connection connection = factory.newConnection(); Channel channel = connection.createChannel()) { channel.exchangeDeclare(EXCHANGE_NAME, BuiltinExchangeType.FANOUT); // 模拟日志消息 String message = "Log message"; // 发布日志消息到交换机 channel.basicPublish(EXCHANGE_NAME, "", null, message.getBytes("UTF-8")); System.out.println(" [x] Sent '" + message + "'"); } } }
在这个示例中,
LogProducer
类代表日志发布者,它将日志消息发送到名为logs
的交换机中。日志消费者可以绑定到这个交换机上来接收日志消息。下面是一个日志消费者的示例代码:import com.rabbitmq.client.*; import java.io.IOException; import java.util.concurrent.TimeoutException; public class LogConsumer { private final static String EXCHANGE_NAME = "logs"; public static void main(String[] args) throws IOException, TimeoutException { ConnectionFactory factory = new ConnectionFactory(); factory.setHost("localhost"); try (Connection connection = factory.newConnection(); Channel channel = connection.createChannel()) { channel.exchangeDeclare(EXCHANGE_NAME, BuiltinExchangeType.FANOUT); // 声明一个非持久化的、唯一的队列 String queueName = channel.queueDeclare().getQueue(); // 将队列绑定到交换机 channel.queueBind(queueName, EXCHANGE_NAME, ""); System.out.println(" [*] Waiting for logs. To exit press CTRL+C"); // 设置消息的消费者回调函数 DeliverCallback deliverCallback = (consumerTag, delivery) -> { String message = new String(delivery.getBody(), "UTF-8"); System.out.println(" [x] Received '" + message + "'"); }; // 开始消费消息 channel.basicConsume(queueName, true, deliverCallback, consumerTag -> { }); } } }
在这个示例中,
LogConsumer
类代表日志消费者,它创建了一个临时队列,并将它绑定到logs
交换机上。当日志发布者发布日志消息时,该消息将通过交换机广播到所有绑定的队列中,而这个日志消费者就会接收到这些消息并输出到控制台。 -
实时数据处理:通过RabbitMQ实现实时数据的收集、传输和处理,例如在大数据处理、实时监控等场景中应用广泛。
要使用RabbitMQ实现实时数据处理,可以设置一个数据发布者,将实时数据发送到RabbitMQ队列,然后有一个或多个数据消费者从队列中接收数据并进行实时处理。下面是一个简单的Java代码示例:
import com.rabbitmq.client.*; import java.io.IOException; import java.util.concurrent.TimeoutException; public class DataProducer { private final static String QUEUE_NAME = "data_queue"; public static void main(String[] args) throws IOException, TimeoutException { ConnectionFactory factory = new ConnectionFactory(); factory.setHost("localhost"); try (Connection connection = factory.newConnection(); Channel channel = connection.createChannel()) { channel.queueDeclare(QUEUE_NAME, false, false, false, null); while (true) { // 模拟产生实时数据 String data = generateData(); // 发布实时数据到队列 channel.basicPublish("", QUEUE_NAME, null, data.getBytes("UTF-8")); System.out.println(" [x] Sent '" + data + "'"); // 模拟数据产生间隔 Thread.sleep(1000); } } catch (InterruptedException e) { Thread.currentThread().interrupt(); } } private static String generateData() { // 模拟产生实时数据的逻辑 return "Real-time data"; } }
DataProducer
类代表数据发布者,它将实时数据发送到名为data_queue
的队列中。数据消费者可以从这个队列中接收实时数据并进行相应的处理。下面是一个数据消费者的示例代码:import com.rabbitmq.client.*; import java.io.IOException; import java.util.concurrent.TimeoutException; public class DataConsumer { private final static String QUEUE_NAME = "data_queue"; public static void main(String[] args) throws IOException, TimeoutException { ConnectionFactory factory = new ConnectionFactory(); factory.setHost("localhost"); try (Connection connection = factory.newConnection(); Channel channel = connection.createChannel()) { channel.queueDeclare(QUEUE_NAME, false, false, false, null); System.out.println(" [*] Waiting for messages. To exit press CTRL+C"); // 设置消息的消费者回调函数 DeliverCallback deliverCallback = (consumerTag, delivery) -> { String message = new String(delivery.getBody(), "UTF-8"); System.out.println(" [x] Received '" + message + "'"); // 模拟实时数据处理 processData(message); }; // 开始消费消息 channel.basicConsume(QUEUE_NAME, true, deliverCallback, consumerTag -> { }); } } private static void processData(String data) { // 模拟实时数据处理的逻辑 System.out.println(" [x] Processed '" + data + "'"); } }
-
应用解耦:将不同模块之间的通信通过消息队列实现解耦,提高系统的灵活性和可维护性。
首先,我们创建一个消息生产者(Producer),负责向队列发送消息:
import com.rabbitmq.client.ConnectionFactory; import com.rabbitmq.client.Connection; import com.rabbitmq.client.Channel; import java.io.IOException; import java.util.concurrent.TimeoutException; public class Producer { private final static String QUEUE_NAME = "hello"; public static void main(String[] argv) throws IOException, TimeoutException { ConnectionFactory factory = new ConnectionFactory(); factory.setHost("localhost"); try (Connection connection = factory.newConnection(); Channel channel = connection.createChannel()) { channel.queueDeclare(QUEUE_NAME, false, false, false, null); String message = "Hello World!"; channel.basicPublish("", QUEUE_NAME, null, message.getBytes()); System.out.println(" [x] Sent '" + message + "'"); } } }
然后,我们创建一个消息消费者(Consumer),负责从队列接收消息并进行处理:
import com.rabbitmq.client.*; import java.io.IOException; import java.util.concurrent.TimeoutException; public class Consumer { private final static String QUEUE_NAME = "hello"; public static void main(String[] argv) throws IOException, TimeoutException { ConnectionFactory factory = new ConnectionFactory(); factory.setHost("localhost"); try (Connection connection = factory.newConnection(); Channel channel = connection.createChannel()) { channel.queueDeclare(QUEUE_NAME, false, false, false, null); System.out.println(" [*] Waiting for messages. To exit press CTRL+C"); DeliverCallback deliverCallback = (consumerTag, delivery) -> { String message = new String(delivery.getBody(), "UTF-8"); System.out.println(" [x] Received '" + message + "'"); }; channel.basicConsume(QUEUE_NAME, true, deliverCallback, consumerTag -> { }); } } }
Producer 将消息发送到队列,而 Consumer 则从队列中接收消息并进行处理。两者之间通过 RabbitMQ 实现解耦,彼此不直接耦合。
-
流量削峰:通过消息队列实现流量削峰,将请求缓存到消息队列中,然后由消费者按照一定的速率处理,避免系统因突发高峰而崩溃。
首先,我们创建一个消息生产者(Producer),负责向队列发送请求消息:
import com.rabbitmq.client.ConnectionFactory; import com.rabbitmq.client.Connection; import com.rabbitmq.client.Channel; import java.io.IOException; import java.util.concurrent.TimeoutException; public class Producer { private final static String QUEUE_NAME = "requests"; public static void main(String[] argv) throws IOException, TimeoutException { ConnectionFactory factory = new ConnectionFactory(); factory.setHost("localhost"); try (Connection connection = factory.newConnection(); Channel channel = connection.createChannel()) { channel.queueDeclare(QUEUE_NAME, false, false, false, null); // 模拟发送请求消息 for (int i = 1; i <= 100; i++) { String message = "Request " + i; channel.basicPublish("", QUEUE_NAME, null, message.getBytes()); System.out.println(" [x] Sent '" + message + "'"); } } } }
然后,我们创建多个消息消费者(Consumer),负责从队列接收请求消息并进行处理。这里我们创建两个消费者以模拟多个处理节点:
import com.rabbitmq.client.*; import java.io.IOException; import java.util.concurrent.TimeoutException; public class Consumer { private final static String QUEUE_NAME = "requests"; public static void main(String[] argv) throws IOException, TimeoutException { ConnectionFactory factory = new ConnectionFactory(); factory.setHost("localhost"); try { final Connection connection = factory.newConnection(); for (int i = 1; i <= 2; i++) { final int consumerNumber = i; new Thread(() -> { try { final Channel channel = connection.createChannel(); channel.queueDeclare(QUEUE_NAME, false, false, false, null); System.out.println(" [*] Consumer " + consumerNumber + " waiting for messages. To exit press CTRL+C"); DeliverCallback deliverCallback = (consumerTag, delivery) -> { String message = new String(delivery.getBody(), "UTF-8"); System.out.println(" [x] Consumer " + consumerNumber + " Received '" + message + "'"); // 模拟处理请求消息 try { Thread.sleep(1000); // 模拟处理请求的耗时 } catch (InterruptedException e) { Thread.currentThread().interrupt(); } finally { System.out.println(" [x] Consumer " + consumerNumber + " Done"); channel.basicAck(delivery.getEnvelope().getDeliveryTag(), false); } }; channel.basicConsume(QUEUE_NAME, false, deliverCallback, consumerTag -> { }); } catch (IOException e) { e.printStackTrace(); } }).start(); } } catch (IOException e) { e.printStackTrace(); } } }
在这个示例中,我们创建了两个消费者,并且每个消费者会从队列中接收请求消息,并模拟处理请求的过程。通过多个消费者并行处理请求,可以达到流量削峰的效果。
-
微服务通信:在微服务架构中,通过RabbitMQ实现微服务之间的通信,例如服务调用、事件通知等。
首先,我们创建一个消息生产者(Producer),负责向队列发送消息:
import com.rabbitmq.client.ConnectionFactory; import com.rabbitmq.client.Connection; import com.rabbitmq.client.Channel; import java.io.IOException; import java.util.concurrent.TimeoutException; public class Producer { private final static String QUEUE_NAME = "microservice_queue"; public static void main(String[] argv) throws IOException, TimeoutException { ConnectionFactory factory = new ConnectionFactory(); factory.setHost("localhost"); try (Connection connection = factory.newConnection(); Channel channel = connection.createChannel()) { channel.queueDeclare(QUEUE_NAME, false, false, false, null); String message = "Hello Microservice!"; channel.basicPublish("", QUEUE_NAME, null, message.getBytes()); System.out.println(" [x] Sent '" + message + "'"); } } }
然后,我们创建一个消息消费者(Consumer),负责从队列接收消息并进行处理:
import com.rabbitmq.client.*; import java.io.IOException; import java.util.concurrent.TimeoutException; public class Consumer { private final static String QUEUE_NAME = "microservice_queue"; public static void main(String[] argv) throws IOException, TimeoutException { ConnectionFactory factory = new ConnectionFactory(); factory.setHost("localhost"); try (Connection connection = factory.newConnection(); Channel channel = connection.createChannel()) { channel.queueDeclare(QUEUE_NAME, false, false, false, null); System.out.println(" [*] Waiting for messages. To exit press CTRL+C"); DeliverCallback deliverCallback = (consumerTag, delivery) -> { String message = new String(delivery.getBody(), "UTF-8"); System.out.println(" [x] Received '" + message + "'"); }; channel.basicConsume(QUEUE_NAME, true, deliverCallback, consumerTag -> { }); } } }
创建一个简单的生产者和消费者,它们通过RabbitMQ进行消息的发送和接收,实现了微服务间的通信。