SpringBoot 连接多个Rabbitmq数据源


前言

最近工作上需要在一个项目里面使用两个RabbitMq作为数据源,记录一下大概的实现过程,加深影响。

一、单个RabbitMq数据源

一般情况下,都是依赖单个RabbitMQ作为单个数据,以SpringBoot官网实例为例,在SpringBoot项目的application.properties中配置rabbitmq的连接信息即可,实例代码如下:

package com.example.messagingrabbitmq;

import org.springframework.amqp.core.Binding;
import org.springframework.amqp.core.BindingBuilder;
import org.springframework.amqp.core.Queue;
import org.springframework.amqp.core.TopicExchange;
import org.springframework.amqp.rabbit.connection.ConnectionFactory;
import org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer;
import org.springframework.amqp.rabbit.listener.adapter.MessageListenerAdapter;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;

@SpringBootApplication
public class MessagingRabbitmqApplication {

  static final String topicExchangeName = "spring-boot-exchange";

  static final String queueName = "spring-boot";

  @Bean
  Queue queue() {
    return new Queue(queueName, false);
  }

  @Bean
  TopicExchange exchange() {
    return new TopicExchange(topicExchangeName);
  }

  @Bean
  Binding binding(Queue queue, TopicExchange exchange) {
    return BindingBuilder.bind(queue).to(exchange).with("foo.bar.#");
  }

  @RabbitListener(queues = "${spring.queue}")
  public void receiveQueueFromXalarm(Message message) {
     handlerMessage(message);
  }

  public static void main(String[] args) throws InterruptedException {
    SpringApplication.run(MessagingRabbitmqApplication.class, args).close();
  }

}

二、多个RabbitMQ数据源

从单个RabbitMQ的数据源,我们了解到SpringBoot使用约定优于配置的方式,简化了RabbitMQ开发。在spring-boot-starter-amqp中,根据application.properties中的rabbitmq连接信息,自动创建了连接信息,创建@Queue,@Exchange,@Binding。但是要在一个项目里面使用多个RabbitMQ数据源,需要手动创建其他的RabbitMQ数据源的连接工作,大概的步骤如下:

2.1 增加配置信息

spring.rabbitmq.main.host=192.168.10.223
spring.rabbitmq.main.port=5672
spring.rabbitmq.main.username=admin
spring.rabbitmq.main.password=admin

spring.rabbitmq.second.used=true
spring.rabbitmq.second.host=192.168.10.224
spring.rabbitmq.second.port=5672
spring.rabbitmq.second.username=admin
spring.rabbitmq.second.password=admin
@Configuration
public class RabbitMqConfig {

    @Value("${direct.exchange.alarm}")
    private String directExchangeAlarm;
    @Value("${queue.alarm}")
    private String queueAlarm;
    @Value("${routing.key.alarm}")
    private String routingKeyAlarm;

    @Value("${direct.exchange.alarm.second}")
    private String directExchangeAlarmSecond;
    @Value("${queue.alarm.second}")
    private String queueAlarmSecond;
    @Value("${routing.key.alarm.second}")
    private String routingKeyAlarmSecond;

    @Value("${spring.rabbitmq.main.ip}")
    private String mainHost;

    @Value("${spring.rabbitmq.main.port}")
    private int mainPort;

    @Value("${spring.rabbitmq.main.username}")
    private String mainUserName;

    @Value("${spring.rabbitmq.main.password}")
    private String mainPassword;

    @Value("${spring.rabbitmq.second.ip}")
    private String secondHost;

    @Value("${spring.rabbitmq.second.port}")
    private int secondPort;

    @Value("${spring.rabbitmq.second.username}")
    private String secondUserName;

    @Value("${spring.rabbitmq.second.password}")
    private String secondPassword;

    @Bean(name = "mainConnectionFactory")
    @Primary
    public  ConnectionFactory mainConnectionFactory(){
        return connectionFactory(mainHost, mainPort, mainUserName, mainPassword);
    }

    @Bean(name = "secondConnectionFactory")
    @ConditionalOnProperty(prefix  = "spring.rabbitmq.second", value = "used", havingValue = "true")
    public  ConnectionFactory secondConnectionFactory(){
        return connectionFactory(secondHost, secondPort, secondUserName, secondPassword);
    }

    @Bean(name = "mainFactory")
    @Primary
    public SimpleRabbitListenerContainerFactory mainFactory(SimpleRabbitListenerContainerFactoryConfigurer configurer,
                                                            @Qualifier("mainConnectionFactory") ConnectionFactory connectionFactory){
        SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
        configurer.configure(factory, connectionFactory);

        return factory;
    }

    @Bean(name = "secondFactory")
    @ConditionalOnProperty(prefix  = "spring.rabbitmq.second", value = "used", havingValue = "true")
    public SimpleRabbitListenerContainerFactory secondFactory(SimpleRabbitListenerContainerFactoryConfigurer configurer,
                                                              @Qualifier("secondConnectionFactory") ConnectionFactory connectionFactory){
        SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
        configurer.configure(factory, connectionFactory);

        return factory;
    }


    //xalarm行为分析
    @Bean
    public Queue alarmQueue() {
        return new Queue(queueAlarm);
    }
    @Bean
    DirectExchange alarmExchange() {
        return new DirectExchange(directExchangeAlarm);
    }
    @Bean
    Binding alarmBinding() {
        return BindingBuilder.bind(alarmQueue()).to(alarmExchange()).with(routingKeyAlarm);
    }


    @Bean
    @ConditionalOnProperty(prefix  = "spring.rabbitmq.second", value = "used", havingValue = "true")
    public String alarmTQueueSecond(@Qualifier("secondConnectionFactory") ConnectionFactory connectionFactory) throws IOException {
        HashMap<String, Object> map = new HashMap<>();
        map.put("x-expires", 86400000); // 当队列在指定的时间没有被访问(consume, basicGet, queueDeclare…)就会被删除,Features=Exp
        map.put("x-max-length", 100000); // 限定队列的消息的最大值长度,超过指定长度将会把最早的几条删除掉
        map.put("x-max-length-bytes", 209715200); // 限定队列最大占用的空间大小, 一般受限于内存、磁盘的大小, Features=Lim B, 设置为200M

        connectionFactory.createConnection().createChannel(false).queueDeclare(queueAlarmSecond, true, false, false, map);
        connectionFactory.createConnection().createChannel(false).queueBind(queueAlarmSecond, directExchangeAlarmSecond, routingKeyAlarmSecond);
        return  "alarmTQueueSecond";
    }

    private CachingConnectionFactory connectionFactory(String host, int port, String username, String password){
        CachingConnectionFactory connectionFactory = new CachingConnectionFactory();
        connectionFactory.setHost(host);
        connectionFactory.setPort(port);
        connectionFactory.setUsername(username);
        connectionFactory.setPassword(password);
        return connectionFactory;
    }

}

2.2 代码实现

将两个数据处理的公共部分抽取到抽象类中,具体如下所示:

public abstract class AbstractAlarmConsumer {

    @Autowired
    private XXXService xxxService;

    protected void handlerMessage(Message message) {
        try {
            String strFromRMQ = new String(message.getBody());
            
            // TODO 公共业务逻辑
			
            transformsMessage(analysisResult);

            // TODO 公共业务逻辑
        } catch (Exception e) {
            throw new ImmediateAcknowledgeAmqpException(e);
        }
    }

    protected abstract void transformsMessage(AnalysisResult analysisResult) throws ExecutionException;
}
@Component
public class XXXAlarmConsumer extends AbstractAlarmConsumer {

    @Override
    protected void transformsMessage(AnalysisResult analysisResult) throws ExecutionException {
		// TODO
    }

    @RabbitListener(queues = "${queue.xalarm}")
    public void receiveQueueFromXalarm(Message message) {
        handlerMessage(message);
    }
}
@Component
@ConditionalOnProperty(prefix  = "spring.rabbitmq.second", value = "used", havingValue = "true")
public class ThirdAiBehaviorConsumer extends AbstractAiBehaviorConsumer {
    @RabbitListener(queues = "${queue.xalarm.second}", containerFactory = "secondFactory")
    public void receiveSecondQueueFromXalarm(Message message) {
        handlerMessage(message);
       }

    @Override
    protected void transformsMessage(AnalysisResult analysisResult) throws ExecutionException {
        // TODO 
    }
}

queue

三、RabbitMQ的optional queue arguments 和 policies

3.1 optional queue agruments

queueDeclare(String queue, boolean durable, boolean exclusive, Map<String, Object> arguments);
是指该方法中设置arguments,主要用于设置单个queue,主要类型如下:

  • Auto Expire(x-expires): 当队列在指定的时间没有被访问(consume, basicGet, queueDeclare…)就会被删除,Features=Exp
  • Max Length(x-max-length): 限定队列的消息的最大值长度,超过指定长度将会把最早的几条删除掉, 类似于mongodb中的固定集合,例如保存最新的100条消息, Feature=Lim
  • Max Length Bytes(x-max-length-bytes): 限定队列最大占用的空间大小, 一般受限于内存、磁盘的大小, Features=Lim B
  • Dead letter exchange(x-dead-letter-exchange): 当队列消息长度大于最大长度、或者过期的等,将从队列中删除的消息推送到指定的交换机中去而不是丢弃掉,Features=DLX
  • Dead letter routing key(x-dead-letter-routing-key):将删除的消息推送到指定交换机的指定路由键的队列中去, Feature=DLK
  • Maximum priority(x-max-priority):优先级队列,声明队列时先定义最大优先级值(定义最大值一般不要太大),在发布消息的时候指定该消息的优先级, 优先级更高(数值更大的)的消息先被消费
  • Lazy mode(x-queue-mode=lazy): Lazy Queues: 先将消息保存到磁盘上,不放在内存中,当消费者开始消费的时候才加载到内存中

3.2 policies

主要用来设置一组queue,属于全局设置的参数,主要包括:

  • expires
  • message-ttl
  • max-length
  • max-length-bytes
    example
    应对磁盘满或rabbitmq消息严重堆积策略:
rabbitmqctl set_policy Main-policy ".*" '{"max-length-bytes":10485760000,"max-length":500000,"message-ttl":259200000}' --apply-to queues

Main-policy:策略名称
“.*”:通配符,所有队列
max-length-bytes:队列最大堆积上线(配置为10G 10485760000)
max-length:最大堆积数量(配置的为50w条)
message-ttl:消息过期时间(默认3天 259200000)
该策略配置后,消息堆积以最先到达条件为准(10G或者50w条),若到达消息堆积上限,默认会将最早的消息丢弃(若到达50w条后,队列中就只会堆积50w条,再有消息进来老的消息会被丢弃)。

  • 0
    点赞
  • 3
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值