RocketMQ的定时消息设计与思考

rocketMQ的定时消息与局限

本文rocketmq的版本是基于4.5.2
熟悉rocketmq的朋友都知道rocketmq有一个延迟消息机制，该机制在消息重试过程中就有使用，基本原理为：用一个内部的SCHEDULE_TOPIC_XXXX来存放所有的延迟消息，该topic里的Queue与延迟级别一一对应，也就是说有多少个延迟级别该topic就有多少个Queue。利用Timer定时任务管理来对每个Queue里的消息进行扫描，时间到了的就放回到原本的topic里去，等待消费者执行。局限性也就是只能在rocketmq设定好的延迟级别中挑选，或者自定义延迟级别。

根据延迟级别确定QueueId

// Delay Delivery，根据延迟级别确定QueueId
if (msg.getDelayTimeLevel() > 0) {
    if (msg.getDelayTimeLevel() > this.defaultMessageStore.getScheduleMessageService().getMaxDelayLevel()) {
        msg.setDelayTimeLevel(this.defaultMessageStore.getScheduleMessageService().getMaxDelayLevel());
    }

    topic = ScheduleMessageService.SCHEDULE_TOPIC;
    queueId = ScheduleMessageService.delayLevel2QueueId(msg.getDelayTimeLevel());

    // Backup real topic, queueId
    MessageAccessor.putProperty(msg, MessageConst.PROPERTY_REAL_TOPIC, msg.getTopic());
    MessageAccessor.putProperty(msg, MessageConst.PROPERTY_REAL_QUEUE_ID, String.valueOf(msg.getQueueId()));
    msg.setPropertiesString(MessageDecoder.messageProperties2String(msg.getProperties()));

    msg.setTopic(topic);
    msg.setQueueId(queueId);
}

启动Timer定时任务对每个queue进行扫描

public void start() {
    if (started.compareAndSet(false, true)) {
        this.timer = new Timer("ScheduleMessageTimerThread", true);
        for (Map.Entry<Integer, Long> entry : this.delayLevelTable.entrySet()) {
            Integer level = entry.getKey();
            Long timeDelay = entry.getValue();
            Long offset = this.offsetTable.get(level);
            if (null == offset) {
                offset = 0L;
            }

            if (timeDelay != null) {
                this.timer.schedule(new DeliverDelayedMessageTimerTask(level, offset), FIRST_DELAY_TIME);
            }
        }

        this.timer.scheduleAtFixedRate(new TimerTask() {

            @Override
            public void run() {
                try {
                    if (started.get()) ScheduleMessageService.this.persist();
                } catch (Throwable e) {
                    log.error("scheduleAtFixedRate flush exception", e);
                }
            }
        }, 10000, this.defaultMessageStore.getMessageStoreConfig().getFlushDelayOffsetInterval());
    }
}

定时消息的设计方案

在需要使用定时消息的场景下，延迟消息的级别并不能满足定时消息的需要，那么我们可不可以将当前时间与定时时间之间的差用自定义的延迟级别来表示，也就是将需要定时的时间划分成n个延迟级别，依次延迟下去，这样当延迟级别列表里的延迟级别延迟完了也就是到了定时的时间，从而达到定时消息的需要。
首先需要在Message中添加两个属性：

    /**
     * 设置定时时间
     * @param execTime
     */
    public void setExecTime(String execTime) {
        this.putProperty("TIMING", execTime);
    }

    public String getExecTime() {
        return this.getProperty("TIMING");
    }

    /**
     * 根据当前时间与定时时间计算延迟级别列表
     * @param allLevel
     */
    public void setTimingLevelList(String allLevel) {
        this.putProperty("TIMING-LIST", allLevel);
    }

    public String getTimingLevelList() {
        return this.getProperty("TIMING-LIST");
    }

我这里总共设置了32个延迟级别来表示24小时内的时间差，分别是：

1s,3s,5s,7s,9s,10s,20s,30s,40s,50s,1m,3m,5m,7m,9m,10m,20m,30m,40m,50m,1h,3h,5h,7h,9h,11h,13h,15h,17h,19h,21h,23h

修改CommitLog代码，在putMessage时依次执行延迟级别：

if (StringUtils.isNotBlank(msg.getTimingLevelList())) {
    String levelList = msg.getTimingLevelList();
    //取一个延迟级别，延迟级别是用逗号隔开的
    msg.setDelayTimeLevel(Integer.parseInt(levelList.split(",")[0]));
    //levelList减去一个响应的延迟级别
    msg.setTimingLevelList(levelList.substring(levelList.indexOf(",") + 1, levelList.length()));
}

接下来就是如何计算两个时间之间的时间差了，以及如何将时间差转换为延迟级别列表，下面的代码是计算一个给定时间点的时间差，并转换为延迟级别列表，读者可以根据自己的实际情况设定消息的开始时间和结速时间，即只在某一段时间内处理消息。代码中的ScheduleMessageService.delayTimeToLevelTable是自定义的延迟级别，根据自定义的时间片段来获取相应的delayLevel。

    private String getDelayLevelListByTime(String execTime) {
        if (StringUtils.isBlank(execTime)) return null;
        StringBuffer sb = new StringBuffer();
        int borrow = 0;
        String[] execTimeArr = execTime.split(":");
        if (execTimeArr.length != 3) return null;

        DateTime dateTime = new DateTime();
        //秒的处理
        int secondDiff = Integer.parseInt(execTimeArr[2]) - dateTime.getSecondOfMinute();
        if (secondDiff < 0) {
            borrow = 1;
            secondDiff = secondDiff + 60;
        }
        getLevelList(sb, secondDiff, "s");
        //分的处理
        int minuteDiff = Integer.parseInt(execTimeArr[1]) - borrow - dateTime.getMinuteOfHour();
        if (minuteDiff < 0) {
            borrow = 1;
            minuteDiff = minuteDiff + 60;
        } else {
            borrow = 0;
        }
        getLevelList(sb, minuteDiff, "m");
        //小时的处理
        int hourDiff = Integer.parseInt(execTimeArr[0]) - borrow - dateTime.getHourOfDay();
        if (hourDiff < 0) {
            hourDiff = hourDiff + 24;
        }
        getLevelList(sb, hourDiff, "h");

        System.out.println(hourDiff + ":" + minuteDiff + ":" + secondDiff);
        return sb.toString();
    }

    private void getLevelList(StringBuffer sb, int timeDiff, String timeUnite) {
        if (timeUnite.equals("h")) {
            if (timeDiff % 2 == 1) {
                sb.append(ScheduleMessageService.delayTimeToLevelTable.get(timeDiff + "h")).append(",");
            } else if (timeDiff != 0) {
                sb.append(ScheduleMessageService.delayTimeToLevelTable.get(timeDiff - 1 + "h")).append(",");
                sb.append(ScheduleMessageService.delayTimeToLevelTable.get(1 + "h")).append(",");
            }
        } else {
            if (timeDiff / 10 != 0) {
                sb.append(ScheduleMessageService.delayTimeToLevelTable.get(timeDiff / 10 * 10 + timeUnite)).append(",");
            }
            if (timeDiff % 10 % 2 == 1) {
                //奇数取level
                sb.append(ScheduleMessageService.delayTimeToLevelTable.get(timeDiff % 10 + timeUnite)).append(",");
            } else if (timeDiff % 10 != 0) {
                //不为0的偶数取level
                sb.append(ScheduleMessageService.delayTimeToLevelTable.get(timeDiff % 10 - 1 + timeUnite)).append(",");
                sb.append(ScheduleMessageService.delayTimeToLevelTable.get(1 + timeUnite)).append(",");
            }
        }
    }