Disruptor消息队列

一、简介

        Disruptor是一个高性能的消息队列，它通过无锁算法实现高并发、高性能以及非阻塞。

1.核心概览

a.ringbuffer

    ringbuffer 是一个大小固定的环形结构。基于cas更新producer和consumer位移实现无锁操作。

b.sequence

    sequence用于跟踪生产者和消费者在RingBuffer的位置。

c.sequencer

   管理sequence，序号生成逻辑，包括producer发布事件序号，consumer等待消费序号

d.sequence barrier

    sequence barrier用于协调生产者和消费者。生产者需要等到队列未满时候才能生产，消费者需要等到队列未空时才能消费。

e.wait strategy

    当队列为空时，消费者的处理策略，包括busy spin(忙轮训)、sleeping、yielding、blocking(基于事件唤醒)

f.event processor

  负责从RingBuffer中读取事件并交给event handler进行处理。

二、样例

a. event定义

public class LongEvent
{
    private long value;

    public void set(long value)
    {
        this.value = value;
    }

    @Override
    public String toString()
    {
        return "LongEvent{" + "value=" + value + '}';
    }
}



public class LongEventFactory implements EventFactory<LongEvent>
{
    @Override
    public LongEvent newInstance()
    {
        return new LongEvent();
    }
}

b. 消费者逻辑

public class LongEventHandler implements EventHandler<LongEvent>
{
    ThreadLocal<List<LongEvent>> batchEventsThreadLocal = new ThreadLocal<>(){
        @Override
        protected List<LongEvent>initialValue() {
            return new ArrayList<>();
        }
    };
    @Override
    public void onEvent(LongEvent event, long sequence, boolean endOfBatch)
    {
        List<LongEvent>batchEvents = batchEventsThreadLocal.get();
        batchEvents.add(event);
        if (endOfBatch) { // 批处理
           
            processBatchEvents(batchEvents);
            batchEventsThreadLocal.set(new ArrayList<>());
        }
    }
    
    private void processBatchEvents(List<LongEvent> events){
         System.out.println(events.size());
    }
}

c.装配逻辑

import com.lmax.disruptor.dsl.Disruptor;
import com.lmax.disruptor.RingBuffer;
import com.lmax.disruptor.examples.longevent.LongEvent;
import com.lmax.disruptor.util.DaemonThreadFactory;
import java.nio.ByteBuffer;

public class LongEventMain
{
    public static void main(String[] args) throws Exception
    {
        int ringBufferSize = 1024;

        Disruptor<LongEvent> disruptor =
                new Disruptor<LongEvent>(new LongEventFactory(), ringBufferSize, Executors.newFixedThreadPool(5));

        disruptor.handleEventsWith(new LongEventHandler());
        disruptor.start();


        RingBuffer<LongEvent> ringBuffer = disruptor.getRingBuffer();
       
        for (long l = 0; true; l++)
        {
            long finalL = l;
            ringBuffer.publishEvent((event, sequence, buffer) -> event.set(finalL));
           
        }
    }
}

三、系统设计

1.菱形消费结构

a.每一个消费者有自己的sequce。

b.每一层级的消费者共用一个sequenceBarrier。

   上图中c3的sequenceBarrier对应可获取位移由 c1,c2的共同消费完成位移决定，为min(c1FinOff,c2FinOff)

   而c1和c2可获取位移由P1的已提交位移且未自己消费位移决定。