buffer-slayer源码说明

/**
 * @author 
 * @Description buffer-slayer 的实现类
 * @date 
 */
@Slf4j
@Service
public class BufferSlayerService {
    private static final String INSERTION = "INSERT INTO test(data, time) VALUES(?, ?);";

    static String randomString() {
        return String.valueOf(ThreadLocalRandom.current().nextLong());
    }
    @Resource
    private JdbcTemplate jdbcTemplate;

    public void doBufferSlayer(){

        AsyncReporterProperties reporterProperties = new AsyncReporterProperties()
                .setFlushThreads(5)
                .setSharedSenderThreads(10)
                .setBufferedMaxMessages(500)
                .setPendingMaxMessages(10000)
                .setMetrics("inmemory")
                .setMetricsExporter("http");

        BatchJdbcTemplate template = new BatchJdbcTemplate(jdbcTemplate, reporterProperties);

        MessageFuture<Integer> future = template.update(INSERTION, randomString(), new Date());
        future.addListener(f -> {
            log.info("返回值：" + f.get());
        });
        /*future.addListener(new GenericFutureListener<Future<? super Integer>>() {
            @Override
            public void operationComplete(Future<? super Integer> future) throws Exception {
                log.info("返回值：" + future.get());
            }
        });*/
    }
}

首先调用BatchJdbcTemplate的构造方法。
之后调用update方法

public MessageFuture<Integer> update(String sql, Object... args) throws DataAccessException {
    return reporter.report(Sql.builder()
        .sql(sql)
        .args(args)
        .build());
  }

看到了reporter，是buffer-slayer的组件之一
reporter是一个接口，实现他的是AsyncReporter

public class AsyncReporter<M extends Message, R> extends TimeDriven<MessageKey> implements Reporter<M, R>, Flushable {
....
}

AsyncReporter中report方法。

@Override
  @SuppressWarnings("unchecked")
  public MessageFuture<R> report(M message) {
    checkNotNull(message, "message");
    metrics.incrementMessages(1);

    if (REPORTER_STATE_UPDATER.get(this) == REPORTER_STATE_SHUTDOWN) { // Drop the message when closed.
      MessageDroppedException cause =
          dropped(new IllegalStateException("closed!"), singletonList(message));
      MessageFuture<R> future = (MessageFuture<R>) message.newFailedFuture(cause);
      setFailListener(future);
      return future;
    }

    // Lazy initialize flush threads
    if (REPORTER_STATE_UPDATER.compareAndSet(this, REPORTER_STATE_INIT, REPORTER_STATE_STARTED) &&
        messageTimeoutNanos > 0) {
      startFlushThreads();
    }
    //队列冲的记录，如果达到限制，则阻止当前线程，直到发出信号
    memoryLimiter.waitWhenMaximum();
    // If singleKey is true, ignore original message key. 默认是false
    Message.MessageKey key = singleKey ?
        Message.SINGLE_KEY : message.asMessageKey();

    // Offer message to pending queue.  向挂起队列提供消息。
    AbstractSizeBoundedQueue pending = queueManager.getOrCreate(key);// key 即队列的名字，如没有队列则新建一个队列，有的话返回
    MessagePromise<R> promise = message.newPromise();
    pending.offer(promise);//如果元素可以添加，则通知延迟；如果由于其他原因拒绝
    setFailListener(promise);//失败的监听

    if (pending.size() >= bufferedMaxMessages)//大于一次提交的最大数量的时候
      synchronizer.offer(pending);
    return promise;
  }

getOrCreate方法是得到一个队列，key就是队列的名字。之后调用pending.offer(promise);将执行的sql语句加入到队列中。
到这里就要抛出问题了，sql语句加入对了是怎么执行的呢？（大神们都说看源码要带着问题去看。）
请看下面：
AsyncReporter的构造方法

AsyncReporter(Builder<M, R> builder) {
    this.sender = toAsyncSender(builder);
    this.metrics = builder.metrics;
    this.memoryLimiter = MemoryLimiter.maxOf(builder.totalQueuedMessages, metrics);
    this.messageTimeoutNanos = builder.messageTimeoutNanos;
    this.bufferedMaxMessages = builder.bufferedMaxMessages;
    this.singleKey = builder.singleKey;

    this.flushThreads = builder.flushThreads;
    this.timerThreads = builder.timerThreads;

    this.queueManager =
        new QueueManager(builder.pendingMaxMessages, builder.overflowStrategy,
            builder.pendingKeepaliveNanos, this,
            metrics, initHashedWheelTimer(builder));

    this.flushThreadFactory = new FlushThreadFactory(this);
    this.bufferPool = new BufferPool(MAX_BUFFER_POOL_ENTRIES,
        builder.bufferedMaxMessages, builder.singleKey);

    if (messageTimeoutNanos > 0) {
      this.queueManager.onCreate(new CreateCallback() {
        @Override
        public void call(AbstractSizeBoundedQueue queue) {
          schedulePeriodically(queue.key, messageTimeoutNanos);
        }
      });
    }
  }

这里注意构造方法中的最后一个方法，用了QueueManager 组件

接下来看看QueueManager 中onCreate中实现的方法。
这是一个抽象类中的方法

  void schedulePeriodically(final K timerKey, long intervalNanos) {
    cancelTimer(timerKey);
    long id = timerIdGen.getAndIncrement();
    ScheduledFuture<?> task = scheduler().scheduleWithFixedDelay(new Runnable() {
      @Override
      public void run() {
        onTimer(timerKey);
      }
    }, intervalNanos, intervalNanos, TimeUnit.NANOSECONDS);
    keyToTimer.put(timerKey, new Timer(id, task));
  }

这里需要注意一个类ScheduledFuture
这里使用了ScheduledExecutorService 延迟/周期执行线程池

继续回到方法，上面的方法调用onTimer，onTimer是一个抽象方法。
查看实现

可以看到AsyncReporter实现了这个抽象类中的方法

 @Override
  protected void onTimer(MessageKey timerKey) {
    AbstractSizeBoundedQueue q = queueManager.get(timerKey);
    if (q != null && q.size() > 0) flush(q);
  }

可以看到又一大组件出现：SizeBoundedQueue 。这里是取队列中值，然后刷新出去执行。

@Override
  public void flush() {
    for (AbstractSizeBoundedQueue pending : queueManager.elements()) {
      Future<?> future = flush(pending);
      future.awaitUninterruptibly();
    }
  }

  Future<?> flush(AbstractSizeBoundedQueue pending) {
    CompositeFuture result;
    Buffer buffer = bufferPool.acquire();
    try {
      int drained = pending.drainTo(buffer);
      if (drained == 0) {
        return new SucceededFuture<>(null, null);
      }

      List<MessagePromise<R>> promises = buffer.drain();
      result = sender.send(promises);
    } finally {
      bufferPool.release(buffer);
    }
    // Update metrics
    metrics.updateQueuedMessages(pending.key, pending.size());
    // Signal producers
    if (!memoryLimiter.isMaximum())
      memoryLimiter.signalAll();

    logWhenFailed(result);
    return result;
  }

这里咱们看到了又一大组件Sender -批量发送缓冲区消息。调用send方法
AsyncSenderAdaptor

@Override
  public CompositeFuture send(final List<MessagePromise<R>> promises) {
    logger.debug("Sending {} messages.", promises.size());
    final List<M> messages = extractMessages(promises);

    executor.execute(new Runnable() {
      @Override
      public void run() {
        try {
          List<R> result = delegate.send(messages);
          Promises.allSuccess(result, promises);
        } catch (Throwable t) {
          Promises.allFail(t, promises, messages);
        }
      }
    });
    return CompositeFuture.all(promises);
  }

JdbcTemplateSender

@Override
  public List<Integer> send(List<Sql> sqls) {
    Preconditions.checkArgument(sqls != null && sqls.size() > 0, "Sqls must not be empty.");
    if (!ofTheSameSql(sqls)) {
      throw new UnsupportedOperationException("Different sqls not supported");
    }

    boolean prepared = allPreparedStatement(sqls);
    int[] rowsAffected;
    if (prepared) {
      rowsAffected = underlying.batchUpdate(sqls.get(0).sql, batchPreparedStatementSetter(sqls));
    } else {
      rowsAffected = underlying.batchUpdate(extractSqls(sqls).toArray(new String[0]));
    }

    List<Integer> ret = new ArrayList<>(rowsAffected.length);
    for (int aRowsAffected : rowsAffected) {
      ret.add(aRowsAffected);
    }
    return ret;
  }

在这里执行了sql语句。

到此sql语句执行了，返回结果。

这里 AsyncReporter(Builder<M, R> builder) 中初始化 queueManager用到了哈希轮盘算法

this.queueManager =
        new QueueManager(builder.pendingMaxMessages, builder.overflowStrategy,
            builder.pendingKeepaliveNanos, this,
            metrics, initHashedWheelTimer(builder));

 private static HashedWheelTimer initHashedWheelTimer(Builder<?, ?> builder) {
    synchronized (AsyncReporter.class) {
      if (hashedWheelTimer == null) {
        hashedWheelTimer = new HashedWheelTimer(
            hashedWheelTimerThreadFactory,
            builder.tickDurationNanos, TimeUnit.NANOSECONDS,
            builder.ticksPerWheel);
      }
    }
    return hashedWheelTimer;
  }

有兴趣的同学可以研究下
欢迎捐赠