Memory Channel
使用LinkedBlockingDeque实现,
数据基于双向队列存储,可从从两端取出,插入。
主要属性:
LinkedBlockingDeque queue: 存放Event
Semaphore queueStored : queue中已使用空间
private volatile Integer transCapacity :一个事务中Event的最大数目
Semaphore queueRemaning : queue中剩余空间
private volatile int keepAlive: 向queue中添加、移除Event的等待时间
private volatile int byteCapacity:queue中,所有Event所能占用的最大空间
private volatile int lastByteCapacity:
private volatile int byteCapacityBufferPercentage:queue中,所有Event的header所能占用的最大空间占byteCapacity的比例
private Semaphore bytesRemaining:用于标示byteCapacity中剩余空间的信号量
private ChannelCounter channelCounter:用于记录Memory Channel的一些指标,后面可以通过配置监控来观察Flume的运行情况
内部类:MemoryTransaction
继承BasicTransactionSemantics,实现doPut,doTake,doCommit,doRollback方法。
源码:
private class MemoryTransaction extends BasicTransactionSemantics {
//和MemoryChannel一样,内部使用LinkedBlockingDeque来保存没有commit的Event
private LinkedBlockingDeque<Event> takeList;
private LinkedBlockingDeque<Event> putList;
private final ChannelCounter channelCounter;
//下面两个变量用来表示put的Event的大小、take的Event的大小
private int putByteCounter = 0;
private int takeByteCounter = 0;
public MemoryTransaction(int transCapacity, ChannelCounter counter) {
//用transCapacity来初始化put、take的队列
putList = new LinkedBlockingDeque<Event>(transCapacity);
takeList = new LinkedBlockingDeque<Event>(transCapacity);
channelCounter = counter;
}
@Override
protected void doPut(Event event) throws InterruptedException {
//doPut操作,先判断putList中是否还有剩余空间,有则把Event插入到该队列中,同时更新putByteCounter
//没有剩余空间的话,直接报ChannelException
channelCounter.incrementEventPutAttemptCount();
int eventByteSize