Flume 源码学习（一）Source介绍

Source是Flume NG三大组件的第一个，是数据收集的来源，也是我们今天学习的主题。

类图

先来看一下Source组件类图：

从上面类图，可以清晰看出，Source接口实现了NamedComponent接口和LifecycleWare接口。而AbstractSource抽象类可以看做是适配器模式的一种应用，实现了Source接口的所有业务不相关方法。

关键接口声明

接口名	作用
NamedComponent	Enables a component to be tagged with a name so that it can be referred to uniquely within the configuration system.
LifecycleAware	An interface implemented by any class that has a defined, stateful,lifecycle.

AbstractSource

private ChannelProcessor channelProcessor;
  private String name;

  private LifecycleState lifecycleState;

  public AbstractSource() {
    lifecycleState = LifecycleState.IDLE;
  }

  @Override
  public synchronized void start() {
    Preconditions.checkState(channelProcessor != null,
        "No channel processor configured");

    lifecycleState = LifecycleState.START;
  }

  @Override
  public synchronized void stop() {
    lifecycleState = LifecycleState.STOP;
  }

  @Override
  public synchronized void setChannelProcessor(ChannelProcessor cp) {
    channelProcessor = cp;
  }

  @Override
  public synchronized ChannelProcessor getChannelProcessor() {
    return channelProcessor;
  }

  @Override
  public synchronized LifecycleState getLifecycleState() {
    return lifecycleState;
  }

  @Override
  public synchronized void setName(String name) {
    this.name = name;
  }

  @Override
  public synchronized String getName() {
    return name;
  }

  public String toString() {
      return this.getClass().getName() + "{name:" + name + ",state:" + lifecycleState +"}";
  }  

实现了最基本的LifecycyleAware接口和NamedComponentNamed定义的方法，子类只需要专注于具体业务。

ExecSource

接下来就是具体的Source实现了，我们先来看用的最多的ExecSource

public class ExecSource extends AbstractSource implements EventDrivenSource,
Configurable {
   

  private static final Logger logger = LoggerFactory
      .getLogger(ExecSource.class);

  private String shell;//exec执行的是shell
  private String command;//exec执行的是command
  private SourceCounter sourceCounter;//用于JMX查看的计数器
  private ExecutorService executor;//具体的Exec执行器
  private Future<?> runnerFuture;//Exec执行的结果Future
  private long restartThrottle;//重新执行的间隔
  private boolean restart;//判断是否命令一次执行成功后需要重新执行
  private boolean logStderr;//是否输出stderr
  private Integer bufferCount;//缓冲区数量
  private long batchTimeout;//批量flush的timeout
  private ExecRunnable runner;//具体命令的执行代码
  private Charset charset;//编码

  @Override
  public void start() {
  //重写了父类的start()方法
    logger.info("Exec source starting with command:{}", command);

    executor = Executors.newSingleThreadExecutor();//单线程的执行器

    runner = new ExecRunnable(shell, command, getChannelProcessor(), sourceCounter,
        restart, restartThrottle, logStderr, bufferCount, batchTimeout, charset);

    // FIXME: Use a callback-like executor / future to signal us upon failure.
    runnerFuture = executor.submit(runner);

    /*
     * NB: This comes at the end rather than the beginning of the method because
     * it sets our state to running. We want to make sure the executor is alive
     * and well first.
     */
    sourceCounter.start();
    super.start();//调用父类start()方法更改Source的状态为started

    logger.debug("Exec source started");
  }

  @Override
  public void stop()