Flume Event

event类的定义

event是一个接口.

实现了四个方法.headers是map类型,里面是键值对.
body的类型是byte[]

public interface Event {

  /**
   * Returns a map of name-value pairs describing the data stored in the body.
   */
  public Map<String, String> getHeaders();

  /**
   * Set the event headers
   * @param headers Map of headers to replace the current headers.
   */
  public void setHeaders(Map<String, String> headers);

  /**
   * Returns the raw byte array of the data contained in this event.
   */
  public byte[] getBody();

  /**
   * Sets the raw byte array of the data contained in this event.
   * @param body The data.
   */
  public void setBody(byte[] body);

}

event类的实现

public class JSONEvent implements Event {
  private Map<String, String> headers;
  private String body;
  private transient String charset = "UTF-8";

  @Override
  public Map<String, String> getHeaders() {
    return headers;
  }

  @Override
  public void setHeaders(Map<String, String> headers) {
    this.headers = headers;
  }

  @Override
  public byte[] getBody() {
    if (body != null) {
      try {
        return body.getBytes(charset);
      } catch (UnsupportedEncodingException ex) {
        throw new FlumeException(String.format("%s encoding not supported", charset), ex);
      }
    } else {
      return new byte[0];
    }

  }

  @Override
  public void setBody(byte[] body) {
    if (body != null) {
      this.body = new String(body);
    } else {
      this.body = "";
    }
  }

  public void setCharset(String charset) {
    this.charset = charset;
  }

}

event如何构建

event由 LineDeserializer类 通过readLine方法构建,其中line就是真正想要的数据.将其封装到event的body中,然后header为null. 当然,如果header中自己设置了一些键值对,这时候就有header了,可以用于后面的分流.但是默认情况下,header里面是没啥东西的. 至于header的定义,可以依据host,time之类的.也可以自定义~ 比如解析body,根据body的不同定义不同的header.

/**
   * Reads a line from a file and returns an event
   * @return Event containing parsed line
   * @throws IOException
   */
  @Override
  public Event readEvent() throws IOException {
    ensureOpen();
    String line = readLine();
    if (line == null) {
      return null;
    } else {
      return EventBuilder.withBody(line, outputCharset);
    }
  }

readline又是如何读取文件的呢? 可以看出一个char一个char读取的,遇到换行符\n就break.

private String readLine() throws IOException {
    StringBuilder sb = new StringBuilder();
    int c;
    int readChars = 0;
    while ((c = in.readChar()) != -1) {
      readChars++;

      // FIXME: support \r\n
      if (c == '\n') {
        break;
      }

      sb.append((char)c);

      if (readChars >= maxLineLength) {
        logger.warn("Line length exceeds max ({}), truncating line!",
            maxLineLength);
        break;
      }
    }

    if (readChars > 0) {
      return sb.toString();
    } else {
      return null;
    }
  }

其中EventBuilder.withBody(line, outputCharset); 最终追溯到

 /**
   * Instantiate an Event instance based on the provided body and headers.
   * If <code>headers</code> is <code>null</code>, then it is ignored.
   * @param body
   * @param headers
   * @return
   */
  public static Event withBody(byte[] body, Map<String, String> headers) {
    Event event = new SimpleEvent();

    if (body == null) {
      body = new byte[0];
    }
    event.setBody(body);

    if (headers != null) {
      event.setHeaders(new HashMap<String, String>(headers));
    }

    return event;
  }