flume agent组件

 

 

0 前提:

地址: http://flume.apache.org/FlumeUserGuide.html

版本: flume1.6.0

 

只摘抄对自己有用的

 

 

 

1  flume 对avro的支持:

RPC

An Avro client included in the Flume distribution can send a given file to Flume Avro source using avro RPC mechanism:

$ bin/flume-ng avro-client -H localhost -p 41414 -F /usr/logs/log.10

The above command will send the contents of /usr/logs/log.10 to to the Flume source listening on that ports.

 

 

 

 

source:  avro, thrift, jms(mq), kafka, http, custom
channel: memory, jdbc, kafka, file,
sink: hdfs, hive(不用于生产), logger, avro, thrift, null, hbase, elastrcsearch, kafka, custom
interceptor: Host, Timestamp, Static , UUID,  Regex Filtering

 

 

2