flume采用架构
exec-source + memory-channel + avro-sink
avro-source + memory-channel + kafka-sink
kafka采用架构
启动zookeeper
zkServer.sh start
启动kafka(启动一个“篮子”)
kafka-server-start.sh -daemon $KAFKA_HOME/config/server.properties &
模拟
启动一个kafka消费者监听日志文件
kafka-console-consumer.sh –zookeeper 192.168.145.128:2181 –from-beginning –topic firstTopic
向日志文件中添加数据
echo hi,flume-kafka framework >> flume-kafka.txt
exec-memory-avro.conf
flume-ng agent \
--name exec-memory-avro \
--conf $FLUME_HOME/conf \
--conf-file