1.使用官方的组件, 搭配一个 从 netcat source -> memory channel -> logger sink 的 demo
启动agent:
]# ./bin/flume-ng agent --conf conf --conf-file ./conf/1.flume_netcat.conf --name a1 -Dflume.root.logger=INFO,console
发数据:
]# telnet master 44444
flume_netcat.conf文件内容如下
a1.sources = r1 a1.sinks = k1 a1.channels = c1
# Describe/configure the source a1.sources.r1.type = netcat a1.sources.r1.bind = master a1.sources.r1.port = 44444
# 定义sink,输出到logger日志中 a1.sinks.k1.type = logger
# Use a channel which buffers events in memory a1.channels.c1.type = memory #存储在通道中的事件的最大数量 a1.channels.c1.capacity = 1000 #通道在每个事务中从源获取或给予接收器的事件的最大数量 a1.channels.c1.transactionCapacity = 100
# Bind the source and sink to the channel a1.sources.r1.channels = c1 a1.sinks.k1.channel = c1 |
2.使用官方的组件, 搭配一个 从 exec source -> memory channel -> logger sink 的 demo
启动agent:
]# ./bin/flume-ng agent --conf conf --conf-file ./conf/2.flume_exec.conf --name a1 -Dflume.root.logger=INFO,console
tail -f /root/Data/10.flume-lt/test_data/1.log监控文件,有新内容就采集
发数据:
]# echo 'abcabc123' >> 1.log
2.flume_exec.conf 配置文件内容:
# Name the components on this agent a1.sources = r1 a1.sinks = k1 a1.channels = c1
# Describe/configure the source a1.sources.r1.type = exec a1.sources.r1.command = tail -f /root/Data/10.flume-lt/test_data/1.log
# Describe the sink a1.sinks.k1.type = logger
# Use a channel which buffers events in memory a1.channels.c1.type = memory a1.channels.c1.capacity = 1000 a1.channels.c1.transactionCapacity = 100
# Bind the source and sink to the channel a1.sources.r1.channels = c1 a1.sinks.k1.channel = c1 |