flume
青眼酷白龙
这个作者很懒,什么都没留下…
展开
-
Flume当中的事务问题
原创 2020-07-19 20:20:22 · 144 阅读 · 0 评论 -
HDFS sink参数解析文件滚动和文件夹滚动
原创 2020-07-01 09:09:58 · 382 阅读 · 0 评论 -
flume案例--实时采集文件的内容变化到hdfs
1 配置tail-hdfs.conf文件# Name the components on this agenta1.sources = r1a1.sinks = k1a1.channels = c1# Describe/configure the sourcea1.sources.r1.type = execa1.sources.r1.command = tail -F /root/logs/test.log# Describe the sinka1.sinks.k1.type = h原创 2020-06-26 22:09:37 · 346 阅读 · 0 评论 -
flume案例--监听本地网络端口 把网络端口的数据采集到 输出到console控制台上
1 在node-1 上配置vim netcat-logger.conf# Name the components on this agenta1.sources = r1a1.sinks = k1a1.channels = c1# Describe/configure the sourcea1.sources.r1.type = netcata1.sources.r1.bind = node-1a1.sources.r1.port = 44444# Describe the sink原创 2020-06-26 22:08:48 · 434 阅读 · 0 评论 -
flume案例--监控文件夹采集数据
1 配置spool-hdfs.conf文件# Name the components on this agenta1.sources = r1a1.sinks = k1a1.channels = c1# Describe/configure the source##注意:不能往监控目中重复丢同名文件a1.sources.r1.type = spooldira1.sources.r1.spoolDir = /root/logs2a1.sources.r1.fileHeader = true原创 2020-06-26 22:08:12 · 552 阅读 · 0 评论 -
flume案例--flume load balance集群搭建案例
1 分发flume到node-2 和node-3scp -r /export/servers/flume/ root@node-2://export/servers/scp -r /export/servers/flume/ root@node-3://export/servers/2 在node-1 配置 exec-avro.conf文件#agent1 nameagent1.channels = c1agent1.sources = r1agent1.sinks = k1 k2#set原创 2020-06-26 22:07:26 · 151 阅读 · 0 评论