flume_kafka联合使用

最新推荐文章于 2024-11-24 06:00:00 发布

韩利鹏

最新推荐文章于 2024-11-24 06:00:00 发布

阅读量1.3k

点赞数

分类专栏： flume kafka 文章标签： flume-kafk

本文链接：https://blog.csdn.net/HANLIPENGHANLIPENG/article/details/53327003

版权

kafka 同时被 2 个专栏收录

4 篇文章 0 订阅

订阅专栏

flume

3 篇文章 0 订阅

订阅专栏

在大数据的使用中常常将flume和kafka联合起来使用，实用的时候主要是一个配置的文件信息
配置文件信息如下

#agent section
producer.sources = s
producer.channels = c
producer.sinks = r

#source section
#producer.sources.s.type = seq
producer.sources.s.type = exec
producer.sources.s.command =tail -F /tmp/test/a.txt 
producer.sources.s.channels = c
# Each sink's type must be defined
producer.sinks.r.type = org.apache.flume.sink.kafka.KafkaSink
producer.sinks.r.kafka.bootstrap.servers = localhost:9092
producer.sinks.r.kafka.topic=test
producer.sinks.r.kafka.flumeBatchSize = 20
producer.sinks.r.kafka.producer.acks = 1
#Specify the channel the sink should use
producer.sinks.r.channel = c

# Each channel's type is defined.
producer.channels.c.type = memory
producer.channels.c.capacity = 1000