flume几种使用配置 avro spooling exec thrift

最新推荐文章于 2022-08-09 10:55:58 发布

JAVA日常笔记精选

最新推荐文章于 2022-08-09 10:55:58 发布

阅读量350

点赞数 1

分类专栏： flume

本文链接：https://blog.csdn.net/qq_29082603/article/details/103959952

版权

这篇博客详细介绍了Apache Flume的四种使用配置：Avro，Spooling，Exec和Thrift。通过这些配置，Flume能够高效地收集、聚合和移动大量日志数据。

摘要由CSDN通过智能技术生成

参考官网：

http://flume.apache.org/releases/content/1.9.0/FlumeUserGuide.html

# avro source
a1.sources = r1
a1.channels = c1
a1.sinks = k1
a1.sources.r1.type = avro
a1.sources.r1.channels = c1
a1.sources.r1.bind = 0.0.0.0
a1.sources.r1.port = 4141
a1.sinks.k1.type = logger
a1.channels.c1.type = memory
a1.channels.c1.capacity = 1000
a1.channels.c1.transactionCapacity = 100
# Bind the source and sink to the channel
a1.sinks.k1.channel = c1


# thrift source
a1.sources = r1
a1.channels = c1
a1.sinks = k1
a1.sources.r1.type = thrift
a1.sources.r1.channels = c1
a1.sources.r1.bind = 0.0.0.0
a1.sources.r1.port = 4141
a1.channels.c1.type = memory
a1.channels.c1.capacity = 1000
a1.channels.c1.transactionCapacity = 100
a1.sinks.k1.channel = c1
a1.sinks.k1.type &