spooling source - Flie channel- Kafka sink配置

爱学习的小同志

已于 2022-07-14 20:18:59 修改

阅读量242

点赞数

分类专栏： flrume配置文章标签：大数据

于 2022-07-01 17:15:07 首次发布

本文链接：https://blog.csdn.net/m0_58648737/article/details/125561645

版权

flrume配置专栏收录该内容

2 篇文章 0 订阅

订阅专栏

配置篇日志

具体配置

hniu.sources=r2
hniu.channels=c1
hniu.sinks=s1

hniu.sources.r2.type=spooldir
hniu.sources.r2.spoolDir= /home/zy32
hniu.sources.r2.fileSuffix=mxt
hniu.sources.r2.includePattern=\\S*.(txt|json)
hniu.sources.r2.ignorePattern=\\S*.(mxt|tmp)
hniu.sources.r2.basenameHeader=true
hniu.sources.r2.basenameHeaderKey=name
hniu.sources.r2.recursiveDirectorySearch=true

hniu.channels.c1.type=file
hniu.channels.c1.dataDirs=/home/zy/data
hniu.channels.c1.checkpointDir=/home/zy/checkpointdir
hniu.channels.c1.capacity=1000
hniu.channels.c1.transactionCapacity=100

hniu.sinks.s1.type = org.apache.flume.sink.kafka.KafkaSink
hniu.sinks.s1.brokerList = 192.168.15.32:9092
hniu.sinks.s1.topic = one_test

hniu.sources.r2.channels=c1
hniu.sinks.s1.channel=c1