对接kafka_flume对接kafka多路径同时收集日志,配置怎么写?

flume安装目录的conf下

配置文件如下:

# define

#比如我们有8个路径要收集

a1.sources = r1 r2 r3 r4 r5 r6 r7 r8

a1.sinks = k6

a1.channels = c6

# source

a1.sources.r1.type = exec

a1.sources.r2.type = exec

a1.sources.r3.type = exec

a1.sources.r4.type = exec

a1.sources.r5.type = exec

a1.sources.r6.type = exec

a1.sources.r7.type = exec

a1.sources.r8.type = exec

a1.sources.r1.command = tail -F -c +0 /logs/bbcc-bs-1/baseBusiness.log

a1.sources.r2.command = tail -F -c +0 /logs/bbcc-bs-2/baseBusiness.log

a1.sources.r3.command = tail -F -c +0 /logs/bbcc-bs-3/baseBusiness.log

a1.sources.r4.command = tail -F -c +0 /logs/bbcc-bs-4/baseBusiness.log

a1.sources.r5.command = tail -F -c +0 /logs/bbcc-bs-5/baseBusiness.log

a1.sources.r6.command = tail -F -c +0 /logs/bbcc-bs-6/baseBusiness.log

a1.sources.r7.command = tail -F -c +0 /logs/bbcc-bs-7/baseBusiness.log

a1.sources.r8.command = tail -F -c +0 /logs/bbcc-bs-8/baseBusiness.log

a1.sources.r1.shell = /bin/bash -c

a1.sources.r2.shell = /bin/bash -c

a1.sources.r3.shell = /bin/bash -c

a1.sources.r4.shell = /bin/bash -c

a1.sources.r5.shell = /bin/bash -c

a1.sources.r6.shell = /bin/bash -c

a1.sources.r7.shell = /bin/bash -c

a1.sources.r8.shell = /bin/bash -c

# sink

a1.sinks.k6.type = org.apache.flume.sink.kafka.KafkaSink

a1.sinks.k6.kafka.bootstrap.servers = ip(服务器对应的ip):9092,ip(服务器对应的ip):9092,ip(服务器对应的ip):9092

a1.sinks.k6.kafka.topic = produupie

a1.sinks.k6.kafka.flumeBatchSize = 20

a1.sinks.k6.kafka.producer.acks = 1

a1.sinks.k6.kafka.producer.linger.ms = 1

# channel

a1.channels.c6.type = memory

a1.channels.c6.capacity = 10000

a1.channels.c6.transactionCapacity = 100

# bind

a1.sources.r1.channels = c6

a1.sources.r2.channels = c6

a1.sources.r3.channels = c6

a1.sources.r4.channels = c6

a1.sources.r5.channels = c6

a1.sources.r6.channels = c6

a1.sources.r7.channels = c6

a1.sources.r8.channels = c6

a1.sinks.k6.channel = c6

我们可以考虑到sources路径配置到一起,希望这个配置对你有所帮助

f3db7f6ca8eda966b11210951059aa17.png
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值