Flume-RocketMQ整合

    此篇文章,主要是借鉴了码云上某位大佬的项目,我这边根据自己的需求做了一个整合.需求是flume采集日志,sink到阿里的RocketMQ组件.

    下面我先贴上我的flume配置,其中***为必填项,且是你RocketMQ的配置(在你开通的时候能获取到)

gent1.sources=source1
agent1.channels=channel1
agent1.sinks=sink1

agent1.sources.source1.type=exec
agent1.sources.source1.command = tail -F /data/click_access.log

agent1.sinks.sink1.type=com.handu.flume.sink.rocketmq.RocketMQSink
agent1.sinks.sink1.namesrvAddr=****
agent1.sinks.sink1.group=***
agent1.sinks.sink1.topic=***
agent1.sinks.sink1.tag=tagA
agent1.sinks.sink1.accessKey=***
agent1.sinks.sink1.secretKey=***
agent1.sinks.sink1.channel=channel1

agent1.channels.channel1.type=memory
agent1.channels.channel1.capacity=100
agent1.channels.channel1.transactionCapacity=100
agent1.channels.channel1.keep-alive=3

agent1.sources.source1.channels = channel1