agent 01
a1.sources = r1
a1.channels = c1
a1.sinks = k1
a1.sources.r1.type = TAILDIR
a1.sources.r1.filegroups = f1
a1.sources.r1.filegroups.f1 = /root/logs01/.*log.*
a1.channels.c1.type = memory
a1.sinks.k1.type = avro
a1.sinks.k1.hostname = HadoopNode00
a1.sinks.k1.port = 8888
a1.sinks.k1.channel = c1
a1.sources.r1.channels = c1
./bin/flume-ng agent --conf conf/ --conf-file conf/agent001.conf --name a1 -Dflume.root.logger=INFO,console
agent 02
a1.sources = r1
a1.channels = c1
a1.sinks = k1
a1.sources.r1.type = TAILDIR
a1.sources.r1.filegroups = f1
a1.sources.r1.filegroups.f1 = /root/logs02/.*log.*
a1.channels.c1.type = memory
a1.sinks.k1.type = avro
a1.sinks.k1.hostname = HadoopNode00
a1.sinks.k1.port = 8888
a1.sinks.k1.channel = c1
a1.sources.r1.channels = c1
./bin/flume-ng agent --conf conf/ --conf-file conf/agent002.conf --name a1 -Dflume.root.logger=INFO,console
agent 03
a1.sources = r1
a1.channels = c1
a1.sinks = k1
a1.sources.r1.type = TAILDIR
a1.sources.r1.filegroups = f1
a1.sources.r1.filegroups.f1 = /root/logs03/.*log.*
a1.channels.c1.type = memory
a1.sinks.k1.type = avro
a1.sinks.k1.hostname = HadoopNode00
a1.sinks.k1.port = 8888
a1.sinks.k1.channel = c1
a1.sources.r1.channels = c1
./bin/flume-ng agent --conf conf/ --conf-file conf/agent003.conf --name a1 -Dflume.root.logger=INFO,console
agent 04
在avro类型的source中 有一个属性叫做 threads ,在单节点上的测试结果表现为不用设置,即可有多个avroclint可以连接
但是在多节点上使用不同的主机名(域名|ip)进行通讯,实测需要加上threads 属性,threads 的个数由一级agent的个数决定
a2.sources = r1
a2.channels = c1
a2.sinks = k1
a2.sources.r1.type = avro
a2.sources.r1.bind = HadoopNode00
a2.sources.r1.port = 8888
a2.channels.c1.type = memory
a2.sinks.k1.type =logger
a2.sinks.k1.channel = c1
a2.sources.r1.channels = c1
./bin/flume-ng agent --conf conf/ --conf-file conf/agent004.conf --name a2 -Dflume.root.logger=INFO,console
.channels.c1.type = memory
a1.sinks.k1.type = avro
a1.sinks.k1.hostname = HadoopNode00
a1.sinks.k1.port = 8888
a1.sinks.k1.channel = c1
a1.sources.r1.channels = c1
> ./bin/flume-ng agent --conf conf/ --conf-file conf/agent003.conf --name a1 -Dflume.root.logger=INFO,console
### agent 04
> 在avro类型的source中 有一个属性叫做 threads ,在单节点上的测试结果表现为不用设置,即可有多个avroclint可以连接
>
> 但是在多节点上使用不同的主机名(域名|ip)进行通讯,实测需要加上threads 属性,threads 的个数由一级agent的个数决定
~~~properties
a2.sources = r1
a2.channels = c1
a2.sinks = k1
a2.sources.r1.type = avro
a2.sources.r1.bind = HadoopNode00
a2.sources.r1.port = 8888
a2.channels.c1.type = memory
a2.sinks.k1.type =logger
a2.sinks.k1.channel = c1
a2.sources.r1.channels = c1
./bin/flume-ng agent --conf conf/ --conf-file conf/agent004.conf --name a2 -Dflume.root.logger=INFO,console
献给每一个正在努力的我们,就算在忙,也要注意休息和饮食哦!我就是我,一个在互联网跌跌撞撞,摸爬滚打的热忱,给个三连吧~ 还有就是不要只看,多动手才行!努力呀!