Flume 史上最全面的大数据学习第十篇（六)Flume完结篇

agent 01

a1.sources = r1
a1.channels = c1
a1.sinks = k1

a1.sources.r1.type = TAILDIR
a1.sources.r1.filegroups = f1 
a1.sources.r1.filegroups.f1 = /root/logs01/.*log.*


a1.channels.c1.type = memory


a1.sinks.k1.type = avro
a1.sinks.k1.hostname = HadoopNode00
a1.sinks.k1.port = 8888

a1.sinks.k1.channel = c1
a1.sources.r1.channels = c1

./bin/flume-ng agent --conf conf/ --conf-file conf/agent001.conf --name a1 -Dflume.root.logger=INFO,console

agent 02

a1.sources = r1
a1.channels = c1
a1.sinks = k1

a1.sources.r1.type = TAILDIR
a1.sources.r1.filegroups = f1 
a1.sources.r1.filegroups.f1 = /root/logs02/.*log.*


a1.channels.c1.type = memory


a1.sinks.k1.type = avro
a1.sinks.k1.hostname = HadoopNode00
a1.sinks.k1.port = 8888

a1.sinks.k1.channel = c1
a1.sources.r1.channels = c1

./bin/flume-ng agent --conf conf/ --conf-file conf/agent002.conf --name a1 -Dflume.root.logger=INFO,console

agent 03

a1.sources = r1
a1.channels = c1
a1.sinks = k1

a1.sources.r1.type = TAILDIR
a1.sources.r1.filegroups = f1 
a1.sources.r1.filegroups.f1 = /root/logs03/.*log.*


a1.channels.c1.type = memory


a1.sinks.k1.type = avro
a1.sinks.k1.hostname = HadoopNode00
a1.sinks.k1.port = 8888

a1.sinks.k1.channel = c1
a1.sources.r1.channels = c1

./bin/flume-ng agent --conf conf/ --conf-file conf/agent003.conf --name a1 -Dflume.root.logger=INFO,console

agent 04

在avro类型的source中 有一个属性叫做 threads ，在单节点上的测试结果表现为不用设置，即可有多个avroclint可以连接

但是在多节点上使用不同的主机名（域名|ip）进行通讯，实测需要加上threads 属性，threads 的个数由一级agent的个数决定

a2.sources = r1
a2.channels = c1
a2.sinks = k1

a2.sources.r1.type = avro
a2.sources.r1.bind = HadoopNode00
a2.sources.r1.port = 8888

a2.channels.c1.type = memory

a2.sinks.k1.type =logger


a2.sinks.k1.channel = c1
a2.sources.r1.channels = c1

./bin/flume-ng agent --conf conf/ --conf-file conf/agent004.conf --name a2 -Dflume.root.logger=INFO,console

.channels.c1.type = memory

a1.sinks.k1.type = avro
a1.sinks.k1.hostname = HadoopNode00
a1.sinks.k1.port = 8888

a1.sinks.k1.channel = c1
a1.sources.r1.channels = c1

> ./bin/flume-ng agent --conf conf/ --conf-file conf/agent003.conf  --name a1  -Dflume.root.logger=INFO,console

### agent 04

> 在avro类型的source中 有一个属性叫做 threads ，在单节点上的测试结果表现为不用设置，即可有多个avroclint可以连接
>
> 但是在多节点上使用不同的主机名（域名|ip）进行通讯，实测需要加上threads 属性，threads 的个数由一级agent的个数决定

~~~properties
a2.sources = r1
a2.channels = c1
a2.sinks = k1

a2.sources.r1.type = avro
a2.sources.r1.bind = HadoopNode00
a2.sources.r1.port = 8888

a2.channels.c1.type = memory

a2.sinks.k1.type =logger


a2.sinks.k1.channel = c1
a2.sources.r1.channels = c1

./bin/flume-ng agent --conf conf/ --conf-file conf/agent004.conf --name a2 -Dflume.root.logger=INFO,console

献给每一个正在努力的我们，就算在忙，也要注意休息和饮食哦！我就是我，一个在互联网跌跌撞撞，摸爬滚打的热忱，给个三连吧~ 还有就是不要只看，多动手才行！努力呀！