Flume的应用场景

实例1
1.使用Flume的关键就是写配置文件，分别配置Source、Channel、Sink，然后把三者串联起来

比如这里写一个配置文件$FLUME_HOME/conf/example.conf，使用netcat source、memory channel、logger sink，example.conf内容如下：

a1.sources = r1
a1.sinks = k1
a1.channels = c1

a1.sources.r1.type = telnet
a1.sources.r1.bind = localhost
a1.sources.r1.port = 44444

a1.sinks.k1.type = logger

a1.channels.c1.type = memory

a1.sources.r1.channels = c1
a1.sinks.k1.channel = c1

2.启动Agent:

3.启动telnet输入数据验证
telnet 127.0.0.1 44444启动后输入内容123就可以在Flume看到如下数据：
Event: { headers:{} body: 31 32 33 0D 123. }

实例2
1.创建exec-memory-logger.conf配置文件
内容如下：
a1.sources = r1
a1.sinks = k1
a1.channels = c1

a1.sources.r1.type = exec
a1.sources.r1.command = /opt/data.log
a1.sources.r1.shell = /bin/sh -c

a1.sinks.k1.type = logger

a1.channels.c1.type = memory

a1.sources.r1.channels = c1
a1.sinks.k1.channel = c1
2.启动 agent
3.向/opt/data.log日志文件追加数据，验证

三.Flume示例3（两个Agent串起来）

1.创建exec-memory-avro.conf和avro-memory-logger.conf配置文件
因为我手头没有两台机器，这里我只是在一台机器（hadoop000）上模拟两台机器的情况
exec-memory-avro.conf:

exec-memory-avro.sources = exec-source
exec-memory-avro.sinks = avro-sink
exec-memory-avro.channels = memory-channel
exec-memory-avro.sources.exec-source.type = exec
exec-memory-avro.sources.exec-source.command = tail -F /home/hadoop/data/data.log
exec-memory-avro.sources.exec-source.shell = /bin/sh -c
exec-memory-avro.sinks.avro-sink.type = avro
exec-memory-avro.sinks.avro-sink.hostname = hadoop000
exec-memory-avro.sinks.avro-sink.port = 44444
exec-memory-avro.channels.memory-channel.type = memory
exec-memory-avro.sources.exec-source.channels = memory-channel
exec-memory-avro.sinks.avro-sink.channel = memory-channel

avro-memory-logger.conf
avro-memory-logger.sources = avro-source
avro-memory-logger.sinks = logger-sink
avro-memory-logger.channels = memory-channel
avro-memory-logger.sources.avro-source.type = avro
avro-memory-logger.sources.avro-source.bind = hadoop000
avro-memory-logger.sources.avro-source.port = 44444
avro-memory-logger.sinks.logger-sink.type = logger
avro-memory-logger.channels.memory-channel.type = memory
avro-memory-logger.sources.avro-source.channels = memory-channel
avro-memory-logger.sinks.logger-sink.channel = memory-channel

2启动Agent
先启动avro-memory-logger
然后启动exec-memory-avro
3向/home/hadoop/data/data.log日志文件追加数据，验证