1、需求:
- 使用Flume1监控一个端口,其sink组中的sink分别对接Flume2和Flume3
- LoadBalancingSinkProcessor可以实现负载均衡的功能,FailoverSinkProcessor可以实现故障转移的功能.
2、需求分析
3、准备
在/data/flume/apache-flume/job目录下创建group2文件夹
cd /data/flume/apache-flume/job
mkdir group2
4、创建flume1 agent配置文件flume-netcat-flume.conf
配置1个netcat source和1个channel、1个sink group(2个sink),分别输送给flume-flume-console1和flume-flume-console2。
vim job/group2/flume-netcat-flume.conf
添加:
# Name the components on this agent
a1.sources = r1
a1.channels = c1
a1.sinkgroups = g1
a1.sinks = k1 k2
# Describe/configure the source
a1.sources.r1.type = netcat
a1.sources.r1.bind = localhost
a1.sources.r1.port = 44444
# 设置sink组故障转移
a1.sinkgroups.g1.processor.type = failover
# 设置sink组中的sink的优先级,较高优先级值的Sink较早被激活。绝对值越大表示优先级越高
a1.sinkgroups.g1