Flume跨服务器收集信息(A->B)

继上次flume实时读取数据到Oracle。https://blog.csdn.net/sileiH/article/details/83023404

1、搭建材料准备

• jdk 1.8

• flume1.8

• 两台服务器

 

2、安装好jdk1.8，然后解压flume1.8，并改名为flume放于/app/中

 

3、编辑配置文件。

# 服务器A(192.168.3.129)

a1.sources = r2

a1.sinks = k2

a1.channels = c2



#################各Server运行错误日志###########################

# 配置读取的日志

a1.sources.r2.type = exec

a1.sources.r2.command = tail -F /app/flume/test/node_err_info



# Describe the sink

a1.sinks.k2.type = avro

a1.sinks.k2.hostname = 192.168.3.129

a1.sinks.k2.port = 9011



# Use a channel which buffers events in memory

a1.channels.c2.type = memory



# Bind the source and sink to the channel

a1.sources.r2.channels = c2

a1.sinks.k2.channel = c2



# 服务器B(192.168.3.130)

a1.sources = r2

a1.sinks = k2

a1.channels = c2



#################各Server运行错误日志###########################

# 配置读取的日志

a1.sources.r2.type = avro

a1.sources.r2.bind= 192.168.3.130

a1.sources.r2.port = 9011



# Describe the sink

a1.sinks.k2.type = logger



# Use a channel which buffers events in memory

a1.channels.c2.type = memory



# Bind the source and sink to the channel

a1.sources.r2.channels = c2

a1.sinks.k2.channel = c2

 

5、执行flume。

bin/flume-ng agent --conf conf --conf-f conf/confA.properties -name a1 -Dflume.root.logger=INFO,console

bin/flume-ng agent --conf conf --conf-f conf/confB.properties -name a1 -Dflume.root.logger=INFO,console