flume 搜集A 服务器 的日志到B 服务器

 

就是服务器A的Sink 类型是AVRO， 而 服务器 B的Source 是AVRO

 

1.服务器A的配置

# Define a memory channel called ch1 on agent1
agent1.channels.ch1.type = memory
agent1.channels.ch1.capacity = 1000
agent1.channels.ch1.transactionCapacity = 100
 

agent1.sources.exec-source1.channels = ch1
agent1.sources.exec-source1.type = exec
agent1.sources.exec-source1.command = tail -F /Users/walle/Documents/D2/testflume.log

 
# Define a logger sink that simply logs all events it receives
# and connect it to the other end of the same channel.
agent1.sinks.avro-sink1.channel = ch1
agent1.sinks.avro-sink1.type = avro
agent1.sinks.avro-sink1.hostname = localhost
agent1.sinks.avro-sink1.port = 44445
 
# Finally, now that we've defined all of our components, tell
# agent1 which ones we want to activate.
agent1.channels = ch1
agent1.sources = exec-source1
agent1.sinks = avro-sink1

 

2.服务器B的配置

 

# Define a memory channel called ch1 on agent1
agent2.channels.ch2.type = memory
agent2.channels.ch2.capacity = 1000
agent2.channels.ch2.transactionCapacity = 100
 
# Define an Avro source called avro-source1 on agent1 and tell it
# to bind to 0.0.0.0:41414. Connect it to channel ch1.
agent2.sources.avro-source1.channels = ch2
agent2.sources.avro-source1.type = avro
agent2.sources.avro-source1.bind = localhost
agent2.sources.avro-source1.port = 44445

 
# Define a logger sink that simply logs all events it receives
# and connect it to the other end of the same channel.
agent2.sinks.log-sink1.channel = ch2
agent2.sinks.log-sink1.type = logger
 
# Finally, now that we've defined all of our components, tell
# agent1 which ones we want to activate.
agent2.channels = ch2
agent2.sources = avro-source1
agent2.sinks = log-sink1

http://www.waitingfy.com/archives/4179