Flume-A：从网络端口采集数据输出到控制台

1概述

了解一个框架就去它的官网上看 http://flume.apache.org/
Flume is a distributed, reliable, and available service for efficiently collecting, aggregating, and moving large amounts of log data. It has a simple and flexible architecture based on streaming data flows. It is robust and fault tolerant with tunable reliability mechanisms and many failover and recovery mechanisms. It uses a simple extensible data model that allows for online analytic application.

使用：使用flume的关键就是写配置文件
A) 配置Source
B) 配置Channel
C) 配置Sink
D) 把它们串联起来

第一种方式：从网络端口采集数据输出到控制台

netcat source +memory channel +logger sink

# Name the components on this agent   定义这个Agent名称为a1,sources,sinks,channels名称为r1,k1,c1
a1.sources = r1
a1.sinks = k1
a1.channels = c1

# Describe/configure the source    
a1.sources.r1.type = netcat							描述source的类型，网络传输则用netcat,指定的，不要写netdog....
a1.sources.r1.bind = 192.168.240.131			
a1.sources.r1.port = 44444

# Describe the sink
a1.sinks.k1.type = logger								sinks类型

# Use a channel which buffers events in memory
a1.channels.c1.type = memory						channel类型

# Bind the source and sink to the channel			将三个组件组装起来
a1.sources.r1.channels = c1								source可以传输不同的channel所以带了s
a1.sinks.k1.channel = c1									而sink上游只能链接一个channel

启动agent

bin/flume-ng  agent \
--name a1  \
--conf  /sda3/softs/flume-1.6.0-cdh5.14.0/conf  \
--conf-file  /sda3/softs/flume-1.6.0-cdh5.14.0/conf/example1.conf  \
-Dflume.root.logger=INFO,console

在另一边的控制台
成功

Event: { headers:{} body: 64 73 64 61 64 0D                               dsdad. }

Event是Flume的数据传输的基本单元
Event=可选的Header+byte array