黄刚的技术博客

每天进步一点点,自我实现

flume ng简单例子

本例子来源于flume官方文档,主要演示部署flume agent单节点,接收信息然后控制台输出。

首先编辑配置文件,在conf目录中新建文件example.conf,然后添加以下内容:

# example.conf: A single-node Flume configuration
# Name the components on this agent
a1.sources = r1
a1.sinks = k1
a1.channels = c1
# Describe/configure the source
a1.sources.r1.type = netcat
a1.sources.r1.bind = localhost
a1.sources.r1.port = 44444
# Describe the sink
a1.sinks.k1.type = logger
# Use a channel which buffers events in memory
a1.channels.c1.type = memory
a1.channels.c1.capacity = 1000
a1.channels.c1.transactionCapacity = 100
# Bind the source and sink to the channel
a1.sources.r1.channels = c1
a1.sinks.k1.channel = c1


配置文件中定义了一个单节点agent名为a1. a1有一个source监听来自端口444444的数据,有一个channel缓冲事件数据在内存中,之后通过一个sink打印出事件内容到控制台。配置文件中同时会定义各个组件自己特有的一些配置参数。


然后启动:

bin/flume-ng agent --conf conf --conf-file conf/example.conf --name a1 -Dflume.root.logger=INFO,console

最后打开另外一个终端,执行命令:

uanggang@huanggangdeMacBook-Pro:~/workspace/dafka-project$ telnet localhost 44444
Trying ::1...
telnet: connect to address ::1: Connection refused
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
hello world
OK

在之前启动flume agent的终端上就会显示刚才输出的内容:

2015-11-18 10:31:17,101 (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO - org.apache.flume.sink.LoggerSink.process(LoggerSink.java:94)] Event: { headers:{} body: 68 65 6C 6C 6F 20 77 6F 72 6C 64 0D             hello world. }
2015-11-18 10:38:03,428 (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO - org.apache.flume.sink.LoggerSink.process(LoggerSink.java:94)] Event: { headers:{} body: 0D                                              . }



阅读更多
个人分类: flume ng
上一篇Kafka Reliability - When it absolutely, positively has to be there
下一篇How to choose the number of topics/partitions in a Kafka cluster?
想对作者说点什么? 我来说一句

mvn flume ng sdk

2017年09月28日 6.4MB 下载

FLUME-FlumeNG-210517-1655-5858

2017年05月22日 22KB 下载

没有更多推荐了,返回首页

关闭
关闭