1.前置准备
hadoop单机伪分布式(或hadoop完全分布式集群)
flume下载安装
2.命令操作
#(1)创建flume文件夹
[xiaokang@hadoop ~]$ mkdir flume
[xiaokang@hadoop ~]$ cd flume
# (2)创建telnet-logger.properties 并输入如下内容
[xiaokang@hadoop flume]$ vim telnet-logger.properties
# 指定agent的sources、channels、sinks
a1.sources = r1
a1.channels = c1
a1.sinks = k1
# 配置source属性
a1.sources.r1.type = netcat
a1.sources.r1.bind = localhost
a1.sources.r1.port = 44444
# 配置channel类型
a1.channels.c1.type = memory
a1.channels.c1.capacity = 10000
a1.channels.c1.transactionCapacity = 100
# 配置sink
a1.sinks.k1.type = logger
# 将sources和channels绑定
a1.sources.r1.channels = c1
# 将sinks和channels绑定
a1.sinks.k1.channel = c1
# (3)启动集群
[xiaokang@hadoop ~]$ start-yarn.sh
Starting resourcemanager
Starting nodemanagers
[xiaokang@hadoop ~]$ start-dfs.sh
Starting namenodes on [hadoop]
Starting datanodes
Starting secondary namenodes [hadoop]
[xiaokang@hadoop ~]$ jps
10356 NameNode
9829 NodeManager
9721 ResourceManager
10634 SecondaryNameNode
10826 Jps
10475 DataNode
# (4)启动flume,运行telnet-logger.properties
[xiaokang@hadoop ~]$ flume-ng agent -n a1 -c /opt/software/flume-1.9.0/conf/ -f flume/telnet-logger.properties -Dflume.root.logger=INFO,console
# 见到下图,说明启动成功
# 在另一个会话窗口上输入如下命令
[xiaokang@hadoop ~]$ telnet localhost 44444
# 如下所示,连接成功
Trying ::1...
telnet: connect to address ::1: Connection refused
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
在该会话窗口输入"hello-xiaoliu",“hello-flume”,如下图所示:
另一个会话窗口出现"hello-xiaoliu",“hello-flume”,说明监听成功,如下图所示: