flume安装

flume执行原理 是什么

三大可新组件

1.源端采集了理数据给他传输了管道中，管道加载的数据传输到下沉端，在传输到

2.

 

安装flume

 压缩包

上传到 cd /usr/local/ 目录下

手动输入 解压 tar xzvf apache-flume-1.8.0-bin.tar

改名 mv apache-flume-1.8.0-bin flume

添加环境变量 vi /etc/profile

export FLUME_HOME=/usr/local/flume

末尾 :$FLIUME_HOME/bin

source /etc/profile 完成

使用：cd /usr/loacl/flume/conf下

vi.netcat-logger.conf 文件名字编辑下面有文件

a1.sources = r1

a1.sinks = k1

a1.channels = c1

 

# Describe/configure the source

a1.sources.r1.type = netcat

a1.sources.r1.bind = localhost

a1.sources.r1.port = 44444

 

# Describe the sink

a1.sinks.k1.type = logger

 

# Use a channel which buffers events in memory

a1.channels.c1.type = memory

a1.channels.c1.capacity = 1000

a1.channels.c1.transactionCapacity = 100

 

# Bind the source and sink to the channel

a1.sources.r1.channels = c1

a1.sinks.k1.channel = c1

 

 

启动命令：flume-ng agent --conf conf --conf-file netcat-logger.conf --name a1 -Dflume.root.logger=INFO,console

克隆窗口

telnet 127.0.0.1 44444

动态模式

————————————————————————————————————————

 

使用在hadoop下创建 vi /var/log.txt

输入hello word

tail -F /var/log.txt

克隆一个页面 echo 1 >>/var/log.txt

 

——-------------------------------------------------------------------------------------------------

vi tail.sh

#!/bin/bath

for i in {1..1000}

do

echo "hello $i" >> /var/log.txt

sleep 2

done

启动 sh tail.sh

 

 

此下面无所谓

配置文件

cd /usr/local/flume/

ls

cd conf/

修改环境cd flume-env.sh.template flume-env.sh

vi flume-env.sh

改为下面

继续

改为

配置完成

 

flume动态使用

pwd

vi 文件名字 比如vi netcat-logger.conf

a1.sources = r1

a1.sinks = k1

a1.channels = c1

 

# Describe/configure the source

a1.sources.r1.type = netcat

a1.sources.r1.bind = localhost

a1.sources.r1.port = 44444

 

# Describe the sink

a1.sinks.k1.type = logger

 

# Use a channel which buffers events in memory

a1.channels.c1.type = memory

a1.channels.c1.capacity = 1000

a1.channels.c1.transactionCapacity = 100

 

# Bind the source and sink to the channel

a1.sources.r1.channels = c1

a1.sinks.k1.channel = c1

启动

flume-ng agent --conf conf --conf-file netcat-logger.conf --name a1 -Dflume.root.logger=INFO,console

 

测试脚本不正确

vi flume.sh

上面输出了10000遍

sh flume.sh

转载于:https://www.cnblogs.com/dasiji/p/11246112.html