flume的安装配置

1. 上传并解压安装包

tar -zxvf apache-flume-1.9.0-bin.tar.gz ../server/

2. 配置环境变量

sudo vim /etc/profile

# 配置flume的环境变量 export FLUME_HOME=/export/server/flume-1.9.0-bin export PATH=$PATH:$FLUME_HOME/bin

分发到其他节点并source /etc/profile

3. 修改flume的配置文件

cp /export/server/flume-1.9.0-bin/conf/ flume-env.sh.template ./flume-env.sh

vim flume-env.sh

export JAVA_HOME=/export/server/jdk1.8.0_131 export JAVA_OPTS="-Xms100m -Xmx2000m -Dcom.sun.management.jmxremote" export JAVA_OPTS="$JAVA_OPTS -Dorg.apache.flume.log.rawdata=true -Dorg.apache.flume.log.printconfig=true "

cp /export/server/flume-1.9.0-bin/conf/log4j.properties.template ./log4j.properties

vim flume-env.sh

# console表示同时将日志输出到控制台 flume.root.logger=INFO,LOGFILE,console  # 指定日志输出路径 flume.log.dir=/export/server/flume/logs # 只修改以上两项保留原文件原内容

4.使用flume-ng version 可查到相关版本信息即安装成功

5.flume的使用

Flume使用demo

在bin目录下新建一个配置文件

vim example.conf

# example.conf: A single-node Flume configuration # Name the components on this agent a1.sources = r1 a1.sinks = k1 a1.channels = c1 # Describe/configure the source a1.sources.r1.type = netcat a1.sources.r1.bind = localhost a1.sources.r1.port = 44444 # Describe the sink a1.sinks.k1.type = logger # Use a channel which buffers events in memory a1.channels.c1.type = memory a1.channels.c1.capacity = 1000 a1.channels.c1.transactionCapacity = 100 # Bind the source and sink to the channel a1.sources.r1.channels = c1 a1.sinks.k1.channel = c1

启动flume

./flume-ng agent --conf conf --conf-file example.conf --name a1

--conf：Flume通用配置目录

--conf-file：配置文件位置

--name：代理名称

2.3连接测试

新起一个终端并连接44444端口

telnet localhost 44444

没有telnet 指令可以先安装

sudo yum install telnet

输入hello flume

在flume控制台看到输出