Flume 环境搭建

Flume 下载

http://www.apache.org/dyn/closer.lua/flume/1.9.0/apache-flume-1.9.0-bin.tar.gz

上传并解压

[root@master flume]# mkdir /home/flume
[root@master flume]# tar -xzvf apache-flume-1.9.0-bin.tar.gz -C /home/flume/

配置环境变量

FLUME_HOME=/home/flume/apache-flume-1.9.0-bin
SPART_HOME=/home/spark/spark-2.4.3-bin-hadoop2.7
SCALA_HOME=/home/scala/scala-2.13.1
ZOOKEEPER_HOME=/home/zookeeper/zookeeper-3.4.14
HBASE_HOME=/home/hbase/hbase-1.4.5
HADOOP_HOME=/home/hadoop/hadoop-2.7.7
JAVA_HOME=/java/jdk1.8.0_161
PATH=$JAVA_HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$HBASE_HOME/bin:$ZOOKEEPER_HOME/bin:$SCALA_HOME/bin:$SPART_HOME/bin:$FLUME_HOME/bin:$PATH
CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export JAVA_HOME
export HBASE_HOME
export PATH
export CLASSPATH
export HADOOP_HOME
export ZOOKEEPER_HOME
export SCALA_HOME
export SPART_HOME
export FLUME_HOME

测试配置是否成功

[root@master apache-flume-1.9.0-bin]# flume-ng version
Flume 1.9.0
Source code repository: https://git-wip-us.apache.org/repos/asf/flume.git
Revision: d4fcab4f501d41597bc616921329a4339f73585e
Compiled by fszabo on Mon Dec 17 20:45:25 CET 2018
From source with checksum 35db629a3bda49d23e9b3690c80737f9
[root@master apache-flume-1.9.0-bin]# 

flume-env.sh.template文件配置

将flume-env.sh.template 复制一份

配置JAVA_HOME

[root@master conf]# cp flume-env.sh.template flume-env.sh

 export JAVA_HOME=/java/jdk1.8.0_161

 

启动Flume

./bin/flume-ng agent -c conf -f ./conf/flume.conf -n a1 -Dflume.root.logger=INFO,console