下载并安装flume,安装目录为/opt/flume
mv flume-conf.properties.template flume-conf.properties
mv flume-env.sh.template flume-env.sh
修改flume-env.sh 环境变量,添加如下:
export JAVA_HOME=/opt/jdk1.8.0_121
FLUME_CLASSPATH="/opt/hadoop-2.7.5/share/hadoop/hdfs/*"
新增文件 hdfs.properties,内容如下:
#spooldir 对监控指定文件夹中新文件的变化,一旦有新文件出现就解析,解析写入channel后完成的文件名将追加后缀为*.COMPLATE
LogAgent.sources.apache.type = spooldir
LogAgent.sources.apache.spoolDir = /tmp/logs
LogAgent.sources.apache.chan