flume 如何连接hdfs

准备

1    启动dfs

2    关闭防火墙

3    主目录下创建需要监控的文件夹 

连接hdfs

在flume下的conf下创建a4.conf

vi conf/a4.test
#注意，此命令在flume文件夹下

编辑a4.conf

#设置agent的source，sink，channel
a4.channels = c1
a4.sinks = k1
a4.sources =s1

#设置source
a4.sources.s1.type = spooldir                      
a4.sources.s1.spoolDir =/home/hadoop/a1_test     #a1_test是之前新建的文件夹名

#设置channel的参数  
a4.channels.c1.type = memory 

#设置sink
a4.sinks.k1.type = hdfs
a4.sinks.k1.hdfs.path = hdfs://python:9000/user/hadoop/flumedir/%y-%m-%d/%H%M/      
a4.sinks.k1.hdfs.filePrefix = Data
a4.sinks.k1.hdfs.round = true
a4.sinks.k1.hdfs.roundValue = 10
a4.sinks.k1.hdfs.roundUnit = minute
a4.sinks.k1.