Streaming flume 的poll push 的配置信息和启动

flume的配置

1.在…/flume/conf下放入：

flume-poll.conf：

a1.sources = r1
a1.sinks = k1
a1.channels = c1
#source
a1.sources.r1.channels = c1
a1.sources.r1.type = spooldir
a1.sources.r1.spoolDir = /opt/flume/data
a1.sources.r1.fileHeader = true
#channel
a1.channels.c1.type =memory
a1.channels.c1.capacity = 20000
a1.channels.c1.transactionCapacity=5000
#sinks
a1.sinks.k1.channel = c1
a1.sinks.k1.type = org.apache.spark.streaming.flume.sink.SparkSin
k
a1.sinks.k1.hostname=linux-star
a1.sinks.k1.port = 8888
a1.sinks.k1.batchSize= 2000

flume-push.conf：

#push mode
a1.sources = r1
a1.sinks = k1
a1.channels = c1
#source
a1.sources.r1.channels = c1
a1.sources.r1.type = spooldir
a1.sources.r1.spoolDir = /opt/flume/data
a1.sources.r1.fileHeader = true
#channel
a1.channels.c1.type =memory
a1.channels.c1.capacity = 20000
a1.channels.c1.transactionCapacity=5000
#sinks
a1.sinks.k1.channel = c1
a1.sinks.k1.type = avro
a1.sinks.k1.hostname=192.168.1.159 #windowsIp
a1.sinks.k1.port = 8888
a1.sinks.k1.batchSize= 2000

windows Ip 的查看命令 ipconfig

2.poll和push的启动

poll: 先启动flume 后启动项目 然后向指定的文件中放入东西 控制台输出
bin/flume-ng agent -n a1 -c conf -f conf/flume-poll.conf -Dflume.root.logger=INFO,console

push: （hostnome ip地址：为windows 的ip 地址） 先启动idea项目 然后启动 flume 然后向指定的文件中放入东西 控制台输出
bin/flume-ng agent -n a1 -c conf -f conf/flume-push.conf -Dflume.root.logger=INFO,console