下载及安装
1.Flume
到Flume官网上http://flume.apache.org/download.html下载软件安装包,如图:
下载、解压
tar -xzf flume-1.5.2-bin.tar.gz
cp -r apache-flume-1.5.2-bin /usr/local
2.jdk1.8的安装
1.到jdk官网http://www.oracle.com/technetwork/java/javase/downloads/index.html 下载软件安装包,如图:
参考:
3 设置/etc/profile参数
编辑/etc/profile文件,声明flume的home路径和在path加入bin的路径:
export FLUME_HOME=/usr/local/apache-flume-1.5.2-bin
export FLUME_CONF_DIR=$FLUME_HOME/con
export PATH=$PATH:$FLUME_HOME/bin
4 配置文件
在$FLUME_HOME/conf 下改名flume-env.sh.template为flume-env.sh,修改conf/flume-env.sh配置文件
/usr/local/apache-flume-1.5.2-bin/conf$ mv flume-env.sh.template flume-env.sh
sudo vi flume-env.sh
加一句即可
JAVA_HOME=/usr/local/jdk1.8.0_201
进入到flume的conf下,创建一个flume.conf
文件
在$FLUME_HOME/conf目录下修改flume-conf.properties.template文件,复制并改名为flume-conf,
agent.sources = seqGenSrc
agent.channels = memoryChannel
agent.sinks = loggerSink
# For each one of the sources, the type is defined
agent.sources.seqGenSrc.type = seq
# The channel can be defined as follows.
agent.sources.seqGenSrc.channels = memoryChannel
# Each sink's type must be defined
agent.sinks.loggerSink.type = logger
#Specify the channel the sink should use
agent.sinks.loggerSink.channel = memoryChannel
# Each channel's type is defined.
agent.channels.memoryChannel.type = memory
# Other config values specific to each type of channel(sink or source)
# can be defined as well
# In this case, it specifies the capacity of the memory channel
agent.channels.memoryChannel.capacity = 100
5 启动flume agent
$ cd flume
$ bin/flume-ng agent --conf conf --conf-file conf/flume.conf --name a1 -Dflume.root.logger=INFO,console
参数 | 作用 | 举例 |
---|---|---|
–conf 或 -c | 指定配置文件夹,包含flume-env.sh和log4j的配置文件 | –conf conf |
–conf-file 或 -f | 配置文件地址 | –conf-file conf/flume.conf |
–name 或 -n | agent名称 | –name a1 |
-z | zookeeper连接字符串 | -z zkhost:2181,zkhost1:2181 |
-p | zookeeper中的存储路径前缀 | -p /flume |