flume log4j

最新推荐文章于 2022-03-24 14:10:06 发布

liu_xiang_chao

最新推荐文章于 2022-03-24 14:10:06 发布

阅读量393

点赞数

分类专栏： flume 文章标签： flume log4j 配置 sinks

本文链接：https://blog.csdn.net/liu_xiang_chao/article/details/45917443

版权

flume 专栏收录该内容

1 篇文章 0 订阅

订阅专栏

最近有业务需求，将log4j 收集的日志发送到 flume ，整合期间遇到很多问题，在这里统一说明下，方便大家快速整合

关于flume 介绍大家可以参考 http://my.oschina.net/leejun2005/blog/288136 这篇文章很详细

我用的 flume 版本是 1.5.2,

目录结构

其中 bin 是启动命令所在

conf 是资源文件，资源文件最开始都是.teltemple 可能拼错了，模板文件。把这个后缀去了

第一步修改 flume-env.sh，文件在conf里面，也是模板文件copy 》》去后缀

1.增加 jdk 路径，修改启动内存大小
2.增加环境变量

# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# If this file is placed at FLUME_CONF_DIR/flume-env.sh, it will be sourced
# during Flume startup.

# Enviroment variables can be set here.

#JAVA_HOME=/usr/lib/jvm/java-
#Java
export JAVA_HOME=/usr/local/jdk
export CLASSPATH=.:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export PATH=$PATH:$JAVA_HOME/bin
# Give Flume more memory and pre-allocate, enable remote monitoring via JMX
JAVA_OPTS="-Xms100m -Xmx500m -Dcom.sun.management.jmxremote"

# Note that the Flume conf directory is always included in the classpath.
#FLUME_CLASSPATH=""
#flume
export FLUME_HOME=/data/flume/
export FLUME_CONF_DIR=$FLUME_HOME/conf
export PATH=$PATH:$FLUME_HOME/bin

修改下 java_home 和 flume_home

第二部创建启动配置文件 arvo.conf 名字随便了

# example.conf: A single-node Flume configuration
# Name the components on this agent
a1.sources = r1
a1.sinks = k1
a1.channels = c1

# Describe/configure the source
a1.sources.r1.type = avro
a1.sources.r1.bind = 192.168.50.28
a1.sources.r1.port = 44444

# Describe the sink
a1.sinks.k1.type =com.wangku.wjf.dubbo.filter.method.flume.sink.DubboMethodInfoSink

# Use a channel which buffers events in memory
a1.channels.c1.type = memory
a1.channels.c1.capacity = 1000
a1.channels.c1.transactionCapacity = 100

# Bind the source and sink to the channel
a1.sources.r1.channels = c1
a1.sinks.k1.channel = c1