PlumeLog分布式日志

之一丶

已于 2023-11-16 16:48:00 修改

阅读量1.7k

点赞数 1

文章标签： java 分布式中间件

于 2021-08-21 13:42:06 首次发布

本文链接：https://blog.csdn.net/qq_34279574/article/details/119419534

版权

PlumeLog官方地址:https://github.com/fayechenlong/plumelog

无入侵的分布式日志系统，基于log4j、log4j2、logback搜集日志，设置链路ID，方便查询关联日志
基于elasticsearch作为查询引擎
高吞吐，查询效率高
全程不占应用程序本地磁盘空间，免维护;对于项目透明，不影响项目本身运行
无需修改老项目，引入直接使用，支持dubbo,支持springcloud

服务器docker安装elasticsearch

需要指定版本

docker pull elasticsearch:7.7.0

docker运行elasticsearch

docker run --name elasticsearch -d -e ES_JAVA_OPTS="-Xms512m -Xmx512m" -e "discovery.type=single-node" -p 9200:9200 elasticsearch:7.7.0

plumelog的配置文件

解压plumelog,并修改配置文件
application.properties:

spring.application.name=plumelog_server
spring.profiles.active=test-confidential
server.port=8891
spring.thymeleaf.mode=LEGACYHTML5
spring.mvc.view.prefix=classpath:/templates/
spring.mvc.view.suffix=.html
spring.mvc.static-path-pattern=/plumelog/**


#值为4种 redis,kafka,rest,restServer,redisCluster,redisSentinel
#redis 表示用redis当队列
#redisCluster 表示用redisCluster当队列
#redisSentinel 表示用redisSentinel当队列
#kafka 表示用kafka当队列
#rest 表示从rest接口取日志
#restServer 表示作为rest接口服务器启动
#ui 表示单独作为ui启动
plumelog.model=redis

#如果使用kafka,启用下面配置
#plumelog.kafka.kafkaHosts=172.16.247.143:9092,172.16.247.60:9092,172.16.247.64:9092
#plumelog.kafka.kafkaGroupName=logConsumer

#队列redis地址，集群用逗号隔开，model配置redis集群模式
plumelog.queue.redis.redisHost=127.0.0.1:6379
#如果使用redis有密码,启用下面配置
plumelog.queue.redis.redisPassWord=5Brm8#qc
plumelog.queue.redis.redisDb=0

#管理端redis地址
plumelog.redis.redisHost=127.0.0.1:6379
#如果使用redis有密码,启用下面配置
plumelog.redis.redisPassWord=5Brm8#qc
#plumelog.queue.redis.redisDb=0

#如果使用rest,启用下面配置
#plumelog.rest.restUrl=http://127.0.0.1:8891/getlog
#plumelog.rest.restUserName=plumelog
#plumelog.rest.restPassWord=123456

#redis解压缩模式，开启后不消费非压缩的队列
#plumelog.redis.compressor=true

#elasticsearch相关配置，Hosts支持携带协议，如：http、https
plumelog.es.esHosts=127.0.0.1:9200
#ES7.*已经去除了索引type字段，所以如果是es7不用配置这个，7.*以下不配置这个会报错
#plumelog.es.indexType=plumelog
plumelog.es.shards=5
plumelog.es.replicas=1
plumelog.es.refresh.interval=30s
#日志索引建立方式day表示按天、hour表示按照小时
plumelog.es.indexType.model=day
#ES设置密码,启用下面配置
#plumelog.es.userName=elastic
#plumelog.es.passWord=elastic
#是否信任自签证书
#plumelog.es.trustSelfSigned=true
#是否hostname验证
#plumelog.es.hostnameVerification=false


#单次拉取日志条数
plumelog.maxSendSize=100
#拉取时间间隔，kafka不生效
plumelog.interval=100

#plumelog-ui的地址 如果不配置，报警信息里不可以点连接
plumelog.ui.url=https://127.0.0.1:8891

#管理密码，手动删除日志的时候需要输入的密码
admin.password=z25kuG4h

#日志保留天数,配置0或者不配置默认永久保留
admin.log.keepDays=30
#链路保留天数,配置0或者不配置默认永久保留
admin.log.trace.keepDays=30
#登录配置，配置后会有登录界面
login.username=username
login.password=password

项目配置文件

plumelog:
  appName: im-service
  redisHost: xxx.x.x.x:6379
  redisAuth: redis密码

pom文件增加:

        <!--分布式日志收集plumelog-->
        <dependency>
            <groupId>com.plumelog</groupId>
            <artifactId>plumelog-logback</artifactId>
            <version>${plumelog.version}</version>
        </dependency>
        <dependency>
            <groupId>com.plumelog</groupId>
            <artifactId>plumelog-trace</artifactId>
            <version>${plumelog.version}</version>
        </dependency>

项目日志配置文件,这里使用logback

日志配置追加appender
logback.xml:

<?xml version="1.0" encoding="UTF-8"?>
<configuration debug="false">
    <conversionRule conversionWord="clr" converterClass="org.springframework.boot.logging.logback.ColorConverter"/>
    <conversionRule conversionWord="wex"
                    converterClass="org.springframework.boot.logging.logback.WhitespaceThrowableProxyConverter"/>
    <conversionRule conversionWord="wEx"
                    converterClass="org.springframework.boot.logging.logback.ExtendedWhitespaceThrowableProxyConverter"/>
    <!-- 彩色日志格式 -->
    <property name="CONSOLE_LOG_PATTERN"
              value="${CONSOLE_LOG_PATTERN:-%clr(%d{yyyy-MM-dd HH:mm:ss.SSS}){faint} %clr(${LOG_LEVEL_PATTERN:-%5p}) %clr(${PID:- }){magenta} %clr(---){faint} %clr([%15.15t]){faint} %clr(%-40.40logger{39}){cyan} %clr(:){faint} %m%n${LOG_EXCEPTION_CONVERSION_WORD:-%wEx}}"/>
    <!--输出到控制台-->
    <appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
        <!--过滤trace日志到控制台-->
        <filter class="com.plumelog.logback.util.FilterSyncLogger">
            <level></level>
        </filter>
        <encoder>
            <Pattern>${CONSOLE_LOG_PATTERN}</Pattern>
            <!-- 设置字符集 -->
            <charset>UTF-8</charset>
        </encoder>
    </appender>
    <springProperty scope="context" name="plumelog.appName" source="plumelog.appName"/>
    <springProperty scope="context" name="plumelog.redisHost" source="plumelog.redisHost"/>
    <springProperty scope="context" name="plumelog.redisAuth" source="plumelog.redisAuth"/>
    <springProperty scope="context" name="plumelog.env" source="spring.config.activate.on-profile"/>

    <!-- plumelog日志 -->
    <appender name="plumelog" class="com.plumelog.logback.appender.RedisAppender">
        <appName>${plumelog.appName}</appName>
        <redisHost>${plumelog.redisHost}</redisHost>
        <redisAuth>${plumelog.redisAuth}</redisAuth>
        <env>${plumelog.env}</env>
    </appender>
    <!-- 日志输出级别 -->
    <root level="info">
        <appender-ref ref="CONSOLE"/>
        <appender-ref ref="plumelog"/>
    </root>
</configuration>

效果图

在plumelog目录下执行./startup.sh启动,如果出现Permission denied权限问题,输入chmod u+x *.sh .
如图所示成功接收两个应用的日志:
在这里插入图片描述

spring日志AOP

import com.zhengshuo.phoenix.model.entity.exception.BusinessException;
import com.zhengshuo.phoenix.web.annotation.Log;
import lombok.extern.slf4j.Slf4j;
import org.aspectj.lang.JoinPoint;
import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.Signature;
import org.aspectj.lang.annotation.AfterThrowing;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.annotation.Aspect;
import org.aspectj.lang.annotation.Pointcut;
import org.aspectj.lang.reflect.MethodSignature;
import org.springframework.stereotype.Component;
import org.springframework.web.context.request.RequestAttributes;
import org.springframework.web.context.request.RequestContextHolder;
import org.springframework.web.context.request.ServletRequestAttributes;

import javax.servlet.http.HttpServletRequest;
import java.lang.reflect.Method;
import java.util.Arrays;

@Aspect
@Component
@Slf4j
public class LogAspect {

    //用StringBuffer定位调用的controller层 定位第一行
    private static String getTargetStack(String tag) {
        StringBuilder sb = new StringBuilder();
        int number = tag.lastIndexOf(".");//找到controller名称中最后一个.的索引
        String controller = tag.substring(number + 1);//截取controller的名字
        sb.append(".(").append(controller).append(".java:1)");//定位在controller类里面的第一行
        return sb.toString();
    }

    // 配置织入点
    @Pointcut("@annotation(com.zhengshuo.phoenix.web.annotation.Log)")
    public void logPointCut() {
    }

    /**
     * @Description : 环绕方法,从请求开始到结束
     */
    @Around("logPointCut()")
    public Object doAround(ProceedingJoinPoint pjp) throws Throwable {
        long startTime = System.currentTimeMillis();
        RequestAttributes ra = RequestContextHolder.getRequestAttributes();
        ServletRequestAttributes sra = (ServletRequestAttributes) ra;
        HttpServletRequest request = sra.getRequest();
        String method = request.getMethod();//请求方式为GET/POST
        String controller = pjp.getTarget().getClass().getName();//请求的是哪个controller
        String uri = request.getRequestURI();//请求接口名
        Object result = pjp.proceed();//获得拦截方法的返回值
        long time = System.currentTimeMillis() - startTime;// 执行耗时
        // 获得注解
        Log controllerLog = getAnnotationLog(pjp);
        if (controllerLog != null && controllerLog.isPrintResult()) {
            log.info("\n" + "Uri         : " + uri + "\n" +
                    "Time        : " + time + "ms" + "\n" +
                    "Params      : " + Arrays.toString(pjp.getArgs()) + "\n" +
                    "Result      : " + result + "\n" +
                    "Token       : " + request.getHeader("Authorization") + "\n" +
                    "Method      : " + method + "\n" +
                    "Controller  : " + controller + getTargetStack(controller) + "\n");
        } else {
            log.info("\n" + "Uri         : " + uri + "\n" +
                    "Time        : " + time + "ms" + "\n" +
                    "Params      : " + Arrays.toString(pjp.getArgs()) + "\n" +
                    //"Result      : " + result + "\n" +
                    "Token       : " + request.getHeader("Authorization") + "\n" +
                    "Method      : " + method + "\n" +
                    "Controller  : " + controller + getTargetStack(controller) + "\n");
        }
        return result;
    }

    /**
     * @Description : 抛出异常时打印异常信息
     */
    @AfterThrowing(throwing = "e", pointcut = "logPointCut()")
    public void doAfterThrowing(JoinPoint point, Exception e) {

        long startTime = System.currentTimeMillis();
        RequestAttributes ra = RequestContextHolder.getRequestAttributes();
        ServletRequestAttributes sra = (ServletRequestAttributes) ra;
        HttpServletRequest request = sra.getRequest();
        String method = request.getMethod();//请求方式为GET/POST
        String controller = point.getTarget().getClass().getName();//请求的是哪个controller
        String uri = request.getRequestURI();//请求接口名
        long time = System.currentTimeMillis() - startTime;// 执行耗时
        if (e instanceof BusinessException) {
            log.warn("\n" + "Uri         : " + uri + "\n" +
                    "Time        : " + time + "ms" + "\n" +
                    "Params      : " + Arrays.toString(point.getArgs()) + "\n" +
                    "Method      : " + method + "\n" +
                    "Controller  : " + controller + getTargetStack(controller) + "\n" +
                    "Token       : " + request.getHeader("Authorization") + "\n" +
                    "Error       : " + e + "\n");
        } else {
            log.error("\n" + "Uri         : " + uri + "\n" +
                    "Time        : " + time + "ms" + "\n" +
                    "Params      : " + Arrays.toString(point.getArgs()) + "\n" +
                    "Method      : " + method + "\n" +
                    "Controller  : " + controller + getTargetStack(controller) + "\n" +
                    "Token       : " + request.getHeader("Authorization") + "\n" +
                    "Error       : " + e + "\n");
        }
    }

    /**
     * 是否存在注解，如果存在就获取
     */
    private Log getAnnotationLog(JoinPoint joinPoint) {
        Signature signature = joinPoint.getSignature();
        MethodSignature methodSignature = (MethodSignature) signature;
        Method method = methodSignature.getMethod();

        if (method != null) {
            return method.getAnnotation(Log.class);
        }
        return null;
    }

}

之一丶

关注

1
点赞
踩
2

收藏

觉得还不错? 一键收藏
打赏
0
评论
PlumeLog分布式日志

安装elasticsearchdocker pull elasticsearch:7.7.0运行docker run --name elasticsearch -d -e ES_JAVA_OPTS="-Xms512m -Xmx512m" -e “discovery.type=single-node” -p 9200:9200 elasticsearch:7.7.0配置plumelogapplication.properties:spring.application.name=plumelog_s
复制链接

扫一扫