# SpringBoot Aop、Kafka、ELK收集日志

SpringBoot Aop、Kafka、ELK收集日志

---

SpringBoot配置Kafka

1.引入Kafka依赖

<dependency>
	<groupId>org.springframework.kafka</groupId>
	<artifactId>spring-kafka</artifactId>
</dependency>

2.Application.yml配置Kafka如下：

 #kafka配置
 spring:
   kafka:
    bootstrap-servers: 127.0.0.1:9092
    producer:
      key-serializer: org.apache.kafka.common.serialization.StringSerializer
      value-serializer: org.apache.kafka.common.serialization.StringSerializer
    consumer:
      group-id: test #Kafka创建的主题
      enable-auto-commit: true
      key-deserializer: org.apache.kafka.common.serialization.StringDeserializer
      value-deserializer: org.apache.kafka.common.serialization.StringDeserializer

3.配置完KafKa后启动项目，kafkak服务需要启动，需要启动生产者和操作者服务

---

SpringAop日志模块

1.SpringBoot引入Aop的Maven依赖

 <!--Spring AOP日志所需要的包-->
 <dependency>
	 <groupId>org.springframework.boot</groupId>
	 <artifactId>spring-boot-starter-aop</artifactId>
 </dependency>

2.定义注解@System如下：

@Target({ElementType.METHOD, ElementType.TYPE})
@Retention(RetentionPolicy.RUNTIME)
public @interface SystemLog {
}

3.对带有@System注解的方法进行处理如下：切面实现

import com.alibaba.fastjson.JSONObject;
import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.annotation.Aspect;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.core.Ordered;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.stereotype.Component;
import org.springframework.web.context.request.RequestContextHolder;
import org.springframework.web.context.request.ServletRequestAttributes;
import javax.servlet.http.HttpServletRequest;
import java.text.SimpleDateFormat;
import java.util.Date;

@Aspect
@Component
public class LogInterceptor implements Ordered{

	//注入KafKa操作类
    @Autowired
    private KafkaTemplate kafkaTemplate;

    @Around("@annotation(systemLog)")
    public Object Log(ProceedingJoinPoint joinPoint,SystemLog systemLog){

        Object result = null;
        try {
            if (joinPoint == null) {
                return null;
            }
            JSONObject message = new JSONObject();
            HttpServletRequest request = ((ServletRequestAttributes) RequestContextHolder.getRequestAttributes()).getRequest();
            Date now = new Date();
            SimpleDateFormat dateFormat = new SimpleDateFormat("yyyy/MM/dd HH:mm:ss");
            //可以方便地修改日期格式
            message.put("time", dateFormat.format(now));
            //时间
            message.put("requestURL", request.getRequestURL().toString());
            message.put("params",request.getQueryString());
            //参数
            result =joinPoint.proceed();
            message.put("return", result);
            //返回结果
            message.put("requestMethod", joinPoint.getSignature().getName());
            message.put("class", joinPoint.getTarget().getClass().getName());
            //发送JSONString到Kafka消费者
            kafkaTemplate.send("test",message.toJSONString());
            System.out.println("message:" + message.toJSONString());
        } catch (Throwable throwable) {
            throwable.getMessage();
        }
        return result;
    }
    @Override
    public int getOrder() {
        return 0;
    }
}

4.给接口的实现类中的方法加上注解

//Kotlin语言写的
@SystemLog
override fun getUserList(req: HttpServletRequest): RetObj {
	try {
		val sql = "select * from t_user where 1=1 and is_delete=1"
		LogUtils.addLogs(req,"获取用户列表")
		return RetObj(true, GetPageList(req, sql, null))
	} catch (e: Exception) {
		e.printStackTrace()
		return RetObj(false, e.message)
	}
}

5.发送的Kafka消息内容Json如下：

{"requestURL":"http://localhost:8003/user/getUserList","ip":"127.0.0.1","requestMethod":"getUserList","time":"2020/04/17 22:36:56","class":"com.li.bigdatacar.service.user.UserServiceImpl","return":{"code":200,"data":{"firstPage":true,"lastPage":true,"list":[{"password":"21232f297a57a5a743894a0e4a801fc3","alter_people":"admin","role_id":1,"id":1,"account":"admin","username":"admin","is_delete":1,"alter_time":1581586148000},{"password":"098f6bcd4621d373cade4e832627b4f6","alter_people":"admin","role_id":2,"id":2,"account":"test","username":"test","is_delete":1,"alter_time":1581683686000},{"password":"8f30526caf7218250c2549e996ac2083","alter_people":"test","role_id":3,"id":3,"account":"cgs","username":"cgs","is_delete":1,"alter_time":1581683886000},{"password":"0d3a7f39161e53007695c84c3e8b94e2","alter_people":"admin","role_id":2,"id":4,"account":"kakou1","username":"kakou1","is_delete":1,"alter_time":1581752317000},{"password":"5cfee4bf96bc45c81de6a0b385dbdf5f","alter_people":"admin","role_id":4,"id":15,"account":"police1","username":"李某某","is_delete":1,"alter_time":1584363732000}],"pageNumber":1,"pageSize":10,"totalPage":1,"totalRow":5}}}

6.在Kafak的消费者服务中可以看到发来的消息则Kafka消息发送成功，接下来要进行同步Kafka中的消息到Es中。

---

Logstash同步Kafka消费者中的消息到Es

1.Logstash的conf文件下新建kafkaToEs.conf文件，内容如下：

input {
  kafka {
    bootstrap_servers  => "127.0.0.1:9092"
    #主题名称
    topics => ["test"]
   }
}
filter{
	json{
		source => "message"
	}
}
output {
  elasticsearch {
    hosts  => "127.0.0.1:9200"
	#Es中的索引为system_log
    index  => "system_log"
    codec => "json"
  }
}

2.启动Elasticsearch
3.在logstash安装目录下启动Logstash

start /d "F:\Logstah\logstash-7.4.0\logstash-7.4.0\bin" logstash  -f  F:\Logstah\logstash-7.4.0\logstash-7.4.0\config\kafkaToEs.conf

4.配置Kibana的文件，启动Kibana,查看system_log如下