Spring使用注解发送端口日志到ElasticSearch8.3.2

沈阳-施立

已于 2022-07-16 13:12:13 修改

阅读量770

点赞数 1

分类专栏： SpringBoot 文章标签： spring elasticsearch 搜索引擎 kibana java

于 2022-07-16 13:07:23 首次发布

本文链接：https://blog.csdn.net/winnershili/article/details/125818320

版权

SpringBoot 专栏收录该内容

17 篇文章 0 订阅

订阅专栏

前言

版本介绍

CentOS： CentOS Linux release 8.0.1905 (Core)

JDK：java version "1.8.0_211"

ElasticSearch: 8.3.2

Kibana：8.3.2

LogStash：8.2

配置ElasticSearch让其支持SSL

1.安装好ElasticSearch后使用如下命令生成一个证书

./bin/elasticsearch-certutil ca

证书的文件名称为elastic-stack-ca.p12

2.根据证书生成一个certificates文件

./elasticsearch-certutil cert --ca elastic-stack-ca.p12

可以获得名字为elastic-certificates.p12的文件

3.最后生成SSL用到的公钥以及PEM文件

./bin/elasticsearch-certutil http

是否生成CSR？

Generate a CSR? [y/N]n

是否使用已经存在的CA证书

Use an existing CA? [y/N]y

输入CA证书的路径

CA Path: /usr/share/elasticsearch/elastic-stack-ca.p12

是否为每个检点生成证书？

Generate a certificate per node? [y/N]y

输入域名

    Enter all the hostnames that you need, one per line.
    When you are done, press <ENTER> once more to move on to the next step.
        es.xxx.com
    Is this correct [Y/n]Y

输入IP地址这里最好输入外网和内网两个IP地址

Enter all the IP addresses that you need, one per line. When you are done, press <ENTER> once more to move on to the next step.

xxx.xx.xx.xx

xx.xx.xx.xx

Is this correct [Y/n]Y

接下来就是询问是否要求密码之类的按照要求来

最后会得到一个名字为elasticsearch-ssl-http.zip的压缩包

解压缩后得到两个文件夹一个ElasticSearch里面有一个http.p12文件

这个JAVA连接会用到

另外一个是Kibana文件夹里面有一个 elasticsearch-ca.pem

这个Kibana和logstash都会用到

将第2步生成的elastic-certificates.p12文件和第3步的http.p12文件拷贝到 ElasticSearch配置文件夹中

修改ElasticSearch.yml配置文件一定要注意证书的路径是否正确

#外网IP
network.publish_host: 

xpack.security.http.ssl:
  enabled: true
  keystore.path: "certs/http.p12"
  truststore.path: "certs/http.p12"

xpack.security.enabled: true


xpack.security.transport.ssl:
  enabled: true
  verification_mode: certificate
  keystore.path: certs/elastic-certificates.p12
  truststore.path: certs/elastic-certificates.p12

     修改证书的密码(如果提示证书密码错误的话就这么做)

./bin/elasticsearch-keystore add xpack.security.transport.ssl.keystore.secure_password
./bin/elasticsearch-keystore add xpack.security.transport.ssl.truststore.secure_password
./bin/elasticsearch-keystore add xpack.security.http.ssl.keystore.secure_password
./bin/elasticsearch-keystore add xpack.security.http.ssl.truststore.secure_password

重启后完成ElasticSearch的配置

配置Kibana和LogStash连接ElasticSearch

这两个配置基本差不多

首先将我生成的elasticsearch-ca.pem 文件分别拷贝到Kibanan和LogStash配置路径

分别修改配置文件如下

kibana.yml 这里的IP地址一般使用内网地址，要和你生成pem时的一样

elasticsearch.username: "kibana"
elasticsearch.password: "xxxxxxx"
elasticsearch.ssl.certificateAuthorities: ["/etc/kibana/certs/elasticsearch-ca.pem"]
elasticsearch.hosts: ["https://172.24.209.49:9200"]
server.port: 5601

logstash.yml 的配置文件

xpack.monitoring.enabled: true
xpack.monitoring.elasticsearch.username: logstash_system
xpack.monitoring.elasticsearch.password: xxxxxxxxxxxx

#这里必须用 https 这里必须用 https 这里必须用 https
xpack.monitoring.elasticsearch.hosts: "https://172.24.209.49:9200"
#你的ca.pem 的所在路径
xpack.monitoring.elasticsearch.ssl.certificate_authority: "/etc/logstash/certs/elasticsearch-ca.pem"
xpack.monitoring.elasticsearch.ssl.verification_mode: certificate
# 探嗅 es节点，设置为 false
xpack.monitoring.elasticsearch.sniffing: false

logstash的插件可以这样配置

  elasticsearch {
    hosts => ["https://172.24.209.49:9200"]
    index => "[seata]"
    cacert => "/etc/logstash/certs/elasticsearch-ca.pem"
    user => "elastic"
    password => "xxxxxxxxxxx"
  }

使用JAVA注解将接口日志提交到ElasticSearch

配置一个RestClient 这里要注意需要使用到上面生成的http.p12文件 IP地址使用的是外网IP

@Configuration
public class LogConfig {

    @Bean(destroyMethod = "close")
    public RestClient restClient() throws CertificateException, IOException, KeyStoreException, NoSuchAlgorithmException, KeyManagementException, InvalidKeyException, NoSuchProviderException, SignatureException {

        final CredentialsProvider credentialsProvider =
                new BasicCredentialsProvider();
        credentialsProvider.setCredentials(AuthScope.ANY,
                new UsernamePasswordCredentials("elastic", "xxxxxxx=X6b"));

        KeyStore keyStore = KeyStore.getInstance("pkcs12");
        //InputStream is = this.getClass().getClassLoader().getResourceAsStream(elasticsearchProperties.getPkcsClientFilePath())
        InputStream is = new FileInputStream("d:\\certs\\http.p12");
        keyStore.load(is,null);

        SSLContextBuilder sslContextBuilder = SSLContexts.custom().loadTrustMaterial(keyStore, (x509Certificates, s) -> true);
        final SSLContext sslContext = sslContextBuilder.build();
        InetAddress ia2=InetAddress.getByName("218.2.2.2");
        RestClientBuilder builder = RestClient.builder(new HttpHost(ia2, 9200, "https"));
        //RestClientBuilder builder = RestClient.builder(new HttpHost("218.2.2.2", 9200, "https"));
        builder.setHttpClientConfigCallback(httpClientBuilder -> {
            httpClientBuilder.setSSLContext(sslContext);
            httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider);
            return httpClientBuilder;
        });

        return builder.build();
    }

}

创建一个注解

/**
 * 系统日志注解 使用这个注解可以增加一段描述
 */
@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
@Documented
public @interface SysLogTag {

    String value() default "";
    boolean recordParams() default true;
}

写一个注解切面

@Aspect
@Component
public class SysLogTagAspect {

    @Autowired
    RestClient restClient;

    @Pointcut("@annotation(com.fangsheng.common.log.annotation.SysLogTag)")
    public void logPointCut() {

    }

    @Around("logPointCut()")
    public Object around(ProceedingJoinPoint point) throws Throwable {
        long beginTime = System.currentTimeMillis();
        //执行方法
        Object result = point.proceed();
        //执行时长(毫秒)
        long time = System.currentTimeMillis() - beginTime;

        //保存日志
        saveSysLog(point, time);

        return result;
    }

    private void saveSysLog(ProceedingJoinPoint joinPoint, long time) {
        MethodSignature signature = (MethodSignature) joinPoint.getSignature();
        Method method = signature.getMethod();

        SysLog sysLog = new SysLog();
        SysLogTag syslog = method.getAnnotation(SysLogTag.class);
        if(syslog != null){
            //注解上的描述
            sysLog.setOperation(syslog.value());
        }

        //请求的方法名
        String className = joinPoint.getTarget().getClass().getName();
        String methodName = signature.getName();
        sysLog.setMethod(className + "." + methodName + "()");

        ObjectMapper mapper = new ObjectMapper();


        //请求的参数
        if(syslog.recordParams()){
            StringBuffer sb = new StringBuffer();
            Object[] args = joinPoint.getArgs();
            for(Object object:args){
                try{
                    //使用GSON时 会出现一些 死循环, 如果非要使用 可以给Entity增加注解 mark by shili at 2018-9-12
                    //String params = new Gson().toJson(args[0]);
                    if(mapper.writeValueAsString(object).length()<1000){
                        sb.append(mapper.writeValueAsString(object));
                        sb.append(",");
                    }else{
                        sb.append("bob");
                        sb.append(",");
                    }
                }catch (Exception e){

                }
            }
            String params = sb.toString();
            params = params.replaceAll("[\ud800\udc00-\udbff\udfff\ud800-\udfff]","");
            sysLog.setParams(params);
        }

        //sysLogEntity.setParams(joinPoint.getArgs().toString());

        //获取request
        HttpServletRequest request = ((ServletRequestAttributes) RequestContextHolder.getRequestAttributes()).getRequest();
        //设置IP地址
        sysLog.setIp(EzWebUtils.getIpAddr(request));
        //sysLogEntity.setIp("");

        //用户名
        String username = "guest";
        String uuid = "";


        LoginUser loginUser = SecurityUtils.getLoginUser();
        if(null!=loginUser) {
            uuid = loginUser.getUserId()+"";
            username = loginUser.getUsername();


        }
        sysLog.setUuid(uuid);
        sysLog.setUsername(username);

        sysLog.setTime(time);
        sysLog.setCreateDate(new Date());

        sysLog.setTid("11111");
        //TODO shili 固定的TID

        //保存系统日志
        System.out.println(sysLog.toString());


        new Thread(() -> {
            try{

                Request elasticSearchRequest = new Request("PUT","/sys_log/_doc/"+UUID.randomUUID().toString());
                elasticSearchRequest.setJsonEntity(JSON.toJSONString(sysLog));
                restClient.performRequest(elasticSearchRequest);

//                Request request2 = new Request("GET","/ess_book/_doc/1");
//                Response response = restClient.performRequest(request2);
//                System.out.print(EntityUtils.toString(response.getEntity()));

            } catch (IOException e) {
                e.printStackTrace();
            }
        }).start();

    }
}