filebeat-5.6.1 + logstash-5.6.2 + elasticsearch2.4 + spring-boot生成报表

最新推荐文章于 2024-07-17 09:36:37 发布
最新推荐文章于 2024-07-17 09:36:37 发布
阅读量1.6k 收藏
点赞数
分类专栏： 架构 文章标签： elasticsearch filebeat logstash spring-boot
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/lishizeng/article/details/78498081
版权

日志格式 
[INFO ] 2017-10-14 19:16:31 --> MyLog:/user/heart|{"appversion":"1.76","data":"{\"A\":\"\"}","deviceModel":"amodel",
"deviceVsn":"AiUI 1.0-1788 for amodel-dev","iP":"192.168.1.1","kernelVsn":
"3.10.0\sdfsdfsdsdfdsf@12333 #772\nTue Aug 9 14:36:31 CST 2016","operate":2,
"providerSoftVsn":"111111_ANDROID4.4.4-SDK_V1.0.0","usbInfoList":[],"uuid":"d7a9ae37-ddd123bb63-e1b49ed86f91","windowId":"26"}|1

filebeat配置 
###################### Filebeat Configuration Example #########################

# This file is an example configuration file highlighting only the most common
# options. The filebeat.full.yml file from the same directory contains all the
# supported options with more comments. You can use it as a reference.
#
# You can find the full configuration reference here:
# https://www.elastic.co/guide/en/beats/filebeat/index.html

#=========================== Filebeat prospectors =============================

filebeat.prospectors:

# Each - is a prospector. Most options can be set at the prospector level, so
# you can use different prospectors for various configurations.
# Below are the prospector specific configurations.

- input_type: log

  # Paths that should be crawled and fetched. Glob based paths.
  paths:
    - /usr/local/sbin/software/xxx/logs/web/info*.log
    #- c:\programdata\elasticsearch\logs\*

  # Exclude lines. A list of regular expressions to match. It drops the lines that are
  # matching any regular expression from the list.
  exclude_lines: ["(MyLog:?getChannlesUrl)|(MyLog:?list)|(MyLog:?uploadlog)"]

  # Include lines. A list of regular expressions to match. It exports the lines that are
  # matching any regular expression from the list.
  #include_lines: ["^ERR", "^WARN"]

  # Exclude files. A list of regular expressions to match. Filebeat drops the files that
  # are matching any regular expression from the list. By default, no files are dropped.
  #exclude_files: [".gz$"]

  # Optional additional fields. These field can be freely picked
  # to add additional information to the crawled log files for filtering
  #fields:
  #  level: debug
  #  review: 1

  ### Multiline options

  # Mutiline can be used for log messages spanning multiple lines. This is common
  # for Java Stack Traces or C-Line Continuation

  # The regexp Pattern that has to be matched. The example pattern matches all lines starting with [
  #multiline.pattern: ^\[

  # Defines if the pattern set under pattern should be negated or not. Default is false.
  #multiline.negate: false

  # Match can be set to "after" or "before". It is used to define if lines should be append to a pattern
  # that was (not) matched before or after or as long as a pattern is not matched based on negate.
  # Note: After is the equivalent to previous and before is the equivalent to to next in Logstash
  #multiline.match: after


#================================ General =====================================

# The name of the shipper that publishes the network data. It can be used to group
# all the transactions sent by a single shipper in the web interface.
#name:

# The tags of the shipper are included in their own field with each
# transaction published.
#tags: ["service-X", "web-tier"]

# Optional fields that you can specify to add additional information to the
# output.
fields:
  log_type: mylog

#================================ Outputs =====================================

# Configure what outputs to use when sending the data collected by the beat.
# Multiple outputs may be used.

#-------------------------- Elasticsearch output ------------------------------
#output.elasticsearch:
  # Array of hosts to connect to.
  # hosts: ["localhost:9200"]

  # Optional protocol and basic auth credentials.
  #protocol: "https"
  #username: "elastic"
  #password: "changeme"

#----------------------------- Logstash output --------------------------------
output.logstash:
  # The Logstash hosts
  hosts: ["192.168.30.248:15144"]

  # Optional SSL. By default is off.
  # List of root certificates for HTTPS server verifications
  #ssl.certificate_authorities: ["/etc/pki/root/ca.pem"]

  # Certificate for SSL client authentication
  #ssl.certificate: "/etc/pki/client/cert.pem"

  # Client Certificate Key
  #ssl.key: "/etc/pki/client/cert.key"

#================================ Logging =====================================

# Sets log level. The default log level is info.
# Available log levels are: critical, error, warning, info, debug
#logging.level: debug

# At debug level, you can selectively enable logging only for some components.
# To enable all selectors use ["*"]. Examples of other selectors are "beat",
# "publish", "service".
#logging.selectors: ["*"]

logstash[./etc/mylogstash .conf]配置 
input {
    beats {
        port => 15144
    }
}

filter {
	if [fields][log_type] == "mylog" {
		grok {
			patterns_dir => "/lunzn/software/report/my-report/logstash-5.6.2/etc/operate-pattern"
			match=>{"message"=>'\[INFO \] %{STR:logdate} --> MyLog:?%{STR:interfaceName}\|%{STR:datajson}\|'}
		}
		json {
			source => "datajson"
                        #target => "doc"
                        remove_field => ["datajson","message","source","@version","input_type","beat","host"]
		}
		json {
			source => "data"
                        #target => "doc"
			remove_field => ["data"]
		}
		ruby {
                        code => "event.set('datatype', event.get('type')); event.set('type', 'log')"
                } 
	}
        ruby {
                code => "event.set('timestamp', 
                event.get('@timestamp').time.localtime + 8*60*60); event.set('@timestamp', event.get('timestamp'))" 
		remove_field => ["@timestamp"]
        }
}

output {
	if [fields][log_type] == "mylog" {
		elasticsearch {
                    hosts => ["192.168.30.248:9200"]
                    index => "my-log"
                    manage_template => true
                    flush_size => 50000
                    idle_flush_time => 10
		}
	}
	if [fields][log_type] == "operate" {
		elasticsearch {
                    hosts => ["192.168.30.248:9200"]
                    index => "my-operate-log"
                    manage_template => true
                    flush_size => 50000
                    idle_flush_time => 10
		}
	}
}
















 

 

 
 


   logstash[./etc/operate-pattern]配置 
 
STR (.*?)
 

 

 
 


   相关命令启动 
 
>>启动elasticsearch：
#进入elasticsearch的bin目录
./elasticsearch -d

>>启动logstash：
#进入logstash目录
#启动（后台运行）logstash
nohup ./bin/logstash -f ./etc/xiaoyu.conf &

>>启动filebeat：
#进入filebeat目录
nohup ./filebeat -e -c filebeat.yml &
 

 

 
 


   spring-boot [pom.xml] 
 
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<groupId>cn.my.report</groupId>
	<artifactId>cn.my.report</artifactId>
	<packaging>war</packaging>
	<version>1.0</version>
	<name>cn.my.report</name>

	<properties>
		<spring.boot.version>1.5.7.RELEASE</spring.boot.version>
	</properties>

	<dependencies>

		<!-- spring-boot -->
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-dependencies</artifactId>
			<version>${spring.boot.version}</version>
			<type>pom</type>
			<scope>import</scope>
		</dependency>

		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-web</artifactId>
			<version>${spring.boot.version}</version>
		</dependency>

		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-data-elasticsearch</artifactId>
			<version>${spring.boot.version}</version>
		</dependency>

		<!-- 邮箱 -->
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-mail</artifactId>
			<version>${spring.boot.version}</version>
		</dependency>

		<!-- mybatis spring-boot -->
		<dependency>
			<groupId>org.mybatis.spring.boot</groupId>
			<artifactId>mybatis-spring-boot-starter</artifactId>
			<version>1.1.1</version>
		</dependency>

		<!-- mysql database driver -->
		<dependency>
			<groupId>mysql</groupId>
			<artifactId>mysql-connector-java</artifactId>
			<version>5.1.36</version>
		</dependency>

		<dependency>
			<groupId>org.elasticsearch</groupId>
			<artifactId>elasticsearch</artifactId>
			<version>2.4.3</version>
		</dependency>

		<!-- mongodb -->
		<!-- <dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-data-mongodb</artifactId>
			<version>${spring.boot.version}</version>
		</dependency> -->

		<!-- <dependency> <groupId>org.elasticsearch.plugin</groupId> <artifactId>delete-by-query</artifactId> 
			<version>2.4.3</version> </dependency> -->

		<dependency>
			<groupId>javax.servlet</groupId>
			<artifactId>javax.servlet-api</artifactId>
			<version>3.1.0</version>
			<scope>provided</scope>
		</dependency>

		<dependency>
			<groupId>com.alibaba</groupId>
			<artifactId>fastjson</artifactId>
			<version>1.2.7</version>
		</dependency>

		<dependency>
			<groupId>org.apache.poi</groupId>
			<artifactId>poi</artifactId>
			<version>3.17</version>
		</dependency>
	</dependencies>

	<build>
		<plugins>
			<plugin>
				<groupId>org.apache.maven.plugins</groupId>
				<artifactId>maven-war-plugin</artifactId>
				<version>2.6</version>
				<configuration>
					<warName>myreport</warName>
					<failOnMissingWebXml>false</failOnMissingWebXml>
				</configuration>
			</plugin>
		</plugins>
	</build>
</project>

 

 

 
 


   spring-boot[application.properties] 
 
 
    
 
     
server.port = 15050
server.context-path = /myreport
# ES
spring.data.elasticsearch.repositories.enabled = true
spring.data.elasticsearch.cluster-nodes = 192.168.30.248:9300
# 1800 = 30m   28800 = 8h
server.session-timeout = 1800
# server.error.path=/error
#spring.devtools.restart.enabled = false
#spring.devtools.restart.exclude = portal/**,theme/**
#spring.devtools.restart.additional-exclude
# xiaoyu database
spring.datasource.url=jdbc:mysql://192.168.30.253:3306/xxx?characterEncoding=utf-8&autoReconnect=true&failOverReadOnly=false&allowMultiQueries=true&useSSL=true
spring.datasource.username=root
spring.datasource.password=root
spring.datasource.driver-class-name=com.mysql.jdbc.Driver
mybatis.mapper-locations=classpath*:mybatis/*.xml
#mybatis.type-aliases-package=org.springboot.sample.entity
# email
spring.mail.host=smtp.exmail.qq.com
spring.mail.username=my_report@xxx.com
#POP3/SMTP service authorization code
spring.mail.password=123456
spring.mail.default-encoding=UTF-8
#mongodb
#spring.data.mongodb.host=192.168.30.253
#spring.data.mongodb.port=27017
#spring.data.mongodb.database=my_report
#spring.http.encoding.charset=utf8
# 接口请求格式:application/json、application/xml、text/html
#spring.freemarker.content-type=application/json;charset=UTF-8
 
     
 
     
 
    
 
 


   elasticsearch配置 
 
## 建索引（建库）
curl -XPOST "http://192.168.30.248:9200/my-log"

## 设置命中查询最大返回数
curl -XPUT http://192.168.30.248:9200/my-log/_settings/ -d '{ "index" : { "max_result_window" : 2147483647}}' 

## 设置type（建表）, "index" : "not_analyzed" 表示不分词，即字符串中含有特殊字符时，
## 如“aaa-bbb-ccc”，elasticsearch会将此字段的值分成"aaa","bbb","ccc"，故而查询不到“aaa-bbb-ccc”数据，
## 不过在代码中也可以指定该字段不分词查询：
## QueryBuilders.matchPhraseQuery(“uuid”, "xxxxxxx-xxxx-xxxxx").slop(0)
curl -XPOST "http://192.168.30.248:9200/my-log/user/_mapping?pretty" -d'
{
	"user" : {
		"properties" : {
			"uuid" : {
				"type" : "string",
				"index" : "not_analyzed"
			},
			"company" : {
				"type" : "string",
				"index" : "not_analyzed"
			},
			"coversion" : {
				"type" : "string",
				"index" : "not_analyzed"
			},
			"appversion" : {
				"type" : "string",
				"index" : "not_analyzed"
			},
			"logdate" : {
				"format" : "yyyy-MM-dd HH:mm:ss",
				"type" : "date"
			},
			"ip" : {
				"type" : "string",
				"index" : "not_analyzed"
			}
		}
	}
}'
 

 

 
 


   java es相关操作 
 
    /**
     * ES-数据源
     */
    @Autowired
    private ElasticsearchTemplate esTemplate;
    public Long dayBoot(String statDate)
    {
        Client client = esTemplate.getClient();
        // CardinalityBuilder 聚合某个字段，去重复得到总数，比如数据：张三、李四、张三、张三
        // 使用后得到 2,precisionThreshold表示提高准确率，取值0-40000，超过按40000表示，慎用CardinalityBuilder 很难100%准确
        CardinalityBuilder agg =
            AggregationBuilders.cardinality("uuid").field(Field.UUID).precisionThreshold(Integer.MAX_VALUE);
        // 分组
        // TermsBuilder appversion = AggregationBuilders.terms("appversion");
        
        // 创建查询块
        BoolQueryBuilder queryBuilder = QueryBuilders.boolQuery();
        
        // 仅查询当天 日志
        queryBuilder.must(QueryBuilders.rangeQuery("logdate")
            .format("yyyy-MM-dd HH:mm:ss")
            .gte(statDate + " 00:00:00")
            .lte(statDate + " 23:59:59"));
        
        // 前缀匹配
        queryBuilder.filter(QueryBuilders.prefixQuery(Field.INTERFACE_NAME, "/user/"));
        // 完全匹配 且 不分词
        // queryBuilder.filter(QueryBuilders.matchPhraseQuery(fieldname, fieldvalue).slop(0));
        
        //StringTerms aggAppversion = response.getAggregations().get("appversion");
        
        //for (Terms.Bucket appversion : aggAppversion.getBuckets())
        //{
        //   Cardinality agg = appversion.getAggregations().get("uuid");
        //   versionPV.put(appversion.getKeyAsString(), agg.getValue());
        //}
        SearchRequestBuilder search =
            client.prepareSearch(Constant.ES_MYLOG_INDEX).setTypes(Constant.ES_MYLOG_TYPE_USER).setSize(0);
        
        SearchResponse response = search.setQuery(queryBuilder).addAggregation(agg).get();
        
        Cardinality c = response.getAggregations().get("uuid");
        return c.getValue();
    }
    
    // 使用spring-boot集成查询删除方法
    DeleteQuery dq = new DeleteQuery();
    dq.setIndex(Constant.ES_MYLOG_INDEX);
    dq.setType(Constant.ES_MYLOG_TYPE_USER);
    dq.setQuery(QueryBuilders.matchAllQuery());
    esTemplate.delete(dq, User.class);
    // Report此类顶部必须设置：
    // @Document(indexName = Constant.ES_MYLOG_INDEX, type = "user", refreshInterval = "-1")
    // public class User
    
    // 使用原生查询删除方法,需要引用jar，此外还要ES安装批量删除插件，不然也无法删除
    // <dependency><groupId>org.elasticsearch.plugin</groupId><artifactId>delete-by-query</artifactId><version>2.4.3</version></dependency>
    Client client2 = TransportClient.builder().addPlugin(DeleteByQueryPlugin.class).build().addTransportAddress(
        new InetSocketTransportAddress(InetAddress.getByName("192.168.30.248"), 9300));
    DeleteByQueryResponse response1 = new DeleteByQueryRequestBuilder(client2, DeleteByQueryAction.INSTANCE)
            .setIndices(Constant.ES_INDEX)
            .setTypes(Constant.ES_TYPE_USER)
            .setQuery(QueryBuilders.matchAllQuery())
            .get();}

    /** 
     * 清理三个月以前的数据
     * @see [类、类#方法、类#成员]
     */
    public void clearThreeMonthsAgoLog()
    {
        // 三月之前
        Calendar threeMonthsAgoDate = Calendar.getInstance();
        threeMonthsAgoDate.add(Calendar.MONTH, -3);
        String threeMonthsAgo = DateUtil.formatDateToString(DateUtil.DATE_FORMAT_DAY_BAR, threeMonthsAgoDate.getTime());
        
        DeleteQuery dq = new DeleteQuery();
        dq.setIndex(Constant.ES_MYLOG_INDEX);
        dq.setType(Constant.ES_MYLOG_TYPE_UESR);
        
        BoolQueryBuilder queryBuilder = QueryBuilders.boolQuery();
        queryBuilder.must(QueryBuilders.rangeQuery(Field.LOG_DATE)
            .format(DateUtil.DATE_FORMAT_SECOND_BAR)
            .lte(DateUtil.endDate(threeMonthsAgo)));
        
        dq.setQuery(queryBuilder);
        
        // 标记删除
        esTemplate.delete(dq, User.class);
        
        try
        {
            // 标记删除之后,强制删除标记删除的文件
            esTemplate.getClient()
                .admin()
                .indices()
                .forceMerge(new ForceMergeRequest(Constant.ES_MYLOG_INDEX).onlyExpungeDeletes(true))
                .get();
            
            logger.info("[Scheduled] Clear 3 month ago logs success...");
        }
        catch (Exception ee)
        {
            logger.error("[Scheduled] Clear 3 month ago logs failed...");
        }
    }

 

 

 

 
 


   常用ES DDL语句，直接在linux中运行，es提供的wagnye查询太垃圾 
 
curl -XGET "http://192.168.30.253:9200/my-log/user/_search" -d'
{
  "size" : 0,
  "query" : {
    "bool" : {
      "must" : {
        "range" : {
          "logdate" : {
            "from" : null,
            "to" : "2017-10-15 23:59:59",
            "format" : "yyyy-MM-dd HH:mm:ss",
            "include_lower" : true,
            "include_upper" : true
          }
        }
      }
    }
  },
  "aggregations" : {
    "appversion" : {
      "terms" : {
        "field" : "appversion",
        "size" : 2147483647
      },
      "aggregations" : {
        "uuid" : {
          "cardinality" : {
            "field" : "uuid",
            "precision_threshold" : 2147483647
          }
        }
      }
    }
  }
}'

curl -XDELETE "http://192.168.30.248:9200/my-log/user/_query" -d'
{
"query": {
"match_all": {}
}
}'
curl -XGET "http://192.168.30.253:9200/my-log/log/_search" -d'
{
  "size" : 0,
  "query" : {
    "bool" : {
      "must" : {
        "range" : {
          "logdate" : {
            "from" : "2017-10-15 00:00:00",
            "to" : "2017-10-15 23:59:59",
            "format" : "yyyy-MM-dd HH:mm:ss",
            "include_lower" : true,
            "include_upper" : true
          }
        }
      },
      "filter" : [ {
        "match" : {
          "interfaceName" : {
            "query" : "/user/log_report",
            "type" : "phrase",
            "slop" : 0
          }
        }
      }, {
        "match" : {
          "opCommandType" : {
            "query" : "1",
            "type" : "phrase",
            "slop" : 0
          }
        }
      } ]
    }
  },
  "aggregations" : {
    "uuid" : {
      "cardinality" : {
        "field" : "uuid",
        "precision_threshold" : 2147483647
      }
    }
  }
}'
 

 

 


                

        

        

    

  


    

      

        

            

              
                
                  lishizeng
                
              
            

            

                关注
              
            

        

        

          
    
            
  • 
              
                
                
                
                
                      0
                
              
              
    点赞
    
            
    • 
            
  • 
              
                
                
                
              
              
    踩
    
            
    • 
            
  • 
              
                
                
                
                
                    0
                
              
              
    
                
    
                  
                    收藏
                  
                
    
              
    
              
    
                
    
                  觉得还不错?
                  
                    一键收藏
                  
                 
                
    
              
    
            
    • 
          
  • 
            
    
              
              
            
    
              
              
              
                  2
              
            
            
    评论
    
          
    • 
          
  • 
          
    • 
          
  • 
            
              
            
              
    
            
              
                
            
    
          
    • 
        

      

      

            

                专栏目录
          










                



	

		

			

				
					
FileBeat+Elasticsearch+Logstash+Kibana日志收集分析系统搭建

				
			

			

				

					canthny的专栏
				

				

					08-15
					
					1874
					
				

			

		

		

			
				
准备阶段

6.0以上版本需要jdk1.8，自行安装

wget https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.0.0-linux-x86_64.tar.gz

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.0.0...

			
		

	



                

              
                



	

		

			

				
					
STM32CubeMX-5.6.1+JRE64位-1.8.0.45.zip

				
			

			

				

					07-14
				

			

		

		

			
				
STM32CubeMX是STMicroelectronics（意法半导体）推出的一款强大的STM32微控制器配置和代码生成工具，它极大地简化了基于STM32微控制器的项目开发过程。5.6.1是STM32CubeMX的一个较新版本，发布于2020年7月14日，提供...

			
		

	



                


  
              

                


	

		

			

				
					
filebeat安装包

				
			

			

				

					11-21
				

			

		

		

			
				
Filebeat是一个日志文件托运工具，在你的服务器上安装客户端后，filebeat会监控日志目录或者指定的日志文件，追踪读取这些文件（追踪文件的变化，不停的读），并且转发这些信息到elasticsearch或者logstarsh中存放。

			
		

	





	

		

			

				
					
集算报表从elasticsearch取数验证

				
			

			

				

					rqgxy的博客
				

				

					12-02
					
					1002
					
				

			

		

		

			
				
Elasticsearch提供了API接口供其他程序调用里边数据，例如可以通过9200端口与 Elasticsearch 的 RESTful API进行通信，返回json格式数据，如直接在浏览器中访问：http://localhost:9200/，则返回数据为：


Json格式数据做为常见的数据交换格式，润乾集算器可以很简便的对其进行解析及加工计算，并且数据来源可以是json文件，也可以来

			
		

	



		

			
		



	

		

			

				
					
基于SpringBoot+Vue的智能报表系统设计和实现(源码+LW+部署讲解)

				
			

			

				

					java李阳勇的博客
				

				

					07-17
					
					1601
					
				

			

		

		

			
				
程序上交给用户进行使用时，需要提供程序的操作流程图，这样便于用户容易理解程序的具体工作步骤，现如今程序的操作流程都有一个大致的标准，即先通过登录页面提交登录数据，通过程序验证正确之后，用户才能在程序功能操作区页面操作对应的功能。​程序操作流程图首先前端通过Vue和axios发送HTTP请求到后端的登录接口。在后端接收登录请求的Controller会使用`@RequestParam Map params`来接收前端传递的用户参数，用户名和密码。

			
		

	





	

		

			

				
					
Springboot/Springcloud整合ELK平台，（Filebeat方式）日志采集及管理（Elasticsearch+Logstash+Filebeat+Kibana）

				
			

			

				

					weixin_43627706的博客
				

				

					04-27
					
					8138
					
				

			

		

		

			
				
前言
最近在搞一套完整的云原生框架，详见 spring-cloud-alibaba专栏，目前已经整合的log4j2，但是想要一套可以实时观察日志的系统，就想到了ELK，然后上一篇文章是socket异步发送给logstash，logstash再输出到elasticsearch索引库中。
logstash是java应用，解析日志是非的消耗cpu和内存，logstash安装在应用部署的机器上显得非常的笨重。最常见的做法是用filebeat部署在应用的机器上，logstash单独部署，然后由filebeat将日志输

			
		

	





	

		

			

				
					
Spring Boot 日志平台 ELK + Filebeat 入门

				
			

			

				

					weixin_42073629的博客
				

				

					06-16
					
					2016
					
				

			

		

		

			
				
1. 概述

在《芋道 Spring Boot 日志集成 Logging 入门》文章中，我们学习了 Spring Boot 如何集成并使用日志框架，进行日志的打印。


友情提示：对于本文来说，即使胖友未阅读过《芋道 Spring Boot 日志集成 Logging 入门》文章，也毫无影响，不要慌~


不过正如我们在文末所说，在生产环境下，我们会集群部署我们的应用。那么我们可能需要登陆多台服务器，查看不用应用节点下的日志，这样会非常不方便。

所以，本文我们就一起来，使用 ELK + Filebeat 组

			
		

	





	

		

			

				
					
Springboot + docker + ELK +Filebeat实现日志收集与展示

				
			

			

				

					皮卡丘的情绪
				

				

					05-11
					
					6759
					
				

			

		

		

			
				
文章目录1、ELKB架构2、本机环境3、ELKB环境搭建3.1、创建自定义网络3.2、elasticsearch环境配置3.2.1、创建elasticsearch目录3.2.2、配置elasticsearch.yml文件3.3、Kibana环境配置3.3.1、创建kibana目录3.3.2、kibana.yml文件配置3.4、Logstash配置3.4.1、Logstash目录创建3.4.2、logstash.yml和logstash.conf文件配置3.5、Filebeat配置3.5.1、Filebeat

			
		

	





	

		

			

				
					
QTOpenCV的安装与配置(win7-64+Qt-5.6.1+opencv-3.1.0)

					
最新发布

				
			

			

				

					08-11
				

			

		

		

			
				
qt下载，QTOpenCV的安装与配置(win7-64+Qt-5.6.1+opencv-3.1.0)

			
		

	





	

		

			

				
					
innosetup-5.6.1+中文语言包

				
			

			

				

					08-24
				

			

		

		

			
				
《InnoSetup 5.6.1：打造专业级别的安装程序》  InnoSetup是一款广泛应用于软件安装包制作的工具，其5.6.1版本是该系列的一个经典版本，深受开发者喜爱。这款软件提供了丰富的功能，使得创建具有专业水准的安装程序...

			
		

	





	

		

			

				
					
spring-security-core-5.6.1-API文档-中英对照版.zip

				
			

			

				

					05-03
				

			

		

		

			
				
赠送jar包：spring-security-core-5.6.1.jar； 赠送原API文档：spring-security-core-5.6.1-javadoc.jar； 赠送源代码：spring-security-core-5.6.1-sources.jar； 赠送Maven依赖信息文件：spring-security-core-...

			
		

	





	

		

			

				
					
日志检测-filebeat+logstash-整理.docx

				
			

			

				

					07-24
				

			

		

		

			
				
filebeat收集日志，转发给logstash过滤整理成json.再转发给socket server处理业务逻辑

			
		

	





	

		

			

				
					
SpringBoot继承LogStash实现日志收集的方法示例

				
			

			

				

					08-26
				

			

		

		

			
				
主要介绍了SpringBoot继承LogStash实现日志收集的方法示例，小编觉得挺不错的，现在分享给大家，也给大家做个参考。一起跟随小编过来看看吧

			
		

	





	

		

			

				
					
SpringBoot非官方教程 | 第二十六篇： sprinboot整合elk，搭建实时日志平台

					
热门推荐

				
			

			

				

					方志朋的专栏
				

				

					05-04
					
					7万+
					
				

			

		

		

			
				
这篇文章主要介绍springboot整合elk.elk 简介
Elasticsearch是个开源分布式搜索引擎，它的特点有：分布式，零配置，自动发现，索引自动分片，索引副本机制，restful风格接口，多数据源，自动搜索负载等。
Logstash是一个完全开源的工具，他可以对你的日志进行收集、过滤，并将其存储供以后使用（如，搜索）。
Kibana 也是一个开源和免费的工具，它Kibana可以为 Lo

			
		

	





	

		

			

				
					
ELK之使用Filebeat插件收集日志到Logstash

				
			

			

				

					zuihongyan518的博客
				

				

					03-14
					
					1721
					
				

			

		

		

			
				
Filebeat是转发和收集数据的轻量级工具，可以监视指定的日志文件或位置，收集日志，并将数据转发到Elasticsearch或Logstash进行存储

			
		

	





	

		

			

				
					
springboot 日志

				
			

			

				

					我家猫叫奥利奥
				

				

					03-24
					
					1707
					
				

			

		

		

			
				
日志默认模板

如果使用pom里面引用了”Starter POMs“的架包，将会使用logback作为日志框架




2016-03-24 15:22:06.991  INFO 13780 --- [           main] s.b.c.e.t.TomcatEmbeddedServletContainer : Tomcat initialized with port(s): 8080 (

			
		

	





	

		

			

				
					
filebeat+kafka+clickhouse+springboot框架搭建及应用

				
			

			

				

					Dcj695632854的博客
				

				

					07-18
					
					1156
					
				

			

		

		

			
				
提前准备好kafka、clickhouse环境。

			
		

	





	

		

			

				
					
ELK+Filebeat+SpringBoot搭建日志分析系统

				
			

			

				

					zhowyunfat的博客
				

				

					12-20
					
					3300
					
				

			

		

		

			
				
ELK安装与启动
ElasticSearch安装
使用Docker进行安装
执行以下命令：
docker run \
--name elasticsearch \
-p 9200:9200 -p 9300:9300 \
-e  "discovery.type=single-node" \
-e ES_JAVA_OPTS="-Xms64m -Xmx2048m" \
-d elasticsearch:7.14.0

将容器内的配置文件拷贝到本地
拷贝的命令：
docker cp 容器名:容器中要拷贝的文件名及其

			
		

	





	

		

			

				
					
ELK教程3：logstash的部署、SpringBoot整合ELK+Filebeat

				
			

			

				

					方志朋的专栏
				

				

					08-03
					
					8648
					
				

			

		

		

			
				
转载请标明出处：
http://blog.csdn.net/forezp/article/details/94343671
本文出自方志朋的博客

本篇文章主要讲解如下安装Logstash，logstash依赖于Java环境，首先安装Java，安装脚本如下：
yum install java

logstash安装
Logstash的安装脚本如下：
# 下载logstash的压缩包
wget h...

			
		

	





	

		

			

				
					
NB-IoT与eMTC单用户定点吞吐量对比测试分析

				
			

			

				

					
				

			

		

		

			
				
在5.6.1章节中，重点在于单用户上行吞吐量性能，这是衡量设备上传数据能力的关键指标。  测试编号5.6.1关注的是在不同覆盖场景下，单用户的定点上行吞吐量性能。这一测试的目的是验证NB-IoT与eMTC在相同物理测试点下...

			
		

	



              




          




        



    





    



      

      

        
      

      





	

		
评论 2

	

	
	




  

    

      添加红包
      
    

    

      

        
        

          
          
        

        
请填写红包祝福语或标题

      

      

        
        

          
          个
        

        
红包个数最小为10个

      

      

        
        

          
          元
        

        
红包金额最低5元

      

      

        
        

          当前余额3.43元
          前往充值 >
        

      

      

        

          需支付：10.00元
        

        
        
      

    

  





  

    
    
  

  

    
    
  








  

    

      

        

          

            
            
成就一亿技术人!

          

          

        

        

          

          

            领取后你会自动成为博主和红包主的粉丝
            规则
          

        

      

      

        

          

            
              
            
            

              
hope_wisdom
 发出的红包
            

          

        

        

          

          

          

        

      

    

    

  



      
      

      
实付元

      
使用余额支付

      

        

          

            
            点击重新获取
          

        

        
扫码支付

      

      

        
          
            
          
          
        
      

      

        
        钱包余额
          0
          

            
            

              

                
抵扣说明：

                
 1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
 2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

              

            

          

      

      余额充值