ELK高级搜索一入门

ELK介绍

ELK是什么？

ELK是一个免费开源的日志分析架构技术栈总称，官网https://www.elastic.co/cn。包含三大基础组件，分别是Elasticsearch、Logstash、Kibana。Logstash用于数据抽取。Elasticsearch是搜索分析。Kibana是数据展现。

ELK架构

组件介绍 

Elasticsearch

是使用java开发，基于Lucene、分布式、通过Restful方式进行交互的近实时搜索平台框架。它的特点有：分布式，零配置，自动发现，索引自动分片，索引副本机制，restful风格接口，多数据源，自动搜索负载等。下载地址：Download Elastic Enterprise Search | Elastic

•    启动ES

yangyanping@ZBMac-WP2HJYDWY bin % ./elasticsearch
[2022-05-29T19:57:27,899][INFO ][o.e.n.Node               ] [ZBMac-WP2HJYDWY] version[7.10.1], pid[26672], build[default/tar/1c34507e66d7db1211f66f3513706fdf548736aa/2020-12-05T01:00:33.671820Z], OS[Mac OS X/10.16/x86_64], JVM[AdoptOpenJDK/OpenJDK 64-Bit Server VM/15.0.1/15.0.1+9]
[2022-05-29T19:57:27,905][INFO ][o.e.n.Node               ] [ZBMac-WP2HJYDWY] JVM home [/Users/yangyanping/Downloads/server/es7/jdk.app/Contents/Home], using bundled JDK [true]

•  检查ES是否启动成功

GET  http://localhost:9200/?Pretty


{
  "name" : "ZBMAC-15aba3b68",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "ZyNrs0u7SmSAMHDLaDl3WA",
  "version" : {
    "number" : "7.10.1",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "1c34507e66d7db1211f66f3513706fdf548736aa",
    "build_date" : "2020-12-05T01:00:33.671820Z",
    "build_snapshot" : false,
    "lucene_version" : "8.7.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

•  查询集群状态请求 

GET http://localhost:9200/_cluster/health?pretty

{
  "cluster_name" : "elasticsearch",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 1,
  "number_of_data_nodes" : 1,
  "active_primary_shards" : 6,
  "active_shards" : 6,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 100.0
}

Logstash

Logstash 基于java开发，是一个数据抽取转化工具。一般工作方式为c/s架构，client端安装在需要收集信息的主机上，server端负责将收到的各节点日志进行过滤、修改等操作，并发往elasticsearch或其他组件上去。

Kibana 

 Kibana 基于nodejs，也是一个开源和免费的可视化工具。Kibana可以为 Logstash 和 ElasticSearch 提供的日志分析友好的 Web 界面，可以汇总、分析和搜索重要数据日志。下载地址：Download Kibana Free | Get Started Now | Elastic

• 启动 Kibana

yangyanping@ZBMac-WP2HJYDWY bin % sudo ./kibana --allow-root
[2022-04-13T14:47:41.163+08:00][INFO ][plugins-service] Plugin "metricsEntities" is disabled.
[2022-04-13T14:47:41.270+08:00][INFO ][http.server.Preboot] http server running at http://localhost:5601
[2022-04-13T14:47:41.376+08:00][INFO ][plugins-system.preboot] Setting up [1] plugins: [interactiveSetup]
[2022-04-13T14:47:41.381+08:00][INFO ][preboot] "interactiveSetup" plugin is holding setup: Validating Elasticsearch connection configuration…
[2022-04-13T14:47:41.431+08:00][INFO ][root] Holding setup until preboot stage is completed.


i Kibana has not been configured.

Go to http://localhost:5601/?code=919355 to get started.

• Kibana的访问

       通过地址：http://localhost:5601/app/dev_tools#/console可以在控制台中发送请求到elastic。

      

        如：发送get请求，查看集群状态GET _cluster/health

• 快速查看集群中有哪些索引 ，发送请求：GET  /_cat/indices?v     
•  创建索引

elasticsearch-head

node安装

下载地址：Download | Node.js

yangyanping@ZBMac-WP2HJYDWY ~ % brew install node
Running `brew update --preinstall`...
==> Auto-updated Homebrew!
Updated 2 taps (homebrew/core and homebrew/cask).
==> Updated Formulae
Updated 73 formulae.
==> Updated Casks
Updated 8 casks.
..........

elasticsearch-head安装 

下载地址：GitHub - mobz/elasticsearch-head: A web front end for an elastic search cluster

yangyanping@ZBMac-WP2HJYDWY elasticsearch-head-master % npm install
npm WARN EBADENGINE Unsupported engine {
npm WARN EBADENGINE   package: 'karma@1.3.0',
npm WARN EBADENGINE   required: { node: '0.10 || 0.12 || 4 || 5 || 6' },
npm WARN EBADENGINE   current: { node: 'v18.2.0', npm: '8.9.0' }
.........


yangyanping@ZBMac-WP2HJYDWY elasticsearch-head-master % npm run start

elasticsearch 配置修改和重启

打开ES的配置文件elasticsearch.yml输入下面的两句

http.cors.enabled: true
http.cors.allow-origin: "*"

vi elasticsearch.yml


# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true


http.cors.enabled: true
http.cors.allow-origin: "*"

 插件运行情况如下截图

 访问地址：http://localhost:9100/

参考：

ELK集中化日志解决方案