ELK系列（三） Kibana8.9.2搭建

ELK系列传送门

ELK系列（一） ElasticSearch 8.9.2集群搭建

ELK系列（二）Kafka集群3.4.0搭建

ELK系列（三） Kibana8.9.2搭建​​​​​​

ELK数据流规划

ELK接入nginx-acces日志数据流规划


数据流向
		filebeat -------> kafka ----> logstash ----> ES --------> kibana
		 收集              mq          处理日志     存储、搜索     展示、分析
                                                       |
										               |
								                   elastalert2---钉钉告警


ES集群搭建已完成 可见传送门ELK系列（一） ElasticSearch 8.9.2集群搭建
Kafka集群搭建已完成 可以传送门ELK系列（二） Kafka集群3.4.0搭建

主机规划

主机名        内网地址      搭建程序

ELK8-1       192.168.0.1  elasticsearch+kafka+zookeeper
ELK8-2       192.168.0.2  elasticsearch+kafka+zookeeper
ELK8-3       192.168.0.3  elasticsearch+kafka+zookeeper
logstash1    192.168.0.4  kibana+logstash

ELK三台主机规划完成 接下来在logstash1上搭建kibana

kibana搭建

LVM目录挂载设置
fdisk /dev/vdc
pvcreate /dev/vdc1
vgcreate vg1 /dev/vdc1
lvcreate -l 25599 -n lv1 vg1
mkfs.xfs /dev/vg1/lv1
mkdir /data
mount /dev/vg1/lv1 /data
mkdir /data/{kibana,logstash}
mkdir -p /data/kibana/{data,logs}
mkdir -p /data/logstash/{data,logs}

安装Kibana
kibana的yum源包含在elastic.repo中已在前文安装
yum install kibana -y
systemctl daemon-reload
systemctl enable kibana
chown kibana:kibana /data/kibana/ -R

访问kibana开启https 自建证书
openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -days 999999
证书密码自定义
chmod 777 cert.pem
chmod 777 key.pem 

[root@logstash-1 kibana]# pwd 
/etc/kibana  #yum安装后的kibana默认目录
[root@logstash-1 kibana]# ls
cert.pem  key.pem  kibana.keystore  kibana.yml  node.options

[root@logstash-1 logs]# cat /etc/kibana/kibana.yml
# For more configuration options see the configuration guide for Kibana in
# https://www.elastic.co/guide/index.html

# =================== System: Kibana Server ===================
# Kibana is served by a back end server. This setting specifies the port to use.
#server.port: 5601 #默认端口

# Specifies the address to which the Kibana server will bind. IP addresses and host names are both valid values.
# The default is 'localhost', which usually means remote machines will not be able to connect.
# To allow connections from remote users, set this parameter to a non-loopback address.
server.host: "0.0.0.0" 
elasticsearch.hosts: ["https://192.168.0.1:9200", "https://192.168.0.2:9200", "https://192.168.0.3:9200"]  #ES集群地址
elasticsearch.username: "kibana_system" #链接ES集群的用户名
elasticsearch.password: "5VlQZzSTGx3Y=0zLQERT" 
 
#kibana_system密码在ELK8-1安装ES完成后有显示 可以在ELK8-1（ES主节点）上重置
#/usr/share/elasticsearch/bin/elasticsearch-reset-password -u kibana_system

server.ssl.enabled: true  #开启SSL              
server.ssl.certificate: /etc/kibana/cert.pem
server.ssl.key: /etc/kibana/key.pem
server.ssl.keyPassphrase: "证书密码"
elasticsearch.ssl.verificationMode: none

# Enables you to specify a path to mount Kibana at if you are running behind a proxy.
# Use the `server.rewriteBasePath` setting to tell Kibana if it should remove the basePath
# from requests it receives, and to prevent a deprecation warning at startup.
# This setting cannot end in a slash.
#server.basePath: ""

# Specifies whether Kibana should rewrite requests that are prefixed with
# `server.basePath` or require that they are rewritten by your reverse proxy.
# Defaults to `false`.
#server.rewriteBasePath: false

# Specifies the public URL at which Kibana is available for end users. If
# `server.basePath` is configured this URL should end with the same basePath.
#server.publicBaseUrl: ""

# The maximum payload size in bytes for incoming server requests.
server.maxPayload: 1048576

# The Kibana server's name. This is used for display purposes.
#server.name: "your-hostname"

# =================== System: Kibana Server (Optional) ===================
# Enables SSL and paths to the PEM-format SSL certificate and SSL key files, respectively.
# These settings enable SSL for outgoing requests from the Kibana server to the browser.
#server.ssl.enabled: false
#server.ssl.certificate: /path/to/your/server.crt
#server.ssl.key: /path/to/your/server.key

# =================== System: Elasticsearch ===================
# The URLs of the Elasticsearch instances to use for all your queries.
#elasticsearch.hosts: ["http://localhost:9200"]

# If your Elasticsearch is protected with basic authentication, these settings provide
# the username and password that the Kibana server uses to perform maintenance on the Kibana
# index at startup. Your Kibana users still need to authenticate with Elasticsearch, which
# is proxied through the Kibana server.
#elasticsearch.username: "kibana_system"
#elasticsearch.password: "pass"

# Kibana can also authenticate to Elasticsearch via "service account tokens".
# Service account tokens are Bearer style tokens that replace the traditional username/password based configuration.
# Use this token instead of a username/password.
# elasticsearch.serviceAccountToken: "my_token"

# Time in milliseconds to wait for Elasticsearch to respond to pings. Defaults to the value of
# the elasticsearch.requestTimeout setting.
#elasticsearch.pingTimeout: 1500

# Time in milliseconds to wait for responses from the back end or Elasticsearch. This value
# must be a positive integer.
#elasticsearch.requestTimeout: 30000

# The maximum number of sockets that can be used for communications with elasticsearch.
# Defaults to `Infinity`.
elasticsearch.maxSockets: 1024

# Specifies whether Kibana should use compression for communications with elasticsearch
# Defaults to `false`.
#elasticsearch.compression: false

# List of Kibana client-side headers to send to Elasticsearch. To send *no* client-side
# headers, set this value to [] (an empty list).
#elasticsearch.requestHeadersWhitelist: [ authorization ]

# Header names and values that are sent to Elasticsearch. Any custom headers cannot be overwritten
# by client-side headers, regardless of the elasticsearch.requestHeadersWhitelist configuration.
#elasticsearch.customHeaders: {}

# Time in milliseconds for Elasticsearch to wait for responses from shards. Set to 0 to disable.
#elasticsearch.shardTimeout: 30000

# =================== System: Elasticsearch (Optional) ===================
# These files are used to verify the identity of Kibana to Elasticsearch and are required when
# xpack.security.http.ssl.client_authentication in Elasticsearch is set to required.
#elasticsearch.ssl.certificate: /path/to/your/client.crt
#elasticsearch.ssl.key: /path/to/your/client.key

# Enables you to specify a path to the PEM file for the certificate
# authority for your Elasticsearch instance.
#elasticsearch.ssl.certificateAuthorities: [ "/path/to/your/CA.pem" ]

# To disregard the validity of SSL certificates, change this setting's value to 'none'.
#elasticsearch.ssl.verificationMode: full

# =================== System: Logging ===================
# Set the value of this setting to off to suppress all logging output, or to debug to log everything. Defaults to 'info'
#logging.root.level: debug

# Enables you to specify a file where Kibana stores log output.
logging:
  appenders:
    file:
      type: file
      fileName: /data/kibana/logs/kibana.log  #kibana日志文件位置
      layout:
        type: json
  root:
    appenders:
      - default
      - file
#  layout:
#    type: json

# Logs queries sent to Elasticsearch.
#logging.loggers:
#  - name: elasticsearch.query
#    level: debug

# Logs http responses.
#logging.loggers:
#  - name: http.server.response
#    level: debug

# Logs system usage information.
#logging.loggers:
#  - name: metrics.ops
#    level: debug

# =================== System: Other ===================
# The path where Kibana stores persistent data not saved in Elasticsearch. Defaults to data
#path.data: data

# Specifies the path where Kibana creates the process ID file.
pid.file: /run/kibana/kibana.pid

# Set the interval in milliseconds to sample system and process performance
# metrics. Minimum is 100ms. Defaults to 5000ms.
#ops.interval: 5000

# Specifies locale to be used for all localizable strings, dates and number formats.
# Supported languages are the following: English (default) "en", Chinese "zh-CN", Japanese "ja-JP", French "fr-FR".
i18n.locale: "zh-CN" #设置kibana的语言为中文

# =================== Frequently used (Optional)===================

# =================== Saved Objects: Migrations ===================
# Saved object migrations run at startup. If you run into migration-related issues, you might need to adjust these settings.

# The number of documents migrated at a time.
# If Kibana can't start up or upgrade due to an Elasticsearch `circuit_breaking_exception`,
# use a smaller batchSize value to reduce the memory pressure. Defaults to 1000 objects per batch.
#migrations.batchSize: 1000

# The maximum payload size for indexing batches of upgraded saved objects.
# To avoid migrations failing due to a 413 Request Entity Too Large response from Elasticsearch.
# This value should be lower than or equal to your Elasticsearch cluster’s `http.max_content_length`
# configuration option. Default: 100mb
#migrations.maxBatchSizeBytes: 100mb

# The number of times to retry temporary migration failures. Increase the setting
# if migrations fail frequently with a message such as `Unable to complete the [...] step after
# 15 attempts, terminating`. Defaults to 15
#migrations.retryAttempts: 15

# =================== Search Autocomplete ===================
# Time in milliseconds to wait for autocomplete suggestions from Elasticsearch.
# This value must be a whole number greater than zero. Defaults to 1000ms
#unifiedSearch.autocomplete.valueSuggestions.timeout: 1000

# Maximum number of documents loaded by each shard to generate autocomplete suggestions.
# This value must be a whole number greater than zero. Defaults to 100_000
#unifiedSearch.autocomplete.valueSuggestions.terminateAfter: 100000

Kibana首次启动注意

systemctl start kibana 
服务器ip+5601端口访问kibana 自定义证书打开网址是提示证书未认证  选择高级 继续前往


[root@ELK8-1 kafka_2.13-3.4.0]# /usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibana
warning: ignoring JAVA_HOME=/usr/local/jdk-20.0.1; using bundled JDK
eyJ2ZXIiOiI4LjkuMiIsImFkDMuMjUwLjE3OS43Mzo5MjAwIl0sImZnciI6ImM2OGM0YjM0OWUyYWFjMmViODhjY2NiMDhmweqwezxcjMxYTllMDg4ODAzYTgwZWIwY2I1NzAxkiOiI4VjJNMDR3QmRCTXJvYkUwVjQ4VzpnQ2ZuTWd0Z1M3S3FnZDBEUkotdUlnIn0=

kibana首次启动需要ES主节点的token 在ELK8-1生成即可 然后用ELK8-1安装ES后保存的超管elastic用户登录

kibana底层基于node 网页渲染加载基于浏览器本身 网页加载慢调整云主机带宽（效果显著）或者优化配置

[root@logstash-1 bin]# pwd
/usr/share/kibana/bin
[root@logstash-1 bin]# cat kibana | grep ^NODE_OPTIONS
#优化Optione
NODE_OPTIONS="--no-warnings --max-http-header-size=65536 --max-old-space-size=200 $KBN_NODE_OPTS $NODE_OPTIONS" NODE_ENV=production exec "${NODE}" "${DIR}/src/cli/dist" "${@}"

[root@logstash-1 kibana]# pwd
/etc/kibana
[root@logstash-1 kibana]# cat node.options
## Node command line options
## See `node --help` and `node --v8-options` for available options
## Please note you should specify one option per line

## max size of old space in megabytes
--max-old-space-size=4096 kibana分配内存 推荐设置1024

## do not terminate process on unhandled promise rejection
 --unhandled-rejections=warn