Don’t modify anything within the data directory or run processes that might interfere with its contents. If something other than Elasticsearch modifies the contents of the data directory, then Elasticsearch may fail, reporting corruption or other data inconsistencies, or may appear to work correctly having silently lost some of your data. Don’t attempt to take filesystem backups of the data directory; there is no supported way to restore such a backup. Instead, use Snapshot and restore to take backups safely. Don’t run virus scanners on the data directory. A virus scanner can prevent Elasticsearch from working correctly and may modify the contents of the data directory. The data directory contains no executables so a virus scan will only find false positives.
不要修改数据目录中的任何内容或运行可能干扰其内容的进程。 如果不是Elasticsearch修改了数据目录的内容，那么Elasticsearch可能会失败，报告损坏或其他数据不一致，或者可能看起来正常工作，但却悄悄地丢失了一些数据。 不要尝试对数据目录进行文件系统备份; 不支持恢复此类备份的方法。 相反，请使用快照和恢复安全地进行备份。 不要在数据目录上运行病毒扫描程序。 病毒扫描程序可能会阻止Elasticsearch正常工作，并可能会修改数据目录的内容。 数据目录不包含可执行文件，因此病毒扫描只会发现误报。
搭建es
搭建es单点
准备es安装包
下载安装包
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-8.1.3-linux-x86_64.tar.gz
下载该安装包对应的sha512值（校验安装包是否被更改用，可以不下载）
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-8.1.3-linux-x86_64.tar.gz.sha512
校验安装包是否被更改
shasum -a 512 -c elasticsearch-8.1.3-linux-x86_64.tar.gz.sha512
输出
elasticsearch-{version}-linux-x86_64.tar.gz: OK.
解压安装包
tar zxvf elasticsearch-8.1.3-linux-x86_64.tar.gz -C /opt/
新建用户及目录授权
新建启动用户
useradd es
passwd es
授权
chown -R es.es /opt/elasticsearch-8.1.3/
新建数据和日志目录并授权
mkdir /mnt/es_1
mkdir /mnt/es_2
mkdir /mnt/es_3
mkdir /mnt/es_logs
chown -R es.es /mnt/es_1/
chown -R es.es /mnt/es_2
chown -R es.es /mnt/es_3
chown -R es.es /mnt/es_logs
设置es命令的环境变量
echo "export PATH=$PATH:/opt/elasticsearch-8.1.3/bin/" >> /etc/profile
配置es
action.auto_create_index: "*"
path:
data:
- "/mnt/es_1"
- "/mnt/es_2"
- "/mnt/es_3"
logs: "/mnt/es_logs"
启动es
前台启动
elasticsearch
后台启动
说明：pid是个文件名，这个文件里会保存这次启动自动生成的pid号。
elasticsearch -d -p pid
注意：启动成功后再看配置文件会发现自动增加了如下配置，并且在config目录下会生成证书文件
增加的配置
#----------------------- BEGIN SECURITY AUTO CONFIGURATION -----------------------
#
# The following settings, TLS certificates, and keys have been automatically
# generated to configure Elasticsearch security features on 27-04-2022 10:31:15
#
# --------------------------------------------------------------------------------
# Enable security features
xpack.security.enabled: true
xpack.security.enrollment.enabled: true
# Enable encryption for HTTP API client connections, such as Kibana, Logstash, and Agents
xpack.security.http.ssl:
enabled: true
keystore.path: certs/http.p12
# Enable encryption and mutual authentication between cluster nodes
xpack.security.transport.ssl:
enabled: true
verification_mode: certificate
keystore.path: certs/transport.p12
truststore.path: certs/transport.p12
# Create a new cluster with the current node only
# Additional nodes can still join the cluster later
cluster.initial_master_nodes: ["es-01"]
# Allow HTTP API connections from localhost and local networks
# Connections are encrypted and require user authentication
http.host: [_local_, _site_]
# Allow other nodes to join the cluster from localhost and local networks
# Connections are encrypted and mutually authenticated
#transport.host: [_local_, _site_]
#----------------------- END SECURITY AUTO CONFIGURATION -------------------------
生成的证书文件
[es@es-01 elasticsearch-8.1.3]$ ll /opt/elasticsearch-8.1.3/config/certs/
total 24
-rw-rw---- 1 es es 1915 Apr 27 18:31 http_ca.crt
-rw-rw---- 1 es es 10029 Apr 27 18:31 http.p12
-rw-rw---- 1 es es 5822 Apr 27 18:31 transport.p12
带证书访问es
es8.1默认是开启ssl认证的，增删改查的时候要加上证书路径，并且需要输入用户elastic密码。
重置elastic用户密码
[es@es-01 elasticsearch-8.1.3]$ elasticsearch-reset-password -u elastic
This tool will reset the password of the [elastic] user to an autogenerated value.
The password will be printed in the console.
Please confirm that you would like to continue [y/N]y
Password for the [elastic] user successfully reset.
New value: L*fiTZYHBp509nOj=Mod
访问es
[es@es-01 elasticsearch-8.1.3]$ curl --cacert /opt/elasticsearch-8.1.3/config/certs/http_ca.crt -u elastic https://localhost:9200
Enter host password for user 'elastic':#输入重置输入的密码
{
"name" : "es-01",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "oMPfdw-tRXuYCgYt2uUAng",
"version" : {
"number" : "8.1.3",
"build_flavor" : "default",
"build_type" : "tar",
"build_hash" : "39afaa3c0fe7db4869a161985e240bd7182d7a07",
"build_date" : "2022-04-19T08:13:25.444693396Z",
"build_snapshot" : false,
"lucene_version" : "9.0.0",
"minimum_wire_compatibility_version" : "7.17.0",
"minimum_index_compatibility_version" : "7.0.0"
},
"tagline" : "You Know, for Search"
}
删除认证访问es
修改配置
vim /opt/elasticsearch-8.1.3/config/elasticsearch.yml
xpack.security.enabled: false
停止es
[es@es-01 elasticsearch-8.1.3]$ cat /opt/elasticsearch-8.1.3/pid
17183
kill -9 17183
启动es
elasticsearch -d -p pid
访问es
注意：使用http而不是https。
[es@es-01 elasticsearch-8.1.3]$ curl https://localhost:9200
curl: (35) SSL received a record that exceeded the maximum permissible length.
[es@es-01 elasticsearch-8.1.3]$ curl http://localhost:9200
{
"name" : "es-01",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "oMPfdw-tRXuYCgYt2uUAng",
"version" : {
"number" : "8.1.3",
"build_flavor" : "default",
"build_type" : "tar",
"build_hash" : "39afaa3c0fe7db4869a161985e240bd7182d7a07",
"build_date" : "2022-04-19T08:13:25.444693396Z",
"build_snapshot" : false,
"lucene_version" : "9.0.0",
"minimum_wire_compatibility_version" : "7.17.0",
"minimum_index_compatibility_version" : "7.0.0"
},
"tagline" : "You Know, for Search"
}
es启动报错
elasticsearch.yml开启了network参数后（如下）启动es报错。
network.host: 192.168.103.232
启动报错
ERROR: [2] bootstrap checks failed. You must address the points described in the following [2] lines before starting Elasticsearch.
bootstrap check failure [1] of [2]: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65535]
bootstrap check failure [2] of [2]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
解决：
增加启动用户（这里是es）打开文件数
[root@es-01 metricbeat-8.1.3-linux-x86_64]# cat /etc/security/limits.conf
es soft nofile 65535
es hard nofile 65535
修改vm.max_map_count值
cat /etc/sysctl.conf
vm.max_map_count=262144
使修改生效
sysctl -p
输出
vm.max_map_count = 262144
kibana
准备kibana安装包
curl -O https://artifacts.elastic.co/downloads/kibana/kibana-8.1.3-linux-x86_64.tar.gz
tar -xzf kibana-8.1.3-linux-x86_64.tar.gz -C /opt
授权
chown -R es.es /opt/kibana-8.1.3/
设置kibana环境变量
echo "export PATH=$PATH:/opt/elasticsearch-8.1.3/bin/:/opt/kibana-8.1.3/bin/" >> /etc/profile
source /etc/profile
启动kibana
说明：无论es是怎么配置的，都这么启动。不要改kibana.yml配置文件。
[es@es-01 kibana-8.1.3]$ kibana -H 192.168.103.232
[2022-05-05T15:58:22.637+08:00][INFO ][plugins-service] Plugin "metricsEntities" is disabled.
[2022-05-05T15:58:22.724+08:00][INFO ][http.server.Preboot] http server running at http://localhost:5601
[2022-05-05T15:58:22.764+08:00][INFO ][plugins-system.preboot] Setting up [1] plugins: [interactiveSetup]
[2022-05-05T15:58:22.766+08:00][INFO ][preboot] "interactiveSetup" plugin is holding setup: Validating Elasticsearch connection configuration…
[2022-05-05T15:58:22.805+08:00][INFO ][root] Holding setup until preboot stage is completed.
i Kibana has not been configured.
Go to http://localhost:5601/?code=241770 to get started.
后台启动命令
nohup /opt/kibana-8.1.3/bin/kibana -H 192.168.103.232 > logs/kibana.log 2>&1 &
访问kibana
说明：如果es配置文件elasticsearch.yml中做了如下配置，访问过程如下。
xpack.security.enabled: true
如果token失效，重新生成token，命令如下
elasticsearch-create-enrollment-token -s kibana --url "https://192.168.103.232:9200"
输出
eyJ2ZXIiOiI4LjEuMyIsImFkciI6WyIxOTIuMTY4LjEwMy4yMzI6OTIwMCIsIjEwLjAuMC4yOjkyMDAiXSwiZmdyIjoiYjE0MGU1NDg0YTJjNzUzMzNlYWIwMGRjMjRlY2M5OWMzNzcwMmIwOTAwN2NmYzJjMTI4YzZkNDAwNzM4N2IyNyIsImtleSI6Ik11dS1kSUFCb2FGd05FdWF4V1ZnOkdhZnFtMHFRU3ZXc2lvLXJheWFsZ0EifQ==
生成序列码
kibana-verification-code
Your verification code is: 860 099
当一切完成后，kibana文件内会自动生成配置。配置如下
cat /opt/kibana-8.1.3/config/kibana.yml
elasticsearch.hosts: ['https://192.168.103.232:9200']
elasticsearch.serviceAccountToken: AAEAAWVsYXN0aWMva2liYW5hL2Vucm9sbC1wcm9jZXNzLXRva2VuLTE2NTE3Mzc4NzUwNzk6NlJKUzN4VEZTSHVtN3hkWGxmWXE0UQ
elasticsearch.ssl.certificateAuthorities: [/opt/kibana-8.1.3/data/ca_1651737875620.crt]
xpack.fleet.outputs: [{id: fleet-default-output, name: default, is_default: true, is_default_monitoring: true, type: elasticsearch, hosts: ['https://192.168.103.232:9200'], ca_trusted_fingerprint: b140e5484a2c75333eab00dc24ecc99c37702b09007cfc2c128c6d4007387b27}]
生成token
bin/elasticsearch-service-tokens create elastic/kibana my-token
SERVICE_TOKEN elastic/kibana/my-token = AAEAAWVsYXN0aWMva2liYW5hL215LXRva2VuOkJPVWVVMDF0UlBpOUx6X1pLcEpEZ1E
[2022-05-05T15:35:52.962+08:00][ERROR][elasticsearch-service] Unable to retrieve version information from Elasticsearch nodes. self signed certificate in certificate chain
kibana启动报错
[2022-05-05T16:19:31.197+08:00][ERROR][elasticsearch-service] Unable to retrieve version information from Elasticsearch nodes. self signed certificate in certificate chain
解决:
删除kibana.yml中的配置
elasticsearch.hosts: ['https://192.168.103.232:9200']
metricbeat
登录到kibana界面。
点击上图中的链接，开始安装metricbeat
准备metricbeat安装包
curl -L -O https://artifacts.elastic.co/downloads/beats/metricbeat/metricbeat-8.1.3-linux-x86_64.tar.gz
tar xzvf metricbeat-8.1.3-linux-x86_64.tar.gz -C /opt
授权
chown -R root /opt/metricbeat-8.1.3-linux-x86_64/
配置
查看es证书的指纹
openssl x509 -fingerprint -sha256 -in /opt/elasticsearch-8.1.3/config/certs/http_ca.crt
SHA256 Fingerprint=B1:40:E5:48:4A:2C:75:33:3E:AB:00:DC:24:EC:C9:9C:37:70:2B:09:00:7C:FC:2C:12:8C:6D:40:07:38:7B:27
-----BEGIN CERTIFICATE-----
MIIFWTCCA0GgAwIBAgIUCi1tsyMQxyphk3ml6YRRA3rrPzAwDQYJKoZIhvcNAQEL
BQAwPDE6MDgGA1UEAxMxRWxhc3RpY3NlYXJjaCBzZWN1cml0eSBhdXRvLWNvbmZp
Z3VyYXRpb24gSFRUUCBDQTAeFw0yMjA0MjcxMDMxMjBaFw0yNTA0MjYxMDMxMjBa
MDwxOjA4BgNVBAMTMUVsYXN0aWNzZWFyY2ggc2VjdXJpdHkgYXV0by1jb25maWd1
cmF0aW9uIEhUVFAgQ0EwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQCZ
fuPQrGMX3XhpBJBTXpLcyQvXw8jqCWfU+Gv/9jaaOHBrYrFE9YVkYwZcZbJ/+RfP
9mie0B46rq8kEs2TnoAw5sGmaCpeZnuCeI99IFRYzmoDTFGOg5+7GE0UFtnFvL0j
6uxyhwPYIOTkKo9ayontGZyUI9Yov9GgYoPkS6xw7+zDAWupRcjpWbM3O6M7k0HO
hrc8KJQhWByzlYFmDFejIJuzAJbcnVJBmwnFbR0q5W+5BvSgIhH2NcR3P+Kazpyq
qL9jKXQ1UjuKI/7Xs2Khhn3YDvwNEQWo22mdNt7FI9LFM+rG8jqPUULo+PMRnZNi
cCdO362XjM7XWxlv2OFjnZ6nme6I6j/wHgi8AsLtBcl5iI7+OxDhRvoFLP8sAwt0
75EoSD1mw2URiY8PQdcxR60wa1aPqv4XyOhmEXJISKsfWmEbJ+1vGOQAwocUC18G
XasOYrWaoXGqYOi7RdrvK5ka1CdYy/+xsjryNZZMvd04hNMamiq3opPYff9GGK8U
B2t22JEuX57vvy8mr3AegyCErD0IbcVUUSYBj1lIlVz2ZVhQvCIj94v9U1XU5Res
zethyKbVGt3Zkfy7ffyRGar2FThi+QlD+xAz8pV2P7YZDNDyVUB+ERzB31428fnj
UZ6qcp/J/aOBKEWWDq+HqREYextjbaBq9WStaaDn5QIDAQABo1MwUTAdBgNVHQ4E
FgQUFo0TjPfM1k2qBSXPVgRZTk1AiwwwHwYDVR0jBBgwFoAUFo0TjPfM1k2qBSXP
VgRZTk1AiwwwDwYDVR0TAQH/BAUwAwEB/zANBgkqhkiG9w0BAQsFAAOCAgEAZ1PP
BZGTP1M6Y42RLxgSgfB95vNxglMWiBmAfcfSKi/INkmHMsOkoGFzgOiZG+FXlnh+
gQfzc+oXeo1nb0yAf+Bpnu9YRRsAXioGUSz/ncKgt7t5In+Td8HFa20Ga8jIplYL
KZrwnrAP33OHy09x7Vrm4ScJXKojxL6iF0H8sDTiHq7ONVnBwRQz6Lfs7n3ff9HG
7XyQL4lHbR0OA/QdWNNhZ1Ph4JB33ubaP8QdCZt1R7OeZ4A4MTDw0P1tYDn/nMnH
E+WPiktomn8hMTNYy+uHY3bPAUvm1nonw0ZAsQL6D88ADAFr49v1WT2n0RelOYiv
UcS1rjIbjQiiPkWgaKD+qRSde6+/8zfqbaUXIHofWZSuBprSbeq4zYbB4n34hQ6p
lMdx5gXGCr07mNCNBTpzD8JP6ZI9UqC879hHyGcj6RZ7+Uk9LLDrvFrjMGjE1f5F
Tp13KCe/N2e6DgH7ePn41Qfiendt0Ynw4fa0cWSiYwwXJF6Ll26A5d8GegQ5eOWG
D+dNi37emHYuzZMCDnCK0U4LQ7cTOS+VS1BmxeAHFIA+oQQRXYGN+qKIYDJk5aM3
8Hf9Fx4LNFhs0khFuMWS8qHUSgdJK+2mYRhg1o1vA/+ytazouDYdbalixnKT3v0s
HZgH46uRV5+kKf2CoRCiDuzKvUV/MFb7hd13Pjc=
-----END CERTIFICATE-----
从输出可以看出es证书的指纹为
B1:40:E5:48:4A:2C:75:33:3E:AB:00:DC:24:EC:C9:9C:37:70:2B:09:00:7C:FC:2C:12:8C:6D:40:07:38:7B:27
配置metricbeat.yml
cat metricbeat.yml
output.elasticsearch:
# Array of hosts to connect to.
hosts: ["https://192.168.103.232:9200"]
# Protocol - either `http` (default) or `https`.
protocol: "https"
# Authentication credentials - either API key or username/password.
#api_key: "id:api_key"
username: "elastic"
password: "9RO=5cuqxgOTcD3TZt3d"
ssl.enabled: true
ca_trusted_fingerprint: "B1:40:E5:48:4A:2C:75:33:3E:AB:00:DC:24:EC:C9:9C:37:70:2B:09:00:7C:FC:2C:12:8C:6D:40:07:38:7B:27"
如果你计划使用我们预建的Kibana仪表板，配置Kibana端点。 如果Kibana与Elasticsearch运行在同一主机上，则跳过此步骤。
setup.kibana:
host: "mykibanahost:5601"
username: "my_kibana_user"
password: "{pwd}"
Metricbeat使用模块收集指标。 每个模块定义了从特定服务(如Redis或MySQL)收集数据的基本逻辑。 一个模块由获取和构造数据的度量集组成。
模块如何工作，参考链接
https://www.elastic.co/guide/en/beats/metricbeat/8.1/how-metricbeat-works.html
查询可用模块
确定需要启用的模块。 要查看可用的模块列表，运行
cd /opt/metricbeat-8.1.3-linux-x86_64/
[es@es-01 metricbeat-8.1.3-linux-x86_64]$ ./metricbeat modules list
从安装目录中启用一个或多个模块。 如果您接受默认配置而不启用其他模块，则Metricbeat只收集系统度量。
下面的命令启用模块中的nginx配置。
开启模块
./metricbeat modules enable system
更多命令参考链接
https://www.elastic.co/guide/en/beats/metricbeat/8.1/command-line-options.html#modules-command
检查配置
./metricbeat test config -e -c /opt/metricbeat-8.1.3-linux-x86_64/metricbeat.yml
说明:最后有OK字样输出即为配置正确。
[es@es-01 metricbeat-8.1.3-linux-x86_64]$ ./metricbeat test config -e -c /opt/metricbeat-8.1.3-linux-x86_64/metricbeat.yml
{"log.level":"info","@timestamp":"2022-05-05T17:35:13.650+0800","log.origin":{"file.name":"instance/beat.go","file.line":669},"message":"Home path: [/opt/metricbeat-8.1.3-linux-x86_64] Config path: [/opt/metricbeat-8.1.3-linux-x86_64] Data path: [/opt/metricbeat-8.1.3-linux-x86_64/data] Logs path: [/opt/metricbeat-8.1.3-linux-x86_64/logs]","service.name":"metricbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-05T17:35:13.650+0800","log.origin":{"file.name":"instance/beat.go","file.line":677},"message":"Beat ID: c2a2095a-303b-47a5-acc4-a2c5cca0c499","service.name":"metricbeat","ecs.version":"1.6.0"}
{"log.level":"warn","@timestamp":"2022-05-05T17:35:16.652+0800","log.logger":"add_cloud_metadata","log.origin":{"file.name":"add_cloud_metadata/provider_aws_ec2.go","file.line":80},"message":"read token request for getting IMDSv2 token returns empty: Put \"http://169.254.169.254/latest/api/token\": context deadline exceeded (Client.Timeout exceeded while awaiting headers). No token in the metadata request will be used.","service.name":"metricbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-05T17:35:16.653+0800","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1047},"message":"Beat info","service.name":"metricbeat","system_info":{"beat":{"path":{"config":"/opt/metricbeat-8.1.3-linux-x86_64","data":"/opt/metricbeat-8.1.3-linux-x86_64/data","home":"/opt/metricbeat-8.1.3-linux-x86_64","logs":"/opt/metricbeat-8.1.3-linux-x86_64/logs"},"type":"metricbeat","uuid":"c2a2095a-303b-47a5-acc4-a2c5cca0c499"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2022-05-05T17:35:16.653+0800","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1056},"message":"Build info","service.name":"metricbeat","system_info":{"build":{"commit":"271435c21bfd4e2e621d87c04f4b815980626978","libbeat":"8.1.3","time":"2022-04-19T09:42:04.000Z","version":"8.1.3"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2022-05-05T17:35:16.653+0800","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1059},"message":"Go runtime info","service.name":"metricbeat","system_info":{"go":{"os":"linux","arch":"amd64","max_procs":2,"version":"go1.17.8"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2022-05-05T17:35:16.654+0800","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1063},"message":"Host info","service.name":"metricbeat","system_info":{"host":{"architecture":"x86_64","boot_time":"2022-04-29T16:25:30+08:00","containerized":false,"name":"es-01","ip":["127.0.0.1/8","::1/128","192.168.103.232/16","fe80::a622:3db0:e230:e7b6/64","10.0.0.2/24","fe80::7e82:ad8a:1f8b:c63a/64","172.17.0.1/16"],"kernel_version":"3.10.0-862.el7.x86_64","mac":["00:0c:29:7e:93:b6","00:0c:29:7e:93:c0","02:42:df:31:e5:cf"],"os":{"type":"linux","family":"redhat","platform":"centos","name":"CentOS Linux","version":"7 (Core)","major":7,"minor":5,"patch":1804,"codename":"Core"},"timezone":"CST","timezone_offset_sec":28800,"id":"4d1ddead2f6540798cf79d9465e8cf9a"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2022-05-05T17:35:16.655+0800","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1092},"message":"Process info","service.name":"metricbeat","system_info":{"process":{"capabilities":{"inheritable":null,"permitted":null,"effective":null,"bounding":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend"],"ambient":null},"cwd":"/opt/metricbeat-8.1.3-linux-x86_64","exe":"/opt/metricbeat-8.1.3-linux-x86_64/metricbeat","name":"metricbeat","pid":46365,"ppid":46271,"seccomp":{"mode":"disabled"},"start_time":"2022-05-05T17:35:13.140+0800"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2022-05-05T17:35:16.655+0800","log.origin":{"file.name":"instance/beat.go","file.line":323},"message":"Setup Beat: metricbeat; Version: 8.1.3","service.name":"metricbeat","ecs.version":"1.6.0"}
{"log.level":"warn","@timestamp":"2022-05-05T17:35:16.656+0800","log.logger":"cfgwarn","log.origin":{"file.name":"tlscommon/config.go","file.line":102},"message":"DEPRECATED: Treating the CommonName field on X.509 certificates as a host name when no Subject Alternative Names are present is going to be removed. Please update your certificates if needed. Will be removed in version: 8.0.0","service.name":"metricbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-05T17:35:16.656+0800","log.logger":"esclientleg","log.origin":{"file.name":"eslegclient/connection.go","file.line":105},"message":"elasticsearch url: https://192.168.103.232:9200","service.name":"metricbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-05T17:35:16.656+0800","log.logger":"publisher","log.origin":{"file.name":"pipeline/module.go","file.line":113},"message":"Beat name: es-01","service.name":"metricbeat","ecs.version":"1.6.0"}
Config OK
加载
Metricbeat带有预定义的用于解析、索引和可视化数据的资产。 加载这些资产 。
进到Metricbeat安装目录。
./metricbeat setup -e
启动
./metricbeat -e
后台启动
nohup /opt/metricbeat-8.1.3-linux-x86_64/metricbeat -e > /opt/metricbeat-8.1.3-linux-x86_64/logs/metricbeat.log 2>&1 &
kibana界面查看
连接es报错
执行的命令
./metricbeat setup -e
报错
{"log.level":"error","@timestamp":"2022-05-05T17:41:29.129+0800","log.logger":"esclientleg","log.origin":{"file.name":"transport/logging.go","file.line":37},"message":"Error dialing x509: certificate signed by unknown authority","service.name":"metricbeat","network":"tcp","address":"192.168.103.232:9200","ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2022-05-05T17:41:29.129+0800","log.logger":"esclientleg","log.origin":{"file.name":"eslegclient/connection.go","file.line":231},"message":"error connecting to Elasticsearch at https://192.168.103.232:9200: Get \"https://192.168.103.232:9200\": x509: certificate signed by unknown authority","service.name":"metricbeat","ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2022-05-05T17:41:29.129+0800","log.origin":{"file.name":"instance/beat.go","file.line":1022},"message":"Exiting: couldn't connect to any of the configured Elasticsearch hosts. Errors: [error connecting to Elasticsearch at https://192.168.103.232:9200: Get \"https://192.168.103.232:9200\": x509: certificate signed by unknown authority]","service.name":"metricbeat","ecs.version":"1.6.0"}
Exiting: couldn't connect to any of the configured Elasticsearch hosts. Errors: [error connecting to Elasticsearch at https://192.168.103.232:9200: Get "https://192.168.103.232:9200": x509: certificate signed by unknown authority]
解决：
在metricbeat.yml文件中增加配置
output.elasticsearch:
ssl.certificate_authorities: "/opt/elasticsearch-8.1.3/config/certs/http_ca.crt"
连接kibana报错
执行的命令
./metricbeat setup -e
报错
{"log.level":"error","@timestamp":"2022-05-05T18:29:59.802+0800","log.origin":{"file.name":"instance/beat.go","file.line":1022},"message":"Exiting: error connecting to Kibana: fail to get the Kibana version: HTTP GET request to http://localhost:5601/api/status fails: fail to execute the HTTP GET request: Get \"http://localhost:5601/api/status\": dial tcp [::1]:5601: connect: connection refused. Response: .","service.name":"metricbeat","ecs.version":"1.6.0"}
Exiting: error connecting to Kibana: fail to get the Kibana version: HTTP GET request to http://localhost:5601/api/status fails: fail to execute the HTTP GET request: Get "http://localhost:5601/api/status": dial tcp [::1]:5601: connect: connection refused. Response: .
解决：
在metricbeat.yml文件中将连接kibana的地址改成kibana监听的地址（启动kibana的时候-H指定的地址）。
cat metricbeat.yml
setup.kibana:
host: "192.168.103.232:5601"
filebeat
注意下图中日志没法输出
现在配置检测日志工具filebeat。
准备安装包
curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-8.1.3-linux-x86_64.tar.gz
tar zxvf filebeat-8.1.3-linux-x86_64.tar.gz -C /opt/
授权
chown -R root /opt/filebeat-8.1.3-linux-x86_64/
启用模块
cd /opt/filebeat-8.1.3-linux-x86_64/
./filebeat modules list
./filebeat modules enable elasticsearch
配置模块
cat modules.d/elasticsearch.yml
# Module: elasticsearch
# Docs: https://www.elastic.co/guide/en/beats/filebeat/8.1/filebeat-module-elasticsearch.html
- module: elasticsearch
# Server log 这个server只是个日志标签名字而已，任意写即可
server:
enabled: true
# Set custom paths for the log files. If left empty,
# Filebeat will choose the paths depending on your OS.
var.paths: ["/mnt/es_logs/elasticsearch*"]
配置filebeat
配置输出到es
output.elasticsearch:
# Array of hosts to connect to.
hosts: ["192.168.103.232:9200"]
# Protocol - either `http` (default) or `https`.
protocol: "https"
# Authentication credentials - either API key or username/password.
#api_key: "id:api_key"
#username: "elastic"
#password: "changeme"
username: "elastic"
password: "9RO=5cuqxgOTcD3TZt3d"
ssl.enabled: true
ca_trusted_fingerprint: "B1:40:E5:48:4A:2C:75:33:3E:AB:00:DC:24:EC:C9:9C:37:70:2B:09:00:7C:FC:2C:12:8C:6D:40:07:38:7B:27"
ssl.certificate_authorities: "/opt/elasticsearch-8.1.3/config/certs/http_ca.crt"
配置输出到kibana
# =================================== Kibana ===================================
# Starting with Beats version 6.0.0, the dashboards are loaded via the Kibana API.
# This requires a Kibana endpoint configuration.
setup.kibana:
# Kibana Host
# Scheme and port can be left out and will be set to the default (http and 5601)
# In case you specify and additional path, the scheme is required: http://localhost:5601/path
# IPv6 addresses should always be defined as: https://[2001:db8::1]:5601
host: "192.168.103.232:5601"
# Kibana Space ID
# ID of the Kibana Space into which the dashboards should be loaded. By default,
# the Default Space will be used.
#space.id:
加载
/opt/filebeat-8.1.3-linux-x86_64/filebeat setup -e
启动
后台启动
nohup /opt/filebeat-8.1.3-linux-x86_64/filebeat -e > /opt/filebeat-8.1.3-linux-x86_64/logs/filebeat.log 2>&1 &
kibana界面查看
1152
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
