Linux安装ELK
废话不多说，直接上手
我这次准备安装的有ElasticSearch，Kibana，filebeat，apm，metricbeat.
先下载需要的压缩包，具体下载链接自己百度，因为我这是前辈留下的😳.
两台服务器，一台作为资源服务器（监控装在这），一台项目服务器（信息采取在这）.
1. ElasticSearch安装
这里提一句，安装ELK需要首先安装jdk环境，这个很简单，网上有安装步骤，或者参考：https://blog.csdn.net/liu1160848595/article/details/102838318。
1.1 解压安装包
#-C为指定解压目录
[ect@localhost software]$ tar -zxvf elasticsearch-7.8.0-linux-x86_64.tar.gz -C /home/ect/elasticsearch/
解压后
[ect@localhost elasticsearch]$ ll
total 568
drwxr-xr-x. 2 ect ect 4096 Jun 15 03:38 bin
drwxr-xr-x. 3 ect ect 169 Nov 6 09:48 config
drwxr-xr-x. 9 ect ect 107 Jun 15 03:38 jdk
drwxr-xr-x. 3 ect ect 4096 Jun 15 03:38 lib
-rw-r--r--. 1 ect ect 13675 Jun 15 03:34 LICENSE.txt
drwxr-xr-x. 2 ect ect 6 Jun 15 03:37 logs
drwxr-xr-x. 47 ect ect 4096 Jun 15 03:39 modules
-rw-r--r--. 1 ect ect 544318 Jun 15 03:37 NOTICE.txt
drwxr-xr-x. 2 ect ect 6 Jun 15 03:37 plugins
-rw-r--r--. 1 ect ect 8165 Jun 15 03:34 README.asciidoc
1.2 修改配置文件
[ect@localhost elasticsearch]$ vim config/elasticsearch.yml
修改内容包括node.name、path.data、path.logs、network.host、http.port等配置
node.name: node-121
path.logs: /home/ect/elasticsearch/logs
path.data: /home/ect/elasticsearch/data
network.host: 0.0.0.0
http.port: 9200
1.3 启动
[ect@localhost elasticsearch]$ ./bin/elasticsearch
1.4 处理异常
出现异常，心塞
ERROR: [3] bootstrap checks failed
[1]: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65535]
[2]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
[3]: the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured
ERROR: Elasticsearch did not exit normally - check the logs at /home/ect/elasticsearch/logs/elasticsearch.log
三个问题
问题1
## max file descriptors [4096] for elasticsearch process is too low, increase to at least [65535]
[root@localhost ~]# vim /etc/security/limits.conf
## 65535修改为65536
* soft nofile 65536
* hard nofile 65536
## 文件末尾追加
* soft nproc 4096
* hard nproc 4096
问题2
## max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
[root@localhost ~]# vim /etc/sysctl.conf
## 在文件末尾追加：vm.max_map_count=655360
[root@localhost ~]# sysctl -p
## vm.max_map_count = 655360
问题3
## the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured
[ect@localhost elasticsearch]$ vim config/elasticsearch.yml
# 取消注释，并保留一个节点
cluster.initial_master_nodes: ["node-121"]
解决后，断开连接，重新登录服务器，不然问题一不起作用
测试是否启动成功
[ect@localhost elasticsearch]$ ./bin/elasticsearch
http://ip:port
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-uP8AQfQL-1604905286363)(C:\Users\taozheng4\AppData\Roaming\Typora\typora-user-images\image-20201107151741066.png)]
success！
做个sh启动命令，后台启动！
#!/bin/bash
./bin/elasticsearch -d
1.5 设置初始密码
[ect@localhost elasticsearch]$ ./bin/elasticsearch-setup-passwords interactive
future versions of Elasticsearch will require Java 11; your Java version from [/home/ect/java/jre] does not meet this requirement
gg，自带了jdk11和本地jdk8冲突了，该咋办嘞，我让他别用我本地jdk，就用他自带的
[ect@localhost elasticsearch]$ vim bin/elasticsearch-env
###能发现这一段
# now set the path to java
if [ ! -z "$JAVA_HOME" ]; then
JAVA="$JAVA_HOME/bin/java"
JAVA_TYPE="JAVA_HOME"
else
if [ "$(uname -s)" = "Darwin" ]; then
# macOS has a different structure
JAVA="$ES_HOME/jdk.app/Contents/Home/bin/java"
else
JAVA="$ES_HOME/jdk/bin/java"
fi
JAVA_TYPE="bundled jdk"
fi
###大概意思就是本地有全局jdk就用全局的，没有就用自带的，我来个偷梁换柱，不管有没有，全用es自带的（其他服务器不知道，windows和Linux应该是没问题的）
if [ ! -z "$JAVA_HOME" ]; then
JAVA="$ES_HOME/jdk/bin/java"
#JAVA="$JAVA_HOME/bin/java"
JAVA_TYPE="JAVA_HOME"
else
if [ "$(uname -s)" = "Darwin" ]; then
# macOS has a different structure
JAVA="$ES_HOME/jdk.app/Contents/Home/bin/java"
else
JAVA="$ES_HOME/jdk/bin/java"
fi
JAVA_TYPE="bundled jdk"
fi
得，还需要加配置
Unexpected response code [500] from calling GET http://10.4.60.121:9200/_security/_authenticate?pretty
It doesn't look like the X-Pack security feature is enabled on this Elasticsearch node.
Please check if you have enabled X-Pack security in your elasticsearch.yml configuration file.
ERROR: X-Pack Security is disabled by configuration.
再在elasticsearch.yml添加上这段，然后重启，重启后莫急，稍等一会
xpack.security.enabled: true
xpack.license.self_generated.type: basic
xpack.security.transport.ssl.enabled: true
然后就是愉快的设置密码了
[ect@localhost elasticsearch]$ ./bin/elasticsearch-setup-passwords interactive
Initiating the setup of passwords for reserved users elastic,apm_system,kibana,kibana_system,logstash_system,beats_system,remote_monitoring_user.
You will be prompted to enter passwords as the process progresses.
Please confirm that you would like to continue [y/N]y
Enter password for [elastic]:
Reenter password for [elastic]:
Enter password for [apm_system]:
Reenter password for [apm_system]:
Enter password for [kibana_system]:
Reenter password for [kibana_system]:
Enter password for [logstash_system]:
Reenter password for [logstash_system]:
Enter password for [beats_system]:
Reenter password for [beats_system]:
Enter password for [remote_monitoring_user]:
Reenter password for [remote_monitoring_user]:
Changed password for user [apm_system]
Changed password for user [kibana_system]
Changed password for user [kibana]
Changed password for user [logstash_system]
Changed password for user [beats_system]
Changed password for user [remote_monitoring_user]
Changed password for user [elastic]
2. kibana安装
2.1 同款解压
2.2 修改配置文件
根据实际情况设置server.port、server.host、server.name、elasticsearch.hosts、elasticsearch.username、elasticsearch.password、i18n.locale等配置
搞这个有点迷，一直以为没部署成功，结果好像一直是地址输错了，贴出我elasticsearch.yml和kibana.yml
elasticsearch.yml；
# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
# Before you set out to tweak and tune the configuration, make sure you
# understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: node-121
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: node-121
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /home/ect/elasticsearch/data
#
# Path to log files:
#
path.logs: /home/ect/elasticsearch/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 0.0.0.0
#
# Set a custom port for HTTP:
#
http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
discovery.seed_hosts: ["127.0.0.1"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
cluster.initial_master_nodes: ["node-121"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
xpack.security.enabled: true
xpack.license.self_generated.type: basic
xpack.security.transport.ssl.enabled: true
http.cors.enabled: true
http.cors.allow-origin: http://localhost:5601
kibana.yml；
# Kibana is served by a back end server. This setting specifies the port to use.
#端口
server.port: 5601
# Specifies the address to which the Kibana server will bind. IP addresses and host names are both valid values.
# The default is 'localhost', which usually means remote machines will not be able to connect.
# To allow connections from remote users, set this parameter to a non-loopback address.
#地址
server.host: "10.4.60.121"
# Enables you to specify a path to mount Kibana at if you are running behind a proxy.
# Use the `server.rewriteBasePath` setting to tell Kibana if it should remove the basePath
# from requests it receives, and to prevent a deprecation warning at startup.
# This setting cannot end in a slash.
#server.basePath: ""
# Specifies whether Kibana should rewrite requests that are prefixed with
# `server.basePath` or require that they are rewritten by your reverse proxy.
# This setting was effectively always `false` before Kibana 6.3 and will
# default to `true` starting in Kibana 7.0.
#server.rewriteBasePath: false
# The maximum payload size in bytes for incoming server requests.
#server.maxPayloadBytes: 1048576
# The Kibana server's name. This is used for display purposes.
#服务名称
server.name: "CT"
# The URLs of the Elasticsearch instances to use for all your queries.
#elasticsearch地址
elasticsearch.hosts: ["http://10.4.60.121:9200"]
# When this setting's value is true Kibana uses the hostname specified in the server.host
# setting. When the value of this setting is false, Kibana uses the hostname of the host
# that connects to this Kibana instance.
#elasticsearch.preserveHost: true
# Kibana uses an index in Elasticsearch to store saved searches, visualizations and
# dashboards. Kibana creates a new index if the index doesn't already exist.
#kibana.index: ".kibana"
# The default application to load.
#kibana.defaultAppId: "home"
# If your Elasticsearch is protected with basic authentication, these settings provide
# the username and password that the Kibana server uses to perform maintenance on the Kibana
# index at startup. Your Kibana users still need to authenticate with Elasticsearch, which
# is proxied through the Kibana server.
#elasticsearch用户名和密码
elasticsearch.username: "elastic"
elasticsearch.password: "elastic"
# Enables SSL and paths to the PEM-format SSL certificate and SSL key files, respectively.
# These settings enable SSL for outgoing requests from the Kibana server to the browser.
#server.ssl.enabled: false
#server.ssl.certificate: /path/to/your/server.crt
#server.ssl.key: /path/to/your/server.key
# Optional settings that provide the paths to the PEM-format SSL certificate and key files.
# These files are used to verify the identity of Kibana to Elasticsearch and are required when
# xpack.security.http.ssl.client_authentication in Elasticsearch is set to required.
#elasticsearch.ssl.certificate: /path/to/your/client.crt
#elasticsearch.ssl.key: /path/to/your/client.key
# Optional setting that enables you to specify a path to the PEM file for the certificate
# authority for your Elasticsearch instance.
#elasticsearch.ssl.certificateAuthorities: [ "/path/to/your/CA.pem" ]
# To disregard the validity of SSL certificates, change this setting's value to 'none'.
#elasticsearch.ssl.verificationMode: full
# Time in milliseconds to wait for Elasticsearch to respond to pings. Defaults to the value of
# the elasticsearch.requestTimeout setting.
#elasticsearch.pingTimeout: 1500
# Time in milliseconds to wait for responses from the back end or Elasticsearch. This value
# must be a positive integer.
#elasticsearch.requestTimeout: 30000
# List of Kibana client-side headers to send to Elasticsearch. To send *no* client-side
# headers, set this value to [] (an empty list).
#elasticsearch.requestHeadersWhitelist: [ authorization ]
# Header names and values that are sent to Elasticsearch. Any custom headers cannot be overwritten
# by client-side headers, regardless of the elasticsearch.requestHeadersWhitelist configuration.
#elasticsearch.customHeaders: {}
# Time in milliseconds for Elasticsearch to wait for responses from shards. Set to 0 to disable.
#elasticsearch.shardTimeout: 30000
# Time in milliseconds to wait for Elasticsearch at Kibana startup before retrying.
#elasticsearch.startupTimeout: 5000
# Logs queries sent to Elasticsearch. Requires logging.verbose set to true.
#elasticsearch.logQueries: false
# Specifies the path where Kibana creates the process ID file.
#pid.file: /var/run/kibana.pid
# Enables you specify a file where Kibana stores log output.
#logging.dest: stdout
# Set the value of this setting to true to suppress all logging output.
#logging.silent: false
# Set the value of this setting to true to suppress all logging output other than error messages.
#logging.quiet: false
# Set the value of this setting to true to log all events, including system usage information
# and all requests.
#logging.verbose: false
# Set the interval in milliseconds to sample system and process performance
# metrics. Minimum is 100ms. Defaults to 5000.
#ops.interval: 5000
# Specifies locale to be used for all localizable strings, dates and number formats.
# Supported languages are the following: English - en , by default , Chinese - zh-CN .
i18n.locale: "zh-CN"
xpack.security.encryptionKey: "122333444455555666666777777788888888"
xpack.reporting.encryptionKey: "122333444455555666666777777788888888"
xpack.encryptedSavedObjects.encryptionKey: "122333444455555666666777777788888888"
后面有机会补一次
3. apm安装
apm是项目监控，安装相对简单
3.1 同款解压
3.2 修改apm-server.yml
修改elasticsearch地址，及将账号密码填入
apm-server:
host: "10.4.60.121:8200"
output.elasticsearch:
hosts: ["localhost:9200"]
username: "elastic"
password: "elastic"
3.3 启动
后台启动
nohub ./apm-server -e >> /dev/null 2>&1 &
查看端口是否在运行
[ect@localhost apm-server-7.8.0-linux-x86_64]$ netstat -anp | grep 8200
3.4 下载apm-agent.jar
https://search.maven.org/#search%7Cga%7C1%7Ca%3Aelastic-apm-agent
这个需要放到需要启动的项目同服务器下，我将这个jar放到根目录下
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-5NsQguGi-1604905286365)(C:\Users\taozheng4\AppData\Roaming\Typora\typora-user-images\image-20201109143235169.png)]
3.5 项目启动
在项目启动命令中添加apm的参数
-javaagent:/home/ect/elastic-apm-agent-1.12.0.jar -Delastic.apm.service_name=eureka-java-application -Delastic.apm.server_url=http://10.4.60.121:8200 -Delastic.apm.secret_token= -Delastic.apm.application_packages=org.example
如
java -javaagent:/home/ect/elastic-apm-agent-1.12.0.jar -Delastic.apm.service_name=eureka-java-application -Delastic.apm.server_url=http://10.4.60.121:8200 -Delastic.apm.secret_token= -Delastic.apm.application_packages=org.example -Xms1024m -Xmx1024m -Xss256k -XX:NewSize=768m -XX:MaxNewSize=768m -XX:MetaspaceSize=128m -XX:+HeapDumpOnOutOfMemoryError -XX:-OmitStackTraceInFastThrow -jar eureka.jar > /dev/null 2>&1 &
显示这个就说明已经成功监控到了
2020-11-09 14:06:34.429 [apm-server-healthcheck] INFO co.elastic.apm.agent.report.ApmServerHealthChecker - Elastic APM server is available: { "build_date": "2020-06-14T17:10:16Z", "build_sha": "06c58bf4e5b675d04314bf44961ffd6b0e13f544", "version": "7.8.0"}
4. metricbeat安装
4.1 同款解压
4.2 修改metricbeat.yml
output.elasticsearch:
hosts: ["<es_url>"]
username: "elastic"
password: "<password>"
setup.kibana:
host: "<kibana_url>"
4.3 启动你想监控的部分
./metricbeat modules enable system
如服务器系统监控
4.4 启动metricbeat
./metricbeat setup
./metricbeat -e
5 filebeat安装
不想写了，和metricbeat一模一样
“build_date”: “2020-06-14T17:10:16Z”, “build_sha”: “06c58bf4e5b675d04314bf44961ffd6b0e13f544”, “version”: “7.8.0”}
### 4. metricbeat安装
#### 4.1 同款解压
#### 4.2 修改metricbeat.yml
```shell
output.elasticsearch:
hosts: ["<es_url>"]
username: "elastic"
password: "<password>"
setup.kibana:
host: "<kibana_url>"
4.3 启动你想监控的部分
./metricbeat modules enable system
如服务器系统监控
4.4 启动metricbeat
./metricbeat setup
./metricbeat -e
1537
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
