2021-04-01

ELK日志分析系统

环境准备

规划3个节点，其中1个作为主节点，2个作为数据节点

修改主机名

使用hostnamectl命令修改3个主机名
elk-1节点：

[root@localhost ~]# hostnamectl set-hostname elk-1
// 修改完后ctrl+d退出后重新连接
[root@elk-1 ~]# 

剩下两个同理

配置hosts文件

3个节点配置相同(以elk-1节点为例)

[root@elk-1 ~]# vi /etc/hosts
[root@elk-1 ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.91.138 elk-1
192.168.91.139 elk-2
192.168.91.136 elk-3

安装JDK

部署ELK环境需要jdk1.8以上的JDK版本软件环境，我们使用opnejdk1.8，3节点全部安装（以elk-1节点为例）

[root@elk-1 ~]# yum install -y java-1.8.0-openjdk java-1.8.0-openjdk-devel
……
[root@elk-1 ~]# java -version
openjdk version "1.8.0_242"
OpenJDK Runtime Environment (build 1.8.0_242-b08)
OpenJDK 64-Bit Server VM (build 25.242-b08, mixed mode)

安装Elasticserach

将提供的rpm包上传至3台节点的/root/目录下，或者上传至一节点后使用scp进行拷贝，之后使用rpm命令进行安装，3节点全部安装
软件包下载链接：https://mirrors.tuna.tsinghua.edu.cn/elasticstack/6.x/yum/6.0.0/elasticsearch-6.0.0.rpm
scp复制命令：

[root@elk-1 ~]# scp elasticsearch-6.0.0.rpm elk-3:/root/ 	
The authenticity of host 'elk-3 (192.168.40.13)' can't be established.
ECDSA key fingerprint is f3:72:41:05:79:cd:52:9b:a6:98:f0:5b:e8:5f:26:3d.
Are you sure you want to continue connecting (yes/no)? y  
// 第一次连接会询问你确定连接？第二次连接就会只让你输入密码。
Please type 'yes' or 'no': yes
Warning: Permanently added 'elk-3,192.168.40.13' (ECDSA) to the list of known hosts.
root@elk-3's password:   
// 连接的机器的密码，就是elk-3这台机器root登入的密码。
elasticsearch-6.0.0.rpm      100%  298     0.3KB/s   00:00  

elk-3节点查看是否复制过去：

[root@elk-3 ~]# ls
anaconda-ks.cfg  elasticsearch-6.0.0.rpm

elk-1节点：

[root@elk-1 ~]# rpm -ivh elasticsearch-6.0.0.rpm 
// 参数含义：i表示安装，v表示显示安装过程，h表示显示进度
warning: elasticsearch-6.0.0.rpm: Header V4 RSA/SHA512 Signature, key ID d88e42b4: NOKEY
Preparing... 
################################# [100%]
Creating elasticsearch group... OK
Creating elasticsearch user... OK
Updating / installing...
   1:elasticsearch-0:6.0.0-1          ################################# [100%]
### NOT starting on installation, please execute the following statements to configure elasticsearch service to start automatically using systemd
 sudo systemctl daemon-reload
 sudo systemctl enable elasticsearch.service
### You can start elasticsearch service by executing
 sudo systemctl start elasticsearch.service

配置Elasticserach

配置elasticsearch的配置文件，配置文件：/etc/elasticsearch/elasticsearch.yml

[root@elk-1 ~]# vi /etc/elasticsearch/elasticsearch.yml 
[root@elk-1 ~]# cat /etc/elasticsearch/elasticsearch.yml 
# ======= Elasticsearch Configuration ===========
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ------------------Cluster --------------------
# Use a descriptive name for your cluster:
cluster.name: ELK  
# ------------------------Node -----------------
# Use a descriptive name for the node:
node.name: elk-1	

> node.master: true	 
> node.data: false

# ----------------- Paths ----------------
# Path to directory where to store the data (separate multiple locations by comma):
path.data: /var/lib/elasticsearch 
# Path to log files:
path.logs: /var/log/elasticsearch 
# --------------- Network ------------------
# Set the bind address to a specific IP (IPv4 or IPv6):
network.host: 192.168.40.11  
//设置绑定的ip地址，可以是ipv4或ipv6的，默认为0.0.0.0。
# Set a custom port for HTTP:
http.port: 9200  
# For more information, consult the network module documentation.
# --------------------Discovery ----------------
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#discovery.zen.ping.unicast.hosts: ["host1", "host2"]
discovery.zen.ping.unicast.hosts: ["elk-1","elk-2","elk-3"] 

elk-2，elk-3节点：

[root@elk-2 ~]# vi /etc/elasticsearch/elasticsearch.yml 
[root@elk-2 ~]# cat /etc/elasticsearch/elasticsearch.yml |grep -v ^# |grep -v ^$
cluster.name: ELK
node.name: elk-2
node.master: false
node.data: true
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
network.host: 192.168.40.12
http.port: 9200
discovery.zen.ping.unicast.hosts: ["elk-1","elk-2","elk-3"]

启动服务

通过命令启动es服务，启动后使用ps命令查看进程是否存在或者使用netstat命令查看是否端口启动。

[root@elk-1 ~]# systemctl start elasticsearch
[root@elk-1 ~]# ps -ef |grep elasticsearch
[root@elk-1 ~]# netstat -lntp
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      1446/sshd           
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      1994/master         
tcp6       0      0 192.168.40.11:9200      :::*                    LISTEN      19280/java          
tcp6       0      0 192.168.40.11:9300      :::*                    LISTEN      19280/java          
tcp6       0      0 :::22                   :::*                    LISTEN      1446/sshd           
tcp6       0      0 ::1:25                  :::*                    LISTEN      1994/master 

有以上端口或者进程存在，证明es服务启动成功。

检测集群状态

[root@elk-1 ~]# curl '192.168.40.11:9200/_cluster/health?pretty'
{
  "cluster_name" : "ELK",
  "status" : "green",	
//为green则代表健康没问题，yellow或者red	则是集群有问题
  "timed_out" : false,	
//是否有超时
  "number_of_nodes" : 3, 
//集群中的节点数量
  "number_of_data_nodes" : 2,	
//集群中data节点的数量
  "active_primary_shards" : 1,
  "active_shards" : 2,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 100.0
}