Linux centOS7安装ELK技术栈Linux系统安装ElasticSearch8.1.3、Kibana8.1.3、Logstash8.1.3、Filebeat8.1.3

ElasticSearch安装

1、创建目录、进入目录下载压缩包，解压文件

sudo mkdir -p /usr/local/ELK/es
cd /usr/local/ELK/es
sudo wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-8.1.3-linux-x86_64.tar.gz
sudo tar zxvf elasticsearch-8.1.3-linux-x86_64.tar.gz 

2、设置文件夹权限、进入安装的bin目录、启动服务启动时加上 -d 则为后台启动

sudo chown -R manager:manager /usr/local/ELK/es
cd /usr/local/ELK/es/elasticsearch-8.1.3/bin/
./elasticsearch

3、如果远程服务没有配置SSL证书则需要将yml中的 xpack.security.enabled 修改 fasle 才能进行访问

cd /usr/local/ELK/es/elasticsearch-8.1.3/config
sudo vim elasticsearch.yml

Kibana安装

1、创建存储目录、下载Kibanna、解压下载压缩包

sudo mkdir 
sudo wget https://artifacts.elastic.co/downloads/kibana/kibana-8.1.3-linux-x86_64.tar.gz
sudo tar zxvf kibana-8.1.3-linux-x86_64.tar.gz

2、进入安装的Kibana的bin目录启动Kibana

cd /usr/local/ELK/Kibana/kibana-8.1.3/bin/
./kibana 

后台启动、并且查看运行进程，Kibana是一个节点服务器所以搜索 node

nohup  ./kibana-8.1.3/bin/kibana > /usr/local/ELK/Kibana/kibana.log 2>&1 &
ps -ef | grep node

3、设置server.host 进行外网访问，如果不设置的话默认启动说localhost

Logstash安装

1、创建存储目录、下载logstash、解压下载压缩包

sudo mkdir /usr/local/ELK/logstash
cd /usr/local/ELK/logstash
sudo wget https://artifacts.elastic.co/downloads/logstash/logstash-8.1.3-linux-x86_64.tar.gz
sudo tar -zxvf logstash-8.1.3-linux-x86_64.tar.gz

2、配置用户的文件夹权限

sudo chown -R manager:manager /usr/local/ELK/logstash
cd /usr/local/ELK/logstash/logstash-8.1.3/bin/

3、配置 logstash-sample.conf 中的filter 、启动logstash

cd /usr/local/ELK/logstash/logstash-8.1.3/config
vim logstash-sample.conf

match 中的表达式要与 patterns 中内容可以匹配，否则启动时会出现表达式未定义错误

input {
  beats {
    port => 5044
  }
}

filter {
    grok {
       patterns_dir => ["./patterns"]
  match=>{"message"=>"%{DATE1:time1} %{JAVA_SOURCE:source1} %{LEVEL:level1} %{JAVALOGMESSAGE:doc}" }
    }
}

output {
  elasticsearch {
    hosts => ["http://localhost:9200"]
    index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
    #user => "elastic"
    #password => "changeme"
  }
}

patterns文件配置

DATE1    [0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2}
LEVEL   (INFO)|(DEBUG)|(ERROR)|(WARN)|(FATAL)
JAVA_SOURCE [a-zA-Z.<>():0-9]*

启动logstash

cd /usr/local/ELK/logstash/logstash-8.1.3/bin
nohup ./logstash -f /usr/local/ELK/logstash/logstash-8.1.3/config/logstash-sample.conf > /usr/local/ELK/logstash/logstash.log 2>&1 &

Filebeat安装

1、创建存放filebeat目录、进入目录下载二进制安装文件

sudo mkdir /usr/local/ELK/filebeat
cd /usr/local/ELK/filebeat
sudo wget https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-8.1.3-linux-x86_64.tar.gz

2、解压目录，并且为安装目录设置用户权限

sudo tar zxvf filebeat-8.1.3-linux-x86_64.tar.gz
sudo chown -R manager:manager /usr/local/ELK/filebeat

3、设置filebeat.yml配置文件，当前文件为yml格式，需要注意格式上下级，multiline 内容是自定义的

- type: log
  enabled: true
  paths: #这个地址要确保当前账号拥有目录文件的读写创建的权限，否则会启动失败
   - /usr/local/ELK/filebeat/log/*.log
  multiline.pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}'
  multiline.negate: true
  multiline.match: after

4、配置yml管理启动 Filebeat

cd /usr/local/ELK/filebeat/filebeat-8.1.3-linux-x86_64
nohup ./filebeat -e -c filebeat.yml -d "publish"