centos7 安装 logstash 7+

下载并安装公开签名密钥

GPG在Linux上的应用主要是实现官方发布的包的签名机制。
GPG分为公钥及私钥。
公钥：顾名思意，即可共享的密钥，主要用于验证私钥加密的数据及签名要发送给私钥方的数据。
私钥：由本地保留的密钥，用于签名本地数据及验证用公钥签名的数据。

实现原理(以Red Hat签名为例)：
1>RH在发布其官方的RPM包时（如本地RHEL光盘及FTP空间包），会提供一个GPG密钥文件，即所谓的公钥。
2>用户下载安装这个RPM包时，引入RH官方的这个RPM GPG公钥，用来验证RPM包是不是RH官方签名的。
导入GPG-KEY:
可以去https://www.redhat.com/security/team/key/或/etc/pki/rpm-gpg查找相应的GPG密钥，并导入到RPM:
rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY

sudo rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

如果出现error: https://artifacts.elastic.co/GPG-KEY-elasticsearch: import read failed(2)访问超时的方式
则用以下方法

sudo wget https://artifacts.elastic.co/GPG-KEY-elasticsearch --no-check-certificate
sudo rpm --import GPG-KEY-elasticsearch

配置yum源

vi /etc/yum.repos.d/logstash.repo

[logstash-7.x]
name=Elastic repository for 7.x packages
#baseurl=https://artifacts.elastic.co/packages/7.x/yum  因为国外站点速度很慢改用下面地址
baseurl=https://mirror.tuna.tsinghua.edu.cn/elasticstack/7.x/yum/
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

安装logstash

[root@master2 src]# sudo yum install logstash
Loaded plugins: fastestmirror, langpacks
base                                                                                                                                                                                                                  | 3.6 kB  00:00:00     
elrepo                                                                                                                                                                                                                | 2.9 kB  00:00:00     
extras                                                                                                                                                                                                                | 2.9 kB  00:00:00     
logstash-7.x                                                                                                                                                                                                          | 2.9 kB  00:00:00     
updates                                                                                                                                                                                                               | 2.9 kB  00:00:00     
logstash-7.x/primary_db                                                                                                                                                                                               | 274 kB  00:00:02     
Loading mirror speeds from cached hostfile
 * base: ftp.sjtu.edu.cn
 * elrepo: mirrors.tuna.tsinghua.edu.cn
 * extras: mirrors.163.com
 * updates: mirrors.163.com
Resolving Dependencies
--> Running transaction check
---> Package logstash.noarch 1:7.8.0-1 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

=============================================================================================================================================================================================================================================
 Package                                                 Arch                                                  Version                                                     Repository                                                   Size
=============================================================================================================================================================================================================================================
Installing:
 logstash                                                noarch                                                1:7.8.0-1                                                   logstash-7.x                                                160 M

Transaction Summary
=============================================================================================================================================================================================================================================
Install  1 Package

Total download size: 160 M
Installed size: 160 M
Is this ok [y/d/N]: 
Downloading packages:
logstash-7.8.0.rpm                                                                                                                                                                                                    | 160 MB  00:00:12     
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : 1:logstash-7.8.0-1.noarch                                                                                                                                                                                                 1/1 
Using provided startup.options file: /etc/logstash/startup.options
OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/pleaserun-0.0.31/lib/pleaserun/platform/base.rb:112: warning: constant ::Fixnum is deprecated
Successfully created system startup script for Logstash
  Verifying  : 1:logstash-7.8.0-1.noarch                                                                                                                                                                                                 1/1 

Installed:
  logstash.noarch 1:7.8.0-1                                                                                                                                                                                                                  

Complete!

软连接各安装目录到指定目录，日常规范，不是必要步骤

ln -s /etc/logstash/ /apps/conf/logstash
ln -s /usr/share/logstash/ /apps/svr/logstash

配置logstash获取nginx日志内容

cd /apps/conf/logstash/conf.d
vi nginx.conf
input{
        file{
               path => "/apps/logs/nginx/kibana-access.log"
               codec => json
               start_position => "beginning"
               type => "nginx-log"
        }
}
output{
        stdout{
                codec => rubydebug
               }
}

运行logstash

直接用命令运行，进行配置文件检测

[root@master2 conf.d]# /apps/svr/logstash/bin/logstash -f /apps/conf/logstash/conf.d/nginx.conf --config.reload.automatic

{
            "request_method" => "POST",
              "request_time" => "0.055",
                   "request" => "POST /api/console/proxy?path=_mapping&method=GET HTTP/1.1",
               "remote_user" => "-",
             "http_referrer" => "http://kibana.liumaster.com/app/kibana",
                    "status" => "200",
                  "@version" => "1",
      "http_x_forwarded_for" => "-",
           "upstream_status" => "200",
                      "path" => "/apps/logs/nginx/kibana-access.log",
                      "type" => "nginx-log",
                      "host" => "kibana.liumaster.com",
           "body_bytes_sent" => "21149",
           "http_user_agent" => "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.113 Safari/537.36",
               "remote_addr" => "192.168.254.1",
                "@timestamp" => 2020-07-03T08:41:26.000Z,
    "upstream_response_time" => "0.054"
}
正常输出，没问题。

新增输出至elasticsearch配置

input{
        file{
               path => "/apps/logs/nginx/kibana-access.log"
               codec => json
               start_position => "beginning"
               type => "nginx-log"
        }
}
output{
           elasticsearch{
                hosts => ["192.168.254.130:9200"]
                user => 'elastic'
                password => 'sHq5wTnRc08yrCcqU9gD'
                index => "nginx-log-%{+YYYY.MM.dd}"
        }
      #  stdout{
      #         codec => rubydebug
      #         }
}

设置开机启动

systemctl enable logstash.service

启动服务

systemctl start logstash.service