ELK centos7

最新推荐文章于 2024-07-20 04:30:00 发布

yuan-ONE

最新推荐文章于 2024-07-20 04:30:00 发布

阅读量1.6k

点赞数 1

原著文章地址：https://www.digitalocean.com/community/tutorials/how-to-install-elasticsearch-logstash-and-kibana-elk-stack-on-centos-7

转载文章2：http://blog.sina.com.cn/s/blog_6f2d2e310102wa41.html

ELK官方地址：https://www.elastic.co/products

官方的下载源在国外比较缓慢本人下载好的安装包：http://pan.baidu.com/s/1o7EIZv8 密码：5zme 最新版本5.1系列

@@@@@@@@@@@@@@@@@@@@@首先安装elasticsearch //@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

groupadd elk && useradd -g elk elk  // elasticsearch 是不能直接用root用户启动新加elk用户启动elasticsearch
chown -R elk.elk  /opt/elasticsearch  // "/opt/elasticsearch " 是我elasticsearch的安装目录（安装包请在百度云下载或者在官方下载）

修改conf下的配置文件elasticsearch.yml 使只能本机访问 network.host: localhost

nohup sh bin/elasticsearch & //后台运行且终端关闭后仍然运行

2017-01-05T02:28:28,702][INFO ][o.e.n.Node ] version[5.1.1], pid[13420], build[5395e21/2016-12-06T12:36:15.409Z], OS[Linux/3.10.0-229.el7.x86_64/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_92/25.92-b14]

[2017-01-05T02:28:29,756][INFO ][o.e.p.PluginsService ] [ZYem2PN] loaded module [aggs-matrix-stats]
[2017-01-05T02:28:29,756][INFO ][o.e.p.PluginsService ] [ZYem2PN] loaded module [ingest-common]
[2017-01-05T02:28:29,756][INFO ][o.e.p.PluginsService ] [ZYem2PN] loaded module [lang-expression]
[2017-01-05T02:28:29,757][INFO ][o.e.p.PluginsService ] [ZYem2PN] loaded module [lang-groovy]
[2017-01-05T02:28:29,757][INFO ][o.e.p.PluginsService ] [ZYem2PN] loaded module [lang-mustache]
[2017-01-05T02:28:29,757][INFO ][o.e.p.PluginsService ] [ZYem2PN] loaded module [lang-painless]
[2017-01-05T02:28:29,757][INFO ][o.e.p.PluginsService ] [ZYem2PN] loaded module [percolator]

curl下本机看下返回结果：

curl 127.0.0.1:9200
结果：

{
  "name" : "ZYem2PN",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "Kpt3lcQDRl-7rq8oQEGZ6Q",
  "version" : {
    "number" : "5.1.1",
    "build_hash" : "5395e21",
    "build_date" : "2016-12-06T12:36:15.409Z",
    "build_snapshot" : false,
    "lucene_version" : "6.3.0"
  },
  "tagline" : "You Know, for Search"
}

&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&安装kibana&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&

sudo rpm -ivh kibana-5.1.1-x86_64.rpm
rpm -qc kibana //查看kibbaba的配置文件
返回结果：/etc/kibana/kibana.yml

更改配置文件：server.host: "localhost"

systemctl enable kibana.service  //开机启动kibana
 systemctl start kibana.service   //启动kibana

%%%%%%%%%%%%%%%%%%%%%安装nginx代理本机kibbaba，elasticsearch ，并增加认证%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

将上面提供下载的epel源添加到/etc/yum.repos.d/ 更新yum源：

yum makecache fast
yum install nginx httpd-tools -y  //yum安装nginx和认证工具：
htpasswd -c /etc/nginx/htpasswd.users admin  //生成密码认证

vim /etc/nginx/conf.d/kibana.conf //创建server 用nginx代理本地访问记得把nginx.conf 的server注释掉

         server {
listen 80;
server_name localhost;
auth_basic "Restricted Access";
auth_basic_user_file /etc/nginx/htpasswd.users;
 
           location / {
proxy_pass http://localhost:5601;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;   
}
}

nginx -t 检查nginx语法没有报错就可以启动nginx systemctl start nginx && systemctl enable nginx

同理可以代理elasticsearch服务的9200端口 // 然后就可以通过web访问kibana

*****************************************************安装logstash*****************************

ln -s /opt/jdk1.8.0_92/bin/java /usr/bin/java  //增加java的软连接 不然安装要报错 我这里jdk的路径是/opt/jdk1.8.0_92/

[root@localhost elk]# rpm -ivh logstash-5.1.1.rpm

Preparing... ################################# [100%]
Updating / installing...
1:logstash-1:5.1.1-1 ################################# [100%]
Using provided startup.options file: /etc/logstash/startup.options
Successfully created system startup script for Logstash

Logstash配置文件使用JSON格式，路径为 /etc/logstash/conf.d/

包含 inputs | filters | outputs 三部分

# vim /etc/logstash/conf.d/02-filebeat-input.conf

input {
  beats {
    port => 5044
    type => "logs"
    ssl => true
    ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt"
    ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"
  }
}

# vim /etc/logstash/conf.d/10-syslog.conf

filter {
  if [type] == "syslog" {
    grok {
      match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
      add_field => [ "received_at", "%{@timestamp}" ]
      add_field => [ "received_from", "%{host}" ]
    }
    syslog_pri { }
    date {
      match => [ "syslog_timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
    }
  }
}

将日志存储到 Elasticsearch [ 跑在本机 9200 端口 ]

# vim /etc/logstash/conf.d/30-elasticsearch-output.conf

output {
    elasticsearch { hosts => ["localhost:9200"] }
    stdout { codec => rubydebug }
}

chmod 777 /var/log/logstash/logstash.log
systemctl start logstash
systemctl enable logstash

！！！！！！！！！！！！！！！！！！！！！！创建通信证书！！！！！！！！！！！！！！！！！！！！！！！！！！！！！！！

请先修改/etc/hosts 文件增加服务器和客服端的解析

cd /etc/pki/tls  
openssl req -subj '/CN=yoursername/' -x509 -days 3650 -batch -nodes -newkey rsa:2048  -keyout private/logstash-forwarder.key -out certs/logstash-forwarder.crt
scp /etc/pki/tls/certs/logstash-forwarder.crt root@youagent:/tmp/    //将证书拷贝到你的agent(agent需要配置证书通信)

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^客户端配置^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

rpm --import http://packages.elastic.co/GPG-KEY-elasticsearch  //导入证书
rpm -ivh filebeat-5.1.1-x86_64.rpm     //安装filebeat
cp /tmp/logstash-forwarder.crt /etc/pki/tls/certs/     //复制server创建的证书

grpe -v“^#” /etc/filebeat/filebeat.yml

filebeat.prospectors:
- input_type: log
  paths:
    - /var/log/*.log
output.logstash:
  hosts: ["elkserver:5044"]


  ssl.certificate_authorities: ["/etc/pki/tls/certs/logstash-forwarder.crt"]

systemctl start filebeat.service 
systemctl enable filebeat.service
filebeat.sh -e -c filebeat.yml -d "Publish"  查看是否能够通向server 请根据日志进行排查

如果不能通信server

@@@@@@@@@@@@@@@@@@@@@@@@@配置kibana信息@@@@@@@@@@@@@@@@@@@@@@@@@

yuan-ONE

关注

1
点赞
踩
0

收藏

觉得还不错? 一键收藏
0
评论
复制链接

分享到 QQ

分享到新浪微博

扫一扫