https://blog.csdn.net/guyan0319/article/details/78749639
版权声明:转载请注明出处 http://blog.csdn.net/guyan0319 https://blog.csdn.net/guyan0319/article/details/78749639 </div>
<link rel="stylesheet" href="https://csdnimg.cn/release/phoenix/template/css/ck_htmledit_views-3019150162.css">
<div id="content_views" class="markdown_views">
<!-- flowchart 箭头图标 勿删 -->
<svg xmlns="http://www.w3.org/2000/svg" style="display: none;">
<path stroke-linecap="round" d="M5,0 0,2.5 5,5z" id="raphael-marker-block" style="-webkit-tap-highlight-color: rgba(0, 0, 0, 0);"></path>
</svg>
<p>架构图 <br>
Elasticsearch:搜索,提供分布式全文搜索引擎;
Logstash: 日志收集,管理,存储;
Kibana :日志的过滤web 展示;
Filebeat:监控日志文件、转发,其已取代 logstash forwarder;
一、准备工作
设置 yum源,采用官网提供的源
https://www.elastic.co/guide/en/elasticsearch/reference/current/rpm.html
下载并安装公共签名密钥:
rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch
- 1
- 2
创建yum的repo文件
vim /etc/yum.repos.d/elasticsearch.repo
- 1
内容如下
[elasticsearch-6.x]
name=Elasticsearch repository for 6.x packages
baseurl=https://artifacts.elastic.co/packages/6.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
二、elasticsearch安装
elasticsearch依赖Java开发环境支持,先安装JDK。
yum -y install java-1.8.0-openjdk
- 1
查看java安装情况
安装Elasticsearch
yum -y install elasticsearch
systemctl start elasticsearch
- 1
- 2
ElasticSearch默认的对外服务的HTTP端口是9200,节点间交互的TCP端口是9300。
ss -tlnp |grep -E '9200|9300'
- 1
测试服务
curl -X GET http://localhost:9200
- 1
三、安装Logstash
yum -y install logstash
systemctl start logstash
- 1
- 2
四、安装Kibana
yum -y install kibana
systemctl start kibana
- 1
- 2
五、浏览器http://localhost:5601
六、配置nginx 访问
vim /etc/nginx/conf.d/kibana.conf
server {
listen 80;
server_name kb.com;
access_log /var/log/nginx/kibana.aniu.co.access.log;
error_log /var/log/nginx/kibana.aniu.co.access.log;
#auth_basic "Restricted Access";
#auth_basic_user_file /etc/nginx/htpasswd.users;
location / {
proxy_pass http://localhost:5601;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
}
}
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
systemctl reload nginx
- 1
七、安装filebeat
yum -y install filebeat
systemctl start filebeat
systemctl enable filebeat
- 1
- 2
- 3
配置 Filebeat
vim /etc/filebeat/filebeat.yml ##配置filebeat
#============= Filebeat prospectors ===============
filebeat.prospectors:
- input_type: log
enabled: true #更改为true以启用此prospectors配置。
paths:
#- /var/log/*.log
- /var/log/messages
#==================== Outputs =====================
#------------- Elasticsearch output ---------------
#output.elasticsearch:
# Array of hosts to connect to.
#hosts: ["localhost:9200"]
#---------------- Logstash output -----------------
output.logstash:
# The Logstash hosts
hosts: ["localhost:5044"]
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
注意:要注释以下两行
output.elasticsearch和output.logstash只能同时开启一个
并且设置
enabled: true #更改为true以启用此prospectors配置。
systemctl restart filebeat
- 1
八、配置logstash
创建配置文件
vim /etc/logstash/conf.d/01-logstash-initial.conf
input {
beats {
port => 5044
type => "logs"
}
}
filter {
#if [type] == "sy" {
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
add_field => [ "received_at", "%{@timestamp}" ]
add_field => [ "received_from", "%{host}" ]
}
geoip {
source => "clientip"
}
syslog_pri {}
date {
match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
# }
}
output {
elasticsearch { hosts => ["localhost:9200"] }
stdout { codec => rubydebug }
}
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
查看端口
ss -tlnp|grep -E ‘5044|9600’
验证logstash配置文件
/usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/01-logstash-initial.conf --config.test_and_exit
- 1
显示Configuration OK 证明配置成功
如果报错:WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using –path.settings. Continuing using
解决办法:
cd /usr/share/logstash
ln -s /etc/logstash ./config
- 1
- 2
九、配置kibana
添加索引
查看状态图
参考资料:
https://www.elastic.co/guide/index.html
http://www.cnblogs.com/hanyifeng/p/5509985.html
http://blog.51cto.com/wangzhijian/1878636#comment