ELK日志系统部署
部署准备
环境
系统Centos7
关闭防火墙、selinux
安装包
链接
ELS:
http://39.105.114.151/elasticsearch-7.6.2-x86_64.rpm
Filebeat:
http://39.105.114.151/filebeat-7.6.2-x86_64.rpm
Kibana:
http://39.105.114.151/kibana-7.6.2-x86_64.rpm
配置好hosts直接wget即可。
ES单机版安装部署
ES的安装：
yum -y localinstall elasticsearch-7.6.2-x86_64.rpm
JVM的内存限制更改，根据服务器内存情况来改
vi /etc/elasticsearch/jvm.options，
-Xms200M
-Xmx200M
ES单实例配置文件
vi /etc/elasticsearch/elasticsearch.yml，single-node代表单机运行
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
network.host: 0.0.0.0
http.port: 9200
前面都是配置文件里面有的，后面是需要添加的。
xpack.security.enabled: true
discovery.type: single-node
启动ES
systemctl enable elasticsearch
systemctl restart elasticsearch
观察日志，检查端口
netstat -lntp |grep 9200
ES启动后第一步需要设置密码123456，采用自己设置密码
ES自己设置密码/usr/share/elasticsearch/bin/elasticsearch-setup-passwords interactive
ES设置随机密码/usr/share/elasticsearch/bin/elasticsearch-setup-passwords auto
验证启动是否成功
curl -u elastic:123456 http://localhost:9200
http://xxx:9200/_cat/indices?v
curl -u elastic:sjgpwd -X POST http://192.168.237.50:9200/sjg/_doc -H ‘Content-Type: application/json’ -d ‘{“name”: “sjg”, “age”: 30}’
http://xxx:9200/sjg/_search?q=*
`[root@centos7 tmp]# curl -u elastic:123456 http://localhost:9200
{
"name" : "centos7",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "b1G9XzzGSeGlQRvjlYyAGw",
"version" : {
"number" : "7.6.2",
"build_flavor" : "default",
"build_type" : "rpm",
"build_hash" : "ef48eb35cf30adf4db14086e8aabd07ef6fb113f",
"build_date" : "2020-03-26T06:34:37.794943Z",
"build_snapshot" : false,
"lucene_version" : "8.4.0",
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}`ELS重置密码
1、修改配置文件config/elasticsearch.yml，注释掉支持x-pack的xpack.security.enabled: true配置，保存退出；
2、重启ELS服务Elasticsearch进程
3、启动成功后，使用curl查看当前Elasticsearch的索引：
curl -XGET “127.0.0.1:9200/_cat/indices” -H ‘Content-Type: application/json’
4、删除.security-7的索引
curl -XDELETE 127.0.0.1:9200/.security-7
5、有如下打印，表示已成功删除：
{“acknowledged”:true}
6、修改配置文件config/elasticsearch.yml，放开x-pack的xpack.security.enabled: true配置，保存退出；
7、重启ELS服务Elasticsearch进程
8、ES自己设置密码/usr/share/elasticsearch/bin/elasticsearch-setup-passwords interactive
Kibana的安装及配置
Kibana的安装
Kibana包含前端展示、es操作简化
yum localinstall kibana-7.6.2-x86_64.rpm -y
[root@centos7 tmp]# yum localinstall kibana-7.6.2-x86_64.rpm -y
已加载插件：fastestmirror
正在检查 kibana-7.6.2-x86_64.rpm: kibana-7.6.2-1.x86_64
kibana-7.6.2-x86_64.rpm 将被安装
正在解决依赖关系
--> 正在检查事务---> 软件包 kibana.x86_64.0.7.6.2-1 将被 安装--> 解决依赖关系完成
依赖关系解决
======================================================================================================================================================================================
Package 架构 版本 源 大小
======================================================================================================================================================================================
正在安装:
kibana x86_64 7.6.2-1 /kibana-7.6.2-x86_64 677 M
事务概要
======================================================================================================================================================================================
安装 1 软件包
总计：677 M
安装大小：677 M
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
正在安装 : kibana-7.6.2-1.x86_64 1/1
验证中 : kibana-7.6.2-1.x86_64 1/1
已安装:
kibana.x86_64 0:7.6.2-1
完毕！Kibana配置修改kibana.yml，连接es的用户名密码需要正确
vi /etc/kibana/kibana.yml
server.port: 5601
server.host: “0.0.0.0”
elasticsearch.hosts: [“http://localhost:9200“]
elasticsearch.username: “elastic”
elasticsearch.password: “123456”
logging.dest: /tmp/kibana.log
Kibana的启动和访问
systemctl enable kibana
systemctl restart kibana
检查端口、访问kibana、登录尝试
[root@centos7 tmp]# systemctl enable kibana
Created symlink from /etc/systemd/system/multi-user.target.wants/kibana.service to /etc/systemd/system/kibana.service.
[root@centos7 tmp]# systemctl restart kibana
[root@centos7 tmp]# netstat -lntp |grep 5601
tcp 0 0 0.0.0.0:5601 0.0.0.0:* LISTEN 2195/node
[root@centos7 tmp]# curl -u elastic:123456 http://localhost:9200
{
"name" : "centos7",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "b1G9XzzGSeGlQRvjlYyAGw",
"version" : {
"number" : "7.6.2",
"build_flavor" : "default",
"build_type" : "rpm",
"build_hash" : "ef48eb35cf30adf4db14086e8aabd07ef6fb113f",
"build_date" : "2020-03-26T06:34:37.794943Z",
"build_snapshot" : false,
"lucene_version" : "8.4.0",
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}Kibana简化ES的操作
验证集群是否成功
curl -u elastic:123456 http://localhost:9200 -> GET /
curl -u elastic:123456 http://localhost:9200_cat/nodes?v -> GET /_cat/nodes?v
curl -u elastic:123456 http://localhost:9200_cat/indices?v -> GET /_cat/indices?v
由于地址、用户名、密码已经配置在Kibana，所以可以直接简化访问
登录网络页面
Filebeat轻量客户端的引入
不需要使用正则的时候
可直接用filebeat发送日志给es，用得比较少
用得比较多filebeat -> Logstash做一些日志的分析提取
Filebeat的二进制安装
yum localinstall filebeat-7.6.2-x86_64.rpm -y
先创建索引模板，不然filebeat创建的索引模板无用内容较多
PUT _template/sjgtemplate
{
“index_patterns”: [“sjg*”],
“settings”:{
“number_of_shards”: 2,
“number_of_replicas”: 0
}
}
Filebeat发送日志到ES配置filebeat.yml
vi /etc/filebeat/filebeat.yml
filebeat.inputs:
- type: log
tail_files: true
backoff: "1s"
paths:
/usr/local/nginx/logs/*.log
processors:
- drop_fields:
fields: ["agent","ecs","input"]
output:
elasticsearch:
hosts: ["127.0.0.1:9200"]
username: elastic
password: 123456
index: "nginx-secure-%{+YYYY.MM.dd}"
setup.template.name: "sjgtemplate"
setup.template.pattern: "sjg*"
setup.ilm.enabled: false验证
要在kibana上面添加索引模板和filebeat的自定义配置文件
添加自定义配置文件
添加索引模板
最后检索nginx访问状态在404的
至此一个简单的单机版ELK日志系统搭建完成。
扫一扫
65
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
