一、Elasticserach安装
1、Installation(elastic 6.3.2 版本 依赖 java JDK8)
下载合适的版本:
curl -L -O https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.4.0.tar.gz
tar -xvf elasticsearch-6.4.0.tar.gz
cd elasticsearch-6.4.0/bin
./elasticsearch
2、集群健康检查
curl -XGET http://127.0.0.1:9200/_cat/health?v 集群健康查看
curl -XGET http://127.0.0.1:9200/_cat/nodes?v 节点状态查看
curl -XGET http://127.0.0.1:9200/_cat/indices?v 查看索引
curl -XPUT http://127.0.0.1:9200/customer?pretty 添加customer索引
curl -XDELETE http://127.0.0.1:9200/customer?pretty 删除索引
3、Config
cluster.name: 集群名
node.name: 节点名
path.data: 数据存储路径
path.logs: 日志存储路径
network.host: 监听地址
http.port: 监听端口
JVM 配置、logging配置 参考官方文档
重要参数:
Path settings
Cluster name
Node name
Network host
Discovery settings
Heap size
Heap dump path
GC logging
Temp directory
重要的系统参数:
Disable swapping
Increase file descriptors
Ensure sufficient virtual memory
Ensure sufficient threads
JVM DNS cache settings
4、Running as daemon
./bin/elasticsearch -d -p pid 启动
kill cat pid
停止
5、Set up X-Pack
参考x-pack破解方法
二、kibana安装及配置
wget https://artifacts.elastic.co/downloads/kibana/kibana-6.4.0-linux-x86_64.tar.gz
tar -xzf kibana-6.4.0-linux-x86_64.tar.gz
cd kibana-6.4.0-linux-x86_64/
1、启动 ./bin/kibana
2、config:
server.port: 5601
server.host: "192.168.12.81"
kibana.index: ".kibana"
elasticsearch.username: "elastic"
elasticsearch.password: "dinpay"
三、logstash安装及配置
wget https://artifacts.elastic.co/downloads/logstash/logstash-6.4.0.tar.gz
配置 filebeat 收集Nginx日志: (Nginx.conf)
input {
beats {
port => 5044
codec => "json"
}
}
output {
elasticsearch {
hosts => ["127.0.0.1:9200"]
index => "test1-nginx-access-%{+YYYY.MM.dd}"
template_overwrite => true
user => elastic
password => dinpay
}
}
配置logstash.yml elastic安装在本机(没有配置https)
xpack.monitoring.enabled: true
xpack.monitoring.elasticsearch.username: elastic
xpack.monitoring.elasticsearch.password: dinpay
xpack.monitoring.elasticsearch.url: ["http://127.0.0.1:9200"]
四、filebeat安装及配置
tar -xf filebeat-6.4.0-linux-x86_64.tar.gz
cd filebeat-6.4.0-linux-x86_64
配置 filebeat.yml
-
type: log
paths:- /var/log/nginx/access.log
output.logstash:
hosts: ["192.168.12.81:5044"]
xpack.monitoring:
enabled: true
elasticsearch:
hosts: ["http://192.168.12.81:9200", "http://192.168.12.81:9200"]
username: elastic
password: dinpay
五、Nginx 日志格式配置:
log_format json '{"@timestamp":"$time_iso8601",'
'"host":"$server_addr",'
'"clientip":"$clientRealIp",'
'"remote_user":"$remote_user",'
'"request":"$request",'
'"http_user_agent":"$http_user_agent",'
'"size":$body_bytes_sent,'
'"responsetime":$request_time,'
'"upstreamtime":"$upstream_response_time",'
'"upstreamhost":"$upstream_addr",'
'"http_host":"$host",'
'"url":"$uri",'
'"domain":"$host",'
'"xff":"$http_x_forwarded_for",'
'"referer":"$http_referer",'
'"status":"$status",'
'"ss":"$upstream_status"}';
access_log /var/log/nginx/access.log json;
map $http_x_forwarded_for $clientRealIp {
"" $remote_addr;
~^(?P<firstAddr>[0-9\.]+),?.*$ $firstAddr;
}
转载于:https://blog.51cto.com/hjt353/2174863