logstash 对于初学者来说是最容易出问题的,所以一下要开 2 个 SSH,一个是命令操作,一个用
看查看日志,要保证不要报错。
1、安装
我们已经配置过Elasticsearch的yum源,这里可以直接使用。
# dnf -y install --enablerepo=elasticsearch logstash
安装完成后查看
# rpm -ql logstash|grep 'logstash/bin'
2、配置
# mv /etc/logstash/logstash.yml /etc/logstash/logstash.yml.demo
# vi /etc/logstash/logstash.yml
内容如下:
http.host: "192.168.1.247"
path.data: /data/logstash
path.logs: /log/logstash
配置pipeline文件(只是一个测试配置,获取messages信息)
配置文件也可暂时不建立也行,不影响,默认情况是没有任何配置的,没配置启动服务没意义!根据默认配置,pipeline实例文件默认应放置于/etc/logstash/conf.d 目录,此时目录下无实例文件,可根据实际情况新建实例,以处理本机messages信息为例,如下:
# vi /etc/logstash/conf.d/messages.conf
内容如下:
# Sample Logstash configuration for creating a simple
# Nginx -> Logstash -> Elasticsearch pipeline.
input {
file {
path => "/log/domain/localhost/access_json.log"
codec => json
start_position => "beginning"
type => "nginx-log"
}
}
output {
if [type] == "nginx-log"{
elasticsearch {
hosts => ["192.168.1.241:9200", "192.168.1.242:9200", "192.168.1.243:9200"]
index => "nginx-log-%{+YYYY.MM.dd}"
}
}
}
目录和服务相关:
# mkdir -p /data/logstash
# mkdir -p /log/logstash
# chown logstash.logstash -R /data/logstash
# chown logstash.logstash -R /log/logstash
# systemctl daemon-reload
# systemctl enable logstash
# systemctl start logstash
# systemctl status logstash
启动发现错误“could not find java; set JAVA_HOME or ensure java is in PATH”。
我们把配置文件中的内容改为绝对路径:
# vi /usr/share/logstash/bin/logstash.lib.sh
修改内容如下:
JAVACMD="/usr/local/jdk/bin/java"
if [ ! -x "$JAVACMD" ]; then
echo "could not find java; set JAVA_HOME or ensure java is in PATH"
exit 1
fi
注:“echo “could not find java; set JAVA_HOME or ensure java is in PATH””是新加的内容
# systemctl start logstash
再次启动,成功了。
logstash收集nginx访问日志
nginx.conf相关内容修改如下:
user nginx;
……
http {
log_format access '$remote_addr - $remote_user [$time_local] $server_name "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent"';
log_format json '{"@timestamp":"$time_iso8601", "@version":"1","client":"$remote_addr",'
'"url":"$uri", "status":"$status", "domain":"$host", "host":"$server_addr",'
'"size":$body_bytes_sent, "responsetime":$request_time, "referer": "$http_referer",'
'"ua": "$http_user_agent"}';
………
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name _;
root /usr/share/nginx/html;
include /etc/nginx/default.d/*.conf;
access_log /log/domain/localhost/access_json.log json;
location / {
}
}
}
重启服务:
# systemctl restart nginx
# systemctl stop logstash
# systemctl start logstash
注:重启服务命令:systemctl restart logstash尽量不要使用,失败的几率很大。
# tail -f /var/log/messages