ELK集群+日志采集

实验逻辑图

1.在IP地址为192.168.153.185的主机上安装nginx和logstash

1.1 安装并开启nginx服务

链接>> https://blog.csdn.net/m0_46674735/article/details/111125960

1.2 安装并开启logstash服务

链接>. https://blog.csdn.net/m0_46674735/article/details/112004006

1.2.1 在 /etc/logstash/conf.d/ 目录下编写conf配置文件，将日志信息输出到Redis数据库中

[root@localhost ~]# cd /etc/logstash/conf.d/
[root@localhost conf.d]# ls
system_nginx.conf
[root@localhost conf.d]# cat system_nginx.conf 
input {
  file {
	path => "/var/log/messages"
	type => "system.log"
	start_position => "beginning"
  }
  file {
	path => "/usr/local/nginx/logs/*.log"
	type => "nginx.log"
	start_position => "beginning"
  }
}

output {
  if [type] == "system.log" {
	  redis {
	    host => "192.168.153.149:6379"
	    data_type => "list"
	    key => "logstash:system-log"
	  }
     }

  if [type] == "nginx.log" {
	  redis {
	    host => "192.168.153.149:6379"
		data_type => "list"
		key => "logstash:nginx-log"
	  }	  
     }
}
[root@localhost conf.d]# 

1.2.2 修改保存之后重启logstash服务

systemctl restart logstash

2.在IP地址为192.168.153.149的主机上安装Redis和logstash

2.1 Redis安装并开启服务

安装链接>> https://blog.csdn.net/m0_46674735/article/details/109823943

2.1.1 使用Redis管理工具查看185主机上的日志信息是否成功存储到Redis数据库中

2.2 logstash安装

安装链接>. https://blog.csdn.net/m0_46674735/article/details/112004006

2.2.1 在 /etc/logstash/ 目录下修改pipelines.yml文件如下：

[root@localhost logstash]# vim pipelines.yml 
- pipeline.id: system
  path.config: "/etc/logstash/conf.d/system.conf"
- pipeline.id: nginx
  path.config: "/etc/logstash/conf.d/nginx.conf"

2.2.2 在 /etc/logstash/conf.d 目录下分别编写nginx.conf 文件和system.conf 文件

nginx.conf 文件内容为：

[root@localhost conf.d]# cat nginx.conf 
input {
  redis { 
     host => "192.168.153.149"
     port => "6379"
     type => "nginx-log"
     data_type => "list"
     key => "logstash:nginx-log"
  }
}
output {
      elasticsearch {
        hosts => "192.168.153.183:9200"
        index => "nginx-log"
      }
}

system.conf 文件内容为：

[root@localhost conf.d]# cat system.conf 
input {
  redis {
     host => "192.168.153.149"
     port => "6379"
     type => "system-log"
     data_type => "list"
     key => "logstash:system-log"       	
  }
}
output {
      elasticsearch {
	hosts => "192.168.153.183:9200"
	index => "system-log"
      }
}

2.2.3 修改保存之后重启logstash服务

systemctl restart logstash

2.2.4 在Redis管理工具中刷新，发现Redis数据库已经清空

3.在IP地址为192.168.153.183的主机上安装elasticsearch服务并开启

安装链接>> https://blog.csdn.net/m0_46674735/article/details/112004006

4.在IP地址为192.168.153.184的主机上安装kibana服务并开启

安装链接>> https://blog.csdn.net/m0_46674735/article/details/112004006

5. 浏览器输入 IP:5601 访问kibana

例：http://192.168.153.184:5601/

5.1 创建索引

5.2 可以看到配置的两个索引

5.3 添加nginx.conf 索引

5.4

5.5 添加成功

5.6 添加system.conf 索引操作相同

5.7 nginx日志监测成功

5.7 系统日志监测成功