kibana数据可视化

kibana简介

Kibana 核心产品搭载了一批经典功能:柱状图、线状图、饼图、旭日图,等等。

将地理数据融入任何地图

精选的时序性 UI,对您Elasticsearch 中的数据执行高级时间序列分析。

利用 Graph 功能分析数据间的关系

Kibana 开发工具为开发人员提供了多种强大方法来帮助其与Elastic Stack 进行交互。

kibana安装与配置

kibana下载

https://elasticsearch.cn/download/

kibana配置:

[root@server5 ~]# rpm -ivh kibana-7.6.1-x86_64.rpm
[root@server5 kibana]# vim kibana.yml
[root@server5 kibana]# systemctl start kibana.service 
[root@server5 kibana]# netstat -antlp| grep :5601

访问kibana 172.25.3.5:5601

创建索引匹配

注意时间范围的选择

基于ES索引创建可视化

生成访问量可视化

创建折线图

仪表盘实时检测访问

保证数据采集打开，可以在其他主机进行压测

[root@server5 conf.d]# logstash -f apache.conf
[root@zhenji Desktop]# ab -c1 -n100 http://172.25.3.5/index.html

轻量级数据采集(filebeat)

实验环境，给server4安装httpd

[root@server4 ~]# yum install httpd -y
[root@server4 ~]# systemctl start httpd
[root@server4 ~]# cd /var/www/html/
[root@server4 html]# echo server4 > index.html
[root@server4 html]# chmod 755 /var/log/httpd/

下载安装配置filebeat

get filebeat-7.6.1-x86_64.rpm
[root@server4 ~]# rpm -ivh filebeat-7.6.1-x86_64.rpm
[root@server4 ~]# cd /etc/filebeat/
[root@server4 filebeat]# vim filebeat.yml 
[root@server4 filebeat]# systemctl start filebeat.service

效果展示

多索引

[root@server4 filebeat]# vim filebeat.yml
setup.template.name: "server4"
setup.template.pattern: "server4-*"
setup.ilm.enabled: false
output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["172.25.3.1:9200"]
  index: "server4-%{+yyy.MM.dd}"

为了可以看到效果，在其他节点也安装

[root@server1 ~]# rpm -ivh filebeat-7.6.1-x86_64.rpm
[root@server1 ~]# cd /etc/filebeat/
[root@server1 filebeat]# vim filebeat.yml 
setup.template.name: "server1"
setup.template.pattern: "server1-*"
setup.ilm.enabled: false
output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["172.25.3.1:9200"]
  index: "server1-%{+yyy.MM.dd}"
[root@server1 filebeat]# systemctl start filebeat.service
[root@server1 ~]# cd /var/www/html/
[root@server1 html]# echo server1 > index.html
[root@server1 html]# systemctl start httpd.service

[root@zhenji Desktop]# ab -c1 -n100 http://172.25.3.1/index.html
[root@zhenji Desktop]# ab -c1 -n100 http://172.25.3.4/index.html

这样做，将根据主机名将索引分离开来。

output.logstash模式

[root@server1 filebeat]# vim filebeat.yml
[root@server1 filebeat]# systemctl restart filebeat.service

[root@server5 conf.d]# vim apache.conf 
input {
	beats {
		port => 5044
}
	#file {
	#path => "/var/log/httpd/access_log"
	#start_position => "beginning"
	#}
	#syslog {}
}

filter{
	grok {
	 match => {"message" => "%{HTTPD_COMBINEDLOG}"}
}
}
output {
	#file {
	#	path => "/tmp/logstash.txt"
	#	codec => line { format => "custom format: %{message}"} 
	#}
	elasticsearch {
	hosts => ["172.25.3.1:9200"]
	index => "apachelogsss-%{+yyyy.MM.dd}"
	}
	stdout { }
}
[root@server5 conf.d]# logstash -f apache.conf

启用xpack安全验证

集群模式需要先创建证书:

# cd /usr/share/elasticsearch/
# bin/elasticsearch-certutil ca
# bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12
# cp elastic-certificates.p12 elastic-stack-ca.p12 /etc/elasticsearch
# cd /etc/elasticsearch
# chown elasticsearch elastic-certificates.p12 elastic-stack-ca.p12

配置所有的elasticsearch集群节点:

# vim /etc/elasticsearch/elasticsearch.yml
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: /etc/elasticsearch/elastic-
certificates.p12
xpack.security.transport.ssl.truststore.path: /etc/elasticsearch/elastic-
certificates.p12

ES集群重启正常后,设置用户密码:

设置kibana连接ES的用户密码:

# vim /etc/kibana/kibana.yml
elasticsearch.username: "kibana"
elasticsearch.password: "westos"

设置Logstash连接ES用户密码:

output {
elasticsearch {
hosts => "172.25.0.13:9200"
index => "apachelog-%{+YYYY.MM.dd}"
user => "elastic"
password => "westos"
}
}

head访问:
• http.cors.allow-headers: Authorization,X-Requested-With,Content-Length,Content-Type //添加参数到es配置
• http://172.25.0.13:9100/?auth_user=elastic&auth_password=westos