fiebeat+elasticsearch+kibana 分析nginx日志

 1. 软件版本(注意软件版本一致性)
nginx-1.14.0
filebeat-5.6.4
elasticsearch-5.6.4
kibana-5.6.4
-----------------------------------------------------------------------------
 2. 安装方法
nginx安装过程省略，配置nginx log格式
nginx 的日志里含有中文的话，会将中文转为 Unicode 编码，如果不转的话，加入 escape=json 参数就可以了
log_format json escape=json'{ "@timestamp": "$time_iso8601", '
				              '"time": "$time_iso8601", '
			                   '"remote_addr": "$remote_addr", '
			                   '"remote_user": "$remote_user", '
			                   '"body_bytes_sent": "$body_bytes_sent", '
			                   '"request_time": "$request_time", '
			                   '"status": "$status", '
			                   '"host": "$host", '
			                   '"request": "$request", '
			                   '"request_method": "$request_method", '
			                   '"uri": "$uri", '
			                   '"http_referrer": "$http_referer", '
			                   '"body_bytes_sent":"$body_bytes_sent", '
			                   '"http_x_forwarded_for": "$http_x_forwarded_for", '
			                   '"http_user_agent": "$http_user_agent" '
			                   '}';
access_log  /opt/nginx/logs/access.log  json; 
   
filebeat安装配置
	#fiebeat.yml配置内容
	filebeat.prospectors:
	- input_type: log
	  enabled: true
	  paths:
	    - /opt/nginx/logs/*access*.log
	  tags: ["nginx-accesslog"]
	  document_type: nginxaccess
	  json.keys_under_root: true
	  json.overwrite_keys: true
	- input_type: log
	  enabled: true
	  paths:
	    - /opt/nginx/logs/*error*.log
	  tags: ["nginx-errorlog"]
	  document_type: nginxerr
	output.elasticsearch:
	  hosts: ["192.168.10.6:9200"]
	  index: "nginx_%{+yyyy.MM.dd}"
启动filebeat ,进入软件安装目录
nohup sudo ./filebeat -e -c filebeat.yml >/dev/null 2>&1 &

elasticsearch安装配置
#elasticsearch.yml配置内容（/opt/elasticsearch-5.6.4/config/elasticsearch.yml）

# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
#cluster.name: my-application
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: node-1
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /data
#
# Path to log files:
#
path.logs: /logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
bootstrap.memory_lock: false
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 192.168.10.6
#
# Set a custom port for HTTP:
#
http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.zen.ping.unicast.hosts: ["host1", "host2"]
#
# Prevent the "split brain" by configuring the majority of nodes (total number of master-eligible nodes / 2 + 1):
#
#discovery.zen.minimum_master_nodes: 3
#
# For more information, consult the zen discovery module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
#allow origin
http.cors.enabled: true
http.cors.allow-origin: "*"

修改jvm内存（/opt/elasticsearch-5.6.4/config/jvm.options）
-Xms1g
-Xmx1g
启动应用  su - es -c "/opt/elasticsearch-5.6.4/bin/elasticsearch -d"
安装head插件
5.0以上，elasticsearch-head 不能放在elasticsearch的 plugins、modules 目录下，否则elasticsearch启动会报错
git clone https://github.com/mobz/elasticsearch-head.git
安装node js
curl -sL https://rpm.nodesource.com/setup_8.x | bash 
yum install -y nodejs
安装grunt
npm install -g grunt-cli 
npm install
vi /opt/elasticsearch-5.6.4/elasticsearch-head/Gruntfile.js
 connect: {
                        server: {
                                options: {
                                        port: 9100,
                                        hostname: '*',  #此行为新增加
                                        base: '.',
                                        keepalive: true
                                }
                        }
                }

        })；
vi /opt/elasticsearch-5.6.4/elasticsearch-head/_site/app.js
this.base_uri = this.config.base_uri || this.prefs.get("app-base_uri") || "http://192.168.10.6:9200";#修改localhost为定义的地址
运行head
grunt server &

kibana图形展示