目录
1.开启json格式所需的字段概述
filebeat配置input要有以下字段
json.keys_under_root: true json.overwrite_keys: true # 默认情况下,解码后的 JSON 位于输出文档中的“json”键下。如果启用此设置,则键将在输出文档中的顶层复制。默认值为 false # 如果启用了此设置,则解码的 JSON 对象中的值将覆盖 Filebeat 在发生冲突时通常添加的字段(类型、源、偏移量等) ## 总的来说就是开启json格式
processors: - decode_json_fields: fields: ["message"] target: "json" # 指定日志字段,头部以json标注,如果不要json标注则设置为空如:target: "" # 注意:如果还需要解析message里面的某个字段则需要把target设置成空fields指定解析字段名:如下: processors: - decode_json_fields: fields: ["message"] target: "" - decode_json_fields: fields: ["o"] target: ""
2.配置示例
filebeat.inputs: - type: log enabled: true paths: - "/data/nginx/access_log" encoding: utf-8 tail_files: false close_inactive: 5m ignore_older: 24h clean_inactive: 96h clean_removed: true fields: hosts: ${HOST:none} k8s_cluster: shtu-global log_type: nginx_access fields_under_root: true processors: - decode_json_fields: # 指定message字段 fields: ["message"] target: "" - add_kubernetes_metadata: in_cluster: true host: ${NODE_NAME} default_indexers.enabled: false default_matchers.enabled: false include_pod_uid: true indexers: - pod_uid: matchers: - logs_path: logs_path: /data/kubelet/pods/ resource_type: pod - type: log enabled: true paths: - "/data/nginx/error.log" encoding: utf-8 tail_files: false close_inactive: 5m ignore_older: 24h clean_inactive: 96h clean_removed: true fields: hosts: ${HOST:none} k8s_cluster: shtu-global log_type: nginx_error fields_under_root: true processors: - decode_json_fields: # 指定message字段 fields: ["message"] target: "" - add_kubernetes_metadata: in_cluster: true host: ${NODE_NAME} default_indexers.enabled: false default_matchers.enabled: false include_pod_uid: true indexers: - pod_uid: matchers: - logs_path: logs_path: /data/kubelet/pods/ resource_type: pod setup.template.enabled: false # 不用模块版,只用我这里配置的 setup.ilm.enabled: false json.keys_under_root: true # 开启json格式 json.overwrite_keys: true filebeat.modules: - module: nginx - module: elasticsearch - module: system - module: kibana output.elasticsearch: hosts: ["elasticsearch-master:9200"] index: tapd-%{[log_type]} indices: when.equals: log_type: ['nginx_access'] when.equals: log_type: ['nginx_error'] logging.level: info logging.to_syslog: false logging.metrics.period: 300s logging.to_files: true logging.files: path: /usr/share/filebeat/logs/ name: "filebeat.log" rotateeverybytes: 10485760 keepfiles: 7
注:我这是多日志监控
查看:
1)首先要增加点数据,比如多访问几次
2)登录kibana创建索引
3)查看数据
3.如果问题没有解决可点击官网
HTTP JSON input |文件点对应 [7.17] |弹性的 (elastic.co)
日志输入|文件点对应 [7.4] |弹性的 (elastic.co)
注:如果看不懂英文鼠标右击翻译成中问,不过也要看浏览器·,win10的默认就可以