filebeat自定义日期类型

项目输出日志格式如下：

DEBUG|2022-10-12 17:13:00|maas|test|http|/api/order/orderusedList|d67dae34340e91e37d5a33e23327|2f8416a9-df83-4910-b71f-f534ef1a9d|ssd|request|{"start_id":0,"count":20}

---

按以下grok收集到es的时候，所有字段都是text类型，现需要将logdate字段转化为date field

      "grok" :{
        "field" : "message",
        "patterns" : ["%{DATA:level}\\|%{DATA:logdate}\\|%{DATA:app}\\|%{DATA:env}\\|%{DATA:type}\\|%{DATA:site}\\|%{DATA:device_id}\\|%{DATA:request_id}\\|%{DATA:from}\\|%{DATA:message}\\|%{DATA:context}"]
      }

---

需要使用date对logdate字段进行处理，自定义pipeline如下：

[root@test filebeat]#cat php-log.json
{
  "description" : "php_log",
  "processors" : [
    {
      "grok" :{
        "field" : "message",
        "patterns" : ["%{DATA:level}\\|%{DATA:logdate}\\|%{DATA:app}\\|%{DATA:env}\\|%{DATA:type}\\|%{DATA:site}\\|%{DATA:device_id}\\|%{DATA:request_id}\\|%{DATA:from}\\|%{DATA:message}\\|%{DATA:context}"]
      }
    },
    {
      "date": {
       "on_failure" : [
         {
           "append" : {
             "field" : "error.message",
             "value" : "{{ _ingest.on_failure_message }}"
           }
         }
          ],
        "field": "logdate",   # 需要处理的字段名
        "target_field": "time",  # 这里会新生成一个time字段
        "timezone": "Asia/Shanghai",  # 不加时区结果会相差8小时
        "formats": [
          "yyyy-MM-dd H:m:s"  # 如果时间格式写HH:mm:ss会与实际时间相差9个月
        ]
      }
    }
  ]
}

#更新pipeline
curl -X PUT "http://192.168.0.171:9200/_ingest/pipeline/php-log" -H 'Content-Type: application/json' -d@php-log.json

---

kibana中修改日期显示格式

---

成功将text类型改为date类型