Logql入门

logql 简单示例

Log Queries

基本的日志搜索语句包含两个部分

• log stream selector
• log pipeline

loki的设计规定，log stream selector是必不可少的。log stream selector决定了多少日志文件被查找，一个好的log stream selector可以减少被查找的日志，并且也会影响查找性能。

一个完整的查询示例

{container="query-frontend",namespace="tempo-dev"} |= "metrics.go" | logfmt | duration > 10s and throughput_mb < 500

• log stream selector {container=“query-frontend”,namespace=“tempo-dev”}
• log pipeline |= “metrics.go” | logfmt | duration > 10s and throughput_mb < 500

label matching operator

• = 直接相等
• != 不等于
• =~ 正则匹配
• !~ 正则不匹配

Log pipeline

log pipeline可以有以下几方面组成

• Line Filter Expression
• Parser Expression
• Label Filter Expression
• Line Format Expression
• Labels Format Expression
• Unwrap Expression

Line Filter Expression

• |= 包含
• != 不包含
• |~ 正则匹配
• !~ 正则不匹配

Parser Expression

可以从日志中解析和提取标签关键词，目前支持json、logfmt和正则解析

• | json

示例

{ "a.b": {c: "d"}, e: "f" }

转换为

{a_b_c="d", e="f"}

• | logfmt

示例

at=info method=GET path=/ host=grafana.net fwd="124.133.124.161" connect=4ms service=8ms status=200	

转换为

"at" => "info"
"method" => "GET"
"path" => "/"
"host" => "grafana.net"
"fwd" => "124.133.124.161"
"service" => "8ms"
"status" => "200"

• | regexp

示例

| regexp "(?P<method>\\w+) (?P<path>[\\w|/]+) \\((?P<status>\\d+?)\\) (?P<duration>.*)"

POST /api/prom/api/v1/query_range (200) 1.5s

转换为

"method" => "POST"
"path" => "/api/prom/api/v1/query_range"
"status" => "200"
"duration" => "1.5s"

Label Filter Expression

可以使用原生或者提取的标签来过滤日志，可以同时包含多个过滤操作

• String
• Duration
• Number
• Bytes

示例

logfmt | duration > 1m and bytes_consumed > 20MB

Line Format Expression

使用text/template格式可以重写log

Labels Format Expression

| label_format 表达式可以重命名、修改、增加标签

Unwrap Expression

Metric Queries

Log Range Aggregations

• rate(log-range) : calculates the number of entries per second
• count_over_time(log-range) : counts the entries for each log stream within the given range.
• bytes_rate(log-range) : calculates the number of bytes per second for each stream.
• bytes_over_time(log-range) : counts the amount of bytes used by each log stream for a given range.

logql官方文档