怎么安装Grafana-Loki

How to install Grafana-Loki to query logs from your k8s servers?

Requirements:
System: ubuntu
Helm3 (Assuming installed)
Grafana
Loki
Promtail (send data[logs] to Loki)

Step to install Grafana-loki

1. Install Grafana on k8s:

Refer: https://grafana.com/docs/grafana/latest/setup-grafana/installation/kubernetes/

2. Install data source – Loki by helm3

Refer: https://grafana.com/docs/loki/latest/setup/install/helm/install-monolithic/

Notice:
After installed, You can get a gateway address of Loki, which is used by Promtail later

***********************************************************************
Sending logs to Loki
***********************************************************************

Loki has been configured with a gateway (nginx) to support reads and writes from a single component.

You can send logs from inside the cluster using the cluster DNS:

http://loki-gateway.default.svc.cluster.local/loki/api/v1/push

3. Install Promtail by helm3

Refer: https://grafana.com/docs/loki/latest/send-data/promtail/installation/

Notice:
Put your Loki-Gateway address here, For Loki-Auth required, Add X-Scope-OrgId: foo [Important]

```yaml
# values.yaml
config:
# publish data to loki
  clients:
    - url: http://loki-gateway/loki/api/v1/push
      X-Scope-OrgId: foo
```

Finally, All Requirements are done.

Now you can use Grafana to see all logs you want, But set up Data sources first.

The Problems may appear && How to deal:

1. Label amounts out of limit

# from Promtail you can get the error like below
error: loki has 16 label names; limit 15

Solution: Update your values of Promtail, Try official values for helm install Promtail

2. HTTP 401

level=error ts=2024-09-05T07:53:52.821695784Z caller=client.go:430 component=client host=loki-gateway.default.svc.cluster.local msg="final error sending batch" status=401 tenant= error="server returned HTTP status 401 Unauthorized (401): no org id"

Solution: Add X-Scope-OrgId to values.yaml of Promtail
If this happening when you add data source by using Grafana, Also, add X-Scope-OrgId to Custom Header