基于kubernetes的jaeger环境部署

    本地部署非all-in-one的jaeger，主要分为以下几个步骤。下述步骤中的code均被验证，且image均来自于docker hub。

1. configmap的部署

configmap.yml:

 

apiVersion: v1

kind: ConfigMap

metadata:

  name: jaeger-configuration

  labels:

    app: jaeger

    jaeger-infra: configuration

data:

  span-storage-type: elasticsearch

  collector: |

    es:

      server-urls: http://elasticsearch:9200

      username: elastic

      password: changeme

    collector:

      zipkin:

        http-port: 9411

  query: |

    es:

      server-urls: http://elasticsearch:9200

      username: elastic

      password: changeme

2. jaeger query pod的安装

jaeger-query-deployment.yml:

 

  apiVersion: extensions/v1beta1

  kind: Deployment

  metadata:

    name: jaeger-query

    labels:

      app: jaeger

      jaeger-infra: query-deployment

  spec:

    replicas: 1

    strategy:

      type: Recreate

    template:

      metadata:

        labels:

          app: jaeger

          jaeger-infra: query-pod

        annotations:

          prometheus.io/scrape: "true"

          prometheus.io/port: "16686"

      spec:

        containers:

        - image: jaegertracing/jaeger-query:latest

          name: jaeger-query

          args: ["--config-file=/conf/query.yaml"]

          ports:

          - containerPort: 16686

            protocol: TCP

          readinessProbe:

            httpGet:

              path: "/"

              port: 16687

          volumeMounts:

          - name: jaeger-configuration-volume

            mountPath: /conf

          env:

          - name: SPAN_STORAGE_TYPE

            valueFrom:

              configMapKeyRef:

                name: jaeger-configuration

                key: span-storage-type

        volumes:

          - configMap:

              name: jaeger-configuration

              items:

                - key: query

                  path: query.yaml

            name: jaeger-configuration-volume

       

3. jaeger query servic的安装

jaeger-query-service.yml:

  apiVersion: v1

  kind: Service

  metadata:

    name: jaeger-query

    labels:

      app: jaeger

      jaeger-infra: query-service

  spec:

    ports:

    - name: jaeger-query

      port: 80

      protocol: TCP

      targetPort: 16686

    selector:

      jaeger-infra: query-pod

type: LoadBalancer

4. jaeger collector pod的安装

jaeger-query-collector.yml:

 

  apiVersion: extensions/v1beta1

  kind: Deployment

  metadata:

    name: jaeger-collector

    labels:

      app: jaeger

      jaeger-infra: collector-deployment

  spec:

    replicas: 1

    strategy:

      type: Recreate

    template:

      metadata:

        labels:

          app: jaeger

          jaeger-infra: collector-pod

        annotations:

          prometheus.io/scrape: "true"

          prometheus.io/port: "14268"

      spec:

        containers:

        - image: jaegertracing/jaeger-collector:latest

          name: jaeger-collector

          args: ["--config-file=/conf/collector.yaml"]

          ports:

          - containerPort: 14267

            protocol: TCP

          - containerPort: 14268

            protocol: TCP

          - containerPort: 9411

            protocol: TCP

          readinessProbe:

            httpGet:

              path: "/"

              port: 14269

          volumeMounts:

          - name: jaeger-configuration-volume

            mountPath: /conf

          env:

          - name: SPAN_STORAGE_TYPE

            valueFrom:

              configMapKeyRef:

                name: jaeger-configuration

                key: span-storage-type

        volumes:

          - configMap:

              name: jaeger-configuration

              items:

                - key: collector

                  path: collector.yaml

            name: jaeger-configuration-volume

5. jaeger collector service的安装

jaeger-query-service.yml:

 

apiVersion: v1

kind: List

items:

- apiVersion: v1

  kind: Service

  metadata:

    name: jaeger-collector

    labels:

      app: jaeger

      jaeger-infra: collector-service

  spec:

    ports:

    - name: jaeger-collector-tchannel

      port: 14267

      protocol: TCP

      targetPort: 14267

    - name: jaeger-collector-http

      port: 14268

      protocol: TCP

      targetPort: 14268

    - name: jaeger-collector-zipkin

      port: 9411

      protocol: TCP

      targetPort: 9411

    selector:

      jaeger-infra: collector-pod

    type: ClusterIP

- apiVersion: v1

  kind: Service

  metadata:

    name: zipkin

    labels:

      app: jaeger

      jaeger-infra: zipkin-service

  spec:

    ports:

    - name: jaeger-collector-zipkin

      port: 9411

      protocol: TCP

      targetPort: 9411

    selector:

      jaeger-infra: collector-pod

    type: ClusterIP

6. elasticsearch pod的安装

elasticsearch-statefulset.yml:

 

  apiVersion: apps/v1beta1

  kind: StatefulSet

  metadata:

    name: elasticsearch

    labels:

      app: jaeger

      jaeger-infra: elasticsearch-statefulset

  spec:

    serviceName: elasticsearch

    replicas: 1

    template:

      metadata:

        labels:

          app: jaeger-elasticsearch

          jaeger-infra: elasticsearch-replica

      spec:

        containers:

          - name: elasticsearch

            image: docker.elastic.co/elasticsearch/elasticsearch:5.6.0

            imagePullPolicy: Always

            command:

              - bin/elasticsearch

            args:

              - "-Ehttp.host=0.0.0.0"

              - "-Etransport.host=127.0.0.1"

            volumeMounts:

              - name: data

                mountPath: /data

            readinessProbe:

              exec:

                command:

                - curl

                - --fail

                - --silent

                - --output

                - /dev/null

                - --user

                - elastic:changeme

                - localhost:9200

              initialDelaySeconds: 5

              periodSeconds: 5

              timeoutSeconds: 4

        volumes:

          - name: data

            emptyDir: {}

7. elasticsearch service的安装

elasticsearch-service.yml：

 

apiVersion: v1

  kind: Service

  metadata:

    name: elasticsearch

    labels:

      app: jaeger

      jaeger-infra: elasticsearch-service

  spec:

    clusterIP: None

    selector:

      app: jaeger-elasticsearch

    ports:

    - port: 9200

      name: elasticsearch

    - port: 9300

      name: transport

8. 测试用例的部署

本文使用hotrod作为jaeger的测试用例：

 

8.1. deployment的部署

  apiVersion: extensions/v1beta1

  kind: Deployment

  metadata:

    name: hotrod

  spec:

    template:

      metadata:

        labels:

          app: hotrod

      spec:

        containers:

        - image: jaegertracing/example-hotrod:latest

          name: hotrod

          ports:

          - containerPort: 8080

        - image: registry.jaegertracing/jaeger-agent:latest

          name: jaeger-agent

          ports:

          - containerPort: 5775

            protocol: UDP

          - containerPort: 6831

            protocol: UDP

          - containerPort: 6832

            protocol: UDP

          - containerPort: 5778

            protocol: TCP

          command:

            - "/go/bin/agent-linux"

            - "--collector.host-port=jaeger-collector.logging-test:14267"

8.2. service的部署

apiVersion: v1

kind: Service

metadata:

labels:

app: hotrod

name: hotrod

spec:

type: NodePort

ports:

- port: 8080

targetPort: 8080

nodePort: 30000

selector:

app: hotrod

9. 部署kibana

部署kibana为了很好的与elasticsearch集成，校验数据的存储

 

         

9.2. kibana deployment的部署

  apiVersion: apps/v1

  kind: Deployment

  metadata:

    name: kibana-logging

    labels:

      name: kibana-logging

      kubernetes.io/cluster-service: "true"

      addonmanager.kubernetes.io/mode: Reconcile

  spec:

    replicas: 1

    selector:

      matchLabels:

        name: kibana-logging

    template:

      metadata:

        labels:

          name: kibana-logging

        annotations:

          seccomp.security.alpha.kubernetes.io/pod: 'docker/default'

      spec:

        containers:

        - name: kibana-logging

          image: docker.elastic.co/kibana/kibana:5.6.0

          resources:

            # need more cpu upon initialization, therefore burstable class

            limits:

              cpu: 1000m

            requests:

              cpu: 100m

          env:

            - name: ELASTICSEARCH_URL

              value: http://elasticsearch:9200

            - name: ELASTICSEARCH_USERNAME

              value: elastic

            - name: ELASTICSEARCH_PASSWORD

              value: changeme

          ports:

          - containerPort: 5601

            name: ui

            protocol: TCP

9.2. service的部署

  apiVersion: v1

  kind: Service

  metadata:

    name: kibana-logging

    labels:

      name: kibana-logging

      kubernetes.io/cluster-service: "true"

      kubernetes.io/name: "Kibana"

  spec:

    type: NodePort

    ports:

    - port: 5601

      protocol: TCP

      targetPort: ui

      nodePort: 30012

    selector:

      name: kibana-logging

10. 验证部署

部署之后:

    访问 http://localhost:80  即可进入jaeger界面

 

    访问 http://localhost:30000  即可进入hotrod界面

 

    访问 http://localhost:30012  即可进入kibana界面, 如果需要登录 输入admin/admin