kubernetes StatefulSet 部署 elasticsearch cluster

使用Kubernetes StatefulSet部署Elasticsearch.

Service 配置

下面是一份elasticsearch service的配置。

apiVersion: v1
kind: Service
metadata:
  name: es-cluster-node
spec:
  clusterIP: None
  selector:
    app: es-cluster
  ports:
    - name: transport
      port: 9300
      protocol: TCP

---

# cluster ip
apiVersion: v1
kind: Service
metadata:
  name: es-cluster-external
spec:
  selector:
    app: es-cluster
  ports:
    - name: http
      port: 80
      targetPort: 9200
  type: ClusterIP

• 上面的service es-cluster-node把clusterIP设置为None，这种service被称为headless service。具体什么是headless service请自行搜索一下。es-cluster-node设置的端口是9300，用于elasticsearch node之间通信，选举等，走的是tcp协议。
• 下面的service es-cluster-external 是一个常用的service类型，映射9200到80端口，这个service用于外部访问elasticsearch。
• 这2个service的selector都为app: es-cluster，说明这2个service对应的pod都是相同的。

Ingress 配置

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: es-cluster-external
spec:
  rules:
    - host: xxx.com
      http:
        paths:
          - backend:
              serviceName: es-cluster-external
              servicePort: 80
            path: /

ingress配置的service为es-cluster-external，elasticsearch能够让外部访问。因为es-cluster-node只需要内部node之间通信，所以我们不需要将它暴露。

ConfigMap 配置

用来配置elasticsearch.yml。

apiVersion: v1
kind: ConfigMap
metadata:
  name: es-config
data:
  elasticsearch.yml: |
    node.name: ${HOSTNAME}
    cluster.name: my-elastic-cluster
    network.host: "0.0.0.0"
    bootstrap.memory_lock: false
    discovery.zen.ping.unicast.hosts: esnode-0.es-cluster-node.xxx.svc.cluster.local,esnode-1.es-cluster-node.xxx.svc.cluster.local,esnode-2.es-cluster-node.xxx.svc.cluster.local
    discovery.zen.minimum_master_nodes: 1
    xpack.security.enabled: false
    xpack.monitoring.enabled: false

我们要重点专注discovery.zen.ping.unicast.hosts配置，这里配置的值为esnode-0.es-cluster-node.xxx.svc.cluster.local…。

由于headless service能够让pod之间直接通信，而不需要走kube-proxy。使用headless service时，pod的dns规则如下 {stateful-set-name}-{0…N}.{service-name}.{namespace}.svc.cluster.local。

这里配置的service-name是headless service name。

StatefulSet 配置

apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
  name: esnode
  labels:
    app: es-cluster
spec:
  serviceName: es-cluster-node
  replicas: 3
  updateStrategy:
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: es-cluster
    spec:
      securityContext:
        fsGroup: 1000
      initContainers:
        - name: init-sysctl
          image: busybox
          imagePullPolicy: IfNotPresent
          securityContext:
            privileged: true
          # increase mmap limits
          command: ["sysctl", "-w", "vm.max_map_count=262144"]
      containers:
        - name: elasticsearch
          resources:
            requests:
              memory: 300Mi
              cpu: 0.01
            limits:
              memory: 1.5Gi
              cpu: 1
          securityContext:
            privileged: true
            runAsUser: 1000
            capabilities:
              add:
                - IPC_LOCK
                - SYS_RESOURCE
          image: docker.elastic.co/elasticsearch/elasticsearch:6.5.0
          imagePullPolicy: IfNotPresent
          env:
            - name: ES_JAVA_OPTS
              value: "-Xms800m -Xmx800m"
          readinessProbe:
            httpGet:
              scheme: HTTP
              path: /_cluster/health?local=true
              port: 9200
            initialDelaySeconds: 5
          ports:
            - containerPort: 9200
              name: es-http
            - containerPort: 9300
              name: es-transport
          volumeMounts:
            - name: es-data
              mountPath: /usr/share/elasticsearch/data
            - name: elasticsearch-config
              mountPath: /usr/share/elasticsearch/config/elasticsearch.yml
              subPath: elasticsearch.yml
      volumes:
        - name: elasticsearch-config
          configMap:
            name: es-config
            items:
              - key: elasticsearch.yml
                path: elasticsearch.yml
  volumeClaimTemplates:
    - metadata:
        name: es-data
      spec:
        accessModes: [ "ReadWriteOnce" ]
        resources:
          requests:
            storage: 1Gi
        storageClassName: rbd

上面配置一份elasticsearch statefulset配置。

1. volumeClaimTemplates配置了pvc
2. volumeMounts配置了pvc，指定挂在路径。配置了elasticsearch.yml，这个配置内容我们在上面的configmap里指定了。
3. spec.serviceName必须和headless service metadata.name一致。

---

最后通过ingress配置的url，访问xxx.com/_nodes查看是否部署成功。