【presto on kubernates】presto on kubernates解决方案2

同上篇文章，上篇文章使用了两个镜像presto coordinator和worker，这篇文章展示只用一个镜像，并且动态配置presto参数的情况

 

一。 镜像打包

首先删除presto中内存参数，jvm参数的设置文件。保留hive的链接文件（这里hive的链接也可以配置动态，因为hive的链接不会经常变动，所以直接打在镜像里）

接下去编写Dockerfile,与前面类似

FROM python

MAINTAINER 682556

RUN useradd --create-home --no-log-init --shell /bin/bash hive

RUN adduser hive sudo

RUN mkdir -p /opt

RUN mkdir -p /opt/presto/data

ADD java-1.8.0-openjdk-1.8.0.232.b09-0.el8_0.x86_64 /opt/jdk

ADD presto-server-0.220.tar.gz /opt

ADD whiletrue.sh /opt/rcommand.sh

ENV PRESTO_HOME /opt/presto-server-0.220

ENV JAVA_HOME /opt/jdk

ENV PATH $PATH:$JAVA_HOME/bin

EXPOSE 8099

RUN chmod -R 777 /opt

USER hive

ENTRYPOINT ["/opt/rcommand.sh"]

修改启动命令rcommand.sh，这里的/opt/bootstrap路径，会在后面deployment启动时使用configmap来挂载文件倒这个目录，将/opt/bootstrap下的node.properties，jvm.config，config.properties，log.properties复制到presto的etc目录

#! /bin/bash
i=1
while ((i<100))
do
if((i<2)); then
    cd /opt/bootstrap
    cat ./node.properties > $PRESTO_HOME/etc/node.properties
    cat ./jvm.config > $PRESTO_HOME/etc/jvm.config
    cat ./config.properties > $PRESTO_HOME/etc/config.properties
    cat ./log.properties > $PRESTO_HOME/etc/log.properties
    sed -i 's/${COORDINATOR_NODE}/'$COORDINATOR_NODE'/g' $PRESTO_HOME/etc/config.properties
    $PRESTO_HOME/bin/launcher start
    pwd;
else
    sleep 10000000000
fi
let i++
done

打包完成，上传到harbor

 

二。启动presto coordinator和worker

分别编写coordinator和worker的configmap挂载文件

presto-configMap-coordinator.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: presto-config-cm
  labels:
    app: presto-coordinator
data:
  node.properties: |-
    node.environment=staging
    node.data-dir=/opt/presto/data
  jvm.config: |-
    -server
    -Xmx4G
    -XX:+UseG1GC
    -XX:G1HeapRegionSize=32M
    -XX:+UseGCOverheadLimit
    -XX:+ExplicitGCInvokesConcurrent
    -XX:+HeapDumpOnOutOfMemoryError
    -XX:+ExitOnOutOfMemoryError
  config.properties: |-
    coordinator=${COORDINATOR_NODE}
    node-scheduler.include-coordinator=false
    http-server.http.port=8099
    query.max-memory=4GB
    query.max-memory-per-node=0.6GB
    query.max-total-memory-per-node=0.6GB
    discovery-server.enabled=true
    discovery.uri=http://presto-coordinator-service:8099
  log.properties: |-
    com.facebook.presto=INFO

presto-configMap-worker.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: presto-config-worker-cm
  labels:
    app: presto-worker
data:
  node.properties: |-
    node.environment=staging
    node.data-dir=/opt/presto/data
  jvm.config: |-
    -server
    -Xmx2G
    -XX:+UseG1GC
    -XX:G1HeapRegionSize=32M
    -XX:+UseGCOverheadLimit
    -XX:+ExplicitGCInvokesConcurrent
    -XX:+HeapDumpOnOutOfMemoryError
    -XX:+ExitOnOutOfMemoryError
  config.properties: |-
    coordinator=${COORDINATOR_NODE}
    http-server.http.port=8099
    query.max-memory=10GB
    query.max-memory-per-node=0.6GB
    query.max-total-memory-per-node=0.6GB
    discovery.uri=http://presto-coordinator-service:8099
  log.properties: |-
    com.facebook.presto=INFO

编写presto coordinator的deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: presto-coordinator
spec:
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app: presto-coordinator
  template:
    metadata:
      labels:
        app: presto-coordinator
    spec:
      nodeSelector:
        type: lab1
      containers:
        - name: presto-coordinator
          image: 10.38.150.64:8090/library/presto-dy:latest
          ports:
            - name: http-coord
              containerPort: 8099
              protocol: TCP
          env:
            - name: COORDINATOR_NODE
              value: "true"
          volumeMounts:
            - name: presto-config-volume   
              mountPath: /opt/bootstrap
              readOnly: false
            - name: presto-data-volume
              mountPath: /opt/presto/data
              readOnly: false
          readinessProbe:
            initialDelaySeconds: 10
            periodSeconds: 5
            httpGet:
              path: /v1/cluster
              port: http-coord
      volumes:
        - name: presto-config-volume
          configMap:
            name: presto-config-cm   ##使用configmap挂载本地文件到pod
        - name: presto-data-volume
          emptyDir: {}

编写presto worker的deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: presto-worker
spec:
  replicas: 3
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app: presto-worker
  template:
    metadata:
      labels:
        app: presto-worker
    spec:
      nodeSelector:
        type: lab1
      containers:
        - name: presto-worker
          image: 10.38.150.64:8090/library/presto-dy:latest
          resources:
            requests:
              memory: 3000Mi
            limits:
              memory: 3000Mi
          ports:
            - name: http-coord
              containerPort: 8099
              protocol: TCP
          env:
            - name: COORDINATOR_NODE
              value: "false"
          volumeMounts:
            - name: presto-config-volume
              mountPath: /opt/bootstrap
              readOnly: false
            - name: presto-data-volume
              mountPath: /opt/presto/data
              readOnly: false
      volumes:
        - name: presto-config-volume
          configMap:
            name: presto-config-worker-cm  #使用configmap挂载本地文件到pod
        - name: presto-data-volume
          emptyDir: {}

编写coordinator的service

kind: Service
apiVersion: v1
metadata:
  labels:
    app: presto-coordinator
  name: presto-coordinator-service
spec:
  type: NodePort
  sessionAffinity: ClientIP
  ports:
    - port: 8099
      targetPort: http-coord
      name: http-coord
      protocol: TCP
      nodePort: 30012
  selector:
    app: presto-coordinator

编写presto worker的HPA组件

apiVersion: autoscaling/v2beta2
kind: HorizontalPodAutoscaler
metadata:
  name: presto-worker
spec:
  maxReplicas: 10
  minReplicas: 3
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: presto-worker
  metrics:
  - type: Resource
    resource:
      name: memory
      target:
        type: Utilization
        averageUtilization: 50

 

各个组件编写完成按照顺序启动

kubectl apply -f presto-configMap-coordinator.yaml
kubectl apply -f presto-configMap-worker.yaml
kubectl apply -f presto-coordinator.yaml
kubectl apply -f presto-service.yaml
kubectl apply -f presto-worker.yaml
kubectl apply -f presto-hpa.yaml

捎带片刻，启动完成