一、Zookeeper集群搭建

爬也要爬着前进

已于 2023-07-12 09:57:39 修改

阅读量106

点赞数

分类专栏： k8s-elk 文章标签： zookeeper linux 分布式

于 2023-06-26 17:31:10 首次发布

本文链接：https://blog.csdn.net/JackNiMaBaBa/article/details/131401527

版权

k8s-elk 专栏收录该内容

4 篇文章 0 订阅

订阅专栏

该文详细描述了如何在Kubernetes环境下构建Zookeeper集群，包括重命名压缩包、编写Dockerfile、配置zoo.cfg、创建headless服务和StatefulSetYAML文件，以及检查集群的运行状态。通过Docker镜像和K8sStatefulSet保证了Zookeeper的有状态服务特性。

摘要由CSDN通过智能技术生成

思路：

zk属于有状态服务
zk集群存在角色之分
zk集群每个节点都需要存储自己的数据
zk集群每个节点都需要有一个唯一的地址

准备文件：

apache-zookeeper-3.8.0-bin.tar.gz
Dockerfile
entrypoint.sh
zoo.cfg
zk_headless.yaml
zk_sts.yaml

1、重命名apache-zookeeper-3.8.0-bin.tar.gz

[admin@ansible-001 ~]$mv apache-zookeeper-3.8.0-bin.tar.gz   zookeeper.tar.gz

2、编写Dockerfile

[admin@ansible-001 ~]$vim Dockerfile
FROM openjdk:8-jre

ENV ZK_HOME /zookeeper
RUN mkdir -p "$ZK_HOME"
WORKDIR $ZK_HOME

#1.调整时区
RUN /bin/cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime && \
    echo 'Asia/Shanghai' > /etc/timezone

#2.拷贝Zookeeper及配置文件
ADD ./zookeeper.tar.gz .
ADD ./zoo.cfg conf/
ADD ./entrypoint.sh /entrypoint.sh

EXPOSE 2181 2888 3888   //leader、follwer通讯端口：2888， 选举端口：3888 
CMD ["/bin/bash","/entrypoint.sh"]

3、编写entrypoint.sh

[admin@ansible-001 ~]$vim Dockerfile
#1.设置变量
ZK_BIN_DIR=/zookeeper/bin
ZK_CONF=/zookeeper/conf/zoo.cfg

#2.对配置文件中的字符串进行变量替换
sed -i s@{ZK_TICKTIME}@${ZK_TICKTIME:-2000}@g ${ZK_CONF}
sed -i s@{ZK_INIT_LIMIT}@${ZK_INIT_LIMIT:-10}@g ${ZK_CONF}
sed -i s@{ZK_SYNC_LIMIT}@${ZK_SYNC_LIMIT:-5}@g ${ZK_CONF}
sed -i s@{ZK_DATA_DIR}@${ZK_DATA_DIR:-/data}@g ${ZK_CONF}
sed -i s@{ZK_LOG_DIR}@${ZK_LOG_DIR:-/logs}@g ${ZK_CONF}
sed -i s@{ZK_CLIENT_PORT}@${ZK_CLIENT_PORT:-2181}@g ${ZK_CONF}
sed -i s@{ZK_MAX_CLIENT_CNXNS}@${ZK_MAX_CLIENT_CNXNS:-120}@g ${ZK_CONF}

#3.ZK的集群节点地址，后面通过ENV注入进来
for i in ${ZK_SERVERS}
do
  echo $i >> ${ZK_CONF}
done

#4.在datadir目录创建myid的文件，并填入对应的编号
ZK_MYID=$(( $(hostname | sed 's#.*-##g') + 1 ))
echo ${ZK_MYID:-99} > ${ZK_DATA_DIR:-/data}/myid

#5.前台运行ZK
cd ${ZK_BIN_DIR}
./zkServer.sh start-foreground

4、编写 zoo.cfg

[admin@ansible-001 ~]$vim zoo.cfg 
# 服务器之间或客户端与服务器之间维持心跳的时间间隔 tickTime以毫秒为单位。
tickTime={ZK_TICKTIME}

# 集群中的follower服务器(F)与leader服务器(L)之间的初始连接心跳数 10* tickTime
initLimit={ZK_INIT_LIMIT}

# 集群中的follower服务器与leader服务器之间请求和应答之间能容忍的最多心跳数 5 * tickTime
syncLimit={ZK_SYNC_LIMIT}
 
# 数据保存目录
dataDir={ZK_DATA_DIR}

# 日志保存目录
dataLogDir={ZK_LOG_DIR}

# 客户端连接端口
clientPort={ZK_CLIENT_PORT}

# 客户端最大连接数。# 根据自己实际情况设置，默认为60个
maxClientCnxns={ZK_MAX_CLIENT_CNXNS}

# 客户端获取 zookeeper 服务的当前状态及相关信息
4lw.commands.whitelist=*

# 三个接点配置，格式为： server.服务编号=服务地址、LF通信端口、选举端口

5、编写 zk_headless.yaml

[admin@ansible-001 ~]$vim zk_headless.yaml
apiVersion: v1
kind: Service
metadata: 
  name: zk-svc
  namespace: oa
spec:
  clusterIP: None
  selector:
    app: zk 
  ports:
  - name: client
    port: 2181
    targetPort: 2181
  - name: leader-follwer
    port: 2888
    targetPort: 2888
  - name: selection
    port: 3888
    targetPort: 3888

6、编写zk_sts.yaml

[admin@ansible-001 ~]$vim zk_sts.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: zookeeper
  namespace: oa
spec:
  serviceName: "zk-svc"
  replicas: 3
  selector:
    matchLabels:
      app: zk
  template:
    metadata:
      labels:
        app: zk
    spec:
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchExpressions:
                - key: app
                  operator: In
                  values: ["zk"]
              topologyKey: "kubernetes.io/hostname"
      imagePullSecrets:
      - name: harbor-admin
      containers:
      - name: zk
        image: harbor.xx.com/base/zookeeper:3.8.0 
        imagePullPolicy: Always
        ports:
        - name: client
          containerPort: 2181
        - name: leader-follwer
          containerPort: 2888
        - name: selection
          containerPort: 3888
        env:
        - name: ZK_SERVERS
          value: "server.1=zookeeper-0.zk-svc.oa.svc.cluster.local:2888:3888 server.2=zookeeper-1.zk-svc.oa.svc.cluster.local:2888:3888 server.3=zookeeper-2.zk-svc.oa.svc.cluster.local:2888:3888"
        readinessProbe:
          exec:
            command:
            - "/bin/bash"
            - "-c"
            - '[[ "$(/zookeeper/bin/zkServer.sh status 2>/dev/null | grep 2181)" ]] && exit 0 || exit 1'
          initialDelaySeconds: 5
         
        livenessProbe:
          exec:
            command:
            - "/bin/bash"
            - "-c"  
            - '[[ "$(/zookeeper/bin/zkServer.sh status 2>/dev/null | grep 2181)" ]] && exit 0 || exit 1'
          initialDelaySeconds: 5
        volumeMounts:
        - name: data
          mountPath: /data
  volumeClaimTemplates:
  - metadata: 
      name: data
    spec:
      accessModes: ["ReadWriteMany"] 
      storageClassName: "nfs-provisioner-storage"
      resources:
        requests:
          storage: 50Gi

7、检查测试集群

7.1 检查pod和service

[admin@ansible-001 ~]$kubectl get pod -n oa 
NAME          READY   STATUS    RESTARTS   AGE
zookeeper-0   1/1     Running   0          5h37m
zookeeper-1   1/1     Running   0          5h36m
zookeeper-2   1/1     Running   0          5h36m

[zhaowei@master01 ~]$kubectl get svc -n oa 
NAME     TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)                      AGE
zk-svc   ClusterIP   None         <none>        2181/TCP,2888/TCP,3888/TCP   6h46m

7.2 检查集群状态


[admin@ansible-001 ~]$kubectl exec -it zookeeper-0 -n oa -- /zookeeper/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /zookeeper/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost. Client SSL: false.
Mode: follower

[admin@ansible-001 ~]$kubectl exec -it zookeeper-1 -n oa -- /zookeeper/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /zookeeper/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost. Client SSL: false.
Mode: leader

[admin@ansible-001 ~]$kubectl exec -it zookeeper-2 -n oa -- /zookeeper/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /zookeeper/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost. Client SSL: false.
Mode: follower

7.3 链接集群

 [admin@ansible-001 ~]$kubectl   exec -it -n oa zookeeper-0 -- /bin/bash
root@zookeeper-0:/zookeeper#  /zookeeper/bin/zkCli.sh -server zk-svc
[zk: zk-svc(CONNECTED) 0] create /haha aaa
Created /haha
[zk: zk-svc(CONNECTED) 1] get /haha 
aaa