k8s准入控制器Admission Webhook研究

一身轻灬

已于 2023-08-06 16:04:14 修改

阅读量1.5k

点赞数 18

文章标签： kubernetes 容器云原生

于 2023-08-06 16:00:25 首次发布

本文链接：https://blog.csdn.net/weixin_47575974/article/details/132109151

版权

1. 什么是准入控制插件

k8s官方文档解释：

准入控制器是一段代码，它会在请求通过认证和鉴权之后、对象被持久化之前拦截到达 API 服务器的请求。

准入控制器分为两种：验证（Validating）性质和变更（Mutating）性质的准入控制器。变更（mutating）控制器可以根据被其接受的请求更改相关对象；验证（validating）控制器则不行。

通过准入控制器，可以拦截api请求，从而根据自定义逻辑修改或拒绝请求。例如，通过验证性准入控制器，可以验证资源对象的配置是否符合预定义的规则，通过变更性准入控制器，可以对资源对象的进行某些初始化配置等。

2. 默认开启的准入控制插件

k8s默认开启一些准入控制插件，可通过kube-apiserver -h | grep enable-admission-plugins查看。

例如，NamespaceLifecycle这个准入插件可以禁止删除三个系统保留的名字空间，即 default、 kube-system 和 kube-public，这样当删除default命名空间时，这个请求会被拒绝

3. MutatingAdmissionWebhook 和ValidatingAdmissionWebhook 控制器

除了k8s内置的准入控制，k8s提供了验证性质的准入webhook（ValidatingAdmissionWebhook）和变更性质的准入webhook（MutatingAdmissionWebhook ）用于接收准入请求并对其进行处理的 HTTP 回调机制。MutatingAdmissionWebhook 会先调用，它们可以更改发送到 API 服务器的对象以执行自定义的设置默认值操作。在完成所有对象修改并且 API 服务器也验证了所传入的对象之后， ValidatingAdmissionWebhook 会被调用，并通过拒绝请求的方式来强制实施自定义的策略。

4. 编写一个MutatingAdmissionWebhook控制器示例

k8s官方提供了详细的webhook服务器的示例（https://github.com/kubernetes/kubernetes/blob/release-1.21/test/images/agnhost/webhook/main.go）。

接下来，我们构建一个准入控制器示例：

创建pod时必须包含指定的label，并且会将pod对象的镜像进行变更。

代码示例：

关键点就是构建处理AdmissionReview对象，k8s官方文档（动态准入控制 | Kubernetes）对AdmissionReview有着详细的结构说明

import (
	"crypto/tls"
	"encoding/json"
	"fmt"
	"io/ioutil"
	"log"
	"net/http"

	admissionv1 "k8s.io/api/admission/v1"
	corev1 "k8s.io/api/core/v1"
	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
	"k8s.io/apimachinery/pkg/runtime"
	"k8s.io/apimachinery/pkg/runtime/serializer"
)

const (
	port     = 8080
	certFile = "/path/to/tls.crt" // 证书文件路径
	keyFile  = "/path/to/tls.key" // 私钥文件路径
)

var (
	// 定义反序列化器
	decoder runtime.Decoder
	// 定义要求的标签
	requiredLabel = "required-label"
)

func init() {
	// 创建反序列化器
	scheme := runtime.NewScheme()
	codecs := serializer.NewCodecFactory(scheme)
	decoder = codecs.UniversalDeserializer()
}
func main() {
	// 注册处理函数
	http.HandleFunc("/validate", validateHandler)
	// 加载证书和私钥文件
	cert, err := tls.LoadX509KeyPair(certFile, keyFile)
	if err != nil {
		log.Fatalf("Failed to load certificate and key: %v", err)
	}
	// 创建 TLS 配置
	tlsConfig := &tls.Config{
		Certificates: []tls.Certificate{cert},
	}
	// 创建 HTTPS 服务器
	server := &http.Server{
		Addr:      fmt.Sprintf(":%v", port),
		TLSConfig: tlsConfig,
	}
	// 启动 Web 服务器
	log.Printf("Server listening on port %v", port)
	log.Fatal(server.ListenAndServeTLS("", ""))
}
func validateHandler(w http.ResponseWriter, r *http.Request) {
	data, err := ioutil.ReadAll(r.Body)
	if err != nil {
		log.Printf("Failed to read request body: %v", err)
		http.Error(w, "Failed to read request body", http.StatusBadRequest)
		return
	}
	// 解析 admissionReview 请求对象
	admissionReview := admissionv1.AdmissionReview{}
	_, _, err = decoder.Decode(data, nil, &admissionReview)
	if err != nil {
		log.Printf("Failed to decode AdmissionReview: %v", err)
		http.Error(w, "Failed to decode AdmissionReview", http.StatusBadRequest)
		return
	}
	pod := corev1.Pod{}
	err = json.Unmarshal(admissionReview.Request.Object.Raw, &pod)
	if err != nil {
		log.Printf("Failed to unmarshal Pod: %v", err)
		http.Error(w, "Failed to unmarshal Pod", http.StatusBadRequest)
		return
	}
	if !validatePodLabels(&pod) {
		// 标签不符合要求，返回错误响应
		admissionReview.Response = &admissionv1.AdmissionResponse{
			Result: &metav1.Status{
				// 自定义状态码和返回客户端的信息
				Message: "Pod labels do not meet the requirements",
				Code:    403,
			},
			// 通过Allowed字段控制允许请求或禁止请求
			Allowed: false,
		}

	} else {
		// ValidatingAdmissionWebhook和MutatingAdmissionWebhook的区别就在于
		// 处理Mutating Webhook时需要拼接JSONPatch 的数据
		// 当没有此处逻辑时，该示例代码就是个验证性的webhook
		patchTypeConst := admissionv1.PatchTypeJSONPatch
		admissionReview.Response = &admissionv1.AdmissionResponse{
			Allowed:   true,
			PatchType: &patchTypeConst,
			// 当传入指定标签时，通过patch操作修改镜像
			Patch: patchOperation(),
		}
	}
	// 设置 AdmissionReview 的 UID 和 API 版本
	admissionReview.Response.UID = admissionReview.Request.UID
	admissionReview.APIVersion = admissionv1.SchemeGroupVersion.String()
	// 序列化 AdmissionReview 对象
	responseData, err := json.Marshal(admissionReview)
	if err != nil {
		log.Printf("Failed to marshal AdmissionReview: %v", err)
		http.Error(w, "Failed to marshal AdmissionReview", http.StatusInternalServerError)
		return
	}
	// 返回响应数据
	w.Header().Set("Content-Type", "application/json")
	_, err = w.Write(responseData)
	if err != nil {
		log.Printf("Failed to write response: %v", err)
	}

}
func validatePodLabels(pod *corev1.Pod) bool {
	for key, _ := range pod.Labels {
		if key == requiredLabel {
			return true
		}
	}

	return false
}

func patchOperation() []byte {
	str := `[{
		"op":    "replace",
		"path":  "/spec/containers/0/image",
		"value": "nginx:1.16"
	}]`

	return []byte(str)
}

5. 部署自定义的webhook服务

webhook是通过https暴露服务的，因此需要为其生成相关证书。这里用cfssl生成相关证书。

5.1 centos安装cfssl

wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64

wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64

wget https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64

chmod +x cfssl_linux-amd64 cfssljson_linux-amd64 cfssl-certinfo_linux-amd64

mv cfssl_linux-amd64 /usr/bin/cfssl

mv cfssljson_linux-amd64 /usr/bin/cfssljson

mv cfssl-certinfo_linux-amd64 /usr/bin/cfssl-certinfo

cfssl version

5.2 生成ca证书和私钥

cfssl print-defaults config > ca-config.json
cfssl print-defaults csr > ca-csr.json

ca-config.json内容

{
	"signing": {
		"default": {
			"expiry": "168h"
		},
		"profiles": {
			"server": {
				"expiry": "876000h",
				"usages": [
					"signing",
					"key encipherment",
					"client auth",
					"server auth"
				]
			}
		}
	}
}

ca-csr.json内容

{
    "CN": "kubernetes",
    "key": {
        "algo": "ecdsa",
        "size": 256
    },
    "names": [
        {
            "C": "CN",
            "L": "BeiJing",
            "ST": "BeiJing",
            "O": "k8s",
            "OU": "CA"
        }
    ]
}

生成证书

cfssl gencert -initca ca-csr.json | cfssljson -bare ca

5.3 生成server端证书

cfssl print-defaults csr > server.json

server.json内容

{
    "CN": "admission",
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
           "C": "CN",
        "L": "BeiJing",
        "ST": "BeiJing",
        "O": "k8s",
        "OU": "CA"
        }
    ]
}

创建Server 端证书，-hostname修改为webhook的service名字和命名空间

cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json \
-hostname=mypod-webhook.default.svc -profile=server server.json | cfssljson -bare server

5.4 基于server证书和私钥创建secret对象

kubectl create secret tls mypod-webhook-tls --key=server-key.pem  --cert=server.pem

5.5 制作镜像

编译二进制文件

set GOOS=linux
set GOARCH=amd64
go build -o webhook

制作镜像，Dockerfile内容

FROM alpine:3.9.2
COPY webhook webhook
RUN chmod -R 777 webhook
ENTRYPOINT ["/webhook"]

5.6 部署webhook

示例代码中写死了证书位置，直接挂载证书。webhook的yaml为

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mypod-webhook
  labels:
    app: mypod-webhook
spec:
  selector:
    matchLabels:
      app: mypod-webhook
  template:
    metadata:
      labels:
        app: mypod-webhook
    spec:
      containers:
        - name: webhook
          image: mypod-webhook:v1
          imagePullPolicy: IfNotPresent
          ports:
          - containerPort: 8080
          volumeMounts:
          - name: webhook-certs
            mountPath: /path/to
            readOnly: true
      volumes:
        - name: webhook-certs
          secret:
            secretName: mypod-webhook-tls
---
apiVersion: v1
kind: Service
metadata:
  name: mypod-webhook
  labels:
    app: mypod-webhook
spec:
  ports:
  - port: 443
    targetPort: 8080
  selector:
    app: mypod-webhook

5.7 注册MutatingAdmissionWebhook

其中CA_BUNDLE值是ca.pem的base64编码值，yaml为

apiVersion: admissionregistration.k8s.io/v1
kind: MutatingWebhookConfiguration
metadata:
  name: mypod-webhook-muta
webhooks:
- name: webhookmuta.mydomain.io
  rules:
  - apiGroups:   [""]
    apiVersions: ["v1"]
    operations:  ["CREATE"]
    resources:   ["pods"]
  clientConfig:
    service:
      namespace: default
      name: mypod-webhook
      path: "/validate"
    caBundle: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUNLVENDQWMrZ0F3SUJBZ0lUSXlrTjJhMFRZRFlZYkFLVXRILzEva1BkOWpBS0JnZ3Foa2pPUFFRREFqQmgKTVFzd0NRWURWUVFHRXdKRFRqRVFNQTRHQTFVRUNCTUhRbVZwU21sdVp6RVFNQTRHQTFVRUJ4TUhRbVZwU21sdQpaekVNTUFvR0ExVUVDaE1EYXpoek1Rc3dDUVlEVlFRTEV3SkRRVEVUTUJFR0ExVUVBeE1LYTNWaVpYSnVaWFJsCmN6QWVGdzB5TXpBNE1ERXdOalE1TURCYUZ3MHlPREEzTXpBd05qUTVNREJhTUdFeEN6QUpCZ05WQkFZVEFrTk8KTVJBd0RnWURWUVFJRXdkQ1pXbEthVzVuTVJBd0RnWURWUVFIRXdkQ1pXbEthVzVuTVF3d0NnWURWUVFLRXdOcgpPSE14Q3pBSkJnTlZCQXNUQWtOQk1STXdFUVlEVlFRREV3cHJkV0psY201bGRHVnpNRmt3RXdZSEtvWkl6ajBDCkFRWUlLb1pJemowREFRY0RRZ0FFUGZQM3NHSHdONlhBOHlFeFpzNnFXNWZvNlZjZ0FXNEZiMDFiZWhCVXJER2UKUVg5aXV5N0xsTjBjbUh4Snk3VlRrUlhKSUNPQVZCMGRwK3NqYmFYaXQ2Tm1NR1F3RGdZRFZSMFBBUUgvQkFRRApBZ0VHTUJJR0ExVWRFd0VCL3dRSU1BWUJBZjhDQVFJd0hRWURWUjBPQkJZRUZQREUzeis3dW9uSnhqUFFkWE5jCkI1Q1JzVGpDTUI4R0ExVWRJd1FZTUJhQUZQREUzeis3dW9uSnhqUFFkWE5jQjVDUnNUakNNQW9HQ0NxR1NNNDkKQkFNQ0EwZ0FNRVVDSUFuQVg3Q28ycDBKdDZyUGlLTVpqc3c4UHowWGxISmZUeWt6NmN0cjd1WnlBaUVBellkTAp6VTVsMjdxeG1WRFdDNEFIbUQrdFBFUFAyNENIbWFpV0h6UEI5bzQ9Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K
  admissionReviewVersions: ["v1"]
  sideEffects: None

6. 验证自定义的webhook服务

该pod示例没有指定labels，被拒绝

apiVersion: v1
kind: Pod
metadata:
  name: nginx1
  labels:
    app: nginx
spec:
  containers:
  - name: nginx
    image: nginx
    ports:
    - containerPort: 80

该pod示例有指定labels，能够创建，但是镜像被修改为nginx:1.16

apiVersion: v1
kind: Pod
metadata:
  name: nginx
  labels:
    app: nginx
    required-label: "true"
spec:
  containers:
  - name: nginx
    image: nginx
    ports:
    - containerPort: 80