kubernetes CIS 安全基准 Kube-bench 安全工具

开头语

写在前面：如有问题，以你为准，

目前24年应届生，各位大佬轻喷，部分资料与图片来自网络

内容较长，页面右上角目录方便跳转

CIS 介绍

问题：下载pdf后，根据里面的基准来检查K8s集群配置，但内容量太大，一般会采用相关工具来完成这项工作。

Kube-bench是容器安全厂商Aquq推出的工具，以CISK8s基准作为基础，来检查K8s是否安全部署。

主要查找不安全的配置参数、敏感的文件权限、不安全的帐户或公开端口等等。

可以获得各种操作系统的安全最佳实践基准

Kube-bench

GitHub - aquasecurity/kube-bench: Checks whether Kubernetes is deployed according to security best practices as defined in the CIS Kubernetes Benchmark

[root@master kube-bench]# ls

kube-bench_0.6.12_linux_amd64.tar.gz

[root@master kube-bench]# tar -xvf kube-bench_0.6.12_linux_amd64.tar.gz

cfg/ack-1.0/config.yaml

cfg/ack-1.0/controlplane.yaml

cfg/ack-1.0/etcd.yaml

...

[root@master kube-bench]# ls

cfg  kube-bench  kube-bench_0.6.12_linux_amd64.tar.gz

[root@master kube-bench]# cd cfg/

[root@master cfg]# ls

ack-1.0  cis-1.20  cis-1.24  cis-1.6      config.yaml  eks-1.1.0                 gke-1.0    rh-0.7

aks-1.0  cis-1.23  cis-1.5   cis-1.6-k3s  eks-1.0.1    eks-stig-kubernetes-v1r6  gke-1.2.0  rh-1.0

另创目录

mkdir /etc/kube-bench

mv cfg  kube-bench  /etc/kube-bench/

[root@master kube-bench]# cd /etc/kube-bench/

[root@master kube-bench]# cp kube-bench /bin

命令介绍

https://github.com/aquasecurity/kube-bench/blob/main/docs/platforms.md

[root@master cfg]# kube-bench help

This tool runs the CIS Kubernetes Benchmark (https://www.cisecurity.org/benchmark/kubernetes/)



Usage:

  kube-bench [flags]

  kube-bench [command]



Available Commands:

  completion  Generate the autocompletion script for the specified shell

  help        Help about any command

  run         Run tests

  version     Shows the version of kube-bench.



Flags:

      --alsologtostderr                  log to standard error as well as files

...

kube-bench run 常用参数：

-s,--targets 指定要基础测试的目标，这个目标需要匹配cfg/<version>中的

文件名称，已有目标：master,controlplane,node,etcd,policies

--version: 指定k8s版本，如果未指定会自动检测

--benchmark:手动指定CIS基准版本，不能与--version一起使用

检查源代码

type 可以手动设置来让其跳过某一项检测

手动跳过某项检测

--profiling 最新版本被弃用了

重新执行检测

汇总信息输出

运行原理

通过 cfg 文件下的 cis-1.24 目录中的yaml 进行检测（简单的配置比对）

cis-1.24是根据cis安全基准来的

修复

检测 master

[root@master cfg]# ls

ack-1.0  cis-1.20  cis-1.24  cis-1.6      config.yaml  eks-1.1.0                 gke-1.0    rh-0.7

aks-1.0  cis-1.23  cis-1.5   cis-1.6-k3s  eks-1.0.1    eks-stig-kubernetes-v1r6  gke-1.2.0  rh-1.0

[root@master cis-1.24]# ls

config.yaml  controlplane.yaml  etcd.yaml  master.yaml  node.yaml  policies.yaml

这些就是cis的手册安全配置，通过这些配置进行检测集群

[root@master cfg]# kube-bench run --targets=master

[INFO] 1 Control Plane Security Configuration



[INFO] 1.1 Control Plane Node Configuration Files

[PASS] 1.1.1 Ensure that the API server pod specification file permissions are set to 644 or more restrictive (Automated)

[PASS] 1.1.2 Ensure that the API server pod specification file ownership is set to root:root (Automated)

[PASS] 1.1.3 Ensure that the controller manager pod specification file permissions are set to 600 or more restrictive (Automated)

[PASS] 1.1.4 Ensure that the controller manager pod specification file ownership is set to root:root (Automated)

[PASS] 1.1.5 Ensure that the scheduler pod specification file permissions are set to 600 or more restrictive (Automated)

[PASS] 1.1.6 Ensure that the scheduler pod specification file ownership is set to root:root (Automated)

[PASS] 1.1.7 Ensure that the etcd pod specification file permissions are set to 600 or more restrictive (Automated)

[PASS] 1.1.8 Ensure that the etcd pod specification file ownership is set to root:root (Automated)

[WARN] 1.1.9 Ensure that the Container Network Interface file permissions are set to 600 or more restrictive (Manual)

[WARN] 1.1.10 Ensure that the Container Network Interface file ownership is set to root:root (Manual)

[PASS] 1.1.11 Ensure that the etcd data directory permissions are set to 700 or more restrictive (Automated)

[FAIL] 1.1.12 Ensure that the etcd data directory ownership is set to etcd:etcd (Automated)

[PASS] 1.1.13 Ensure that the admin.conf file permissions are set to 600 or more restrictive (Automated)

....

== Remediations master ==

.....



== Summary master ==

40 checks PASS

9 checks FAIL

12 checks WARN

0 checks INFO



== Summary total ==

40 checks PASS

9 checks FAIL

12 checks WARN

0 checks INFO

[PASS]: 测试通过

[FAIL]: 测试未通过，重点关注，在测试结果会给出修复建议

[WARN]: 警告，可做了解

[INFO]: 信息

决安全问题

将没有通过筛选出来

[FAIL] 1.1.12 Ensure that the etcd data directory ownership is set to etcd:etcd (Automated)

[FAIL] 1.2.5 Ensure that the --kubelet-certificate-authority argument is set as appropriate (Automated)

[FAIL] 1.2.17 Ensure that the --profiling argument is set to false (Automated)

[FAIL] 1.2.18 Ensure that the --audit-log-path argument is set (Automated)

[FAIL] 1.2.19 Ensure that the --audit-log-maxage argument is set to 30 or as appropriate (Automated)

[FAIL] 1.2.20 Ensure that the --audit-log-maxbackup argument is set to 10 or as appropriate (Automated)

[FAIL] 1.2.21 Ensure that the --audit-log-maxsize argument is set to 100 or as appropriate (Automated)

== Remediations master == # 下面就会有解决方法

1.1.12 On the etcd server node, get the etcd data directory, passed as an argument --data-dir,

from the command 'ps -ef | grep etcd'.

Run the below command (based on the etcd data directory found above).

For example, chown etcd:etcd /var/lib/etcd

[root@master cis-1.24]# ps -ef | grep etcd | grep data-dir

root        3466    3420  1 Feb23 ?        08:48:04 etcd --advertise-client-urls=https://192.168.100.53:2379 --cert-file=/etc/kubernetes/pki/etcd/server.crt --client-cert-auth=true --data-dir=/var/lib/etcd

[root@master cis-1.24]# ll /var/lib/etcd

total 0

drwx------. 4 root root 29 Feb  2 07:11 member

# 注意etcd现在是pod部署，所有这个项安全不需要改

[FAIL] 1.2.17 Ensure that the --profiling argument is set to false (Automated)

1.2.17 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml

on the control plane node and set the below parameter.

--profiling=false

# 配置文件进行备份

[root@master manifests]# cp kube-apiserver.yaml kube-apiserver.yaml.bak

# 进行修改添加字段

[root@master manifests]# vim kube-apiserver.yaml

spec:

  containers:

  - command:

    - --tls-cert-file=/etc/kubernetes/pki/apiserver.crt

    - --tls-private-key-file=/etc/kubernetes/pki/apiserver.key

    - --enable-aggregator-routing=true

    - --profiling=false #在最下添加这个字段

[root@master manifests]# systemctl restart kubelet

[root@master manifests]# kubectl get node

NAME     STATUS   ROLES           AGE   VERSION

master   Ready    control-plane   42d   v1.26.1

node1    Ready    <none>          42d   v1.26.1

node2    Ready    <none>          42d   v1.26.1

# 再使用 kube-bench 进行检测，注意需要第一次可能还不通过，多执行几次

[root@master cfg]# kube-bench run --targets=master

[PASS] 1.2.17 Ensure that the admission control plugin NodeRestriction is set (Automated)

# 如果没成功是因为 bak文件在这个目录下，不知道为什么kuberneter会识别bak结尾的文件

# 创建一个backup目录，然后将备份的yaml移动进去，就解决了

# 原因：是不是全文件名匹配（强匹配），可能是kube-apiserver*这种，所以会导致在后面加.bak依旧被识别

检测 node

需要将kube-bench也在node上使用，并将命令改为（node上执行）

./kube-bench run --targets node --benchmark cis-1.24