相信你使用k8s有一段时间了,但是你使用kubernetes是否对它的安全了解有多少呢?
在你的生产环境中是否对你集群以及Docker的配置安全了解是否很清晰?
如果对Docker安全配置不清晰,可以通过CIS基准测试这个工具来完成你docker主机上的基准测试,从而获得哪些问题是漏洞,哪些你需要必须解决,哪些不需要解决,哪些配置是安全的,这都是我们需要所去认知的问题。
Docker CIS security是一款脚本工具,用于检查围绕在生产环境中部署Docker容器的数十种常见最佳实践。这些测试都是自动化的,并受到CIS Docker基准测试v1.2.0的启发。
目前它已经作为一种开源工具提供给Docker社区,这样Docker社区就可以轻松地根据这个基准来评估他们的主机和Docker容器。
如果你按照社区的方式去部署它包含了docker run、脚本、docker-compose去部署,但是唯一不足的是缺少k8s-job的形式,另外还有一个问题,1.3.5的版本还在开发当中,你可能去测试的时候还是使用的1.3.4的版本,这里我向社区也提出了issue,目前docker hub的镜像没有更新上去,只能通过脚本去执行最新的1.3.5版本,正因如此,我自己整理了最新的1.3.5的版本的镜像以及适合k8s-job的形式来测试你的docker安全性,也希望你少走一些弯路。
docker/docker-bench-securitygithub.com![198186b9651fbbcf258bbb4149ffd317.png](https://i-blog.csdnimg.cn/blog_migrate/eca08da53b7588b464fa7df07437b57e.png)
这里我帮你列出了它最新版本所检测的选项,一共8项
1) 常规配置
2) Docker守护程序配置
3) Docker守护程序配置文件
4) 容器镜像和构建文件
5) 容器运行时
6) Docker安全运行方式
7) Docker Swarm配置
8) Docker企业配置
检测结果分为4种状态
PASS 通过
INFO 信息输出
WARN 警告
NOTE 请注意
让我们去运行起来检测你的Docker主机安全性的所有问题
---
apiVersion: batch/v1
kind: Job
metadata:
name: docker-bench
namespace: kube-system
spec:
template:
metadata:
labels:
app: docker-bench
spec:
hostPID: true
hostNetwork: true
containers:
- name: docker-bench
image: zhaocheng172/docker-bench-security:v1.3.5
volumeMounts:
- name: var-lib
mountPath: /var/lib
readOnly: true
- name: var-run-docker-sock
mountPath: /var/run/docker.sock
readOnly: true
- name: var-systemd
mountPath: /var/lib/systemd
readOnly: true
- name: etc
mountPath: /etc
readOnly: true
restartPolicy: Never
volumes:
- name: var-lib
hostPath:
path: "/var/lib"
- name: var-run-docker-sock
hostPath:
path: "/var/run/docker.sock"
- name: var-systemd
hostPath:
path: "/var/lib/systemd"
- name: etc
hostPath:
path: "/etc"
通过这条命令来查看运行时的检测结果
# kubectl get pod -n kube-system |awk '{print $1}' |grep docker-bench |xargs kubectl logs -n kube-system -f
可以看到pod日志的输出,此时这个正检查你本机的Docker的安全最佳实践是否合格
# ------------------------------------------------------------------------------
# Docker Bench for Security v1.3.5
#
# Docker, Inc. (c) 2015-
#
# Checks for dozens of common best-practices around deploying Docker containers in production.
# Inspired by the CIS Docker Benchmark v1.2.0.
# ------------------------------------------------------------------------------
Initializing Thu Jul 9 11:10:41 UTC 2020
[INFO] 1 - Host Configuration
[INFO] 1.1 - General Configuration
[NOTE] 1.1.1 - Ensure the container host has been Hardened
[INFO] 1.1.2 - Ensure Docker is up to date
[INFO] * Using 18.09.9, verify is it up to date as deemed necessary
[INFO] * Your operating system vendor may provide support and security maintenance for Docker
[INFO] 1.2 - Linux Hosts Specific Configuration
[WARN] 1.2.1 - Ensure a separate partition for containers has been created
[INFO] 1.2.2 - Ensure only trusted users are allowed to control Docker daemon
[INFO] * docker:x:994
[WARN] 1.2.3 - Ensure auditing is configured for the Docker daemon
[WARN] 1.2.4 - Ensure auditing is configured for Docker files and directories - /var/lib/docker
[WARN] 1.2.5 - Ensure auditing is configured for Docker files and directories - /etc/docker
[INFO] 1.2.6 - Ensure auditing is configured for Docker files and directories - docker.service
[INFO] * File not found
[INFO] 1.2.7 - Ensure auditing is configured for Docker files and directories - docker.socket
[INFO] * File not found
[INFO] 1.2.8 - Ensure auditing is configured for Docker files and directories - /etc/default/docker
[INFO] * File not found
[INFO] 1.2.9 - Ensure auditing is configured for Docker files and directories - /etc/sysconfig/docker
[INFO] * File not found
[WARN] 1.2.10 - Ensure auditing is configured for Docker files and directories - /etc/docker/daemon.json
[INFO] 1.2.11 - Ensure auditing is configured for Docker files and directories - /usr/bin/containerd
[INFO] * File not found
[INFO] 1.2.12 - Ensure auditing is configured for Docker files and directories - /usr/sbin/runc
[INFO] * File not found
[INFO] 2 - Docker daemon configuration
[WARN] 2.1 - Ensure network traffic is restricted between containers on the default bridge
[PASS] 2.2 - Ensure the logging level is set to 'info'
[PASS] 2.3 - Ensure Docker is allowed to make changes to iptables
[WARN] 2.4 - Ensure insecure registries are not used
[PASS] 2.5 - Ensure aufs storage driver is not used
[INFO] 2.6 - Ensure TLS authentication for Docker daemon is configured
[INFO] * Docker daemon not listening on TCP
[INFO] 2.7 - Ensure the default ulimit is configured appropriately
[INFO] * Default ulimit doesn't appear to be set
[WARN] 2.8 - Enable user namespace support
[PASS] 2.9 - Ensure the default cgroup usage has been confirmed
[PASS] 2.10 - Ensure base device size is not changed until needed
[WARN] 2.11 - Ensure that authorization for Docker client commands is enabled
[WARN] 2.12 - Ensure centralized and remote logging is configured
[PASS] 2.13 - Ensure live restore is Enabled
[WARN] 2.14 - Ensure Userland Proxy is Disabled
[PASS] 2.15 - Ensure that a daemon-wide custom seccomp profile is applied if appropriate
[PASS] 2.16 - Ensure that experimental features are not implemented in production
[WARN] 2.17 - Ensure containers are restricted from acquiring new privileges
这里列举几个WARN状态不符合安全的最佳实践的例子解决一下,后面的这里不详细举例了,也是由于篇幅过长,如果你想知道更多的安全最佳实践,可以评论
1.2.3-1.2.12的解决方式
Linux auditd 工具可以将审计记录写入日志文件。包括记录系统调用和文件访问。管理员可以检查这些日志,确定是否存在安全漏洞,指定docker文件到linux的审计规则中
- 1.1、确保Docker daemon要将审计的能力进行配置
- 1.2、确保对docker文件和目录进行审计
- 1.3、同样是对Docker文件和目录进行审计的检查
- 1.4、确保启动systemd文件进行审计检查
- 1.5、确保docker client和docker守护进程之间与localhost的通信
- 1.6、确保docker.service文件进行审计
- 1.7、确保docker.socket文件进行审计
- 1.8、确保/usr/sbin/runc容器命令行工具进行审计
针对这8点需要在
cat /etc/audit/rules.d/audit.rules 添加以下配置
-w /usr/bin/dockerd -k docker
-w /var/lib/docker -k docker
-w /etc/docker -k docker
-w /usr/lib/systemd/system/docker.service -k docker
-w /usr/lib/systemd/system/docker.socket -k docker
-w /etc/docker/daemon.json -k docker
-w /usr/bin/containerd -k docker
-w /usr/sbin/runc -k docker
配置完成需要重启才能生效,确保每台docker主机都进行配置相同操作
# service auditd restart
Stopping logging: [ OK ] Redirecting start to /bin/systemctl start auditd.service
[WARN] 2.4 - Ensure insecure registries are not used (Scored)
2.4 -确保不安全的镜像仓库不被使用(评分) https://docs.docker.com/registry/insecure/ 建议使用https安全的镜像仓库
这里可根据官方的配置可以创建https的harbor镜像仓库,不使用不安全的镜像仓库配置daemon.json里面的配置
[WARN] 2.8 - Enable user namespace support (Scored)
2.8 -启用用户名称空间支持(得分)
由于我们使用的root用户运行的容器,这里显示不是安全的,但是要是使用非root用户运行,并使用用户命名空间
重新映射可能会使不少docker功能不兼容
修改etc/docker/daemon.json
"userns-remap": "default", 具体问题详细可以看一下官方的解释 https:// docs.docker.com/engine/ reference/commandline/dockerd/#daem
[WARN] 2.11 - Ensure that authorization for Docker client commands is enabled (Scored)
2.11 -确保Docker客户端命令的授权是启用的(得分)
使用--authorization-plugin=PLUGIN_ID格式中的专用命令行标志启用授权插件。设置访问控制 https://docs.docker.com/engine/extend/plugins_authorization/
[WARN] 2.12 - Ensure centralized and remote logging is configured (Scored)
[WARN] 2.12 -确保配置了集中式和远程日志(评分)
默认情况下使用的json,远程日志管理可选项
syslog
gelf
fluentd
splunk
Logentries
查看默认使用的日志驱动默认是json-file
docker info --format '{{.LoggingDriver}}'
json-file https://docs.docker.com/config/containers/logging/configure/
[WARN] 2.14 - Ensure Userland Proxy is Disabled (Scored)
禁用userland代理
在docker.daemon.json配置 "userland-proxy": false, https://github.com/moby/moby/issues/14856
[WARN] 2.17 - Ensure containers are restricted from acquiring new privileges (Scored)
https://github.com/moby/moby/pull/29984 像有些服务
在docker.daemon.json中配置 "no-new-privileges": true
剩下的有不是很懂的欢迎讨论~~~