kubernetes 集群_Kubernetes集群内部的安全措施

kubernetes 集群

As Kubernetes has rapidly become one of the most popular choices for deploying code in the cloud, I decided to write an article to give you some insights about security measures. Furthermore, those hidden things which you should consider when coming through a production cluster in Kubernetes.

随着KubernetesSwift成为在云中部署代码的最受欢迎选择之一，我决定写一篇文章，为您提供有关安全措施的一些见解。 此外，通过Kubernetes的生产集群时，您应该考虑那些隐藏的东西。

I assume the reader of this article has a familiarity with basic knowledge about the structure of Kubernetes. This article is mainly written for Kubernetes developers or DevOps engineers who want to deploy their services on the production and are looking for some security tips to ensure that their services remain secure after deployment.

我假定本文的读者熟悉有关Kubernetes结构的基本知识。 本文主要是为Kubernetes开发人员或DevOps工程师撰写的，他们想要在生产上部署其服务，并正在寻找一些安全技巧，以确保部署后其服务仍然安全。

There are some security measures about cluster itself which should be considered, especially as a cluster administrator when you want to bring up a cluster from scratch. Even on managed Kubernetes services like GKE, EKS, AKS, etc. you need to be cautious. Although in most cases, managed Kubernetes services help you to secure your stacks like cluster network and access to the cluster API server.

有一些有关群集本身的安全措施应予以考虑，特别是当您要从头启动群集时，作为群集管理员，尤其如此。 即使在托管的Kubernetes服务(例如GKE，EKS，AKS等)上，您也要保持谨慎。 尽管在大多数情况下，托管Kubernetes服务可以帮助您保护堆栈(如群集网络)和对群集API服务器的访问。

If you think technology can solve your security problems, then you do not understand the problems and you do not understand the technology.

如果您认为技术可以解决您的安全问题，那么您就不了解这些问题，也不了​​解该技术。

Bruce Schneier, Applied Cryptography

Bruce Schneier，应用密码学

Container Security

集装箱安全

A big mistake is often made is to think the container is completely isolated.

通常会犯的一个大错误是认为容器是完全隔离的。

Say No To Root: Some exception to run with root could be:

说不对 ：用root运行某些异常可能是：

• Your container needs to edit the host system, for example modifying the kernel’s configuration.
  您的容器需要编辑主机系统，例如修改内核的配置。
• The container needs to bind to privileged ports on the node, for example, nginx serving on port 80, although this can be avoided through port mapping in service abstraction in Kubernetes.
  容器需要绑定到节点上的特权端口，例如，nginx服务于端口80，尽管可以通过Kubernetes中服务抽象中的端口映射来避免这种情况。
• Installing software into a container at runtime. In some cases, the traditional package management system might require root to function or store files in a certain location, although this approach is bad practice since any code install at runtime has not been scanned for vulnerability or other policy requirements.
  在运行时将软件安装到容器中。 在某些情况下，传统的程序包管理系统可能需要root才能在特定位置运行文件或将文件存储在某个位置，尽管这种方法是一种不好的做法，因为尚未对运行时安装的任何代码进行扫描漏洞或其他策略要求。

Scanning Docker image: In order to scan vulnerabilities inside your docker image, you need to take advantage of a container image scanner to inspect the packages included in an image. Image scanner lets you know whether there are any known vulnerabilities included in those packages or not.Well-known docker registries like Docker Trusted Registry, Google Container Registry or even Red Hat Container Catalog already have an image scan result. So, if you want to use any docker image from one of these resources you can check the scan result over there.

扫描Docker映像：为了扫描Docker映像中的漏洞，您需要利用容器映像扫描器来检查映像中包含的软件包。 图像扫描仪可让您知道这些软件包中是否包含任何已知漏洞。著名的Docker注册表(例如Docker Trusted Registry，Google Container Registry甚至Red Hat Container Catalog)已经具有图像扫描结果。 因此，如果您想使用这些资源之一中的任何docker映像，则可以在那检查扫描结果。

Container Utilities: Make sure in your docker file there are utilities as little as possible, this will make the running container less useful to an attacker who manages to compromise it. For example, suppose we have some readable credentials in a file on a container, if the containers do not include utilities like cat or more or even shell(like sh or bash) then it will much much harder to read the credential. Sometimes removing these utilities makes the troubleshooting will be much harder but it could be a trade-off to see whether you need these or not.

容器实用程序：确保您的docker文件中的实用程序越少越好，这将使正在运行的容器对设法妥协它的攻击者的用处不大。 例如，假设我们在容器上的文件中有一些可读的凭证，如果容器中不包含诸如cat或更多甚至shell (如sh或bash )之类的实用程序，那么将很难读取凭证。 有时删除这些实用程序会使故障排除变得更加困难，但是要权衡一下是否需要这些实用程序。

Image Version: Refer to images in pods by using a semantic version tag or even if it is possible to refer to it by its unique digest instead of tags, this always helps you to know which particular version of the container image is deploying. If we ignore image tag or a digest, the image version will be tagged with latest, which is strongly advised to avoid using latest version in production. Because it’s more difficult to keep track of exactly what code is running and what version to use when you want to roll back to the previous version.

映像版本：通过使用语义版本标记来引用pod中的映像，或者即使可以通过其唯一的摘要(而不是标记)来引用它，也始终可以帮助您了解正在部署哪个特定版本的容器映像。 如果我们忽略图片标签或摘要，则图片版本将标记为最新 ，强烈建议避免在生产中使用最新版本。 因为要回滚到以前的版本时，要准确地跟踪正在运行的代码和要使用的版本比较困难。

Pods Security

豆荚安全

Kubernetes gives the ability to have two pod-level security-policy mechanisms, the first one is to allow us to restrict what process can do within a pod(Security Context) and the second one is how pods could communicate with each other(Network Policy).

Kubernetes提供了两种Pod级别的安全策略机制，第一种是允许我们限制Pod内的进程(安全上下文)，第二种是Pod如何相互通信(网络策略) )。

Security Context: what we can do in the security context is to define privilege and access control settings on the pod or container level. For example, imagine you want to have all containers in a pod run under 1001 user through runAsUser setting or in a webserver container prevent setuid binaries from changing the user ID by setting allowPrivileagesEscalation to false. Here is an example to see how to define a security context in a pod:

安全上下文：我们在安全上下文中可以做的是在Pod或容器级别上定义特权和访问控制设置。 例如，假设您想让pod中的所有容器都通过runAsUser设置在1001位用户以下运行，或者要在Web服务器容器中通过将allowPrivileagesEscalation设置为false来阻止setuid二进制文件更改用户ID。 这是查看如何在Pod中定义安全上下文的示例：

apiVersion: v1
kind: Pod
metadata:
  name: security-context-demo
spec:
  securityContext:
    runAsUser: 1001
  volumes:
  - name: sec-ctx-vol
    emptyDir: {}
  containers:
  - name: sec-ctx-demo
    image: busybox
    command: [ "sh", "-c", "sleep 1h" ]
    volumeMounts:
    - name: sec-ctx-vol
      mountPath: /data/demo
    securityContext:
      allowPrivilegeEscalation: false      runAsUser: 1001

Network policy: By default, all kind of incoming and outgoing traffic is allowed but you can control how pods are allowed to communicate with each other by network policy.

网络策略：默认情况下，允许所有类型的传入和传出通信，但是您可以控制网络策略允许pod相互通信的方式。

Network policy adds one more security layer to your cluster, you suppose by any security breach an external attacker is able to reach the cluster network, network policy can stop that attacker from sending traffic to application code running inside the pods.

网络策略为您的群集增加了一层安全层，您假设由于任何安全漏洞，外部攻击者都可以访问群集网络，因此网络策略可以阻止攻击者将流量发送到Pod中运行的应用程序代码。

In addition, if a container somehow becomes compromised, an attacker will typically try to explore the network to move to other containers which by restricting the addresses, ports, and pods than can be much harder. take a look at this sample to see how to define a network policy between the pods:

此外，如果某个容器以某种方式受到威胁，攻击者通常会尝试探索网络以迁移到其他容器，这通过限制地址，端口和吊舱而变得更加困难。 看一下此示例，看看如何在Pod之间定义网络策略：

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: test-network-policy
  namespace: default
spec:
  podSelector:
    matchLabels:
      role: db
  policyTypes:
  - Ingress
  - Egress
  ingress:
  - from:
    - ipBlock:
        cidr: 172.17.0.0/16
        except:
        - 172.17.1.0/24
    - namespaceSelector:
        matchLabels:
          project: myproject
    - podSelector:
        matchLabels:
          role: frontend
    ports:
    - protocol: TCP
      port: 6379
  egress:
  - to:
    - ipBlock:
        cidr: 10.0.0.0/24
    ports:
    - protocol: TCP
      port: 5978

Secrets management

秘密管理

The Twelve-Factor App is suggested to pass configuration information as environment variables. This allows us to separate configuration from code, which is useful when we need to run the code in a different environment like dev, test, acc, and prod.

建议使用十二要素应用程序将配置信息作为环境变量传递。 这使我们能够将配置与代码分开，这在我们需要在不同的环境(例如dev，test，acc和prod)中运行代码时非常有用。

Kubernetes provides secret resources for this purpose but there are different ways of using. By default, secret values are stored alongside other configuration information in the etcd database, Although base64 encode make content unreadable, it does not encrypt it when you simply can decode it with base64 decode. Somehow I can say base64 is plain text to an attacker.Look at this example, username and password are encrypted which you can decrypt easily :

Kubernetes为此提供了秘密资源，但是有不同的使用方式。 默认情况下，秘密值与其他配置信息一起存储在etcd数据库中。尽管base64编码使内容不可读，但是当您仅可以使用base64解码对其进行解码时，它就不会对其进行加密。 我可以说base64是攻击者的纯文本，看这个例子，用户名和密码是加密的，可以轻松解密：

apiVersion: v1
data:
  username: YWRtaW4=
  password: MWYyZDFlMmU2N2Rm
kind: Secret
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: { ... }
  name: mysecret
type: Opaque

As probably you guess storing encrypted values in etcd is not secure enough. Some third-party systems are specifically designed to store secret and sensitive values like HashiCorp Vault. These third-parties usually offer integration with multiple backends secret stores and can also control access to that only specific containers have access to particular secrets.

可能您猜测将加密值存储在etcd中不够安全。 一些第三方系统是专门为存储秘密和敏感值而设计的，例如HashiCorp Vault。 这些第三方通常提供与多个后端秘密存储的集成，并且还可以控制对仅特定容器有权访问特定秘密的访问。

If you want to keep a secret, you should hide it from yourself

如果要保守秘密，应该对自己隐藏

George Orwell, 1984

乔治·奥威尔(George Orwell)，1984年

Another common way of storing secret variables is to store them in a file, Kubernetes supports passing secrets into pods through volume mounts. The containerized code needs to read values out of these files. If the mounted volume is a temporary filesystem, it means files are not writing to the disk but held in memory and also the values in the file are not accessible via docker inspect or kubectl describe. By this example, you can get some insight about how to store the secrets into a file:

存储秘密变量的另一种常见方式是将它们存储在文件中，Kubernetes支持通过卷安装将秘密传递到Pod中。 容器化的代码需要从这些文件中读取值。 如果挂载的卷是临时文件系统，则意味着文件不是写入磁盘，而是保存在内存中，并且文件中的值也无法通过docker inspect或kubectl describe进行访问。 通过此示例，您可以了解有关如何将机密存储到文件中的信息：

apiVersion: v1
kind: Pod
metadata:
  name: mypod
spec:
  containers:
  - name: mypod
    image: redis
    volumeMounts:
    - name: foo
      mountPath: "/etc/foo"
      readOnly: true
  volumes:
  - name: foo
    secret:
      secretName: mysecret
      items:
      - key: username
        path: my-group/my-username

Wrap up

结语

Kubernetes security is complex. It is not possible to cover all the topics in one single article. With this article, I meant to cover common security measures for developers, and please keep in mind to follow and read good practices during deployment on your clusters. Count on my tips as a starting point.

Kubernetes的安全性很复杂。 不可能在一篇文章中涵盖所有主题。 在本文中，我打算介绍针对开发人员的常见安全措施，并请记住在群集上进行部署时遵循并阅读良好做法。 依靠我的技巧作为起点。

I highly recommend reading Kubernetes Security by Liz Rice and Michael Hausenblas which cover all of these topics with much better details.

我强烈建议您阅读Liz Rice和Michael Hausenblas撰写的Kubernetes Security，其中涵盖了所有这些主题，而且细节要好得多。

翻译自: https://itnext.io/security-measures-inside-kubernetes-cluster-e43b3a328473

kubernetes 集群