aws eks_有关AWS eks中kubernetes podsecuritypolicy的详细指南

aws eks

豆荚安全 (Pod Security)

Pods have a variety of different settings that can strengthen or weaken your overall security posture. As a Kubernetes practitioner your chief concern should be preventing a process that’s running in a container from escaping the isolation boundaries of Docker and gaining access to the underlying host.

Pod具有多种不同的设置，可以增强或削弱您的整体安全状况。 作为Kubernetes的从业者，您的主要担心应该是防止容器中运行的进程逃避Docker的隔离边界并获得对基础主机的访问权。

The processes that run within a container run under the context of the Linux root user by default. Although the actions of root within a container are partially constrained by the set of Linux capabilities that Docker assigns to the containers, these default privileges could allow an attacker to escalate their privileges and/or gain access to sensitive information bound to the host, including Secrets and ConfigMaps.

默认情况下，在容器内运行的进程在Linux超级用户的上下文中运行。 尽管容器中root用户的操作部分受到Docker分配给容器的Linux功能的限制，但是这些默认特权可以使攻击者提升其特权和/或访问绑定到主机的敏感信息，包括Secrets。和ConfigMaps。

If you want to know more about Docker, Linux Capabilites and why pod security matters, please read my other articles before continuing reading this one, so that you can fully understand what it is and why we need it:

如果您想了解有关Docker，Linux Capabilites以及Pod安全性为何重要的更多信息，请在继续阅读本文章之前先阅读我的其他文章，以便您完全了解它的含义以及我们为什么需要它：

为什么我们需要它 (Why We Need It)

In most clusters today, by default, all resources (e.g. Deployments and ReplicatSets) and authenticated users have permissions to create pods, even privileged ones, running as root, accessing files and paths from the host machine, making the attack surface much bigger.

在当今的大多数群集中，默认情况下，所有资源(例如Deployments和ReplicatSets)和经过身份验证的用户都具有创建Pod，甚至是特权Pod的权限，这些Pod以root身份运行，可以从主机访问文件和路径，从而使攻击面更大。

If an attacker attacked into the pod, since the pod is already running as root, if he could escape the boundary that is the container by exploiting some vulnerabilities, he could potentially become root on the host machine.

如果攻击者攻击了Pod，则由于Pod已经以root身份运行，如果他可以通过利用某些漏洞来逃避作为容器的边界，则他有可能成为主机上的root。

There are a lot of third party run-time security tools to strengthen up security inside Kubernetes, but if you don’t want to buy something else, with minimum effort, you can already strengthen up the security by simply applying the PodSecurityPolicy which comes native with Kubernetes.

有很多第三方运行时安全工具可以增强Kubernetes内部的安全性，但是如果您不想购买其他产品，只需花费最少的精力，您就可以通过简单地应用原生的PodSecurityPolicy来增强安全性。与Kubernetes。

什么是PodSecurityPolicy (What is PodSecurityPolicy)

A PodSecurityPolicy is an admission controller resource, which enables fine-grained authorization of pod creation and updates.

PodSecurityPolicy是准入控制器资源，可用于对Pod创建和更新进行细粒度的授权。

It is a cluster-level resource that controls security sensitive aspects of the pod specification, and defines a set of conditions, with which a pod must run, in order to be accepted into the system.

它是一个集群级资源，它控制Pod规范的安全敏感方面，并定义了一组条件，pod必须运行这些条件才能被系统接受。

When a request to create or update a Pod does not meet the conditions in the PodSecurityPolicy, that request is rejected and an error is returned.

当创建或更新Pod的请求不符合PodSecurityPolicy中的条件时，该请求将被拒绝并返回错误。

在开始之前 (Before Starting)

In this article we are going to focus mainly on AWS EKS. As of today, the latest version of EKS in AWS is 1.16 which already enables the admission controller with a default privileged policy. So, in short, if you have already upgraded to the latest version of EKS, there is nothing you need to do before you can continue.

在本文中，我们将主要关注AWS EKS。 截至今天，AWS中EKS的最新版本是1.16，这已经为准入控制器启用了默认特权策略。 简而言之，如果您已经升级到最新版本的EKS，则无需执行任何操作即可继续。

A little more on this topic:

关于此主题的更多信息：

Under the hood, in order to use PodSecurityPolicy, you must first create and define policies that new and updated Pods must meet, then enable the PodSecurityPolicy admission controller, which validates requests to create and update Pods against the defined policies.

在幕后，为了使用PodSecurityPolicy，您必须首先创建和定义新的和更新的Pod必须满足的策略，然后启用PodSecurityPolicy准入控制器，该控制器根据定义的策略验证创建和更新Pod的请求。

PodSecurityPolicy became available as early as in Kubernetes 1.5/1.6.

PodSecurityPolicy最早在Kubernetes 1.5 / 1.6中可用。

In Google Compute Platform, GKE clusters running Kubernetes version 1.8.6 or later already enabled it.

在Google Compute Platform中，运行Kubernetes 1.8.6或更高版本的GKE集群已启用它。

In AWS, The pod security policy admission controller is only enabled on Amazon EKS clusters running Kubernetes version 1.13 or later.

在AWS中，仅在运行Kubernetes版本1.13或更高版本的Amazon EKS集群上启用pod安全策略准入控制器。

Note that, when multiple PodSecurityPolicies are available, the admission controller uses the first policy that successfully validates. Policies are ordered alphabetically, and the controller prefers non-mutating policies (policies that don’t change the Pod) over mutating policies.

请注意，当多个PodSecurityPolicies可用时，准入控制器将使用成功验证的第一个策略。 策略按字母顺序排序，并且控制器更喜欢非更改策略(不更改Pod的策略)而不是更改策略。

Another note before you continue: PodSecurityPolicies are enforced by enabling the admission controller, but doing so without authorizing any policies will prevent any pods from being created in the cluster.

在继续之前，请注意另一点：PodSecurityPolicies是通过启用准入控制器来强制执行的，但是在未授权任何策略的情况下这样做将阻止在集群中创建任何Pod 。

PodSecurityPolicy可以做什么 (What PodSecurityPolicy Can Do)

With PodSecurityPolicy, you can control the following:

使用PodSecurityPolicy，您可以控制以下内容：

• Running of privileged containers
  运行特权容器
• Usage of host namespaces
  主机名称空间的用法
• Usage of host networking and ports
  主机网络和端口的使用
• Usage of volume types
  卷类型的用法
• Usage of the host filesystem
  主机文件系统的用法
• Allow specific FlexVolume drivers
  允许特定的FlexVolume驱动程序
• Allocating an FSGroup that owns the pod’s volumes
  分配拥有Pod卷的FSGroup
• Requiring the use of a read only root file system
  要求使用只读根文件系统
• The user and group IDs of the container
  容器的用户和组ID
• Restricting escalation to root privileges
  将升级限制为root特权
• Linux capabilities
  Linux功能
• The SELinux context of the container
  容器的SELinux上下文
• The Allowed Proc Mount types for the container
  容器的允许的Proc Mount类型
• The AppArmor profile used by containers
  容器使用的AppArmor配置文件
• The seccomp profile used by containers
  容器使用的seccomp配置文件
• The sysctl profile used by containers
  容器使用的sysctl配置文件

While this seems to be an overwhelmingly long list, chances are, you might have already used a few of them when you are using Kubernetes, for example:

虽然这似乎是一个庞大的清单，但有可能，当您使用Kubernetes时，您可能已经使用了其中一些，例如：

• You need to mount some storage to the pod like PVC
  您需要像PVC那样将一些存储空间安装到Pod
• When you don’t need/want to run a pod with root user, you use security context to run as a user/group
  当您不需要/不想与root用户一起运行pod时，可以使用安全上下文以用户/组的身份运行
• For some of your applications, you need to mount some volumes to it
  对于您的某些应用程序，您需要在其中装载一些卷
• For some logging applications, you need to access the logs from a path that lives on the host
  对于某些日志记录应用程序，您需要从主机上存在的路径访问日志。

You can do this, because either your cluster does not enable the PodSecurityPolicy admission controller, or it is enabled but there is a default policy that allows everything.

您可以执行此操作，因为您的群集没有启用PodSecurityPolicy准入控制器，或者已启用，但是有一个默认策略允许所有操作。

In the case of AWS EKS, the clusters with Kubernetes version 1.13 and higher have a default pod security policy named eks.privileged. This policy has no restriction on what kind of pod can be accepted into the system, which is equivalent to running Kubernetes with the PodSecurityPolicy controller disabled (or there is one that allows you to do everything).

对于AWS EKS，具有Kubernetes版本1.13和更高版本的集群具有一个名为eks.privileged的默认Pod安全策略。 该策略对可以将哪种Pod接受到系统没有任何限制，这等效于在禁用PodSecurityPolicy控制器的情况下运行Kubernetes(或者有一种方法可以执行所有操作)。

PodSecurityPolicy无法做什么 (What PodSecurityPolicy Can Not Do)

However, PodSecurityPolicy can’t do everything.

但是，PodSecurityPolicy不能做所有事情。

If it’s not in the list above, it can’t do it.

如果不在上面的列表中，则无法执行。

Also, due to the nature of the admission controller, the policy only works when you are creating or updating the pod. If the pod violates the policy, it won’t be created.

同样，由于准入控制器的性质，该策略仅在您创建或更新Pod时有效。 如果pod违反了该政策，则不会创建。

However, if you modify the policy after pods are already up and running, making the pods violating the new policy, the pods won’t be shut down.

但是，如果在Pod已启动并运行后修改策略，使Pod违反新策略，则Pod不会关闭。

PodSecurityPolicy, as the name suggests, is only a set of policies that are enforced when creating/updating pod. It is not container run-time security platform that can detects violations and shutdown pods. This is important to know, and this explains why you might want to consider tools like aqua, sysdig, falco when you want more control over Kubernetes run-time security.

顾名思义，PodSecurityPolicy只是在创建/更新Pod时强制执行的一组策略。 不是容器运行时安全平台可以检测违规和关闭容器。 要知道这一点很重要，这解释了为什么当您想对Kubernetes运行时安全性进行更多控制时，可能要考虑使用aqua，sysdig，falco之类的工具。

准备环境 (Preparing Your Environment)

For AWS EKS latest version 1.16 (or from1.13), the admission controller is already enabled, so there is nothing you need to do. To verify the default policy is there, run:

对于AWS EKS最新版本1.16(或从1.13开始)，已经启用了准入控制器，因此您无需执行任何操作。 要验证是否存在默认策略，请运行：

kubectl get psp eks.privileged

Example output:

输出示例：

NAME             PRIV   CAPS   SELINUX    RUNASUSER   FSGROUP    SUPGROUP   READONLYROOTFS   VOLUMES
eks.privileged   true   *      RunAsAny   RunAsAny    RunAsAny   RunAsAny   false            *

If you don’t feel like ruining your running cluster, testing from minikube might be a good idea.

如果您不想破坏正在运行的集群，那么从minikube进行测试可能是个好主意。

创建PodSecurityPolicy (Creating PodSecurityPolicy)

A default, privileged PodSecurityPolicy looks like this:

默认的特权PodSecurityPolicy如下所示：

---
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: privileged
  annotations:
    kubernetes.io/description: 'privileged allows full unrestricted access to
      pod features, as if the PodSecurityPolicy controller was not enabled.'
    seccomp.security.alpha.kubernetes.io/allowedProfileNames: '*'
  labels:
    kubernetes.io/cluster-service: "true"
    eks.amazonaws.com/component: pod-security-policy
spec:
  privileged: true
  allowPrivilegeEscalation: true
  allowedCapabilities:
  - '*'
  volumes:
  - '*'
  hostNetwork: true
  hostPorts:
  - min: 0
    max: 65535
  hostIPC: true
  hostPID: true
  runAsUser:
    rule: 'RunAsAny'
  seLinux:
    rule: 'RunAsAny'
  supplementalGroups:
    rule: 'RunAsAny'
  fsGroup:
    rule: 'RunAsAny'
  readOnlyRootFilesystem: false

This is the one from AWS EKS default PodSecurityPolicy. It’s a Standard Kubernetes resource definition format.

这是AWS EKS默认PodSecurityPolicy中的一个。 这是标准的Kubernetes资源定义格式。

As you can see, most entries corresponds to the list above, and you can also see from the values, that, this policy does not actually limit anything. Running your cluster with this policy is identical with running your cluster with PodSecurityPolicy admission controller disabled.

如您所见，大多数条目都与上面的列表相对应，并且您还可以从值中看到，该策略实际上并没有限制任何内容。 使用此策略运行群集与在禁用PodSecurityPolicy准入控制器的情况下运行群集相同。

Another example of a slightly limited PodSecurityPolicy:

PodSecurityPolicy稍微受限制的另一个示例：

apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: example
spec:
  privileged: false  # Don't allow privileged pods!
  # The rest fills in some required fields.
  seLinux:
    rule: RunAsAny
  supplementalGroups:
    rule: RunAsAny
  runAsUser:
    rule: RunAsAny
  fsGroup:
    rule: RunAsAny
  volumes:
  - '*'

In this example above, the only limit is, privileged pods are not allowed, but regarding user id, volumes, etc, are not limited.

在上面的示例中，唯一的限制是，不允许特权吊舱，但是关于用户标识，卷等的限制不受限制。

创建RoleBinding — PodSecurityPolicy的工作方式 (Creating RoleBinding — How PodSecurityPolicy Works)

When a PodSecurityPolicy resource is created, it actually does nothing.

创建PodSecurityPolicy资源时，它实际上不执行任何操作。

In order to use the policy, the requesting user (or target pod’s service account) must have permission to “use” this policy, by allowing the use verb on the policy.

为了使用该策略，发出请求的用户(或目标Pod的服务帐户 )必须具有允许通过使用策略上的use动词来“使用”此策略的权限。

So, besides the PodSecurityPolicy, you also need to create a Role/ClusterRole that allows you to use that policy, and a RoleBinding/ClusterRoleBinding to bind to the role.

因此，除了PodSecurityPolicy外，您还需要创建一个Role / ClusterRole来允许您使用该策略，并创建一个RoleBinding / ClusterRoleBinding来绑定到该角色。

For example, if you created a super PodSecurityPolicy which allows everything (the first example above), but you want to limit it to a certain namespace, say, kube-system, then you can create a Role in kube-system with permission to use your PodSecurityPolicy, then create a RoleBinding to bind a service account to that Role:

例如，如果您创建了一个允许所有内容的超级PodSecurityPolicy(上面的第一个示例)，但您想将其限制为某个名称空间(例如kube-system)，则可以在kube-system中创建一个具有使用权限的角色您的PodSecurityPolicy，然后创建RoleBinding将服务帐户绑定到该Role：

---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: eks:podsecuritypolicy:privileged
  namespace: kube-system
  labels:
      kubernetes.io/cluster-service: "true"
      eks.amazonaws.com/component: pod-security-policy
rules:
- apiGroups:
  - policy
  resourceNames:
  - privileged
  resources:
  - podsecuritypolicies
  verbs:
  - use
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: eks:podsecuritypolicy:privileged
  namespace: kube-system
  annotations:
    kubernetes.io/description: 'Allow service account aws-node to use awsnode psp in kube-system namespace.'
  labels:
    kubernetes.io/cluster-service: "true"
    eks.amazonaws.com/component: pod-security-policy
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: eks:podsecuritypolicy:privileged
subjects:
  - kind: ServiceAccount
    name: aws-node
    namespace: kube-system

In this example above, a service account “aws-node” from kube-system namespace is bound to a role which can “use” the “privileged” policy. So when the pod is using this service account in kube-system namespace, it can have any privileges.

在上面的示例中，来自kube-system名称空间的服务帐户“ aws-node”绑定到可以“使用”“特权”策略的角色。 因此，当pod在kube-system名称空间中使用此服务帐户时，它可以具有任何特权。

For another example, if you created a restricted/unprivileged PodSecurityPolicy that only allows basic permission and you want to make this as a default to all authenticated users, you can create a ClusterRole, giving it access to use this policy, then create a ClusterRoleBinding to bind all authenticated users to this role.

再举一个例子，如果您创建了一个仅允许基本权限的受限/非特权PodSecurityPolicy，并且希望将其作为所有经过身份验证的用户的默认权限，则可以创建一个ClusterRole，为其提供使用此策略的访问权限，然后创建一个ClusterRoleBinding将所有经过身份验证的用户绑定到该角色。

The only thing you need to pay attention is, if you want to by default not allow privileged pods, in the ClusterRoleBinding, you can bind a group “system:authenticated” instead of to a specific service account.

您唯一需要注意的是，如果您默认情况下不想允许特权Pod，则在ClusterRoleBinding中，可以将组“ system：authenticated”绑定到特定的服务帐户，而不是绑定。

For a fully restricted PodSecurityPolicy and ClusterRole/ClusterRoleBinding, see here.

有关完全受限制的PodSecurityPolicy和ClusterRole / ClusterRoleBinding的信息，请参见此处 。

测试中 (Testing)

Now let’s have a test of pod creation with restricted policy.

现在让我们测试受限制策略的pod的创建。

First, delete the default privileged PodSecurityPolicy from AWS EKS:

首先，从AWS EKS删除默认特权PodSecurityPolicy：

kubectl delete psp eks.privileged

Then create the restricted policy:

然后创建受限策略：

git clone https://github.com/IronCore864/ekspsp.git
kubectl apply -f restricted.yaml

Now let’s run a test:

现在让我们进行测试：

apiVersion: v1
kind: Pod
metadata:
  name: test
spec:
  containers:
    - name: test
      # run as user 0
      image: docker.io/ironcore864/go-hello-http:root

A very simple image with a hello-world app, but in the Dockerfile, it runs as user ID 0 (root). You can try applying the above pod, and you will get error:

带有hello-world应用程序的非常简单的映像，但是在Dockerfile中，它以用户ID 0(根)运行。 您可以尝试应用上述吊舱，但会出现错误：

Error: container has runAsNonRoot and image will run as root

For an image that runs as other user ID:

对于以其他用户ID运行的图像：

apiVersion: v1
kind: Pod
metadata:
  name: test
spec:
  containers:
    - name: test
      # run as user 1000
      image: docker.io/ironcore864/go-hello-http:latest

If you apply this one, it would work, because in this image it runs as user ID 1000.

如果您应用此选项，它将起作用，因为在此图像中，它以用户ID 1000运行。

最佳实践 (Best Practice)

限制可以特权运行的容器 (Restrict the containers that can run as privileged)

Containers that run as privileged inherit all of the Linux capabilities assigned to root on the host. Seldom do containers need these types of privileges to function properly. You can reject pods with containers configured to run as privileged.

以特权身份运行的容器继承了分配给主机根用户的所有Linux功能。 容器很少需要这些类型的特权才能正常运行。 您可以拒绝具有配置为以特权方式运行的容器的容器。

However, there are a few pods in EKS that need some type of privilege, be it run as root or capabilities. For example, the coredns needs NET_BIND_SERVICE, but it doesn’t need to run as root; while aws-node, kube-proxy requires different access to different paths on the host machine.

但是，EKS中有一些Pod需要某种特权，无论它是以root身份还是以功能运行。 例如，coredns需要NET_BIND_SERVICE，但是它不需要以root身份运行； 而aws-node时，kube-proxy需要对主机上不同路径的不同访问。

So the recommendation here is to create one policy for each pod, bind it to that service account, so that each pod has exactly the required minimum set of permissions.

因此，这里的建议是为每个Pod创建一个策略，并将其绑定到该服务帐户，以便每个Pod都具有所需的最小权限集。

不要以根用户身份在容器中运行进程 (Do not run processes in containers as root)

All containers run as root by default. This could be problematic if an attacker is able to exploit a vulnerability in the application and get shell access to the running container.

默认情况下，所有容器都以root用户身份运行。 如果攻击者能够利用应用程序中的漏洞并使外壳程序访问运行中的容器，则可能会出现问题。

You can mitigate this risk a variety of ways:

您可以通过多种方式减轻这种风险：

First, by removing the shell from the container image.

首先，通过从容器映像中删除外壳。

Second, adding the USER directive to your Dockerfile or running the containers in the pod as a non-root user. The Kubernetes podSpec includes a set of fields under spec.securityContext, that allow to let you specify the user and/or group to run your application as. These fields are runAsUser and runAsGroup respectively. You can mandate the use of these fields by creating a pod security policy.

其次，将USER指令添加到Dockerfile或以非root用户身份运行pod中的容器。 该Kubernetes podSpec包括一组下的字段spec.securityContext ，允许让你指定用户和/或组来运行应用程序的。 这些字段分别是runAsUser和runAsGroup 。 您可以通过创建Pod安全策略来强制使用这些字段。

切勿在Docker中运行Docker或将套接字安装在容器中 (Never run Docker in Docker or mount the socket in the container)

While this conveniently lets you to build/run images in Docker containers, you’re basically relinquishing complete control of the node to the process running in the container.

尽管这使您可以方便地在Docker容器中构建/运行映像，但是基本上是将节点的完全控制权交给了容器中运行的进程。

If you need to build container images inside Kubernetes, don’t. Or, either to use some build service, or use build tools that doesn’t depend on docker daemon like Kaniko.

如果您需要在Kubernetes中构建容器映像，则不需要。 或者，要么使用某些构建服务，要么使用不依赖于docker daemon的构建工具，例如Kaniko 。

限制使用hostPath (Restrict the use of hostPath)

hostPath is a volume that mounts a directory from the host directly to the container. Rarely will pods need this type of access, but if they do, you need to be aware of the risks. By default pods that run as root will have write access to the file system exposed by hostPath. This could allow an attacker to modify the kubelet settings, create symbolic links to directories or files not directly exposed by the hostPath, e.g. /etc/shadow, install ssh keys, read secrets mounted to the host, and other malicious things. To mitigate the risks from hostPath, configure the spec.containers.volumeMounts as readOnly .

hostPath是一个将目录从主机直接装载到容器的卷。 吊舱很少需要这种类型的访问权限，但如果确实需要，则需要意识到其风险。 默认情况下，以root用户身份运行的Pod将具有对hostPath公开的文件系统的写访问权。 这可能使攻击者可以修改kubelet设置，创建未直接由hostPath公开的目录或文件的符号链接，例如/ etc / shadow，安装ssh密钥，读取安装在主机上的机密以及其他恶意内容。 为缓解hostPath风险，配置spec.containers.volumeMounts为readOnly 。

You should also use a pod security policy to restrict the directories that can be used by hostPath volumes. You can see examples in this repo.

您还应该使用pod安全策略来限制hostPath卷可以使用的目录。 您可以在此仓库中查看示例。

不允许特权升级 (Do not allow privileged escalation)

Privileged escalation allows a process to change the security context under which its running. Sudo is a good example of this as are binaries with the SUID or SGID bit. Privileged escalation is basically a way for users to execute a file with the permissions of another user or group. You can prevent a container from using privileged escalation with PodSecurityPolicy as well.

特权升级允许进程更改其运行所在的安全上下文。 Sudo和具有SUID或SGID位的二进制文件就是一个很好的例子。 特权升级基本上是用户在另一个用户或组的许可下执行文件的方式。 您还可以防止容器将特权升级与PodSecurityPolicy一起使用。

摘要 (Summary)

In order to make it work easier and out of the box, I create a GitHub repo with necessary policies, role bindings, for EKS related pods, and for ingress controllers.

为了使它更容易和开箱即用，我创建了一个GitHub存储库，其中包含必要的策略，角色绑定，与EKS相关的Pod和入口控制器。

Detailed steps with the YAML files are documented here: https://github.com/IronCore864/ekspsp, and you can also follow the steps in the repo to setup everything you need.

YAML文件的详细步骤记录在这里： https : //github.com/IronCore864/ekspsp，您也可以按照回购中的步骤来设置所需的一切。

翻译自: https://medium.com/devops-dudes/a-detailed-guide-to-kubernetes-podsecuritypolicy-in-aws-eks-71c66ded6375

aws eks