gke下载_从gke安全访问AWS

gke下载

It is not a rare case when an application running on Google Kubernetes Engine (GKE) needs to access Amazon Web Services (AWS) APIs. Any application has needs. Maybe it needs to run an analytics query on Amazon Redshift, access data stored in Amazon S3 bucket, convert text to speech with Amazon Polly or use any other AWS service. This multi-cloud scenario is common nowadays, as companies are working with multiple cloud providers.

在Google Kubernetes Engine( GKE )上运行的应用程序需要访问Amazon Web Services( AWS )API时，这种情况并非罕见。 任何应用程序都有需求。 也许它需要在Amazon Redshift上运行分析查询，访问存储在Amazon S3存储桶中的数据，使用Amazon Polly将文本转换为语音或使用任何其他AWS服务。 随着公司与多家云提供商合作，这种多云场景在当今很普遍。

Cross-cloud access introduces a new challenge; how to manage cloud credentials, required to access from one cloud provider to services running in the other. The naive approach, distributing and saving cloud provider secrets is not the most secure approach; distributing long-term credentials to each service, that needs to access AWS services, is challenging to manage and a potential security risk.

跨云访问带来了新挑战。 如何管理云凭据，这是从一个云提供商访问另一提供商中运行的服务所必需的。 幼稚的方法，分发和保存云提供商的机密不是最安全的方法。 为需要访问AWS服务的每项服务分配长期凭证对于管理来说是一项挑战，并且存在潜在的安全风险。

当前解决方案 (Current Solutions)

Each cloud provides it’s own unique solution to overcome this challenge, and if you are working with a single cloud provider, it’s more than enough.

每个云都提供了自己独特的解决方案来克服这一挑战，并且如果您与单个云提供商合作，那就足够了。

Google Cloud announced a Workload Identity, the recommended way for GKE applications to authenticate to and consume other Google Cloud services. Workload Identity works by binding Kubernetes service accounts and Cloud IAM service accounts, so you can use Kubernetes-native concepts to define which workloads run as which identities, and permit your workloads to automatically access other Google Cloud services, all without having to manage Kubernetes secrets or IAM service account keys! Read DoiT Kubernetes GKE Workload Identity blog post.

Google Cloud宣布了Workload Identity ，这是GKE应用程序认证和使用其他Google Cloud服务的推荐方式。 Workload Identity通过绑定Kubernetes服务帐户和Cloud IAM服务帐户来工作，因此您可以使用Kubernetes本机概念来定义哪些工作负载以哪些身份运行，并允许您的工作负载自动访问其他Google Cloud服务，而无需管理Kubernetes秘密或IAM服务帐户密钥！ 阅读DoiT Kubernetes GKE工作负载身份博客文章。

Amazon Web Services supports a similar functionality with IAM Roles for Service Accounts feature. With IAM roles for service accounts on Amazon EKS clusters, you can associate an IAM role with a Kubernetes service account. This service account can then provide AWS permissions to the containers in any pod that uses that service account. With this feature, you no longer need to provide extended permissions to the worker node IAM role so that pods on that node can call AWS APIs.

Amazon Web Services通过“ IAM服务帐户角色”功能支持类似的功能。 借助Amazon EKS集群上服务帐户的IAM角色，您可以将IAM角色与Kubernetes服务帐户关联。 然后，该服务帐户可以向使用该服务帐户的任何吊舱中的容器提供AWS权限。 使用此功能，您不再需要为工作节点IAM角色提供扩展权限，以便该节点上的Pod可以调用AWS API。

But what if you are running your application workload on GKE cluster and would like to access AWS services without compromising on security?

但是，如果您在GKE集群上运行应用程序工作负载，并且希望在不影响安全性的情况下访问AWS服务，该怎么办？

用例定义 (Use Case Definition)

Let’s assume that you already have an AWS account, and a GKE cluster and your company has decided to run a microservice-based application on GKE cluster, but still wants to use resources in the AWS account (Amazon S3 and SNS services) to integrate with other systems deployed on AWS.

假设您已经有一个AWS账户和一个GKE集群，并且您的公司已决定在GKE集群上运行基于微服务的应用程序，但仍想使用AWS账户中的资源(Amazon S3和SNS服务)与在AWS上部署的其他系统。

For example, the orchestration job (deployed as a Kubernetes Job) is running inside a GKE cluster and needs to upload a data file into a S3 bucket and send a message to an Amazon SNS topic. The equivalent command-line might be:

例如，业务流程作业 (部署为Kubernetes作业)在GKE集群中运行，需要将数据文件上传到S3存储桶中并向Amazon SNS主题发送消息。 等效的命令行可能是：

Pretty simple example. In order for these commands to succeed, the orchestration job must have AWS credentials available to it, and those credentials must be able to make the relevant API calls.

很简单的例子。 为了使这些命令成功执行， 编排作业必须具有可用的AWS凭证，并且这些凭证必须能够进行相关的API调用。

天真(非安全)方法：IAM长期证书 (The Naive (and non-secure) Approach: IAM long-term credentials)

Export AWS Access Key and Secret Key for some AWS IAM User, and inject AWS credentials into the orchestration job, either as a credentials file or environment variables. Probably not doing this directly, but using Kubernetes Secrets resource protected with RBAC authorization policy.

导出某些AWS IAM用户的AWS Access Key和Secret Key，并将AWS凭证作为凭证文件或环境变量注入到业务流程作业中 。 可能不是直接执行此操作，而是使用受RBAC授权策略保护的Kubernetes Secrets资源。

The risk here is that these credentials never expire. They have to be transferred somehow from the AWS environment to the GCP environment, and in most cases, people want them to be stored somewhere so that they can be used to re-create the orchestration job later if required.

这里的风险是这些凭据永远不会过期。 必须将它们从AWS环境转移到GCP环境，并且在大多数情况下，人们希望将它们存储在某个位置，以便在以后需要时可用于重新创建业务流程作业 。

When using long-term AWS credentials, there are multiple ways that your AWS account can be compromised; unintentionally committing AWS credentials into a GitHub repository, keeping them in a Wiki system, reusing credentials for different services and applications, allowing non-restricted access and, so on.

使用长期AWS凭证时，可以通过多种方式来破坏您的AWS账户； 无意中将AWS凭证提交到GitHub存储库中，将其保存在Wiki系统中，针对不同的服务和应用程序重用凭证，并允许无限制访问， 等等 。

While it’s possible to design a proper credentials management solution for issued IAM User credentials, it won’t be required if you will never create these long-term credentials in the first place.

尽管可以为已发布的IAM用户凭据设计适当的凭据管理解决方案，但是如果您永远不会首先创建这些长期凭据，则不需要此解决方案。

拟议方法 (The Proposed Approach)

The basic idea is to assign AWS IAM Role to GKE Pod, similarly to Workload Identity and EKS IAM Roles for Service Accounts cloud-specific features.

基本思想是将AWS IAM角色分配给GKE Pod，类似于服务帐户云特定功能的Workload Identity和EKS IAM角色 。

Luckily for us, AWS allows to create an IAM role for OpenID Connect Federation OIDC identity providers instead of IAM users. On the other hand, Google implements OIDC provider and integrates it tightly with GKE through Workload Identity feature. Providing a valid OIDC token to GKE pod, running under Kubernetes Service Account linked to a Google Cloud Service Account. All these may come in handy to implement GKE-to-AWS secure access.

对我们来说幸运的是，AWS允许为OpenID Connect Federation OIDC身份提供者而不是IAM用户创建IAM角色。 另一方面，Google实现了OIDC提供程序，并通过Workload Identity功能将其与GKE紧密集成。 为GKE pod提供有效的OIDC令牌，该令牌在链接到Google Cloud Service帐户的Kubernetes服务帐户下运行。 所有这些都可以帮助实现GKE-to-AWS安全访问。

将OIDC访问令牌交换为ID令牌 (Exchanging OIDC access token to ID token)

There is one thing missing, required to complete the puzzle. With properly setup Workflow Identity GKE Pod gets an OIDC access token that allows access to Google Cloud services. In order to get temporary AWS credentials from AWS Security Token Service (STS), you need to provide a valid OIDC ID token.

缺少一件事，需要完成拼图。 通过正确设置Workflow Identity， GKE Pod会获得OIDC 访问令牌 ，该令牌可访问Google Cloud服务。 为了从AWS安全令牌服务( STS )获取临时AWS凭证，您需要提供有效的OIDC ID令牌 。

AWS SDK (and aws-cli tool) will automatically request temporary AWS credentials from STS service, when the following environment variables are properly setup:

正确设置以下环境变量后，AWS开发工具包(和aws-cli工具)将自动从STS服务请求临时AWS凭证：

• AWS_WEB_IDENTITY_TOKEN_FILE - the path to the web identity token file (OIDC ID token)

  AWS_WEB_IDENTITY_TOKEN_FILE -Web身份令牌文件的路径(OIDC ID令牌)

• AWS_ROLE_ARN - the ARN of the role to assume by Pod containers

  AWS_ROLE_ARN -Pod容器承担的角色的ARN

• AWS_ROLE_SESSION_NAME - the name applied to this assume-role session

  AWS_ROLE_SESSION_NAME应用于此假定角色会话的名称

This may sound a bit complex, but I will provide a step-by-step guide and supporting open source project dointl/gtoken to simplify the setup.

这听起来可能有些复杂，但是我将提供分步指南并支持开源项目dointl / gtoken来简化设置。

gtoken-webhook Kubernetes突变入网webhook (gtoken-webhook Kubernetes Mutating Admission webhook)

The gtoken-webhook is a Kubernetes mutating admission webhook, that mutates any K8s Pod running under specially annotated Kubernetes Service Account (see details below).

gtoken-webhook是一个Kubernetes突变入场Webhook，它可以突变运行在带有特殊注释的Kubernetes服务帐户下的所有K8s Pod(请参阅下面的详细信息)。

gtoken-webhook突变流 (gtoken-webhook mutation flow)

The gtoken-webhook injects a gtoken initContainer into a target Pod and an additional gtoken sidekick container (to refresh an OIDC ID token a moment before expiration), mounts token volume and injects three AWS-specific environment variables. The gtoken container generates a valid GCP OIDC ID Token and writes it to the token volume. It also injects required AWS environment variables.

所述gtoken-webhook注入一个gtoken initContainer到目标波德和附加gtoken搭档容器(以刷新一个OIDC ID令牌到期之前的力矩)，安装令牌体积并注入3 AWS-特定的环境变量。 gtoken容器会生成有效的GCP OIDC ID令牌，并将其写入令牌卷。 它还会注入必需的AWS环境变量。

The AWS SDK will automatically make the corresponding AssumeRoleWithWebIdentity calls to AWS STS on your behalf. It will handle in-memory caching as well as refreshing credentials as needed.

AWS开发工具包将代表您自动对AWS STS进行相应的AssumeRoleWithWebIdentity调用。 它将处理内存中的缓存以及根据需要刷新凭据。

配置流程指南 (The Configuration Flow Guide)

部署gtoken-webhook (Deploy gtoken-webhook)

1. To deploy the gtoken-webhook server, we need to create a webhook service and a deployment in our Kubernetes cluster. It’s pretty straightforward except one thing, which is the server’s TLS configuration. If you’d care to examine the deployment.yaml file, you’ll find that the certificate and corresponding private key files are read from command line arguments and that the path to these files comes from a volume mount that points to a Kubernetes secret:

   要部署gtoken-webhook服务器，我们需要在Kubernetes集群中创建一个webhook服务和一个部署。 这很简单，只有一件事是服务器的TLS配置。 如果您想检查deploy.yaml文件，则会发现从命令行参数读取了证书和相应的私钥文件，并且这些文件的路径来自指向Kubernetes机密的卷安装：

The most important thing to remember is to set the corresponding CA certificate later in the webhook configuration, so the apiserver will know that it should be accepted. For now, we’ll reuse the script originally written by the Istio team to generate a certificate signing request. Then we’ll send the request to the Kubernetes API, fetch the certificate, and create the required secret from the result.

要记住的最重要的事情是稍后在webhook配置中设置相应的CA证书，因此apiserver将知道应接受该apiserver 。 现在，我们将重用Istio团队最初编写的脚本来生成证书签名请求。 然后，我们会将请求发送到Kubernetes API，获取证书，然后从结果中创建所需的机密。

First, run the webhook-create-signed-cert.sh script and check if the secret holding the certificate and key has been created:

首先，运行webhook-create-signed-cert.sh脚本，并检查是否已创建包含证书和密钥的机密：

./deployment/webhook-create-signed-cert.sh


creating certs in tmpdir /var/folders/vl/gxsw2kf13jsf7s8xrqzcybb00000gp/T/tmp.xsatrckI71
Generating RSA private key, 2048 bit long modulus
.........................+++
....................+++
e is 65537 (0x10001)
certificatesigningrequest.certificates.k8s.io/gtoken-webhook-svc.default created
NAME                         AGE   REQUESTOR              CONDITION
gtoken-webhook-svc.default   1s    alexei@doit-intl.com   Pending
certificatesigningrequest.certificates.k8s.io/gtoken-webhook-svc.default approved
secret/gtoken-webhook-certs configured

Once the secret is created, we can create a deployment and service. These are standard Kubernetes deployment and service resources. Up until this point we’ve produced nothing but an HTTP server that’s accepting requests through service on port 443:

一旦创建了机密，我们就可以创建部署和服务。 这些是标准的Kubernetes部署和服务资源。 到目前为止，我们只生产了HTTP服务器，该服务器通过端口443上的服务接受请求：

配置突变入场webhook (Configure Mutating Admission webhook)

Now that our webhook server is running, it can accept requests from the apiserver. However, we should create some configuration resources in Kubernetes first. Let’s start with our validating webhook, then we’ll configure the mutating webhook later. If you take a look at the webhook configuration, you’ll notice that it contains a placeholder for CA_BUNDLE:

现在我们的webhook服务器正在运行，它可以接受来自apiserver请求。 但是，我们应该首先在Kubernetes中创建一些配置资源。 让我们从验证Webhook开始，然后再配置变异Webhook。 如果看一下webhook配置 ，您会注意到它包含CA_BUNDLE的占位符：

There is a small script that substitutes the CA_BUNDLE placeholder in the configuration with this CA. Run this command before creating the validating webhook configuration:

有一个小的脚本用此CA替换配置中的CA_BUNDLE占位符。 在创建验证Webhook配置之前，运行以下命令：

Create a mutating webhook configuration:

创建一个变异的webhook配置：

为gtoken-webhook配置RBAC (Configure RBAC for gtoken-webhook)

Create Kubernetes Service Account to be used with gtoken-webhook:

创建与gtoken-webhook一起使用的Kubernetes服务帐户：

kubectl create -f deployment/service-account.yaml

Define RBAC permission for webhook service account:

定义webhook服务帐户的RBAC权限：

# create a cluster role
kubectl create -f deployment/clusterrole.yaml
# define a cluster role binding
kubectl create 0f deployment/clusterrolebinding.yaml

流量变量 (Flow Variables)

Some of the following variables should be provided by the user, others will be automatically generated and reused in the following steps.

用户应提供以下某些变量，其他变量将在以下步骤中自动生成并重新使用。

• PROJECT_ID - GCP project ID (provided by the user)

  PROJECT_ID -GCP项目ID(由用户提供)

• CLUSTER_NAME - GKE cluster name (provided by the user)

  CLUSTER_NAME -GKE群集名称(由用户提供)

• GSA_NAME - Google Cloud Service Account name (provided by the user)

  GSA_NAME -Google Cloud Service帐户名(由用户提供)

• GSA_ID - Google Cloud Service Account unique ID (generated by Google)

  GSA_ID -Google云服务帐户的唯一ID(由Google生成)

• KSA_NAME - Kubernetes Service Account name (provided by the user)

  KSA_NAME -Kubernetes服务帐户名(由用户提供)

• KSA_NAMESPACE - Kubernetes namespace (provided by the user)

  KSA_NAMESPACE -Kubernetes命名空间(由用户提供)

• AWS_ROLE_NAME - AWS IAM role name (provided by the user)

  AWS_ROLE_NAME -AWS IAM角色名称(由用户提供)

• AWS_POLICY_NAME - an AWS IAM policy to assign to IAM role (provided by the user)

  AWS_POLICY_NAME分配给IAM角色的AWS IAM策略(由用户提供)

• AWS_ROLE_ARN - AWS IAM Role ARN identifier (generated by AWS)

  AWS_ROLE_ARN -AWS IAM角色ARN标识符(由AWS生成)

Google云端：启用GKE工作负载身份 (Google Cloud: Enable GKE Workload Identity)

Create a new GKE cluster with Workload Identity enabled:

创建一个启用了工作负载标识的新GKE集群：

gcloud beta container clusters create ${CLUSTER_NAME} --identity-namespace=${PROJECT_ID}.svc.id.goog

or update an existing cluster:

或更新现有集群：

gcloud beta container clusters update ${CLUSTER_NAME} --identity-namespace=${PROJECT_ID}.svc.id.goog

Google Cloud：创建一个Google Cloud Service帐户 (Google Cloud: Create a Google Cloud Service Account)

Create a Google Cloud Service Account:

创建一个Google Cloud Service帐户：

# create GCP Service Account
gcloud iam service-accounts create ${GSA_NAME}


# get GCP SA UID to be used for AWS Role with Google OIDC Web Identity
GSA_ID=$(gcloud iam service-accounts describe --format json ${GSA_NAME}@${PROJECT_ID}.iam.gserviceaccount.com  | jq -r '.uniqueId')

Update GSA_NAME Google Service Account with following roles:

使用以下角色更新GSA_NAME Google服务帐户：

• roles/iam.workloadIdentityUser - impersonate service accounts from GKE Workloads

  roles/iam.workloadIdentityUser模拟来自GKE Workloads的服务帐户

• roles/iam.serviceAccountTokenCreator - impersonate service accounts to create OAuth2 access tokens, sign blobs, or sign JWT tokens

  roles/iam.serviceAccountTokenCreator模拟服务帐户以创建OAuth2访问令牌，签名Blob或签名JWT令牌

gcloud iam service-accounts add-iam-policy-binding \
  --role roles/iam.workloadIdentityUser \
  --role roles/iam.serviceAccountTokenCreator \
  --member "serviceAccount:${PROJECT_ID}.svc.id.goog[${K8S_NAMESPACE}/${KSA_NAME}]" \
  ${GSA_NAME}@${PROJECT_ID}.iam.gserviceaccount.com

AWS：使用Google OIDC Federation创建AWS IAM角色 (AWS: Create AWS IAM Role with Google OIDC Federation)

Prepare a role trust policy document for Google OIDC provider:

为Google OIDC提供者准备角色信任策略文档：

cat > gcp-trust-policy.json << EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "accounts.google.com"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "accounts.google.com:sub": "${GSA_SA}"
        }
      }
    }
  ]
}
EOF

Create AWS IAM Role with Google Web Identity:

使用Google Web Identity创建AWS IAM角色：

aws iam create-role --role-name ${AWS_ROLE_NAME} --assume-role-policy-document file://gcp-trust-policy.json

Assign AWS Role desired policies:

分配所需的AWS角色策略：

aws iam attach-role-policy --role-name ${AWS_ROLE_NAME} --policy-arn arn:aws:iam::aws:policy/${AWS_POLICY_NAME}

Get AWS Role ARN to be used in K8s SA annotation:

获取要在K8s SA批注中使用的AWS Role ARN：

AWS_ROLE_ARN=$(aws iam get-role --role-name ${ROLE_NAME} --query Role.Arn --output text)

GKE：创建一个Kubernetes服务帐户 (GKE: Create a Kubernetes Service Account)

Create K8s namespace:

创建K8s命名空间：

kubectl create namespace ${K8S_NAMESPACE}

Create K8s Service Account:

创建K8s服务帐户：

kubectl create serviceaccount --namespace ${K8S_NAMESPACE} ${KSA_NAME}

Annotate K8s Service Account with GKE Workload Identity (GCP Service Account email):

用GKE Workload Identity(GCP服务帐户电子邮件)注释K8s服务帐户：

kubectl annotate serviceaccount --namespace ${K8S_NAMESPACE} ${KSA_NAME}
  iam.gke.io/gcp-service-account=${GSA_NAME}@${PROJECT_ID}.iam.gserviceaccount.com

Annotate K8s Service Account with AWS Role ARN:

使用AWS Role ARN注释K8s服务帐户：

kubectl annotate serviceaccount --namespace ${K8S_NAMESPACE} ${KSA_NAME}
  amazonaws.com/role-arn=${AWS_ROLE_ARN}

运行演示 (Run Demo)

Run a new K8s Pod with K8s ${KSA_NAME}` Service Account:

使用K8s ${KSA_NAME}`服务帐户运行一个新的K8s Pod：

# run a pod (with AWS CLI onboard) in interactive mod
kubectl run -it --rm --generator=run-pod/v1 --image mikesir87/aws-cli --serviceaccount ${KSA_NAME} test-pod


# in Pod shell: check AWS assumed role
aws sts get-caller-identity


# the output should look similar to below
{
    "UserId": "AROA9GB4GPRFFXVHNSLCK:gtoken-webhook-gyaashbbeeqhpvfw",
    "Account": "906385953612",
    "Arn": "arn:aws:sts::906385953612:assumed-role/bucket-full-gtoken/gtoken-webhook-gyaashbbeeqhpvfw"
}

外部参考 (External References)

• GitHub: Securely access AWS services from GKE cluster with doitintl/gtoken

  GitHub：使用doitintl / gtoken从GKE集群安全访问AWS服务

• AWS Docs: Creating a Role for Web Identity or OpenID Connect Federation

  AWS Docs： 为Web Identity或OpenID Connect Federation创建角色

• Blog: Kubernetes GKE Workload Identity link

  博客： Kubernetes GKE工作负载身份链接

• AWS Blog: Introducing fine-grained IAM roles for service accounts link

  AWS博客：为服务帐户链接引入细粒度的IAM角色

• GitHub: AWS Auth using Web Identity Federation from Google Cloud shrikant0013/gcp-aws-webidentityfederation GitHub project

  GitHub：使用来自Google Cloud的Web Identity Federation的 AWS Auth shrikant0013 / gcp-aws-webidentityfederation GitHub项目

• Blog: Using GCP Service Accounts to access AWS IAM Roles blog post by Colin Panisset

  博客：使用GCP服务帐户访问Colin Panisset的AWS IAM角色博客文章

摘要 (Summary)

I hope, you find this post useful. I look forward to your comments and any questions you have.

希望您对这篇文章有用。 我期待您的意见和任何疑问。

Want more stories? Check our blog on Medium, or follow Alexei on Twitter.

需要更多故事吗？ 在Medium上查看我们的博客，或在Twitter上关注Alexei 。

翻译自: https://blog.doit-intl.com/securely-access-aws-from-gke-dba1c6dbccba

gke下载