业务需求需要加密线上各种key,开源工具里面vault最合适,最快上手的方式还是使用demo测试一下
不做过多介绍,主要是简单理解软件的工作方式
参考 https://www.qikqiak.com/post/deploy-vault-on-k8s/
一、 helm安装vault
$ helm repo add hashicorp https://helm.releases.hashicorp.com
# 这里最新版本就不装了
$ helm search repo hashicorp/vault
NAME CHART VERSION APP VERSION DESCRIPTION
hashicorp/vault 0.28.1 1.17.2 Official HashiCorp Vault Chart
hashicorp/vault-secrets-operator 0.8.1 0.8.1 Official Vault Secrets Operator Chart
# 搜索其他版本
helm search repo hashicorp/vault -l
# 创建部署vault的ns
kubectl create namespace vault
# 创建values.yaml
$ cat custom-values.yaml
server:
dev:
enabled: true # 开启开发模式
ui:
enabled: true # 启用 Vault UI,但我没有使用
# 安装最终版本
helm install vault hashicorp/vault --version 0.26.1 --namespace vault -f custom-values.yaml
二、vault创建策略和Role
安装以后进入容器,dev模式初始化操作都跳过了
kubectl exec -it vault-0 -n vault sh
internal路径开启kv-v2 secrets引擎
vault secrets enable -path=internal kv-v2
查看默认的secrets path
秘钥都会存储在path里面
$ vault secrets list
Path Type Accessor Description
---- ---- -------- -----------
cubbyhole/ cubbyhole cubbyhole_4cc7d038 per-token private secret storage
identity/ identity identity_14eadf80 identity store
internal/ kv kv_37a41a78 n/a
secret/ kv kv_008d47de key/value secret storage
sys/ system system_cdde0317 system endpoints used for control, policy and debugging
在路径下添加一个用户名密码的秘钥
vault kv put internal/database/config username="baga" password="bagapass"
创建好以后去get秘钥
这个路径像是bucket路径一样
$ vault kv get internal/database/config
======== Secret Path ========
internal/data/database/config
======= Metadata =======
Key Value
--- -----
created_time 2024-08-30T09:49:42.354666141Z
custom_metadata <nil>
deletion_time n/a
destroyed false
version 1
====== Data ======
Key Value
--- -----
password bagapass
username baga
三、开启kubernetes认证
开启k8s验证
vault auth enable kubernetes
vault接受k8s集群中任何客户端服务的Token验证
这些值都是容器固定的
vault write auth/kubernetes/config \
token_reviewer_jwt="$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)" \
kubernetes_host="https://$KUBERNETES_PORT_443_TCP_ADDR:443" \
kubernetes_ca_cert=@/var/run/secrets/kubernetes.io/serviceaccount/ca.crt
提示:Success! Data written to: auth/kubernetes/config
查看auth list里面已经有了k8s
$ vault auth list
Path Type Accessor Description Version
---- ---- -------- ----------- -------
kubernetes/ kubernetes auth_kubernetes_cbe9eac9 n/a n/a
token/ token auth_token_20470ebe token based credentials n/a
创建vault的策略,读取之前的secret
vault policy write internal-app - <<EOH
path "internal/data/database/config" {
capabilities = ["read"]
}
EOH
查看当前策略 list
$ vault policy list
default
internal-app
root
创建k8s认证角色,这里service_account的ns选择到app的ns下.role未创建,init容器里报错invaild role
vault write auth/kubernetes/role/internal-app \
bound_service_account_names=internal-app \
bound_service_account_namespaces=sijia-test \
policies=internal-app \
ttl=24h
查看role
$ vault list auth/kubernetes/role
四、集成vault role到k8s
创建service account
这里值很关键,要和vault中bound_service_account_names
和bound_service_account_namespaces
一致
vim service-account-test.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: internal-app
namespace: sijia-test
kubectl apply -f service-account-test.yaml
使用应用去测试 vault-demo.yaml
镜像是demo镜像
apiVersion: apps/v1
kind: Deployment
metadata:
name: vault-demo
namespace: sijia-test
labels:
app: vault-demo
spec:
selector:
matchLabels:
app: vault-demo
template:
metadata:
annotations:
vault.hashicorp.com/agent-inject: "true"
vault.hashicorp.com/role: "internal-app"
vault.hashicorp.com/agent-inject-secret-database-config.txt: "internal/data/database/config"
labels:
app: vault-demo
spec:
serviceAccountName: internal-app # 使用上面创建的 serviceaccount 对象
containers:
- name: vault
image: cnych/vault-demo:0.0.1
如果容器报错了 查看init容器日志
kubectl logs vault-demo-cf79bb8b-sgdh4 -n sijia-test -c vault-agent-init
查看容器日志
root@eks-manage-c100:/data/vault# kubectl logs -n sijia-test vault-demo-cf79bb8b-fq77p vault-agent
==> Vault Agent started! Log data will stream in below:
==> Vault Agent configuration:
Api Address 1: http://bufconn
Cgo: disabled
Log Level: info
Version: Vault v1.15.1, built 2023-10-20T19:16:11Z
2024-08-30T11:51:25.991Z [INFO] agent.sink.file: creating file sink
2024-08-30T11:51:25.991Z [INFO] agent.sink.file: file sink configured: path=/home/vault/.vault-token mode=-rw-r-----
Version Sha: b94e275f25ccd9011146d14c00ea9e49fd5032dc
2024-08-30T11:51:25.991Z [INFO] agent.exec.server: starting exec server
2024-08-30T11:51:25.991Z [INFO] agent.exec.server: no env templates or exec config, exiting
2024-08-30T11:51:25.991Z [INFO] agent.sink.server: starting sink server
2024-08-30T11:51:25.991Z [INFO] agent.template.server: starting template server
2024-08-30T11:51:25.991Z [INFO] (runner) creating new runner (dry: false, once: false)
2024-08-30T11:51:25.991Z [INFO] agent.auth.handler: starting auth handler
2024-08-30T11:51:25.991Z [INFO] agent.auth.handler: authenticating
2024-08-30T11:51:25.991Z [INFO] (runner) creating watcher
2024-08-30T11:51:25.995Z [INFO] agent.auth.handler: authentication successful, sending token to sinks
2024-08-30T11:51:25.995Z [INFO] agent.auth.handler: starting renewal process
2024-08-30T11:51:25.995Z [INFO] agent.sink.file: token written: path=/home/vault/.vault-token
2024-08-30T11:51:25.995Z [INFO] agent.template.server: template server received new token
2024-08-30T11:51:25.995Z [INFO] (runner) stopping
2024-08-30T11:51:25.995Z [INFO] (runner) creating new runner (dry: false, once: false)
2024-08-30T11:51:25.995Z [INFO] (runner) creating watcher
2024-08-30T11:51:25.995Z [INFO] (runner) starting
2024-08-30T11:51:25.996Z [INFO] agent.auth.handler: renewed auth token
root@eks-manage-c100:/data/vault#
五、检索vault secret
已经是边车模式了,vault-agent容器会管理token的声明周期和secret数据检索
/vault/secrets/database-config.txt,/vault/secrets路径是固定的。这个文件名称是deployment annotions里面生成的,vault.hashicorp.com/agent-inject-secret-database-config.txt: "internal/data/database/config"
,
kubectl exec -it -n sijia-test vault-demo-cf79bb8b-fq77p -c vault -- cat /vault/secrets/database-config.txt
data: map[password:bagapass username:baga]
metadata: map[created_time:2024-08-30T09:49:42.354666141Z custom_metadata:<nil> deletion_time: destroyed:false version:1]
放到业务中使用还有很多东西需要考虑,目前只是简单使用以下,token ttl需要考虑renew