kubernets-RBAC

RBAC

• Role：角色，它其实是一组规则，定义了一组对 Kubernetes API 对象的操作权限, 但Role只对指定的namespace下的资源生效。
• ClusterRole: 集群橘色，同样也是一组规则，定义了一组对 Kubernetes API 对象的操作权限， ClusterRole 可以对所有namespace下的资源生效。
• Subject：被作用者，既可以是“人”，也可以是“机器”，也可以是你在 Kubernetes 里定义的“用户”。
• RoleBinding：定义了“被作用者”和“角色”的绑定关系

Role

一个权限角色，针对某种资源的权限限制

kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  namespace: mynamespace
  name: example-role
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get", "watch", "list"]

如上面的yml 例子，翻译下来就是允许Subject，对mynamespace下面的Pod对象，进行 GET、WATCH 和 LIST 操作

RoleBinding

权限角色和subject 的绑定，subject 大多数都是user，当然还有其他

kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: example-rolebinding
  namespace: mynamespace
subjects:
- kind: User
  name: example-user
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: example-role
  apiGroup: rbac.authorization.k8s.io

Subject

subject是被操作的对象，如果没有K8S 外部鉴权的情况下，一般用Kubernetes的内置用户管理，即ServiceAccount 就够了

apiVersion: v1
kind: ServiceAccount
metadata:
  namespace: mynamespace
  name: example-sa

基本概念就是如上，拿个flannel的配置做个具体的例子:

# serviceAccount, 创建一个flannel的subject
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: flannel
  namespace: kube-system
---
# clusterRole, 创建一个针对podsecuritypolicies，pod,nodes的权限规则，由于涉及到node,所有不可能在单独的namespace下生效，所以使用role，需要是clusterRole,
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: flannel
rules:
- apiGroups: ['extensions']
  resources: ['podsecuritypolicies']
  verbs: ['use']
  resourceNames: ['psp.flannel.unprivileged']
- apiGroups:
  - ""
  resources:
  - pods
  verbs:
  - get
- apiGroups:
  - ""
  resources:
  - nodes
  verbs:
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - nodes/status
  verbs:
  - patch
---

# ClusterRoleBinding, 将之前的clusterRole 和subject(serviceAccount) 绑定起来
---yaml
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: flannel
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: flannel
subjects:
- kind: ServiceAccount
  name: flannel
  namespace: kube-system
---

在创建好flannel的rbac 之后，在看下flannel对应的daemonset的yml:

# 在spec/template/spec 下，使用的是serviceAccountName，即该POD，使用的是serviceAccount-flannel 所绑定的ClusterRole: flannel 的所有权限
    ...
      serviceAccountName: flannel
      initContainers:
      - name: install-cni
    ...

然后看下实际的POD 运行的yaml

[root@dev001 ~]# kubectl get pod kube-flannel-ds-wkqvm -n kube-system -o yaml | grep secret -A 2
    - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      name: flannel-token-jvvrj
      readOnly: true
--
    - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      name: flannel-token-jvvrj
      readOnly: true
--
    secret:
      defaultMode: 420
      secretName: flannel-token-jvvrj

在实际运行的POD 里，发现在pod 内部有对应serviceAccount的挂载点，以及secret

# 登录pod 看下flannel的的serviceAccount下的内容
[root@dev001 ~]# kubectl exec -it kube-flannel-ds-wkqvm -n kube-system bash 
bash-5.0# ls /var/run/secrets/kubernetes.io/serviceaccount
ca.crt     namespace  token
bash-5.0# cat /var/run/secrets/kubernetes.io/serviceaccount/namespace
kube-system
bash-5.0# cat /var/run/secrets/kubernetes.io/serviceaccount/token
eyJhbGciOiJSUzI1NiIsImtpZCI6InctSGw3MnYzWHVMVEtxbVZiZTZJYnlBU0w3bUc5TU5rb0FQX0U0aVFERTQifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJmbGFubmVsLXRva2VuLWp2dnJqIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImZsYW5uZWwiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiJjN2UwNzI2NC1kMDI3LTQ0ZmItOWZjNS1lMTNjODBmNGYzMWEiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6a3ViZS1zeXN0ZW06Zmxhbm5lbCJ9.pkLd2BDWoVvWis8aHbSeZsKojA8f3nEkpHWmZUaQyezt_ERdWmdVUmxgOyzMBDT26tWPwAn6zaJWk1haoVwi-WzvT-SRvPqOAVt4jT6YxhmEuDqbLOCgHSQIhpCKnHBo65lPHHgr31piSfFl797IUkH7az8V8GKavBUNT4633akkfc7xbJIEIiGg6Q88rA-j-Ee4JJkiXek1vC_jX6OTZJGNf7fD44ahrkkQoQ8kNQeSgRgQC66AXOit0m-s_y7cLGp3lylZWUPzuqTj7USt1dzyUWfLlJjHja8FJvIKt66vSHNiE8FWLsKXTX9QRZEjKRwbPEGKTnLcxPf3M1_Z_gbash-5.0# 
bash-5.0# cat /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
-----BEGIN CERTIFICATE-----
MIICyDCCAbCgAwIBAgIBADANBgkqhkiG9w0BAQsFADAVMRMwEQYDVQQDEwprdWJl
cm5ldGVzMB4XDTIwMTIxNzAzNTM0NloXDTMwMTIxNTAzNTM0NlowFTETMBEGA1UE
AxMKa3ViZXJuZXRlczCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAKPZ
KuDGVFQ1l7l1lnBSpo5FnFw+44Bpwkq9OT34djaR+Uc7HW+V4NSCuU8IcyT7oRZn
U4Cy6I1ggI4GRAmpswchKMKBbLNutAbDgfT93eytGNmms+h5QmAsIf+Aw/WqtzHf
/hHSH88kl0lDnDyg9ptnRSyYcWjGrwTMzQoTVOB0L1EU3iimHNl12rDDRu9Wcn+p
/NtObf0ksNRelqGyahi09cYHoVrk3CnyDJGD1fKk/RE3Hc9qKr/I5/S7kTWJN4xN
+qhi9h+D45aitqL1qkE1/UX9jR/blUTUbtSWLfubswd0UfxmZksqRqZILpukMwfX
GT2FYAoayu6vkCz1BcECAwEAAaMjMCEwDgYDVR0PAQH/BAQDAgKkMA8GA1UdEwEB
/wQFMAMBAf8wDQYJKoZIhvcNAQELBQADggEBAABdWSTxdr/4A0CB/N84wbqckbOv
Sou8YiY/Al94eUClqNPPtDVfGS09pHJb29nSY4GbHzBKZSOEyvhSU10Pq+2KE9ze
icxZ4F8pMpXqSLjrRWIoUic4CMLe3hqFrv9rAof0t94hds/nf+ZKgf/sT7QHK3kD
FGpp4XiLKIwattB6aCugHj/AZF7xiVy3K9iZao/bOtmotAYA1K+ybIfE7wNmSGxI
31o8InN9AGt62lFgJ1AL98qmp02Xr5e/KlowQ/koHy8jIDmdTN3fmoM34ARQBTjz
pMCZELRs/Fg8Y9NoXHjtRGyQ1IApZu9DuRL2/DoyTV3Oa9P2E6WHDBmvHMs=
-----END CERTIFICATE-----

再看下etcd里的信息.

ETCDCTL_API=3 etcdctl --cacert=/etc/kubernetes/pki/etcd/ca.crt --cert=/etc/kubernetes/pki/etcd/peer.crt --key=/etc/kubernetes/pki/etcd/peer.key get /registry/secrets/kube-system/flannel-token-jvvrj
...
flannel-token-jvvrj 
                   kube-system"*$a12ec09f-7498-4ee9-859f-614b1154b53f2º­켐b-
"kubernetes.io/service-account.nameflannelbI
!kubernetes.io/service-account.uid$c7e07264-d027-44fb-9fc5-e13c80f4f31az 
ca.cr-----BEGIN CERTIFICATE-----
MIICyDCCAbCgAwIBAgIBADANBgkqhkiG9w0BAQsFADAVMRMwEQYDVQQDEwprdWJl
cm5ldGVzMB4XDTIwMTIxNzAzNTM0NloXDTMwMTIxNTAzNTM0NlowFTETMBEGA1UE
AxMKa3ViZXJuZXRlczCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAKPZ
KuDGVFQ1l7l1lnBSpo5FnFw+44Bpwkq9OT34djaR+Uc7HW+V4NSCuU8IcyT7oRZn
U4Cy6I1ggI4GRAmpswchKMKBbLNutAbDgfT93eytGNmms+h5QmAsIf+Aw/WqtzHf
/hHSH88kl0lDnDyg9ptnRSyYcWjGrwTMzQoTVOB0L1EU3iimHNl12rDDRu9Wcn+p
/NtObf0ksNRelqGyahi09cYHoVrk3CnyDJGD1fKk/RE3Hc9qKr/I5/S7kTWJN4xN
+qhi9h+D45aitqL1qkE1/UX9jR/blUTUbtSWLfubswd0UfxmZksqRqZILpukMwfX
GT2FYAoayu6vkCz1BcECAwEAAaMjMCEwDgYDVR0PAQH/BAQDAgKkMA8GA1UdEwEB
/wQFMAMBAf8wDQYJKoZIhvcNAQELBQADggEBAABdWSTxdr/4A0CB/N84wbqckbOv
Sou8YiY/Al94eUClqNPPtDVfGS09pHJb29nSY4GbHzBKZSOEyvhSU10Pq+2KE9ze
icxZ4F8pMpXqSLjrRWIoUic4CMLe3hqFrv9rAof0t94hds/nf+ZKgf/sT7QHK3kD
FGpp4XiLKIwattB6aCugHj/AZF7xiVy3K9iZao/bOtmotAYA1K+ybIfE7wNmSGxI
31o8InN9AGt62lFgJ1AL98qmp02Xr5e/KlowQ/koHy8jIDmdTN3fmoM34ARQBTjz
pMCZELRs/Fg8Y9NoXHjtRGyQ1IApZu9DuRL2/DoyTV3Oa9P2E6WHDBmvHMs=
-----END CERTIFICATE-----
...

kubernetes 的基本通信都是通过API Server进行的，而在调用过程中，会需要在header里加上token 和 https的ca证书，比如下面的简单调用:

curl -s $APISERVER/api/v1/namespaces/default/pods/ --header "Authorization: Bearer $TOKEN" --cacert /tmp/ca.crt | jq -rM '.items[].metadata.name'

所有，根据上面的结果可以看到RBAC的基本调用逻辑:

• Pod 的yml内会指定serviceAccount
• Pod在创建过程中会创建该pod的ca.crt 用于和ApiServer 通信，同时也会创建一个token，存放在etcd 中，key就是secret字段的名字
• Pod通过该ca.crt 和ApiServer通信，使用token 进行权限认证
• ApiServer通过token 从etcd 里获取到对应serviceAccount的Role/ClusterRole
• 然后根据Token 查询到的权限，限制该ServiceAccount的资源对象的使用。

个人公众号, 分享一些日常开发，运维工作中的日常以及一些学习感悟，欢迎大家互相学习，交流