k8s的安全机制与RBAC使用方法
文章目录
前言
一:k8s安全机制
安全框架、
传输安全、认证,授权,准入控制
使用rbac授权
1.1:kubernetes安全框架
-
安全框架的流程
流程:kubectl先请求api资源,然后是过三关,第一关是认证(Authentication),第二关是授权(Authorization),第三关是准入控制(Admission Control),只有通过这三关才可能会被k8s创建资源。
K8S安全控制框架主要由下面3个阶段进行控制,每一个阶段都支持插件方式,通过API Server配置来启用插件。
普通用户若要安全访问集群API Server,往往需要证书、Token或者用户名+密码;Pod访问,需要ServiceAccount
-
apiserver使用的是token认证
[root@master ~]# ps aux |grep apiserver root 12636 3.0 23.6 420460 235620 ? Ssl 5月17 156:09 /opt/kubernetes/bin/kube-apiserver --logtostderr=true --v=4 --etcd-servers=https://192.168.233.131:2379,https://192.168.233.132:2379,https://192.168.233.133:2379 --bind-address=192.168.233.131 --secure-port=6443 --advertise-address=192.168.233.131 --allow-privileged=true --service-cluster-ip-range=10.0.0.0/24 --enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota,NodeRestriction --authorization-mode=RBAC,Node --kubelet-https=true --enable-bootstrap-token-auth --token-auth-file=/opt/kubernetes/cfg/token.csv --service-node-port-range=30000-50000 --tls-cert-file=/opt/kubernetes/ssl/server.pem --tls-private-key-file=/opt/kubernetes/ssl/server-key.pem --client-ca-file=/opt/kubernetes/ssl/ca.pem --service-account-key-file=/opt/kubernetes/ssl/ca-key.pem --etcd-cafile=/opt/etcd/ssl/ca.pem --etcd-certfile=/opt/etcd/ssl/server.pem --etcd-keyfile=/opt/etcd/ssl/server-key.pem root 22849 0.0 0.0 112728 976 pts/0 S+ 10:20 0:00 grep --color=auto apiserver '//其中能够查询到token认证等信息'
-
查看ServiceAccount,可以通过ServiceAccount在pod中去访问apiserver
[root@master ~]# kubectl get sa NAME SECRETS AGE default 1 21d '//Service Account它并不是给kubernetes集群的用户使用的,而是给pod里面的进程使用的,它为pod提供必要的身份认证。'
-
传输安全方面:8080用于内部通讯,6443是提供给外部访问的端口
[root@master ~]# netstat -ntap |grep 8080 |grep LISTEN '//默认8080监听本地(是通过master及其他组件连接使用)'
tcp 0 0 127.0.0.1:8080 0.0.0.0:* LISTEN 12636/kube-apiserve
[root@master ~]# netstat -ntap |grep 6443 |grep LISTEN '//对外提供服务端口是6443'
tcp 0 0 192.168.233.131:6443 0.0.0.0:* LISTEN 12636/kube-apiserve
1.2:第一关:Authentication认证
-
三种客户端身份认证:
1、HTTPS 证书认证:基于CA证书签名的数字证书认证
2、HTTP Token认证:通过一个Token来识别用户(生产环境中使用广泛)
3、HTTP Base认证:用户名+密码的方式认证
-
1、HTTPS证书认证
[root@master ~]# cat k8s/k8s-cert/k8s-cert.sh cat > ca-config.json <<EOF { "signing": { "default": { "expiry": "87600h" }, "profiles": { "kubernetes": { "expiry": "87600h", "usages": [ "signing", "key encipherment", "server auth", "client auth" ] } } } } EOF cat > ca-csr.json <<EOF { "CN": "kubernetes", "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "CN", "L": "Beijing", "ST": "Beijing", "O": "k8s", "OU": "System" } ] } EOF cfssl gencert -initca ca-csr.json | cfssljson -bare ca - #----------------------- cat > server-csr.json <<EOF { "CN": "kubernetes", "hosts": [ "10.0.0.1", "127.0.0.1", "192.168.233.131", '//此处直接指定了负载均衡和master的节点' "192.168.233.130", "192.168.233.100", "192.168.233.128", "192.168.233.129", "kubernetes", "kubernetes.default", "kubernetes.default.svc", "kubernetes.default.svc.cluster", "kubernetes.default.svc.cluster.local" ], "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "CN", "L": "BeiJing", "ST": "BeiJing", "O": "k8s", "OU": "System" } ] } EOF cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes server-csr.json | cfssljson -bare server #----------------------- cat > admin-csr.json <<EOF { "CN": "admin", "hosts": [], "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "CN", "L": "BeiJing", "ST": "BeiJing", "O": "system:masters", "OU": "System" } ] } EOF cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes admin-csr.json | cfssljson -bare admin #----------------------- cat > kube-proxy-csr.json <<EOF { "CN": "system:kube-proxy", "hosts": [], "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "CN", "L": "BeiJing", "ST": "BeiJing", "O": "k8s", "OU": "System" } ] } EOF cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes kube-proxy-csr.json | cfssljson -bare kube-proxy
-
2、httpd的token认证
[root@master ~]# cat /opt/kubernetes/cfg/token.csv 7ea8f86b157225fd4b9273765e88a3ca,kubelet-bootstrap,10001,"system:kubelet-bootstrap"
1.3:第二关:Authorization授权
- RBAC(Role-Based Access Control,基于角色的访问控制):负责完成授权(Authorization)工作,允许通过Kubernetes API动态配置策略。
- 使用RBAC授权
-
角色:
1、Role:授权特定命名空间的访问权限
2、ClusterRole:授权所有命名空间的访问权限
角色绑定
1、RoleBinding:将角色绑定到主体(即subject)
2、ClusterRoleBinding:将集群角色绑定到主体
主体(subject)
1、User:用户
2、Group:用户组
3、ServiceAccount:服务账号
1.3.1:RBAC使用测试
-
1、创建名称空间dabao
[root@master ~]# kubectl create ns dabao namespace/dabao created [root@master ~]# kubectl get ns NAME STATUS AGE dabao Active 7s default Active 21d kube-public Active 21d kube-system Active 21d
-
2、创建测试pod
[root@master ~]# kubectl run nginx01 --image=nginx -n dabao kubectl run --generator=deployment/apps.v1beta1 is DEPRECATED and will be removed in a future version. Use kubectl create instead. deployment.apps/nginx-test created [root@master ~]# kubectl get pod -n dabao NAME READY STATUS RESTARTS AGE nginx01-77c44977fd-p4xbh 1/1 Running 0 15m
-
3、扩容成3个副本,使用scale命令
[root@master ~]# kubectl scale deploy/nginx01 --replicas=3 -n dabao deployment.extensions/nginx01 scaled [root@master ~]# kubectl get pod -n dabao NAME READY STATUS RESTARTS AGE nginx01-77c44977fd-98jc4 1/1 Running 0 31s nginx01-77c44977fd-nt424 1/1 Running 0 31s nginx01-77c44977fd-p4xbh 1/1 Running 0 17m
-
4、创建角色
[root@master ~]# vim rbac-role.yaml apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: namespace: dabao name: pod-reader rules: - apiGroups: [""] # "" indicates the core API group resources: ["pods"] '//创建角色只有pod资源的操作权限' verbs: ["get", "watch", "list"] '//只有这些操作可以使用' [root@master ~]# kubectl apply -f rbac-role.yaml role.rbac.authorization.k8s.io/pod-reader created [root@master ~]# kubectl get role -n dabao NAME AGE pod-reader 8s [root@master ~]#
-
5、角色绑定
[root@master ~]# vim rbac-rolebinding.yaml apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: name: read-pods namespace: dabao subjects: - kind: User name: zhangsan apiGroup: rbac.authorization.k8s.io roleRef: kind: Role name: pod-reader apiGroup: rbac.authorization.k8s.io [root@master ~]# kubectl apply -f rbac-rolebinding.yaml rolebinding.rbac.authorization.k8s.io/read-pods created [root@master ~]# kubectl get role,rolebinding -n dabao NAME AGE role.rbac.authorization.k8s.io/pod-reader 2m14s NAME AGE rolebinding.rbac.authorization.k8s.io/read-pods 13s
-
6、创建两个rbac的文件
[root@master ~]# mkdir zhangsan [root@master ~]# cd zhangsan/ [root@master zhangsan]# vim rbac.yaml kind: Role apiVersion: rbac.authorization.k8s.io/v1 metadata: namespace: default name: pod-reader rules: - apiGroups: [""] resources: ["pods"] verbs: ["get", "watch", "list"] --- kind: RoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata: name: read-pods namespace: default subjects: - kind: User name: jane apiGroup: rbac.authorization.k8s.io roleRef: kind: Role name: pod-reader apiGroup: rbac.authorization.k8s.io apiVersion: v1 kind: ServiceAccount metadata: name: pod-reader namespace: default --- kind: RoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata: name: sa-read-pods namespace: default subjects: - kind: ServiceAccount name: pod-reader roleRef: kind: Role name: pod-reader apiGroup: rbac.authorization.k8s.io [root@master zhangsan]# vim rbac-user.sh cat > zhangsan-csr.json <<EOF { "CN": "zhangsan", "hosts": [], "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "CN", "L": "BeiJing", "ST": "BeiJing" } ] } EOF cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes zhangsan-csr.json | cfssljson -bare zhangsan kubectl config set-cluster kubernetes \ --certificate-authority=ca.pem \ --embed-certs=true \ --server=https://192.168.233.100:6443 \ '//修改为负载均衡VIP地址' --kubeconfig=zhangsan-kubeconfig kubectl config set-credentials zhangsan \ --client-key=zhangsan-key.pem \ --client-certificate=zhangsan.pem \ --embed-certs=true \ --kubeconfig=zhangsan-kubeconfig kubectl config set-context default \ --cluster=kubernetes \ --user=zhangsan \ --kubeconfig=zhangsan-kubeconfig kubectl config use-context default --kubeconfig=zhangsan-kubeconfig
-
7、拷贝证书到张三目录并安装格式化工具
[root@master zhangsan]# cp /root/k8s/k8s-cert/ca* ./ [root@master zhangsan]# ls ca-config.json ca.csr ca-csr.json ca-key.pem ca.pem rbac-user.sh rbac.yaml [root@master zhangsan]# yum install dos2unix -y [root@master zhangsan]# dos2unix rbac-user.sh '//执行格式化' dos2unix: converting file rbac-user.sh to Unix format ... [root@master zhangsan]# bash rbac-user.sh '//运行脚本'
-
8、使用zhangsan-kubeconfig访问资源测试
[root@master zhangsan]# cat zhangsan-kubeconfig [root@master zhangsan]# kubectl --kubeconfig=zhangsan-kubeconfig get pod -n dabao NAME READY STATUS RESTARTS AGE nginx01-77c44977fd-98jc4 1/1 Running 0 23m nginx01-77c44977fd-nt424 1/1 Running 0 23m nginx01-77c44977fd-p4xbh 1/1 Running 0 40m '//但是访问不到除了pod的其他资源' [root@master zhangsan]# kubectl --kubeconfig=zhangsan-kubeconfig get svc -n dabao '//service无法访问' Error from server (Forbidden): services is forbidden: User "zhangsan" cannot list resource "services" in API group "" in the namespace "dabao" [root@master zhangsan]# kubectl --kubeconfig=zhangsan-kubeconfig get svc '//默认名称空间也无法访问' Error from server (Forbidden): services is forbidden: User "zhangsan" cannot list resource "services" in API group "" in the namespace "default"
-
UI访问控制
[root@master zhangsan]# kubectl get svc -n kube-system NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes-dashboard NodePort 10.0.0.139 <none> 443:30005/TCP 13d [root@master zhangsan]# kubectl get pod -n kube-system -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE kubernetes-dashboard-7dffbccd68-58qms 1/1 Running 5 13d 172.17.78.3 192.168.233.132 <none> '//接下来访问之前搭建的UI界面'
-
查看令牌
[root@master zhangsan]# kubectl get secret -n kube-system '//这是管理员的令牌' NAME TYPE DATA AGE dashboard-admin-token-zwktc kubernetes.io/service-account-token 3 13d default-token-cnnbv kubernetes.io/service-account-token 3 21d kubernetes-dashboard-certs Opaque 11 13d kubernetes-dashboard-key-holder Opaque 2 13d kubernetes-dashboard-token-qgppd kubernetes.io/service-account-token 3 13d [root@master zhangsan]# vim sa.yaml '//编辑认证yaml文件' apiVersion: v1 kind: ServiceAccount metadata: name: pod-reader namespace: dabao --- kind: RoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata: name: sa-read-pods namespace: dabao subjects: - kind: ServiceAccount name: pod-reader roleRef: kind: Role name: pod-reader apiGroup: rbac.authorization.k8s.io [root@master zhangsan]# kubectl apply -f sa.yaml serviceaccount/pod-reader created rolebinding.rbac.authorization.k8s.io/sa-read-pods created [root@master zhangsan]# kubectl get sa -n dabao NAME SECRETS AGE default 1 60m pod-reader 1 11s [root@master zhangsan]# kubectl get secret -n dabao NAME TYPE DATA AGE default-token-5lkxq kubernetes.io/service-account-token 3 61m pod-reader-token-l49zb kubernetes.io/service-account-token 3 76s [root@master zhangsan]# kubectl describe secret pod-reader-token-l49zb -n dabao Name: pod-reader-token-l49zb Namespace: dabao Labels: <none> Annotations: kubernetes.io/service-account.name: pod-reader kubernetes.io/service-account.uid: b262ec71-9b16-11ea-8c4f-000c294b2dd3 Type: kubernetes.io/service-account-token Data ==== ca.crt: 1359 bytes namespace: 5 bytes token: eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJkYWJhbyIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJwb2QtcmVhZGVyLXRva2VuLWw0OXpiIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6InBvZC1yZWFkZXIiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiJiMjYyZWM3MS05YjE2LTExZWEtOGM0Zi0wMDBjMjk0YjJkZDMiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6ZGFiYW86cG9kLXJlYWRlciJ9.vUdzOLAxq4GE6sRGmS4RkK6_s-NeF7mAeNnuZf9zkacYXZSxxWR6yz7h2-xpSPFJJIvZ1IAmP_G0XO4pazlS_DkLn_cNzLwAT7moOp8CxalATySLY-KtXxbRslwpcyLfyxaZ-PSEiI3fKt5f66C0eL3aoFYIM-xukVQNx_UJE2vsmu93WTuUb8XQjMVGUkQW-p7Mw_2f2wCyGbk_Y__LzXv3dRj0df7EoANHPiadYbntO-_k04LNSfrA1cHRZKBgIddO5tF8olw6rTs99IkSWfMUoF4-qCBDROOBf2h8tSMgz3jrOlhAbvq8kuvR4zjBtwUfI4l0c2IS00HZFW3K3g [root@master zhangsan]#
-
9、使用令牌登陆UI界面
1.4:第三关:准入控制Admission Control
-
Adminssion Control实际上是一个准入控制器插件列表,发送到API Server的请求都需要经过这个列表中的每个准入控制器 插件的检查,检查不通过,则拒绝请求。
-
查看进程信息
[root@master zhangsan]# ps aux |grep apiserver root 12636 3.0 20.9 420460 208584 ? Ssl 5月17 160:57 /opt/kubernetes/bin/kube-apiserver --logtostderr=true --v=4 --etcd-servers=https://192.168.233.131:2379,https://192.168.233.132:2379,https://192.168.233.133:2379 --bind-address=192.168.233.131 --secure-port=6443 --advertise-address=192.168.233.131 --allow-privileged=true --service-cluster-ip-range=10.0.0.0/24 --enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota,NodeRestriction '//此行是准入控制的插件信息'--authorization-mode=RBAC,Node --kubelet-https=true --enable-bootstrap-token-auth --token-auth-file=/opt/kubernetes/cfg/token.csv --service-node-port-range=30000-50000 --tls-cert-file=/opt/kubernetes/ssl/server.pem --tls-private-key-file=/opt/kubernetes/ssl/server-key.pem --client-ca-file=/opt/kubernetes/ssl/ca.pem --service-account-key-file=/opt/kubernetes/ssl/ca-key.pem --etcd-cafile=/opt/etcd/ssl/ca.pem --etcd-certfile=/opt/etcd/ssl/server.pem --etcd-keyfile=/opt/etcd/ssl/server-key.pem root 23615 0.0 0.0 112728 972 pts/0 S+ 12:18 0:00 grep --color=auto apiserver
-
NamespaceLifecycle: 命令空间回收
LimitRanger:配额管理
ServiceAccount: 每个pod中导入方便访问apiserver
ResourceQuota: 基于命名空间的高级配额管理
NodeRestriction: Node加入到k8s群集中以最小权限运行
官网推荐的插件:
1.11版本以上推荐使用的插件:
–enable-admission-plugins= \ NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds, ResourceQuota
ca.pem --etcd-certfile=/opt/etcd/ssl/server.pem --etcd-keyfile=/opt/etcd/ssl/server-key.pem
root 23615 0.0 0.0 112728 972 pts/0 S+ 12:18 0:00 grep --color=auto apiserver
- NamespaceLifecycle: 命令空间回收
LimitRanger:配额管理
ServiceAccount: 每个pod中导入方便访问apiserver
ResourceQuota: 基于命名空间的高级配额管理
NodeRestriction: Node加入到k8s群集中以最小权限运行
官网推荐的插件:
1.11版本以上推荐使用的插件:
–enable-admission-plugins= \ NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds, ResourceQuota
### 如有疑问可评论区交流!