Linux----K8s的安全使用

 

安全机制

k8s的安全框架

流程：kubect先请求apiserver资源，然后是通过三关。

第一关是认证（Authentication）相当于一个凭证。

第二关是授权（Authorization）赋予一个角色，这个角色具体可以使用哪些命令。

第三关是准入控制（Admission Control），可以使用哪些插件，比如可以查看default命名空间的pod资源，只有通过这三关才可能会被k8s创建资源。

K8S安全控制框架主要由下面3个阶段进行控制，每一个阶段都 支持插件方式，通过API Server配置来启用插件。

普通用户若要安全访问集群API Server，往往需要证书、Token  或者用户名+密码；Pod访问，需要ServiceAccount

apiserver使用的是token认证

可以通过ServiceAccount在pod中去查看apiserver 

[root@localhost k8s-cert]# kubectl get sa
NAME      SECRETS   AGE
default   1         20d

 为了安全被访问，对外提供6443端口，有加密协议

[root@localhost demo]# netstat -ntap | grep 6443
tcp        0      0 192.168.179.150:6443    0.0.0.0:*               LISTEN      61722/kube-apiserve 
tcp        0      0 192.168.179.150:6443    192.168.179.151:58236   SYN_RECV    -                   
tcp        0      0 192.168.179.150:6443    192.168.179.155:49638   ESTABLISHED 61722/kube-apiserve 
tcp        0      0 192.168.179.150:34610   192.168.179.150:6443    ESTABLISHED 61722/kube-apiserve 
tcp        0      0 192.168.179.150:6443    192.168.179.150:34610   ESTABLISHED 61722/kube-apiserve 
tcp        0      0 192.168.179.150:6443    192.168.179.155:49646   ESTABLISHED 61722/kube-apiserve 
tcp        0      0 192.168.179.150:6443    192.168.179.155:49550   ESTABLISHED 61722/kube-apiserve 

内部通讯端口为8080

[root@localhost demo]# netstat -natp | grep 8080 | grep LISTEN
tcp        0      0 127.0.0.1:8080          0.0.0.0:*               LISTEN      61722/kube-apiserve 

第一模块：认证

三种客户端身份认证方法：

HTTPS 证书认证：基于CA证书签名的数字证书认证

HTTP Token认证：通过一个Token来识别用户（生产环境中使用广泛）

HTTP Base认证：用户名+密码的方式认证

https证书认证存在于

[root@localhost k8s-cert]# cat server-csr.json 
{
    "CN": "kubernetes",
    "hosts": [
      "10.0.0.1",
      "127.0.0.1",
      "192.168.179.150",
      "192.168.179.153",
      "192.168.179.100",
      "192.168.179.154",
      "192.168.179.155",
      "kubernetes",
      "kubernetes.default",
      "kubernetes.default.svc",
      "kubernetes.default.svc.cluster",
      "kubernetes.default.svc.cluster.local"
    ],
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "L": "BeiJing",
            "ST": "BeiJing",
            "O": "k8s",
            "OU": "System"
        }
    ]
}

httpd的token认证

[root@localhost demo]# cat /opt/kubernetes/cfg/token.csv 
d2b8b9fe6d91ed4b604831ea4b3344ca,kubelet-bootstrap,10001,"system:kubelet-bootstrap"

第二模块：授权

RBAC（Role-Based Access Control，基于角色的访问控制）：负责完成授权（Authorization）工作。

第三模块：准入控制

Adminssion Control实际上是一个准入控制器插件列表，发送到API Server的请求都需要经过这个列表中的每个准入控制器 插件的检查，检查不通过，则拒绝请求

NamespaceLifecycle： 命令空间回收

LimitRanger：配额管理

ServiceAccount： 每个pod中导入方便访问apiserver

ResourceQuota： 基于命名空间的高级配额管理

NodeRestriction： Node加入到k8s群集中以最小权限运行

 设么是RBAC授权？

RBAC（Role-Based Access Control，基于角色的访问控制），允许通过Kubernetes API动态配置策略

流程：绑定一个具体角色如"用户名，组或者是ServiceAccount"，是Role或者ClusterRole其中一种,绑定之后，这个角色具体拥有的权限，比如一用户为“zhangsan”绑定为Role，那么“zhangsan”可以使用"get"命令查看Role中的某一个命名空间的资源，比如pod,node,namespace。

角色

Role：授权特定命名空间的访问权限

ClusterRole：授权所有命名空间的访问权限

角色绑定

RoleBinding：将角色绑定到主体（即subject）

ClusterRoleBinding：将集群角色绑定到主体

主体（subject）

User：用户

Group：用户组

ServiceAccount：服务账号

演示示例

创建命名空间
[root@localhost demo]# kubectl create ns kgc
namespace/kgc created

查看该命名空间
[root@localhost demo]# kubectl get ns
NAME          STATUS   AGE
default       Active   20d
kgc           Active   28s            #创建成功
kube-public   Active   20d
kube-system   Active   20d

创建nginx的pod资源，指定到kgc命名空间中
[root@localhost demo]# kubectl run nginx --image=nginx -n kgc
kubectl run --generator=deployment/apps.v1beta1 is DEPRECATED and will be removed in a future version. Use kubectl create instead.
deployment.apps/nginx created

查看kgc命名空间资源
[root@localhost demo]# kubectl get pods -n kgc
NAME                    READY   STATUS    RESTARTS   AGE
nginx-dbddb74b8-h8z58   1/1     Running   0          17s

扩容成3个副本，在原有的pod增加两个，变为3个pod
[root@localhost demo]#  kubectl scale deploy/nginx --replicas=3 -n kgc
deployment.extensions/nginx scaled

查看kgc命名空间资源
[root@localhost demo]# kubectl get pods -n kgc
NAME                    READY   STATUS    RESTARTS   AGE
nginx-dbddb74b8-49mkm   1/1     Running   0          21s
nginx-dbddb74b8-6nwsg   1/1     Running   0          21s
nginx-dbddb74b8-h8z58   1/1     Running   0          107s

创建role
[root@localhost demo]# vim rbac-role.yaml

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: kgc                         #名称空间
  name: pod-reader                        
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get", "watch", "list"]        #拥有的命令权限

[root@localhost demo]# kubectl apply -f rbac-role.yaml 
role.rbac.authorization.k8s.io/pod-reader created

查看该角色信息
[root@localhost demo]# kubectl get role -n kgc
NAME         AGE
pod-reader   18s

创建rolebinding，绑定一个用户
[root@localhost demo]# vim rbac-rolebinding.yaml

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: read-pods
  namespace: kgc
subjects:
- kind: User
  name: zhangsan
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: pod-reader
  apiGroup: rbac.authorization.k8s.io

[root@localhost demo]# kubectl apply -f rbac-rolebinding.yaml
rolebinding.rbac.authorization.k8s.io/read-pods created
         
查看绑定信息
[root@localhost demo]# kubectl get role,rolebinding -n kgc
NAME                                        AGE
role.rbac.authorization.k8s.io/pod-reader   2m41s

NAME                                              AGE
rolebinding.rbac.authorization.k8s.io/read-pods   27s             

编辑用户信息的脚本

[root@localhost demo]# mkdir zhangsan
[root@localhost demo]# cd zhangsan/
[root@localhost zhangsan]# vim rbac-user.sh 
cat > zhangsan-csr.json <<EOF
{
  "CN": "zhangsan",
  "hosts": [],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "L": "BeiJing",
      "ST": "BeiJing"
    }
  ]
}
EOF

cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes zhangsan-csr.json | cfssljson -bare zhangsan

kubectl config set-cluster kubernetes \
  --certificate-authority=ca.pem \
  --embed-certs=true \
  --server=https://192.168.179.100:6443 \        #指向apiserverIP地址，即为VIP地址
  --kubeconfig=zhangsan-kubeconfig

kubectl config set-credentials zhangsan \
  --client-key=zhangsan-key.pem \
  --client-certificate=zhangsan.pem \
  --embed-certs=true \
  --kubeconfig=zhangsan-kubeconfig

kubectl config set-context default \
"rbac-user.sh" [dos] 38L, 923C      

 拷贝证书

[root@localhost zhangsan]# cp /root/k8s/k8s-cert/ca* ./
[root@localhost zhangsan]# ls
ca-config.json  ca.csr  ca-csr.json  ca-key.pem  ca.pem  rbac-user.sh

安装格式化工具
[root@localhost zhangsan]# yum install dos2unix -y

编辑rbac的yaml文件
[root@localhost zhangsan]# cd ..
[root@localhost demo]# vim rbac.yaml 

kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  namespace: default
  name: pod-reader
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get", "watch", "list"]

---

kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: read-pods
  namespace: default
subjects:
- kind: User
  name: jane
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: pod-reader
  apiGroup: rbac.authorization.k8s.io

apiVersion: v1
kind: ServiceAccount
metadata:
  name: pod-reader
  namespace: default

---

kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: sa-read-pods
  namespace: default
subjects:
- kind: ServiceAccount
  name: pod-reader
roleRef:
  kind: Role
  name: pod-reader
  apiGroup: rbac.authorization.k8s.io

格式化
[root@localhost zhangsan]# dos2unix rbac-user.sh
dos2unix: converting file rbac-user.sh to Unix format ...

执行脚本
[root@localhost zhangsan]# bash rbac-user.sh
2020/05/20 11:28:57 [INFO] generate received request
2020/05/20 11:28:57 [INFO] received CSR
2020/05/20 11:28:57 [INFO] generating key: rsa-2048
2020/05/20 11:28:58 [INFO] encoded CSR
2020/05/20 11:28:58 [INFO] signed certificate with serial number 554903934778595747847838553072709228440415758603
2020/05/20 11:28:58 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").
Cluster "kubernetes" set.
User "zhangsan" set.
Context "default" created.
Switched to context "default".

修改zhangsan-kubeconfigIP地址，指向apiserver
[root@localhost zhangsan]# vim zhangsan-kubeconfig 
......
server: https://192.168.179.100:6443
  name: kubernetes
contexts:
- context:
    cluster: kubernetes
    user: zhangsan
  name: default
current-context: default
kind: Config
preferences: {}
users:
- name: zhangsan
  user:

查看证书
[root@localhost zhangsan]# cat zhangsan-kubeconfig 
......

使用zhangsan-kubeconfig访问pod资源
[root@localhost zhangsan]# kubectl --kubeconfig=zhangsan-kubeconfig get pods -n kgc
NAME                    READY   STATUS    RESTARTS   AGE
nginx-dbddb74b8-49mkm   1/1     Running   0          30m
nginx-dbddb74b8-6nwsg   1/1     Running   0          30m
nginx-dbddb74b8-h8z58   1/1     Running   0          31m

使用zhangsan-kubeconfig访问 svc资源就会被拒绝
[root@localhost zhangsan]#  kubectl --kubeconfig=zhangsan-kubeconfig get svc -n kgc
Error from server (Forbidden): services is forbidden: User "zhangsan" cannot list resource "services" in API group "" in the namespace "kgc"

因为我们只授权zhangsan用户拥有访问pod资源的权限，没有授权查看service资源的权限，同样也无法访问默认的命名空间

[root@localhost zhangsan]# kubectl --kubeconfig=zhangsan-kubeconfig get svc 
Error from server (Forbidden): services is forbidden: User "zhangsan" cannot list resource "services" in API group "" in the namespace "default"

演示示例2，UI访问控制

[root@localhost zhangsan]# kubectl get svc -n kube-system
NAME                   TYPE       CLUSTER-IP   EXTERNAL-IP   PORT(S)         AGE
kubernetes-dashboard   NodePort   10.0.0.153   <none>        443:30001/TCP   12d

访问地址https://192.168.179.151:30001/

查看令牌

[root@localhost dashboard]# kubectl get secret -n kube-system
NAME                               TYPE                                  DATA   AGE
dashboard-admin-token-dlnmm        kubernetes.io/service-account-token   3      10s
default-token-7pgx6                kubernetes.io/service-account-token   3      20d
kubernetes-dashboard-certs         Opaque                                0      7m43s
kubernetes-dashboard-key-holder    Opaque                                2      7m43s
kubernetes-dashboard-token-x5sg4   kubernetes.io/service-account-token   3      7m29s
[root@localhost dashboard]# kubectl describe secret dashboard-admin-token-dlnmm -n kube-system
Name:         dashboard-admin-token-dlnmm
Namespace:    kube-system
Labels:       <none>
Annotations:  kubernetes.io/service-account.name: dashboard-admin
              kubernetes.io/service-account.uid: bd4540f7-9a4e-11ea-9a93-000c29270f97

Type:  kubernetes.io/service-account-token

Data
====
ca.crt:     1359 bytes
namespace:  11 bytes
token:      eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJkYXNoYm9hcmQtYWRtaW4tdG9rZW4tZGxubW0iLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoiZGFzaGJvYXJkLWFkbWluIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiYmQ0NTQwZjctOWE0ZS0xMWVhLTlhOTMtMDAwYzI5MjcwZjk3Iiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50Omt1YmUtc3lzdGVtOmRhc2hib2FyZC1hZG1pbiJ9.D-FSgbur3lZWn1IW1BWNFQPatTUhuILQR9IwysxubmUJM3vnjsll5SYhHB1LDE_GKicJEJw60ze-rbbX-pik5WniQpDOh0HEmtoDqbhCyjUy6iW3crrJXBzfYo9zGqFZ4sUI0ZTbpCvhdIHlZNsULjqufhHzpt-3mCrpO0A0ngZYPQsiFYZM6ngMjF4oshWOiJUNrARYAchxyPROoM7OU-d84tpRKOcdIgPftVqRZW80AW32C9YkOf20Bta-YgJRSG7HfLW1Jdv8qci4C13sc08NptiTKVs6isEpXkwxlUHl5aqr5_zpjEGFiBgtC3VM6gGzz6RSX3_holhsI2zHyg

 

使用zhangsan用户创建一个令牌登录UI界面 

[root@localhost zhangsan]# vim sa.yaml

apiVersion: v1
kind: ServiceAccount
metadata:
  name: pod-reader
  namespace: kgc

---

kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: sa-read-pods
  namespace: kgc
subjects:
- kind: ServiceAccount
  name: pod-reader
roleRef:
  kind: Role
  name: pod-reader
  apiGroup: rbac.authorization.k8s.io

[root@localhost zhangsan]# kubectl apply -f sa.yaml 
serviceaccount/pod-reader created
rolebinding.rbac.authorization.k8s.io/sa-read-pods created

查看token令牌
[root@localhost zhangsan]# kubectl describe secret pod-reader -n kgc
Name:         pod-reader-token-hv5vz
Namespace:    kgc
Labels:       <none>
Annotations:  kubernetes.io/service-account.name: pod-reader
              kubernetes.io/service-account.uid: 6a68ac37-9a4f-11ea-9a93-000c29270f97

Type:  kubernetes.io/service-account-token

Data
====
namespace:  3 bytes
token:      eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrZ2MiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlY3JldC5uYW1lIjoicG9kLXJlYWRlci10b2tlbi1odjV2eiIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJwb2QtcmVhZGVyIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiNmE2OGFjMzctOWE0Zi0xMWVhLTlhOTMtMDAwYzI5MjcwZjk3Iiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50OmtnYzpwb2QtcmVhZGVyIn0.msbU8ongZ25UPa4X5kSg6fiyj_zX5yokGb4YoItMRTCdCVfo9qeyADpkh6Vd6EatKwaZUZh5Owo-pCnhQF1x2z-YIlb5F754KKOIdQUzxb1S2DGKECWxnTrMhF9p4EqbQs1xwhj90CXaDRSKCCex-gIozq8v3BgfjYcN522s_Buh2CENF40ZoRhzz3ezqM5vF71YkNgnIJXK7MH9vhXzMpthw-HKvHiN0TirudgXVasY649qU2VuGeNnvV6GLWiROLqOktfJchqdieKXrLZyA40jq6Oip3-8-VF5TmoNerzz8ZaPFDNKwVk9KH47-X9EyN2ZHMUMOdHpuZB9t6nfgw
ca.crt:     1359 bytes

登录UI界面