serviceaccount 方式部署

很好的文章：https://tonybai.com/2016/11/25/the-security-settings-for-kubernetes-cluster/

https://www.cnblogs.com/charlieroro/p/9791240.html

k8s 1.5.2 版本

 

Expected to load root CA config from /var/run/secrets/kubernetes.io/serviceaccount/ca.crt, but got err: open /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: no such file or directory

产生这个错误是因为Kubernetes默认创建的secrets资源不包含用于访问kube-apiserver的根证书。

# kubectl get secrets --namespace=kube-system
NAME                  TYPE                                  DATA      AGE
default-token-wxzm7   kubernetes.io/service-account-token   2         19h

# kubectl describe secret default-token-wxzm7 --namespace=kube-system
Name:       default-token-wxzm7
Namespace:  kube-system
Labels:     <none>
Annotations:    kubernetes.io/service-account.name=default
        kubernetes.io/service-account.uid=80bc5d75-41e9-11e7-b90e-000c29f6f813

Type:   kubernetes.io/service-account-token

Data
====
namespace:  11 bytes
token:      eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJkZWZhdWx0LXRva2VuLXd4em03Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImRlZmF1bHQiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiI4MGJjNWQ3NS00MWU5LTExZTctYjkwZS0wMDBjMjlmNmY4MTMiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6a3ViZS1zeXN0ZW06ZGVmYXVsdCJ9.m1dIyZiU2vejuUrhjUb3mwykBkp_nrfTQ9kyz6kYQghcJT4iuGNqh3sPBpQ6F4QxCDu_PgKGWr5A7PA3mnvwfmwE8MbLktizf4khOR7gMxp_xwQw8izutdjQZJgtejxzSkBeW3Kh-Xr7YnUt6cpAdkITWJ65rTI5Fp4KmrK-AVMnKr0h3YIbmCTC2-rKJSJw_NUHLYjCELh8c5K2gnn1wTl6QXhgsojtx7cDZZrPBPF6pOX5xtZYN2YEOjjeHS01LA1jbmkaCJiaTT1umICVpGZ8PxRbuuzaUBAdJaxxsE05Jve67E9e6qFIYROsZMIgnoN5t5UBooypBuMkms_31g

 

生成证书和密钥

下面链接是具体生成ca 具体 步骤

https://blog.csdn.net/Michaelwubo/article/details/102583865

在此，使用easyrsa生产证书和密钥。

• 下载easyrsa3

  # curl -L -O https://storage.googleapis.com/kubernetes-release/easy-rsa/easy-rsa.tar.gz
  # tar xzf easy-rsa.tar.gz
  # cd easy-rsa-master/easyrsa3
  # ./easyrsa init-pki
  
  init-pki complete; you may now create a CA or requests.
  Your newly created PKI dir is: /root/k8s/easy-rsa-master/easyrsa3/pki
  

   

• 创建根证书

  # ./easyrsa --batch "--req-cn=192.168.120.121@`date +%s`" build-ca nopass
  Generating a 2048 bit RSA private key
  .......+++
  ................................................................................+++
  writing new private key to '/root/k8s/easy-rsa-master/easyrsa3/pki/private/ca.key'
  

   

• 创建服务端证书和密钥

# ./easyrsa --subject-alt-name="IP:192.168.120.121" build-server-full server nopass
Generating a 2048 bit RSA private key
..............................+++
................................................+++
writing new private key to '/root/k8s/easy-rsa-master/easyrsa3/pki/private/server.key'
-----
Using configuration from /root/k8s/easy-rsa-master/easyrsa3/openssl-1.0.cnf
Check that the request matches the signature
Signature ok
The Subject's Distinguished Name is as follows
commonName            :PRINTABLE:'server'
Certificate is to be certified until May 25 03:23:50 2027 GMT (3650 days)

Write out database with 1 new entries
Data Base Updated

• 拷贝pki/ca.crt、pki/issued/server.crt和pki/private/server.key至指定的目录

# mkdir /etc/kubernetes/pki
# cp pki/ca.crt pki/issued/server.crt pki/private/server.key /etc/kubernetes/pki/
# chmod 644 /etc/kubernetes/pki/*

 

 

kubelet,kube-apiserver,kube-controller-manager参数整体介绍

https://www.kubernetes.org.cn/2540.html

 

 

配置kube-apiserver服务

KUBE_ADMISSION_CONTROL="--admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota"

KUBE_API_ARGS="--service-node-port-range=1-65535  --bind-address=0.0.0.0 --secure-port=6443 --advertise-address=0.0.0.0 --audit-log-path=/var/log/kubernetes/kubernetes.audit --audit-log-maxage=7 --audit-log-maxbackup=4 --audit-log-maxsize=10 --client-ca-file=/etc/kubernetes/pki/ca.crt --tls-cert-file=/etc/kubernetes/pki/server.crt --tls-private-key-file=/etc/kubernetes/pki/server.key  --service-account-key-file=/etc/kubernetes/pki/server.key --basic-auth-file=/etc/kubernetes/pki/basic_auth_file.csv"

参数https://my.oschina.net/u/3797264/blog/2250066详解介绍

 

配置kube-controller-manager服务

参照以下内容编辑/etc/kubernetes/controller-manager：

KUBE_CONTROLLER_MANAGER_ARGS="--service-account-private-key-file=/etc/kubernetes/pki/server.key --root-ca-file=/etc/kubernetes/pki/ca.crt"

 

--service-account-private-key-file：文件名，包含用于签署服务帐户令牌的pem编码的专用rsa或ecdsa密钥

--root-ca-file ：根CA 证书文件路径，如果设置的话将被用于 Service Account 的 token secret 中

 

https://blog.csdn.net/zhonglinzhang/article/details/86062590详细介绍

 

删除旧secrets资源

# kubectl get secrets --all-namespaces
NAMESPACE     NAME                  TYPE                                  DATA      AGE
default       default-token-s1vfh   kubernetes.io/service-account-token   2         5m
kube-system   default-token-jct68   kubernetes.io/service-account-token   2         4m

# systemctl stop kube-controller-manager

# kubectl delete secret default-token-s1vfh
secret "default-token-s1vfh" deleted

# kubectl delete secret default-token-jct68 --namespace=kube-system
secret "default-token-jct68" deleted

重新启动kube-apiserver和kube-controller-manager服务

# systemctl restart kube-apiserver
# systemctl start  kube-controller-manager

检查新创建的secret是否包含根证书

# kubectl get secrets --all-namespaces
NAMESPACE     NAME                  TYPE                                  DATA      AGE
default       default-token-tv69r   kubernetes.io/service-account-token   3         3s
kube-system   default-token-27w5m   kubernetes.io/service-account-token   3         3s

# kubectl describe secret default-token-27w5m --namespace=kube-system
Name:       default-token-27w5m
Namespace:  kube-system
Labels:     <none>
Annotations:    kubernetes.io/service-account.name=default
        kubernetes.io/service-account.uid=80bc5d75-41e9-11e7-b90e-000c29f6f813

Type:   kubernetes.io/service-account-token

Data
====
ca.crt:     1233 bytes
namespace:  11 bytes
token:      eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJkZWZhdWx0LXRva2VuLTI3dzVtIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImRlZmF1bHQiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiI4MGJjNWQ3NS00MWU5LTExZTctYjkwZS0wMDBjMjlmNmY4MTMiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6a3ViZS1zeXN0ZW06ZGVmYXVsdCJ9.CCxdtFRagtEo2eiPZgkiHjLkGDgbvt7VWe2WZGsLKeh_7Z2t-bUawwXGYxgd0MT_lG2HJbmHRTUb57Zw1MGRMZ-u4dBx_J9hXztnrdWcOh8_L_stk64gFQXjXpuZee1ltDksm7pTXtCnG1x8zBBxoZVi0jadPDMC_HP2OzvJXHrUPbCb58PBIqjRjbuJUQgM_hooDoJryK_0wYOd8TWOKUJMqQdJwTozFciDcGE__F3BchgHqfO9064f3ki1qSrZsnTImTpCYsUu4sy1fbL7X-3mVFWNNbsIvscFnBWP1Poj2M_hgqG_e4VCXL6vv61ll1LytWUwqPxosk1Djk7rvQ