K8S-Demo集群实践03:准备集群各组件间HTTPS通讯需要的x509证书

  • 这里一次准备好集群内部通讯需要的所有证书
  • 你可以跳过本篇,在后续部署过程中需要相应证书的时候再创建

一、安装证书生产工具CFSSL

  • 本案例使用CloudFlare的PKI工具集cfssl创建所有证书
  • CSR:Certificate Signing Request 证书签名请求文件
[root@master1 ~]# mkdir -p /opt/install/soft/cfssl
[root@master1 ~]# cd /opt/soft/cfssl
[root@master1 cfssl]# wget https://github.com/cloudflare/cfssl/releases/download/v1.4.1/cfssl_1.4.1_linux_amd64
[root@master1 cfssl]# mv cfssl_1.4.1_linux_amd64 /opt/k8s/bin/cfssl
[root@master1 cfssl]# wget https://github.com/cloudflare/cfssl/releases/download/v1.4.1/cfssljson_1.4.1_linux_amd64
[root@master1 cfssl]# mv cfssljson_1.4.1_linux_amd64 /opt/k8s/bin/cfssljson
[root@master1 cfssl]# wget https://github.com/cloudflare/cfssl/releases/download/v1.4.1/cfssl-certinfo_1.4.1_linux_amd64
[root@master1 cfssl]# mv cfssl-certinfo_1.4.1_linux_amd64 /opt/k8s/bin/cfssl-certinfo
[root@master1 cfssl]# chmod +x /opt/k8s/bin/*
  • 获取缺省配置,可以根据需要在这个基础上修改
[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# cfssl print-defaults config > ca-config.json
[root@master1 cert]# cfssl print-defaults csr > ca-csr.json

二、创建根证书

  • CA(Certificate Authority)是自签名的根证书,用来签名其它证书
  • 如果没有特殊说明,后续操作均在master1节点上执行
  • 证书临时存放目录 /opt/install/cert

1、创建根证书配置文件 ca-config.json

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# cat > ca-config.json <<EOF
{
  "signing": {
    "default": {
      "expiry": "87600h"
    },
    "profiles": {
      "k8s-demo-server": {
        "usages": [
            "signing",
            "key encipherment",
            "server auth",
            "client auth"
        ],
        "expiry": "87600h"
      },
      "k8s-demo-client": {
        "usages": [
            "signing",
            "key encipherment",
            "client auth"
        ],
        "expiry": "87600h"
      }
    }
  }
}
EOF
  • expiry: “87600h”:证书有效期设置为10年
  • signing:表示该证书可用于签名其它证书(生成的 ca.pem 证书中 CA=TRUE)
  • key encipherment:秘钥加密
  • server auth:表示client可以用该该证书对server提供的证书进行验证
  • client auth:表示server可以用该该证书对client提供的证书进行验证

2、创建证书签名请求文件 ca-csr.json

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# cat > ca-csr.json <<EOF
{
  "CN": "k8s-demo-ca",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [{
    "C" : "CN",
    "ST": "GuangDong",
    "L" : "ShenZhen",
    "O" : "k8s-demo",
    "OU": "jason@vip.qq.com"
  }],
  "ca": {
    "expiry": "87600h"
  }
}
EOF
  • CN :Common Name,kube-apiserver从证书中提取该字段作为请求的用户名 (User Name),浏览器使用该字段验证网站是否合法
  • C :国家
  • ST :州,省
  • L : 地区,城市
  • O :Organization,组织名,公司名称,kube-apiserver从证书中提取该字段作为请求用户所属的组 (Group)
  • OU :组织内部单位名称,部门名称
  • 不同证书csr文件的CN、O、C、ST、L、OU组合必须不同,否则可能出现PEER’S CERTIFICATE HAS AN INVALID SIGNATURE错误

3、生成CA 根证书及私钥

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# cfssl gencert -initca ca-csr.json | cfssljson -bare ca
[root@master1 cert]# ls
ca-config.json  ca.csr  ca-csr.json  ca-key.pem  ca.pem

4、分发证书

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# for node_ip in ${ALL_IPS[@]}
  do
    echo ">>> ${node_ip}"
    ssh root@${node_ip} "mkdir -p /opt/k8s/etc/cert"
    scp ca*.pem ca-config.json root@${node_ip}:/opt/k8s/etc/cert
  done

三、生成集群管理员admin的证书(kubectl->kube-apiserver)

1、准备证书签名请求文件 kubectl-admin-csr.json

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# cat > kubectl-admin-csr.json <<EOF
{
  "CN": "k8s-demo-admin",
  "hosts": [
    "192.168.66.10",
    "192.168.66.11",
    "192.168.66.12"
  ],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
 "names": [{
    "C" : "CN",
    "ST": "GuangDong",
    "L" : "ShenZhen",
    "O" : "system:masters",
    "OU": "jason@vip.qq.com"
 }]
}
EOF
  • kube-apiserver 将提取的 User、Group 作为RBAC授权的用户标识
  • kubernetes使用RBAC进行角色权限控制,证书中的CN字段作为User,O字段作为Group
  • 这里管理员名字设定为k8s-demo-admin,你可以根据实际情况修改

2、生成集群管理员证书和私钥

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# cfssl gencert -ca=/opt/install/cert/ca.pem \
  -ca-key=/opt/install/cert/ca-key.pem \
  -config=/opt/install/cert/ca-config.json \
  -profile=k8s-demo-client kubectl-admin-csr.json | cfssljson -bare kubectl-admin
[root@master1 ~]# ls kubectl-admin*
  • 如果授权部分操作权限给一个账户呢?证书如何生成?后面单独一篇文章来说RBAC
  • RBAC:Role Based Access Control
  • K8s还有其它五种的授权方式AlwaysDeny、AlwaysAllow、ABAC、Webhook、Node

3、分发证书到3个Master节点

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# for node_ip in ${MASTER_IPS[@]}
  do
    echo ">>> ${node_ip}"
    scp kubectl-admin*.pem root@${node_ip}:/opt/k8s/etc/cert
  done

四、生成etcd节点之间通讯的证书

1、准备证书签名请求文件

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# cat > etcd-csr.json <<EOF
{
  "CN": "k8s-demo-etcd",
  "hosts": [
    "192.168.66.10",
    "192.168.66.11",
    "192.168.66.12"
  ],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [{
    "C" : "CN",
    "ST": "GuangDong",
    "L" : "ShenZhen",
    "O" : "k8s-demo",
    "OU": "jason@vip.qq.com"
  }]
}
EOF
  • UserName:k8s-demo-etcd
  • Group:k8s-demo

2、生成etcd证书

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# cfssl gencert -ca=/opt/install/cert/ca.pem \
    -ca-key=/opt/install/cert/ca-key.pem \
    -config=/opt/install/cert/ca-config.json \
    -profile=k8s-demo-server etcd-csr.json | cfssljson -bare etcd
ls etcd*pem

3、分发到3个master节点

[root@master1 ~]# cd /opt/install/cert
[root@master1 ~]# for node_ip in ${MASTER_IPS[@]}
  do
    echo ">>> ${node_ip}"
    ssh root@${node_ip} "mkdir -p /opt/k8s/etcd/cert"
    scp etcd*.pem root@${node_ip}:/opt/k8s/etcd/cert/
  done

五、生成kube-apiserver证书

  • kube-apiserver访问etcd、kubelet等接口时使用

1、准备证书签名请求文件 apiserver-csr.json

[root@master1 ~]# cd /opt/install/cert
[root@master1 ~]# cat > apiserver-csr.json <<EOF
{
  "CN": "k8s-demo-apiserver",
  "hosts": [
    "192.168.66.10",
    "192.168.66.11",
    "192.168.66.12",
    "10.8.0.1",
    "kubernetes",
    "kubernetes.default",
    "kubernetes.default.svc",
    "kubernetes.default.svc.cluster",
    "kubernetes.default.svc.${CLUSTER_DNS_DOMAIN}."
  ],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [{
    "C" : "CN",
    "ST": "GuangDong",
    "L" : "ShenZhen",
    "O" : "k8s-demo",
    "OU": "jason@vip.qq.com"
  }]
}
EOF
  • 请打开apiserver-csr.json文件检查使用的环境变量是否正确

2、生成kube-apiserver证书

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# cfssl gencert -ca=/opt/install/cert/ca.pem \
  -ca-key=/opt/install/cert/ca-key.pem \
  -config=/opt/install/cert/ca-config.json \
  -profile=k8s-demo-server kube-apiserver-csr.json | cfssljson -bare kube-apiserver
[root@master1 ~]# ls kube-apiserver*pem

3、分发证书

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# for node_ip in ${MASTER_IPS[@]}
  do
    echo ">>> ${node_ip}"
    ssh root@${node_ip} "mkdir -p /opt/k8s/etc/cert"
    scp kube-apiserver*.pem root@${node_ip}:/opt/k8s/etc/cert/
  done

六、生成kube-controller-manager证书

1、准备证书签名请求文件 controller-manager-csr.json

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# cat > controller-manager-csr.json <<EOF
{
    "CN": "k8s-demo-ctrl-mgr",
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "hosts": [
      "192.168.66.10",
      "192.168.66.11",
      "192.168.66.12"
    ],
    "names": [{
      "C" : "CN",
      "ST": "GuangDong",
      "L" : "ShenZhen",
      "O" : "system:kube-controller-manager",
      "OU": "jason@vip.qq.com"
    }]
}
EOF
  • 用户名 k8s-demo-ctrl-mgr 组名 system:kube-controller-manager

2、生成kube-controller-manager证书

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# cfssl gencert -ca=/opt/install/cert/ca.pem \
  -ca-key=/opt/install/cert/ca-key.pem \
  -config=/opt/install/cert/ca-config.json \
  -profile=k8s-demo-server kube-controller-manager-csr.json | cfssljson -bare kube-controller-manager
[root@master1 ~]# ls kube-controller-manager*pem

3、分发证书到3个master节点

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# for node_ip in ${MASTER_IPS[@]}
  do
    echo ">>> ${node_ip}"
    scp kube-controller-manager*.pem root@${node_ip}:/opt/k8s/etc/cert/
  done

七、生成kube-scheduler证书

1、准备证书签名请求文件 scheduler-csr.json

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# cat > scheduler-csr.json <<EOF
{
    "CN": "k8s-demo-scheduler",
    "hosts": [
      "192.168.66.10",
      "192.168.66.11",
      "192.168.66.12"
    ],
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [{
      "C" : "CN",
      "ST": "GuangDong",
      "L" : "ShenZhen",
      "O" : "system:kube-scheduler",
      "OU": "jason@vip.qq.com"
    }]
}
EOF
  • 用户名 k8s-demo-scheduler 组 system:kube-scheduler

2、生成kube-scheduler证书

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# cfssl gencert -ca=/opt/install/cert/ca.pem \
  -ca-key=/opt/install/cert/ca-key.pem \
  -config=/opt/install/cert/ca-config.json \
  -profile=k8s-demo-server kube-scheduler-csr.json | cfssljson -bare kube-scheduler
[root@master1 ~]# ls kube-scheduler*pem

3、分发证书

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# for node_ip in ${MASTER_IPS[@]}
  do
    echo ">>> ${node_ip}"
    scp kube-scheduler*.pem root@${node_ip}:/opt/k8s/etc/cert/
  done

八、生成kube-proxy证书

1、准备证书签名请求文件 kube-proxy-csr.json

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# cat > kube-proxy-csr.json <<EOF
{
  "CN": "k8s-demo-kube-proxy",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [{
    "C" : "CN",
    "ST": "GuangDong",
    "L" : "ShenZhen",
    "O" : "k8s-demo",
    "OU": "jason@vip.qq.com"
 }]
}
EOF

2、生成kube-proxy证书

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# cfssl gencert -ca=/opt/install/cert/ca.pem \
  -ca-key=/opt/install/cert/ca-key.pem \
  -config=/opt/install/cert/ca-config.json \
  -profile=k8s-demo-client kube-proxy-csr.json | cfssljson -bare kube-proxy
[root@master1 ~]# ls kube-proxy*

3、分发证书到所有节点

[root@master1 ~]# cd /opt/cert
[root@master1 ~]# for node_ip in ${NODE_IPS[@]}
  do
    echo ">>> ${node_ip}"
    scp kube-proxy*.pem root@${node_ip}:/etc/kubernetes/cert/
  done

九、生成插件证书(aggregator.client)

1、准备证书签名请求文件 aggregator-client-csr.json

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# cat > aggregator-client-csr.json <<EOF
{
  "CN": "k8s-demo-aggregator",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
 "names": [{
    "C" : "CN",
    "ST": "GuangDong",
    "L" : "ShenZhen",
    "O" : "k8s-demo",
    "OU": "jason@vip.qq.com"
 }]
}
EOF
  • CN名称需要用于kube-apiserver的–requestheader-allowed-names参数中,否则后续访问metrics时会提示权限不足

2、生成aggregator-client证书

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# cfssl gencert -ca=/etc/kubernetes/cert/ca.pem \
  -ca-key=/opt/install/cert/ca-key.pem  \
  -config=/opt/install/cert/ca-config.json  \
  -profile=k8s-demo-client aggregator-client-csr.json | cfssljson -bare aggregator-client
[root@master1 ~]# ls aggregator-client*.pem

3、分发证书到3个master节点

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# for node_ip in ${MASTER_IPS[@]}
  do
    echo ">>> ${node_ip}"
    scp aggregator-client*.pem root@${node_ip}:/etc/kubernetes/cert/
  done

十、参考

  • https://blog.csdn.net/molixuebeibi/article/details/91873432
  • https://blog.csdn.net/lk2684753/article/details/100160856

附:K8s-Demo集群版本信息

组件版本命令
kubernetes1.18.5kubectl version
docker-ce19.03.11docker version 或者 rpm -qa | grep docker
etcd3.4.3etcdctl version
calico3.13.3calico -v
coredns1.7.0coredns -version

附:专栏链接

K8S-Demo集群实践00:搭建镜像仓库Harbor+安全扫描
K8S-Demo集群实践01:准备VMware虚拟机模板
K8S-Demo集群实践02:准备VMware虚拟机3台Master+3台Node
K8S-Demo集群实践03:准备集群各组件间HTTPS通讯需要的x509证书
K8S-Demo集群实践04:部署etcd三节点高可用集群
K8S-Demo集群实践05:安装kubectl并配置集群管理员账户
K8S-Demo集群实践06:部署kube-apiserver到master节点(3个无状态实例)
K8S-Demo集群实践07:kube-apiserver高可用方案
K8S-Demo集群实践08:部署高可用kube-controller-manager集群
K8S-Demo集群实践09:部署高可用kube-scheduler集群
K8S-Demo集群实践10:部署ipvs模式的kube-proxy组件
K8S-Demo集群实践11:部署ipvs模式的kube-kubelet组件
K8S-Demo集群实践12:部署Calico网络
K8S-Demo集群实践13:部署集群CoreDNS
K8S-Demo集群实践14:部署集群监控服务Metrics Server
K8S-Demo集群实践15:部署Kubernetes Dashboard
K8S-Demo集群实践16:部署Kube-Prometheus
K8S-Demo集群实践17:部署私有云盘owncloud(10.6版本)
K8S-Demo集群实践18:构建宇宙中第一个基础容器镜像


  • 先用起来,通过操作实践认识k8s,积累多了自然就理解了
  • 把理解的知识分享出来,自造福田,自得福缘
  • 追求简单,容易使人理解,知识的上下文也是知识的一部分,例如版本,时间等
  • 欢迎留言交流,也可以提出问题,一般在周末回复和完善文档
  • Jason@vip.qq.com 2021-1-19。
  • 0
    点赞
  • 1
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值