Kubernetes 集群自签证书配置

一、安装Helm3
1、下载htlm

 curl -L -o helm-v3.2.4-linux-amd64.tar.gz https://file.choerodon.com.cn/kubernetes-helm/v3.2.4/helm-v3.2.4-linux-amd64.tar.gz 

2、解压

 tar -zxvf helm-v3.2.4-linux-amd64.tar.gz 

3、 将文件移动到PATH目录中

 mv linux-amd64/helm /usr/local/bin/helm 

二、安装Cert-manager

1、安装crds，根据自己的版本安装
官方cert-manager地址（https://cert-manager.io/docs/releases/）

kubectl apply --validate=false -f https://github.com/cert-manager/cert-manager/releases/download/v1.14.4/cert-manager.crds.yaml

2、创建命名空间，自定义空间名，空间名自定义

 kubectl create namespace cert-manager 

3、添加Jetstack Helm 仓库

 helm repo add jetstack https://charts.jetstack.io 

4、更新helm仓库

 helm repo update 

5、安装Cert Manager

helm install \
  cert-manager jetstack/cert-manager \
  --namespace cluster-service \
  --version v1.14.

查看安装的cert-manager

[root@template train]# kubectl get pod -ncluster-service
NAME                                       READY   STATUS    RESTARTS   AGE
cert-manager-75677b59b6-b6th7              1/1     Running   0          2d23h
cert-manager-cainjector-55b7d68468-4r9hr   1/1     Running   0          2d23h
cert-manager-webhook-6775789b6b-2n9w7      1/1     Running   0          2d23h

三、配置Issuer或者Clusterissuer

Issuer 与 ClusterIssuer 的区别是 ClusterIssuer 可跨命名空间使用，而 Issuer 需在每个命名空间下配置后才可使用。我在此使用 ClusterIssuer，其类型选择 Let‘s Encrypt

配置 staging 环境使用的 Let‘s Encrypt ClusterIssuer，并创建：

-     # cluster-issuer-letsencrypt-staging.yaml
        
        apiVersion: cert-manager.io/v1alpha2
        kind: ClusterIssuer
        metadata:
          name: letsencrypt-staging
        spec:
          acme:
            # 务必将此处替换为你自己的邮箱, 否则会配置失败。当证书快过期时 Let's Encrypt 会与你联系
            email: user@example.com
            server: https://acme-staging-v02.api.letsencrypt.org/directory
            privateKeySecretRef:
              # 将用来存储 Private Key 的 Secret 资源
              name: letsencrypt-staging
            # Add a single challenge solver, HTTP01 using nginx
            solvers:
            - http01:
                ingress:
                  class: nginx
                  
           ```

kubectl create -f cluster-issuer-letsencrypt-staging.yaml

配置 production 环境使用的 Let‘s Encrypt ClusterIssuer，并创建：

  # cluster-issuer-letsencrypt-prod.yaml
  
  apiVersion: cert-manager.io/v1alpha2
  kind: ClusterIssuer
  metadata:
    name: letsencrypt-prod
  spec:
    acme:
      server: https://acme-v02.api.letsencrypt.org/directory
      email: varroreve@gmail.com
      privateKeySecretRef:
        name: letsencrypt-prod
      solvers:
  
     - http01:
       ingress:
         class: nginx

 kubectl create -f cluster-issuer-letsencrypt-prod.yaml 

查看创建clusterissuer状态：

 $ kubectl get clusterissuer

NAME                  READY   AGE
letsencrypt-prod      True    37s
letsencrypt-staging   True    119m

这里分别配置了测试环境与生产环境两个 ClusterIssuer， 原因是 Let’s Encrypt 的生产环境有着非常严格的接口调用限制，最好是在测试环境测试通过后，再切换为生产环境。

测试：
这里已安装好 Nginx Ingress Controller 并已存在一个 Ingress 对象，现在为它开启 TLS 选项

 apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/service-weight: ''
    cert-manager.io/cluster-issuer: letsencrypt-prod  #增加指定的clusterissuer
  name: merchant-client-car-boss-ing
  namespace: saasmerchant
spec:
  rules:
    - host: car-boss.view.sxkemao.com  #域名
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: merchant-client-car-boss-cip
                port:
                  number: 80
  tls:    #开启tls并指定证书名称
    - hosts:
        - car-boss.view.sxkemao.com
      secretName: car-boss-tls  #自定义证书名称

删除ingress并重新发布一下ingress.yaml

查看cert-manager创建的证书

[root@template train]# kubectl get certificate -A
NAMESPACE      NAME               READY   SECRET             AGE
saasmerchant   car-boss-tls       True    car-boss-tls       3h30m
saasmerchant   car-customer-tls   True    car-customer-tls   60m

注：如果你的ingress做的是外部nginx转发需要先把nginx配置写好，确保能够正确访问到正确的域名

server {
    listen 80;
    server_name car-customer.view.sxkemao.com;

    location / {
            proxy_set_header Host car-customer.view.sxkemao.com;
            proxy_set_header Connection "Keep-Alive";
            proxy_set_header   X-Real-IP        $remote_addr;
            proxy_set_header   X-Forwarded-For  $remote_addr;
            proxy_pass http://car-customer.view.sxkemao.com;
    }
}

如果在阿里云上做的配置，需要把解析配置好，确保这个域名能够被公网访问，不然证书会签发失败

此证书加nginx配置只能在内网中访问，公网443需要在nginx配置TCP和UDP流量的负载均衡，需要使用到nginxstream模块。这个模块在nginx1.11.5以后的版本就已经默认安装，如果没有安装请自行安装
This is now possible with the addition of the ngx_stream_ssl_preread module added in Nginx 1.11.5 and the ngx_stream_map module added in 1.11.2.

load_module /usr/lib64/nginx/modules/ngx_stream_module.so;  #此行是加载使用stream，不是所有机器都需要加载，有问题再加
worker_processes 8;

events {
        use epoll;
        accept_mutex off;
    worker_connections  65536;
}

stream {
    map $ssl_preread_server_name $backend_443 {                 #map可配置多个域名
        car-boss.view.sxkemao.com server_view_443;
        default http_nginx_443;                                                      #default在匹配不到上面的域名是，匹配http_nginx_443下面域名
    }

    map $ssl_preread_server_name $backend_80 {
        car-boss.view.sxkemao.com server_view_80;
        default http_nginx_80;
    }

    upstream server_view_443 {
        server 10.50.21.62:443;
    }

    upstream server_view_80 {
        server 10.50.21.62:80;
    }

    upstream http_nginx_443 {
        server 10.50.21.244:443;
    }


    upstream http_nginx_80 {
        server 10.50.21.244:80;
    }

    log_format custom '[$time_local] ';

    access_log /var/log/nginx/stream.log custom;

    server {
        listen 80;
        proxy_pass $backend_80;
        ssl_preread on;
    }

    server {
        listen 443;
        proxy_pass $backend_443;
        ssl_preread on;
    }
}