idea连接远程k8s集群使用kubernetes-client

一. k8s集群修改config

因为默认的是内网IP，复制出来后，后改为外面IP，报错

Unable to connect to the server: x509: certificate is valid for 10.0.1.27, 192.168.0.x, not x.x.x.x

原因从报错信息可以看出来：是证书对访问的IP做了校验，解决的方法就是要重新制作证书（不用动集群）。

1.1 备份当前k8s集群配置文件

cp -r /etc/kubernetes /etc/kubernetes.bak

1.2 删除当前k8s集群的apiserver的cert 和 key

rm -rf /etc/kubernetes/pki/apiserver.*

1.3 生成新的apiserver的cert和key

kubeadm init phase certs apiserver --apiserver-advertise-address ${Internal_IP} --apiserver-cert-extra-sans ${External_IP}

例如：

kubeadm init phase certs apiserver --apiserver-advertise-address 10.0.1.27 --apiserver-cert-extra-sans 122.51.103.44

1.4 刷新admin.conf

kubeadm alpha certs renew admin.conf

1.5 重启apiserver

kubectl -n kube-system delete pod -l component=kube-apiserver

1.6 刷新.kube/config

\cp /etc/kubernetes/admin.conf ~/.kube/config

注意：这里生成的 admin.conf 文件，server填写的依然是master的内网IP，如果需要放在集群外的客户端上使用，将这个IP改为master的外网IP即可。

参考 https://www.cnblogs.com/leozhanggg/p/16243205.html

二. 安装kubectl

2.1 下载kubectl

下载地址 https://www.jianshu.com/go-wild?ac=2&url=https%3A%2F%2Fstorage.googleapis.com%2Fkubernetes-release%2Frelease%2Fv1.21.2%2Fbin%2Fwindows%2Famd64%2Fkubectl.exe

下载kubectl.exe的可执行文件，我这里下载的是对应我们线上kubernetes版本的kubectl。如果想要下载其他版本，可以复制上面链接将版本号改对对应的即可。

将kubectl添加到PATH环境变量
在D盘下创建kube文件夹，将kubectl.exe存放在kube文件夹下。

将D:\kube\kubectl添加到PATH环境变量中

执行命令查看已安装的 kubectl 版本号

C:\Users\16872>kubectl version
Client Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.5", GitCommit:"5c99e2ac2ff9a3c549d9ca665e7bc05a3e18f07e", GitTreeState:"clean", BuildDate:"2021-12-16T08:38:33Z", GoVersion:"go1.16.12", Compiler:"gc", Platform:"windows/amd64"}
Unable to connect to the server: dial tcp 127.0.0.1:6443: connectex: No connection could be made because the target machine actively refused it.

2.2 配置kubectl

将kubectl config文件存放在~/.kube/下。 ~代表当前的用户目录
创建~/.kube文件夹，将master上的config文件拉取到本地， 存放在~/.kube/文件夹下

该config文件支持公网IP连接（一配置）

验证配置结果

PS C:\Users\16872> kubectl get nodes
NAME     STATUS   ROLES                  AGE    VERSION
master   Ready    control-plane,master   4h7m   v1.21.2
node1    Ready    <none>                 4h3m   v1.21.2
node2    Ready    <none>                 4h3m   v1.21.2
PS C:\Users\16872> kubectl get pods -n kube-system
NAME                                           READY   STATUS    RESTARTS   AGE
calico-kube-controllers-69cccc84d-jcc4f        1/1     Running   3          4h1m
calico-node-6qvhg                              1/1     Running   1          4h1m
calico-node-jpqbc                              1/1     Running   1          4h1m
calico-node-xg558                              1/1     Running   2          4h1m
coredns-59d64cd4d4-cb6hg                       1/1     Running   2          4h7m
coredns-59d64cd4d4-w9nt4                       1/1     Running   2          4h7m
etcd-master                                    1/1     Running   1          4h7m
ingress-nginx-controller-qkw2c                 1/1     Running   0          39m
ingress-nginx-defaultbackend-c49866855-dlst8   1/1     Running   0          39m
kube-apiserver-master                          1/1     Running   1          4h7m
kube-controller-manager-master                 1/1     Running   1          4h7m
kube-proxy-4jtzf                               1/1     Running   2          4h3m
kube-proxy-6gbxz                               1/1     Running   1          4h7m
kube-proxy-ljjct                               1/1     Running   1          4h3m
kube-scheduler-master                          1/1     Running   1          4h7m

三. 使用kubernetes-client操作k8s集群

3.1 依赖

        <!--kubernetes-client-->
        <dependency>
            <groupId>io.kubernetes</groupId>
            <artifactId>client-java</artifactId>
            <version>16.0.0</version>
        </dependency>

版本选择参考 https://github.com/kubernetes-client/java/wiki/2.-Versioning-and-Compatibility

3.2 注意（可忽略）

编辑配置添加-Djdk.tls.client.protocols=TLSv1.2，

否则报错

io.kubernetes.client.openapi.ApiException: javax.net.ssl.SSLHandshakeException: extension (5) should not be presented in certificate_request

3.3 创建 StatefulSet

    @Override
    @SneakyThrows
    public Boolean createCpsGo() {
        ApiClient client = Config.defaultClient();
        Configuration.setDefaultApiClient(client);
        // new一个statefulSet
        V1StatefulSet body = new V1StatefulSet()
                .metadata(new V1ObjectMeta().name("pod-go"))
                .spec(new V1StatefulSetSpec()
                        .serviceName("pod-go")
                        .replicas(3)
                        .selector(new V1LabelSelector().matchLabels(Map.of("app", "pod-go")))
                        .template(new V1PodTemplateSpec()
                                .metadata(new V1ObjectMeta().labels(Map.of("app", "pod-go")))
                                .spec(new V1PodSpec().containers(List.of(
                                        new V1Container()
                                                .name("pod-go")
                                                .image("registry.cn-shanghai.aliyuncs.com/wanfei/centos7-golang-sh:v1")
                                                .imagePullPolicy("IfNotPresent")
                                                .stdin(true)
                                                .tty(true)
                                )))
                        )
                );

        // 运行statefulSet
        AppsV1Api appsV1Api = new AppsV1Api();
        appsV1Api.createNamespacedStatefulSet("default", body, null, null, null, null);
        return true;
    }

创建成功

[root@master jenkins]# kubectl get StatefulSet | grep pod-go
pod-go                3/3     17s

3.4 运行shell命令

例如：生成文件，压缩文件

    @Override
    @SneakyThrows
    public Boolean createFile() {
        // 生成 pod-go-0.exe 文件命令
        String shExeCmd = "sh build.sh /app/test/ pod-go-0";
        // 压缩 pod-go-0.exe 为 pod-go-0.zip
        String shZipCmd = "zip /app/test/pod-go-0.zip /app/test/pod-go-0.exe";

        ApiClient client = Config.defaultClient();
        Configuration.setDefaultApiClient(client);

        // 执行kubectl exec在指定pod内运行命令
        String shExeStr = execForStr(shExeCmd);
        log.info("编译exe文件返回: {}", shExeStr);
        String shZipStr = execForStr(shZipCmd);
        log.info("压缩exe文件返回: {}", shZipStr);
        return true;
    }

    /**
     * 执行kubectl exec
     * @param command 命令
     */
    @SneakyThrows
    private static String execForStr(String command) {
        Exec exec = new Exec();
        Process proc = exec.exec("default", "pod-go-0", ArrayUtil.toArray(List.of("/bin/sh", "-c", command), String.class), "pod-go", true, true);
        return RuntimeUtil.getResult(proc);
    }

编译压缩成功

[root@pod-go-0 test]# ls
main.go  pdf1.syso  pod-go-0.exe  pod-go-0.zip

3.5 删除 StatefulSet

    @Override
    @SneakyThrows
    public Boolean deleteCpsGo() {
        ApiClient client = Config.defaultClient();
        Configuration.setDefaultApiClient(client);
        AppsV1Api appsV1Api = new AppsV1Api();
        appsV1Api.deleteNamespacedStatefulSet("pod-go", "default", null, null, null, null, null, null);
        return true;
    }

3.6 线上运行注意

部署到线上后发现报错

进入pod执行

[root@pod-java-67cdc944f8-jknpq app]# kubectl get pods
Error from server (Forbidden): pods is forbidden: User "system:serviceaccount:default:default" cannot list resource "pods" in API group "" in the namespace "default"

创建最高权限的角色

cat <<EOF> client-java-sa.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: client-java-sa
  namespace: default
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: client-java-sa-clusterrolebinding
subjects:
- kind: ServiceAccount
  name: client-java-sa
  namespace: default
roleRef:
  kind: ClusterRole
  name: cluster-admin
  apiGroup: rbac.authorization.k8s.io
EOF

部署java应用的时候使用client-java-sa这个ServiceAccount，否则会使用默认的default这个ServiceAccount

添加serviceAccountName: client-java-sa

部署

kubectl apply -f client-java-sa.yaml

参考 https://blog.51cto.com/zq2599/4907211