kubesphere 卸载 istio后 部署服务一直处理队列中
1.查看发布界面,任务一直卡在队列中
2.找到devops-jenkins,查看日志
WARNING o.c.j.p.k.KubernetesLauncher#launch: Error in provisioning; agent=KubernetesSlave name: maven-4q2zk, template=PodTemplate{id='99462365-6c36-4a3d-8627-7b5912f7a506', name='maven', namespace='kubesphere-devops-worker', label='maven', nodeUsageMode=EXCLUSIVE, workspaceVolume=EmptyDirWorkspaceVolume [memory=false], volumes=[HostPathVolume [mountPath=/var/run/docker.sock, hostPath=/var/run/docker.sock], HostPathVolume [mountPath=/root/.m2, hostPath=/var/data/jenkins_maven_cache], HostPathVolume [mountPath=/root/.sonar/cache, hostPath=/var/data/jenkins_sonar_cache]], containers=[ContainerTemplate{name='maven', image='kubesphere/builder-maven:v3.2.0', command='cat', args='', ttyEnabled=true, resourceRequestCpu='100m', resourceRequestMemory='100Mi', resourceLimitCpu='4000m', resourceLimitMemory='8192Mi'}, ContainerTemplate{name='jnlp', image='jenkins/jnlp-slave:3.27-1', command='jenkins-slave', args='${computer.jnlpmac} ${computer.name}', resourceRequestCpu='50m', resourceRequestMemory='400Mi', resourceLimitCpu='500m', resourceLimitMemory='1536Mi'}]}
io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: POST at: https://kubernetes.default/api/v1/namespaces/kubesphere-devops-worker/pods. Message: Internal error occurred: failed calling webhook "rev.object.sidecar-injector.istio.io": failed to call webhook: Post "https://istiod-1-11-2.istio-system.svc:443/inject?timeout=10s": dial tcp 10.108.87.252:443: connect: connection refused. Received status: Status(apiVersion=v1, code=500, details=StatusDetails(causes=[StatusCause(field=null, message=failed calling webhook "rev.object.sidecar-injector.istio.io": failed to call webhook: Post "https://istiod-1-11-2.istio-system.svc:443/inject?timeout=10s": dial tcp 10.108.87.252:443: connect: connection refused, reason=null, additionalProperties={})], group=null, kind=null, name=null, retryAfterSeconds=null, uid=null, additionalProperties={}), kind=Status, message=Internal error occurred: failed calling webhook "rev.object.sidecar-injector.istio.io": failed to call webhook: Post "https://istiod-1-11-2.istio-system.svc:443/inject?timeout=10s": dial tcp 10.108.87.252:443: connect: connection refused, metadata=ListMeta(_continue=null, remainingItemCount=null, resourceVersion=null, selfLink=null, additionalProperties={}), reason=InternalError, status=Failure, additionalProperties={}).
at io.fabric8.kubernetes.client.dsl.base.OperationSupport.requestFailure(OperationSupport.java:589)
at io.fabric8.kubernetes.client.dsl.base.OperationSupport.assertResponseCode(OperationSupport.java:528)
at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:492)
at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:451)
at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleCreate(OperationSupport.java:252)
at io.fabric8.kubernetes.client.dsl.base.BaseOperation.handleCreate(BaseOperation.java:841)
at io.fabric8.kubernetes.client.dsl.base.BaseOperation.create(BaseOperation.java:332)
at io.fabric8.kubernetes.client.dsl.base.BaseOperation.create(BaseOperation.java:82)
at org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher.launch(KubernetesLauncher.java:133)
at hudson.slaves.SlaveComputer.lambda$_connect$0(SlaveComputer.java:294)
at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:71)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2022-07-26 01:22:21.448+0000 [id=1504432] INFO o.c.j.p.k.KubernetesSlave#_terminate: Terminating Kubernetes instance for agent maven-4q2zk
2022-07-26 01:22:21.454+0000 [id=1504432] WARNING o.c.j.p.k.KubernetesSlave#deleteSlavePod: Failed to delete pod for agent kubesphere-devops-worker/maven-4q2zk: not found
ERROR: Failed to delete pod for agent kubesphere-devops-worker/maven-4q2zk: not found
Disconnected computer maven-4q2zk
2022-07-26 01:22:21.454+0000 [id=1504432] INFO o.c.j.p.k.KubernetesSlave#_terminate: Disconnected computer maven-4q2zk
3.解决方法
先查询准入验证
kubectl get MutatingWebhookConfiguration
kubectl get ValidatingWebhookConfiguration
删除以下两个资源(k8s准入控制)
kubectl delete ValidatingWebhookConfiguration istio-validator-1-11-2-istio-system
kubectl delete MutatingWebhookConfiguration istio-sidecar-injector-1-11-2