Kubeflow v0.6.0 部署采坑记录

最新推荐文章于 2024-09-25 07:58:09 发布
最新推荐文章于 2024-09-25 07:58:09 发布
阅读量1.9k 收藏
点赞数
分类专栏： Kubeflow
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/oscarun/article/details/99107640
版权

文章目录

1 Overview

安装条件，可以在 Kubeflow requirements 找到，具体贴出来，因为版本不同，以及 K8S 一些自定义配置会对部署造成影响。

image_1dhspu8gd1od31u54cnpgb52uq9.png-71.2kB

P.S. 本文部署的是 v0.6.0

2 Deploy

部署提供了集中场景，分别对应 GCP, AWS, ACK, IBM Cloud 等等，本文主要讲的场景是你已经有一套 K8S 集群了，?是我的配置。Deploying Kubeflow on Existing Clusters 后面简称 on EC 好了。

image_1dhsqc07q14en1qf76ef1sabve0m.png-96kB

官方文档给出的 on EC 有两种配置的方案，为什么这么麻烦呢？因为 Kubeflow 本身组件太多了，社区有一个配置方案，另外有个 Vendor 也有一个配置方案，大家可以先都了解下，本文用的是社区的配置方案，也叫做 Community maintained。

image_1dhsqv1hrr0i1mlkjojq2la2713.png-67.5kB

重点在下面的执行脚本的过程。

# Add kfctl to PATH, to make the kfctl binary easier to use.
export PATH=$PATH:"<path to kfctl>"
export KFAPP="<your choice of application directory name>"
# Installs istio by default. Comment out istio components in the config file to skip istio installation. See https://github.com/kubeflow/kubeflow/pull/3663
export CONFIG="https://raw.githubusercontent.com/kubeflow/kubeflow/master/bootstrap/config/kfctl_k8s_istio.yaml"

kfctl init ${KFAPP} --config=${CONFIG} -V
cd ${KFAPP}
kfctl generate all -V
kfctl apply all -V

因为组件太多，配置很容易出问题，所以社区给的是一个统一的配置文件，当然有些组件是插拔式的，不需要的组件可以注释掉。

如果想提前了解都安装了什么组件，建议看看这个 CONFIG 的内容。

image_1dhsrh21t1du01ga9v2p1uba141e1g.png-121kB

如果集群不能访问公网的话，可以将这个文件下载到私有集群上。下图有提到在 kfctl generate 过程中，有下载一些资源。

image_1dhsrprj5fva1blr13it1ig79kq1t.png-216.8kB

按照官网的提示，kubectl -n kubeflow get all 查一下安装的所有 K8S 资源类型。

image_1dhtn73cs1qt61t9016o1qib17qc2q.png-1538.6kB

最后附上，安装和部署的日志，大家可以根据日志来排查部署的问题。

➜  try-kf kfctl init ${KFAPP} --config=${CONFIG} -V
INFO[0000] Downloading https://raw.githubusercontent.com/kubeflow/kubeflow/master/bootstrap/config/kfctl_k8s_istio.yaml to /var/folders/0g/qn4zfbj15m73ccbhz16_b_f00000gn/T/127834711/app.yaml  filename="v1alpha1/application_types.go:334"
WARN[0000] Overriding KfDef.Spec.Name; old value kubeflow_app; new value myapp  filename="coordinator/coordinator.go:274"
WARN[0000] Repo manifests exists in app.yaml ignoring version provided by --package-manager  filename="coordinator/coordinator.go:515"
INFO[0000] Creating directory /tmp/try-kf/myapp          filename="coordinator/coordinator.go:384"
INFO[0000] Writing KfDef to /tmp/try-kf/myapp/app.yaml   filename="coordinator/coordinator.go:401"
INFO[0000] Writing stripped KfDef to /tmp/try-kf/myapp/app.yaml  filename="v1alpha1/application_types.go:626"
INFO[0000] Synchronize cache                             filename="coordinator/coordinator.go:431"
INFO[0000] Creating directory /tmp/try-kf/myapp/.cache   filename="v1alpha1/application_types.go:409"
INFO[0000] Fetching https://github.com/kubeflow/manifests/archive/master.tar.gz to /tmp/try-kf/myapp/.cache/manifests  filename="v1alpha1/application_types.go:449"
INFO[0007] Fetch succeeded; LocalPath /tmp/try-kf/myapp/.cache/manifests/manifests-master  filename="v1alpha1/application_types.go:475"
INFO[0007] Fetching https://github.com/kubeflow/kubeflow/archive/master.tar.gz to /tmp/try-kf/myapp/.cache/kubeflow  filename="v1alpha1/application_types.go:449"
INFO[0013] Fetch succeeded; LocalPath /tmp/try-kf/myapp/.cache/kubeflow/kubeflow-master  filename="v1alpha1/application_types.go:475"
INFO[0013] Writing stripped KfDef to /tmp/try-kf/myapp/app.yaml  filename="v1alpha1/application_types.go:626"
INFO[0013] Downloading /tmp/try-kf/myapp/app.yaml to /var/folders/0g/qn4zfbj15m73ccbhz16_b_f00000gn/T/091930508/app.yaml  filename="v1alpha1/application_types.go:334"
➜  try-kf ls
kfctl myapp
➜  try-kf cd myapp
➜  myapp ls
app.yaml
➜  myapp kfctl generate all -V
INFO[0000] Downloading /tmp/try-kf/myapp/app.yaml to /var/folders/0g/qn4zfbj15m73ccbhz16_b_f00000gn/T/112839703/app.yaml  filename="v1alpha1/application_types.go:334"
INFO[0000] Writing stripped KfDef to /tmp/try-kf/myapp/app.yaml  filename="v1alpha1/application_types.go:626"
INFO[0000] Downloading /tmp/try-kf/myapp/app.yaml to /var/folders/0g/qn4zfbj15m73ccbhz16_b_f00000gn/T/514253450/app.yaml  filename="v1alpha1/application_types.go:334"
INFO[0000] Initializing kustomize package map            filename="kustomize/kustomize.go:149"
INFO[0000] Component path map: {}                        filename="kustomize/kustomize.go:170"
INFO[0000] Writing stripped KfDef to /tmp/try-kf/myapp/app.yaml  filename="v1alpha1/application_types.go:626"
INFO[0000] Processing application: istio-crds            filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: istio-install         filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: istio                 filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: application-crds      filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: application           filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: metacontroller        filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: argo                  filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: centraldashboard      filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: bootstrap             filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: webhook               filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: jupyter-web-app       filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: katib-db              filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: katib-manager         filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: katib-controller      filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: katib-ui              filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: metadata              filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: metrics-collector     filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: suggestion            filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: notebook-controller   filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: pytorch-job-crds      filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: pytorch-operator      filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: spartakus             filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: tensorboard           filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: tf-job-operator       filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: api-service           filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: minio                 filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: mysql                 filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: persistent-agent      filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: pipelines-runner      filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: pipelines-ui          filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: pipelines-viewer      filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: scheduledworkflow     filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: profiles              filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: seldon-core-operator  filename="kustomize/kustomize.go:589"
➜  myapp kfctl apply all -V
INFO[0000] Downloading /tmp/try-kf/myapp/app.yaml to /var/folders/0g/qn4zfbj15m73ccbhz16_b_f00000gn/T/984513046/app.yaml  filename="v1alpha1/application_types.go:334"
INFO[0000] Writing stripped KfDef to /tmp/try-kf/myapp/app.yaml  filename="v1alpha1/application_types.go:626"
INFO[0000] Downloading /tmp/try-kf/myapp/app.yaml to /var/folders/0g/qn4zfbj15m73ccbhz16_b_f00000gn/T/028543101/app.yaml  filename="v1alpha1/application_types.go:334"
INFO[0000] Initializing a default restConfig for Kubernetes  filename="kustomize/kustomize.go:248"
INFO[0000] namespace: kubeflow                           filename="kustomize/kustomize.go:265"
INFO[0000] Creating namespace: kubeflow                  filename="kustomize/kustomize.go:268"
INFO[0000] creating CustomResourceDefinition/adapters.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/apikeys.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/attributemanifests.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/authorizations.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/bypasses.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/certificates.certmanager.k8s.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/challenges.certmanager.k8s.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/checknothings.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/circonuses.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/cloudwatches.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/clusterissuers.certmanager.k8s.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/clusterrbacconfigs.rbac.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/deniers.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/destinationrules.networking.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/dogstatsds.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/edges.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/envoyfilters.networking.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/fluentds.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/gateways.networking.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/handlers.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/httpapispecbindings.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/httpapispecs.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/instances.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/issuers.certmanager.k8s.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/kubernetesenvs.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/kuberneteses.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/listcheckers.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/listentries.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/logentries.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/memquotas.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/meshpolicies.authentication.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/metrics.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/noops.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/opas.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/orders.certmanager.k8s.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/policies.authentication.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/prometheuses.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/quotas.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/quotaspecbindings.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/quotaspecs.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/rbacconfigs.rbac.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/rbacs.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/redisquotas.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/reportnothings.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/rules.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/serviceentries.networking.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/servicerolebindings.rbac.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/serviceroles.rbac.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/sidecars.networking.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/signalfxs.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/solarwindses.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/stackdrivers.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/statsds.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/stdios.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/templates.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/tracespans.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/virtualservices.networking.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/zipkins.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0001] creating Namespace/istio-system               filename="kustomize/kustomize.go:430"
INFO[0001] creating ServiceAccount/istio-citadel-service-account  filename="kustomize/kustomize.go:430"
INFO[0001] creating ServiceAccount/istio-cleanup-secrets-service-account  filename="kustomize/kustomize.go:430"
INFO[0001] creating ServiceAccount/istio-egressgateway-service-account  filename="kustomize/kustomize.go:430"
INFO[0001] creating ServiceAccount/istio-galley-service-account  filename="kustomize/kustomize.go:430"
INFO[0001] creating ServiceAccount/istio-grafana-post-install-account  filename="kustomize/kustomize.go:430"
INFO[0001] creating ServiceAccount/istio-ingressgateway-service-account  filename="kustomize/kustomize.go:430"
INFO[0001] creating ServiceAccount/istio-mixer-service-account  filename="kustomize/kustomize.go:430"
INFO[0001] creating ServiceAccount/istio-multi           filename="kustomize/kustomize.go:430"
INFO[0001] creating ServiceAccount/istio-pilot-service-account  filename="kustomize/kustomize.go:430"
INFO[0001] creating ServiceAccount/istio-security-post-install-account  filename="kustomize/kustomize.go:430"
INFO[0001] creating ServiceAccount/istio-sidecar-injector-service-account  filename="kustomize/kustomize.go:430"
INFO[0001] creating ServiceAccount/kiali-service-account  filename="kustomize/kustomize.go:430"
INFO[0001] creating ServiceAccount/prometheus            filename="kustomize/kustomize.go:430"
INFO[0001] creating Role/istio-ingressgateway-sds        filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRole/istio-citadel-istio-system  filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRole/istio-cleanup-secrets-istio-system  filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRole/istio-egressgateway-istio-system  filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRole/istio-galley-istio-system  filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRole/istio-grafana-post-install-istio-system  filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRole/istio-ingressgateway-istio-system  filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRole/istio-mixer-istio-system  filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRole/istio-pilot-istio-system  filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRole/istio-reader             filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRole/istio-sidecar-injector-istio-system  filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRole/kiali-viewer             filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRole/kiali                    filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRole/prometheus-istio-system  filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRole/istio-security-post-install-istio-system  filename="kustomize/kustomize.go:430"
INFO[0001] creating RoleBinding/istio-ingressgateway-sds  filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRoleBinding/istio-citadel-istio-system  filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRoleBinding/istio-cleanup-secrets-istio-system  filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRoleBinding/istio-egressgateway-istio-system  filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRoleBinding/istio-galley-admin-role-binding-istio-system  filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRoleBinding/istio-grafana-post-install-role-binding-istio-system  filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRoleBinding/istio-ingressgateway-istio-system  filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRoleBinding/istio-kiali-admin-role-binding-istio-system  filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRoleBinding/istio-mixer-admin-role-binding-istio-system  filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRoleBinding/istio-multi       filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRoleBinding/istio-pilot-istio-system  filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRoleBinding/istio-sidecar-injector-admin-role-binding-istio-system  filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRoleBinding/prometheus-istio-system  filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRoleBinding/istio-security-post-install-role-binding-istio-system  filename="kustomize/kustomize.go:430"
INFO[0001] creating ConfigMap/istio-galley-configuration  filename="kustomize/kustomize.go:430"
INFO[0001] creating ConfigMap/istio-grafana-configuration-dashboards-galley-dashboard  filename="kustomize/kustomize.go:430"
INFO[0001] creating ConfigMap/istio-grafana-configuration-dashboards-istio-mesh-dashboard  filename="kustomize/kustomize.go:430"
INFO[0001] creating ConfigMap/istio-grafana-configuration-dashboards-istio-performance-dashboard  filename="kustomize/kustomize.go:430"
INFO[0001] creating ConfigMap/istio-grafana-configuration-dashboards-istio-service-dashboard  filename="kustomize/kustomize.go:430"
INFO[0001] creating ConfigMap/istio-grafana-configuration-dashboards-istio-workload-dashboard  filename="kustomize/kustomize.go:430"
INFO[0001] creating ConfigMap/istio-grafana-configuration-dashboards-mixer-dashboard  filename="kustomize/kustomize.go:430"
INFO[0001] creating ConfigMap/istio-grafana-configuration-dashboards-pilot-dashboard  filename="kustomize/kustomize.go:430"
INFO[0001] creating ConfigMap/istio-grafana-custom-resources  filename="kustomize/kustomize.go:430"
INFO[0001] creating ConfigMap/istio-grafana              filename="kustomize/kustomize.go:430"
INFO[0001] creating ConfigMap/istio-security-custom-resources  filename="kustomize/kustomize.go:430"
INFO[0001] creating ConfigMap/istio-sidecar-injector     filename="kustomize/kustomize.go:430"
INFO[0001] creating ConfigMap/istio                      filename="kustomize/kustomize.go:430"
INFO[0001] creating ConfigMap/kiali                      filename="kustomize/kustomize.go:430"
INFO[0001] creating ConfigMap/prometheus                 filename="kustomize/kustomize.go:430"
INFO[0001] creating Secret/kiali                         filename="kustomize/kustomize.go:430"
INFO[0001] creating Service/grafana                      filename="kustomize/kustomize.go:430"
INFO[0001] creating Service/istio-citadel                filename="kustomize/kustomize.go:430"
INFO[0001] creating Service/istio-egressgateway          filename="kustomize/kustomize.go:430"
INFO[0001] creating Service/istio-galley                 filename="kustomize/kustomize.go:430"
INFO[0001] creating Service/istio-ingressgateway         filename="kustomize/kustomize.go:430"
INFO[0001] creating Service/istio-pilot                  filename="kustomize/kustomize.go:430"
INFO[0001] creating Service/istio-policy                 filename="kustomize/kustomize.go:430"
INFO[0001] creating Service/istio-sidecar-injector       filename="kustomize/kustomize.go:430"
INFO[0001] creating Service/istio-telemetry              filename="kustomize/kustomize.go:430"
INFO[0002] creating Service/jaeger-agent                 filename="kustomize/kustomize.go:430"
INFO[0002] creating Service/jaeger-collector             filename="kustomize/kustomize.go:430"
INFO[0002] creating Service/jaeger-query                 filename="kustomize/kustomize.go:430"
INFO[0002] creating Service/kiali                        filename="kustomize/kustomize.go:430"
INFO[0002] creating Service/prometheus                   filename="kustomize/kustomize.go:430"
INFO[0002] creating Service/tracing                      filename="kustomize/kustomize.go:430"
INFO[0002] creating Service/zipkin                       filename="kustomize/kustomize.go:430"
INFO[0002] creating Deployment/grafana                   filename="kustomize/kustomize.go:430"
INFO[0002] creating Deployment/istio-citadel             filename="kustomize/kustomize.go:430"
INFO[0002] creating Deployment/istio-egressgateway       filename="kustomize/kustomize.go:430"
INFO[0002] creating Deployment/istio-galley              filename="kustomize/kustomize.go:430"
INFO[0002] creating Deployment/istio-ingressgateway      filename="kustomize/kustomize.go:430"
INFO[0002] creating Deployment/istio-pilot               filename="kustomize/kustomize.go:430"
INFO[0002] creating Deployment/istio-policy              filename="kustomize/kustomize.go:430"
INFO[0002] creating Deployment/istio-sidecar-injector    filename="kustomize/kustomize.go:430"
INFO[0002] creating Deployment/istio-telemetry           filename="kustomize/kustomize.go:430"
INFO[0002] creating Deployment/istio-tracing             filename="kustomize/kustomize.go:430"
INFO[0002] creating Deployment/kiali                     filename="kustomize/kustomize.go:430"
INFO[0002] creating Deployment/prometheus                filename="kustomize/kustomize.go:430"
INFO[0002] creating PodDisruptionBudget/istio-egressgateway  filename="kustomize/kustomize.go:430"
INFO[0002] creating PodDisruptionBudget/istio-galley     filename="kustomize/kustomize.go:430"
INFO[0002] creating PodDisruptionBudget/istio-ingressgateway  filename="kustomize/kustomize.go:430"
INFO[0002] creating PodDisruptionBudget/istio-pilot      filename="kustomize/kustomize.go:430"
INFO[0002] creating PodDisruptionBudget/istio-policy     filename="kustomize/kustomize.go:430"
INFO[0002] creating PodDisruptionBudget/istio-telemetry  filename="kustomize/kustomize.go:430"
INFO[0002] creating MutatingWebhookConfiguration/istio-sidecar-injector  filename="kustomize/kustomize.go:430"
INFO[0002] creating HorizontalPodAutoscaler/istio-egressgateway  filename="kustomize/kustomize.go:430"
INFO[0002] creating HorizontalPodAutoscaler/istio-ingressgateway  filename="kustomize/kustomize.go:430"
INFO[0002] creating HorizontalPodAutoscaler/istio-pilot  filename="kustomize/kustomize.go:430"
INFO[0002] creating HorizontalPodAutoscaler/istio-policy  filename="kustomize/kustomize.go:430"
INFO[0002] creating HorizontalPodAutoscaler/istio-telemetry  filename="kustomize/kustomize.go:430"
INFO[0002] creating Job/istio-cleanup-secrets-1.1.6      filename="kustomize/kustomize.go:430"
INFO[0002] creating Job/istio-grafana-post-install-1.1.6  filename="kustomize/kustomize.go:430"
INFO[0002] creating Job/istio-security-post-install-1.1.6  filename="kustomize/kustomize.go:430"
INFO[0002] creating attributemanifest/istioproxy         filename="kustomize/kustomize.go:430"
INFO[0002] creating attributemanifest/kubernetes         filename="kustomize/kustomize.go:430"
INFO[0002] creating handler/kubernetesenv                filename="kustomize/kustomize.go:430"
INFO[0002] creating handler/prometheus                   filename="kustomize/kustomize.go:430"
INFO[0002] creating handler/stdio                        filename="kustomize/kustomize.go:430"
INFO[0002] creating kubernetes/attributes                filename="kustomize/kustomize.go:430"
INFO[0002] creating logentry/accesslog                   filename="kustomize/kustomize.go:430"
INFO[0002] creating logentry/tcpaccesslog                filename="kustomize/kustomize.go:430"
INFO[0002] creating metric/requestcount                  filename="kustomize/kustomize.go:430"
INFO[0002] creating metric/requestduration               filename="kustomize/kustomize.go:430"
INFO[0002] creating metric/requestsize                   filename="kustomize/kustomize.go:430"
INFO[0002] creating metric/responsesize                  filename="kustomize/kustomize.go:430"
INFO[0002] creating metric/tcpbytereceived               filename="kustomize/kustomize.go:430"
INFO[0002] creating metric/tcpbytesent                   filename="kustomize/kustomize.go:430"
INFO[0002] creating metric/tcpconnectionsclosed          filename="kustomize/kustomize.go:430"
INFO[0002] creating metric/tcpconnectionsopened          filename="kustomize/kustomize.go:430"
INFO[0002] creating rule/kubeattrgenrulerule             filename="kustomize/kustomize.go:430"
INFO[0002] creating rule/promhttp                        filename="kustomize/kustomize.go:430"
INFO[0002] creating rule/promtcpconnectionclosed         filename="kustomize/kustomize.go:430"
INFO[0002] creating rule/promtcpconnectionopen           filename="kustomize/kustomize.go:430"
INFO[0002] creating rule/promtcp                         filename="kustomize/kustomize.go:430"
INFO[0002] creating rule/stdiotcp                        filename="kustomize/kustomize.go:430"
INFO[0002] creating rule/stdio                           filename="kustomize/kustomize.go:430"
INFO[0002] creating rule/tcpkubeattrgenrulerule          filename="kustomize/kustomize.go:430"
INFO[0003] creating DestinationRule/istio-policy         filename="kustomize/kustomize.go:430"
INFO[0003] creating DestinationRule/istio-telemetry      filename="kustomize/kustomize.go:430"
INFO[0003] creating ConfigMap/istio-parameters-9m7m4fhdb8  filename="kustomize/kustomize.go:430"
INFO[0003] creating Gateway/kubeflow-gateway             filename="kustomize/kustomize.go:430"
INFO[0003] creating ServiceEntry/google-api-entry        filename="kustomize/kustomize.go:430"
INFO[0003] creating ServiceEntry/google-storage-api-entry  filename="kustomize/kustomize.go:430"
INFO[0003] creating VirtualService/google-api-vs         filename="kustomize/kustomize.go:430"
INFO[0003] creating VirtualService/google-storage-api-vs  filename="kustomize/kustomize.go:430"
INFO[0003] creating VirtualService/grafana-vs            filename="kustomize/kustomize.go:430"
INFO[0003] creating ClusterRbacConfig/default            filename="kustomize/kustomize.go:430"
INFO[0003] creating CustomResourceDefinition/applications.app.k8s.io  filename="kustomize/kustomize.go:430"
INFO[0003] creating ServiceAccount/application-controller-service-account  filename="kustomize/kustomize.go:430"
INFO[0003] creating ClusterRole/application-controller-cluster-role  filename="kustomize/kustomize.go:430"
INFO[0003] creating ClusterRoleBinding/application-controller-cluster-role-binding  filename="kustomize/kustomize.go:430"
INFO[0003] creating ConfigMap/application-controller-parameters  filename="kustomize/kustomize.go:430"
INFO[0003] creating Service/application-controller-service  filename="kustomize/kustomize.go:430"
INFO[0003] creating StatefulSet/application-controller-stateful-set  filename="kustomize/kustomize.go:430"
INFO[0003] creating Application/kubeflow                 filename="kustomize/kustomize.go:430"
INFO[0004] creating CustomResourceDefinition/compositecontrollers.metacontroller.k8s.io  filename="kustomize/kustomize.go:430"
INFO[0004] creating CustomResourceDefinition/controllerrevisions.metacontroller.k8s.io  filename="kustomize/kustomize.go:430"
INFO[0004] creating CustomResourceDefinition/decoratorcontrollers.metacontroller.k8s.io  filename="kustomize/kustomize.go:430"
INFO[0004] creating ServiceAccount/meta-controller-service  filename="kustomize/kustomize.go:430"
INFO[0004] creating ClusterRoleBinding/meta-controller-cluster-role-binding  filename="kustomize/kustomize.go:430"
INFO[0004] creating StatefulSet/metacontroller           filename="kustomize/kustomize.go:430"
INFO[0004] creating CustomResourceDefinition/workflows.argoproj.io  filename="kustomize/kustomize.go:430"
INFO[0004] creating ServiceAccount/argo-ui               filename="kustomize/kustomize.go:430"
INFO[0004] creating ServiceAccount/argo                  filename="kustomize/kustomize.go:430"
INFO[0004] creating ClusterRole/argo-ui                  filename="kustomize/kustomize.go:430"
INFO[0004] creating ClusterRole/argo                     filename="kustomize/kustomize.go:430"
INFO[0004] creating ClusterRoleBinding/argo-ui           filename="kustomize/kustomize.go:430"
INFO[0004] creating ClusterRoleBinding/argo              filename="kustomize/kustomize.go:430"
INFO[0004] creating ConfigMap/workflow-controller-configmap  filename="kustomize/kustomize.go:430"
INFO[0004] creating ConfigMap/workflow-controller-parameters  filename="kustomize/kustomize.go:430"
INFO[0004] creating Service/argo-ui                      filename="kustomize/kustomize.go:430"
INFO[0004] creating Deployment/argo-ui                   filename="kustomize/kustomize.go:430"
INFO[0004] creating Deployment/workflow-controller       filename="kustomize/kustomize.go:430"
INFO[0004] creating VirtualService/argo-ui               filename="kustomize/kustomize.go:430"
INFO[0004] creating ServiceAccount/centraldashboard      filename="kustomize/kustomize.go:430"
INFO[0004] creating Role/centraldashboard                filename="kustomize/kustomize.go:430"
INFO[0004] creating ClusterRole/centraldashboard         filename="kustomize/kustomize.go:430"
INFO[0004] creating RoleBinding/centraldashboard         filename="kustomize/kustomize.go:430"
INFO[0004] creating ClusterRoleBinding/centraldashboard  filename="kustomize/kustomize.go:430"
INFO[0004] creating ConfigMap/parameters                 filename="kustomize/kustomize.go:430"
INFO[0004] creating Service/centraldashboard             filename="kustomize/kustomize.go:430"
INFO[0004] creating Deployment/centraldashboard          filename="kustomize/kustomize.go:430"
INFO[0004] creating VirtualService/centraldashboard      filename="kustomize/kustomize.go:430"
INFO[0005] creating ServiceAccount/admission-webhook-bootstrap-service-account  filename="kustomize/kustomize.go:430"
INFO[0005] creating ClusterRole/admission-webhook-bootstrap-cluster-role  filename="kustomize/kustomize.go:430"
INFO[0005] creating ClusterRoleBinding/admission-webhook-bootstrap-cluster-role-binding  filename="kustomize/kustomize.go:430"
INFO[0005] creating ConfigMap/admission-webhook-bootstrap-config-map  filename="kustomize/kustomize.go:430"
INFO[0005] creating StatefulSet/admission-webhook-bootstrap-stateful-set  filename="kustomize/kustomize.go:430"
INFO[0005] creating CustomResourceDefinition/poddefaults.kubeflow.org  filename="kustomize/kustomize.go:430"
INFO[0005] creating ServiceAccount/admission-webhook-service-account  filename="kustomize/kustomize.go:430"
INFO[0005] creating ClusterRole/admission-webhook-cluster-role  filename="kustomize/kustomize.go:430"
INFO[0005] creating ClusterRoleBinding/admission-webhook-cluster-role-binding  filename="kustomize/kustomize.go:430"
INFO[0005] creating ConfigMap/admission-webhook-admission-webhook-parameters  filename="kustomize/kustomize.go:430"
INFO[0005] creating Service/admission-webhook-service    filename="kustomize/kustomize.go:430"
INFO[0005] creating Deployment/admission-webhook-deployment  filename="kustomize/kustomize.go:430"
INFO[0005] creating MutatingWebhookConfiguration/admission-webhook-mutating-webhook-configuration  filename="kustomize/kustomize.go:430"
INFO[0005] creating ServiceAccount/jupyter-web-app-service-account  filename="kustomize/kustomize.go:430"
INFO[0005] creating Role/jupyter-web-app-jupyter-notebook-role  filename="kustomize/kustomize.go:430"
INFO[0005] creating ClusterRole/jupyter-web-app-cluster-role  filename="kustomize/kustomize.go:430"
INFO[0005] creating RoleBinding/jupyter-web-app-jupyter-notebook-role-binding  filename="kustomize/kustomize.go:430"
INFO[0005] creating ClusterRoleBinding/jupyter-web-app-cluster-role-binding  filename="kustomize/kustomize.go:430"
INFO[0005] creating ConfigMap/jupyter-web-app-config     filename="kustomize/kustomize.go:430"
INFO[0005] creating ConfigMap/jupyter-web-app-parameters  filename="kustomize/kustomize.go:430"
INFO[0005] creating Service/jupyter-web-app-service      filename="kustomize/kustomize.go:430"
INFO[0005] creating Deployment/jupyter-web-app-deployment  filename="kustomize/kustomize.go:430"
INFO[0006] creating Application/jupyter-web-app          filename="kustomize/kustomize.go:430"
INFO[0006] creating VirtualService/jupyter-web-app       filename="kustomize/kustomize.go:430"
INFO[0006] creating Secret/katib-db-secrets              filename="kustomize/kustomize.go:430"
INFO[0006] creating Service/katib-db                     filename="kustomize/kustomize.go:430"
INFO[0006] creating Deployment/katib-db                  filename="kustomize/kustomize.go:430"
INFO[0006] creating PersistentVolumeClaim/katib-mysql    filename="kustomize/kustomize.go:430"
INFO[0006] creating Service/katib-manager-rest           filename="kustomize/kustomize.go:430"
INFO[0006] creating Service/katib-manager                filename="kustomize/kustomize.go:430"
INFO[0006] creating Deployment/katib-manager-rest        filename="kustomize/kustomize.go:430"
INFO[0006] creating Deployment/katib-manager             filename="kustomize/kustomize.go:430"
INFO[0007] creating CustomResourceDefinition/experiments.kubeflow.org  filename="kustomize/kustomize.go:430"
INFO[0007] creating CustomResourceDefinition/trials.kubeflow.org  filename="kustomize/kustomize.go:430"
INFO[0007] creating ServiceAccount/katib-controller      filename="kustomize/kustomize.go:430"
INFO[0007] creating ClusterRole/katib-controller         filename="kustomize/kustomize.go:430"
INFO[0007] creating ClusterRoleBinding/katib-controller  filename="kustomize/kustomize.go:430"
INFO[0007] creating ConfigMap/trial-template             filename="kustomize/kustomize.go:430"
INFO[0007] creating Secret/katib-controller              filename="kustomize/kustomize.go:430"
INFO[0007] creating Service/katib-controller             filename="kustomize/kustomize.go:430"
INFO[0007] creating Deployment/katib-controller          filename="kustomize/kustomize.go:430"
INFO[0007] creating ServiceAccount/katib-ui              filename="kustomize/kustomize.go:430"
INFO[0007] creating ClusterRole/katib-ui                 filename="kustomize/kustomize.go:430"
INFO[0007] creating ClusterRoleBinding/katib-ui          filename="kustomize/kustomize.go:430"
INFO[0007] creating ConfigMap/katib-parameters           filename="kustomize/kustomize.go:430"
INFO[0007] creating Service/katib-ui                     filename="kustomize/kustomize.go:430"
INFO[0007] creating Deployment/katib-ui                  filename="kustomize/kustomize.go:430"
INFO[0007] creating VirtualService/katib-ui              filename="kustomize/kustomize.go:430"
INFO[0007] creating ServiceAccount/metadata-ui           filename="kustomize/kustomize.go:430"
INFO[0007] creating Role/metadata-ui                     filename="kustomize/kustomize.go:430"
INFO[0007] creating RoleBinding/metadata-ui              filename="kustomize/kustomize.go:430"
INFO[0007] creating ConfigMap/metadata-ui-parameters-b6c8ghff7c  filename="kustomize/kustomize.go:430"
INFO[0007] creating Secret/metadata-db-secrets           filename="kustomize/kustomize.go:430"
INFO[0007] creating Service/metadata-db                  filename="kustomize/kustomize.go:430"
INFO[0007] creating Service/metadata-service             filename="kustomize/kustomize.go:430"
INFO[0007] creating Service/metadata-ui                  filename="kustomize/kustomize.go:430"
INFO[0007] creating Deployment/metadata-ui               filename="kustomize/kustomize.go:430"
INFO[0007] creating Deployment/metadata-db               filename="kustomize/kustomize.go:430"
INFO[0008] creating Deployment/metadata-deployment       filename="kustomize/kustomize.go:430"
INFO[0008] creating VirtualService/metadata-ui           filename="kustomize/kustomize.go:430"
INFO[0008] creating PersistentVolumeClaim/metadata-mysql  filename="kustomize/kustomize.go:430"
INFO[0008] creating ServiceAccount/metrics-collector     filename="kustomize/kustomize.go:430"
INFO[0008] creating ClusterRole/metrics-collector        filename="kustomize/kustomize.go:430"
INFO[0008] creating ClusterRoleBinding/metrics-collector  filename="kustomize/kustomize.go:430"
INFO[0008] creating ConfigMap/metrics-collector-template  filename="kustomize/kustomize.go:430"
INFO[0008] creating Service/katib-suggestion-bayesianoptimization  filename="kustomize/kustomize.go:430"
INFO[0008] creating Service/katib-suggestion-grid        filename="kustomize/kustomize.go:430"
INFO[0008] creating Service/katib-suggestion-hyperband   filename="kustomize/kustomize.go:430"
INFO[0008] creating Service/katib-suggestion-nasrl       filename="kustomize/kustomize.go:430"
INFO[0008] creating Service/katib-suggestion-random      filename="kustomize/kustomize.go:430"
INFO[0008] creating Deployment/katib-suggestion-bayesianoptimization  filename="kustomize/kustomize.go:430"
INFO[0008] creating Deployment/katib-suggestion-grid     filename="kustomize/kustomize.go:430"
INFO[0008] creating Deployment/katib-suggestion-hyperband  filename="kustomize/kustomize.go:430"
INFO[0008] creating Deployment/katib-suggestion-nasrl    filename="kustomize/kustomize.go:430"
INFO[0008] creating Deployment/katib-suggestion-random   filename="kustomize/kustomize.go:430"
INFO[0009] creating CustomResourceDefinition/notebooks.kubeflow.org  filename="kustomize/kustomize.go:430"
INFO[0009] creating ServiceAccount/notebook-controller-service-account  filename="kustomize/kustomize.go:430"
INFO[0009] creating ClusterRole/notebook-controller-role  filename="kustomize/kustomize.go:430"
INFO[0009] creating ClusterRoleBinding/notebook-controller-role-binding  filename="kustomize/kustomize.go:430"
INFO[0009] creating ConfigMap/notebook-controller-parameters  filename="kustomize/kustomize.go:430"
INFO[0009] creating Service/notebook-controller-service  filename="kustomize/kustomize.go:430"
INFO[0009] creating Deployment/notebook-controller-deployment  filename="kustomize/kustomize.go:430"
INFO[0009] creating Application/notebook-controller      filename="kustomize/kustomize.go:430"
INFO[0009] creating CustomResourceDefinition/pytorchjobs.kubeflow.org  filename="kustomize/kustomize.go:430"
INFO[0010] creating ServiceAccount/pytorch-operator      filename="kustomize/kustomize.go:430"
INFO[0010] creating ClusterRole/pytorch-operator         filename="kustomize/kustomize.go:430"
INFO[0010] creating ClusterRoleBinding/pytorch-operator  filename="kustomize/kustomize.go:430"
INFO[0010] creating ConfigMap/pytorch-operator-config    filename="kustomize/kustomize.go:430"
INFO[0010] creating Service/pytorch-operator             filename="kustomize/kustomize.go:430"
INFO[0010] creating Deployment/pytorch-operator          filename="kustomize/kustomize.go:430"
INFO[0011] creating ServiceAccount/spartakus             filename="kustomize/kustomize.go:430"
INFO[0011] creating ClusterRole/spartakus                filename="kustomize/kustomize.go:430"
INFO[0011] creating ClusterRoleBinding/spartakus         filename="kustomize/kustomize.go:430"
INFO[0011] creating ConfigMap/spartakus-parameters       filename="kustomize/kustomize.go:430"
INFO[0011] creating Deployment/spartakus-volunteer       filename="kustomize/kustomize.go:430"
INFO[0011] creating ConfigMap/parameters-dgd4h256h5      filename="kustomize/kustomize.go:430"
INFO[0011] creating Service/tensorboard                  filename="kustomize/kustomize.go:430"
INFO[0011] creating Deployment/tensorboard               filename="kustomize/kustomize.go:430"
INFO[0011] creating VirtualService/tensorboard           filename="kustomize/kustomize.go:430"
INFO[0012] creating CustomResourceDefinition/tfjobs.kubeflow.org  filename="kustomize/kustomize.go:430"
INFO[0012] creating ServiceAccount/tf-job-dashboard      filename="kustomize/kustomize.go:430"
INFO[0012] creating ServiceAccount/tf-job-operator       filename="kustomize/kustomize.go:430"
INFO[0012] creating ClusterRole/tf-job-dashboard         filename="kustomize/kustomize.go:430"
INFO[0012] creating ClusterRole/tf-job-operator          filename="kustomize/kustomize.go:430"
INFO[0012] creating ClusterRoleBinding/tf-job-dashboard  filename="kustomize/kustomize.go:430"
INFO[0012] creating ClusterRoleBinding/tf-job-operator   filename="kustomize/kustomize.go:430"
INFO[0012] creating ConfigMap/parameters-dgd4h256h5      filename="kustomize/kustomize.go:430"
INFO[0012] creating ConfigMap/tf-job-operator-config     filename="kustomize/kustomize.go:430"
INFO[0012] creating Service/tf-job-dashboard             filename="kustomize/kustomize.go:430"
INFO[0012] creating Service/tf-job-operator              filename="kustomize/kustomize.go:430"
INFO[0012] creating Deployment/tf-job-dashboard          filename="kustomize/kustomize.go:430"
INFO[0012] creating Deployment/tf-job-operator           filename="kustomize/kustomize.go:430"
INFO[0012] creating VirtualService/tf-job-dashboard      filename="kustomize/kustomize.go:430"
INFO[0013] creating ServiceAccount/ml-pipeline           filename="kustomize/kustomize.go:430"
INFO[0013] creating Role/ml-pipeline                     filename="kustomize/kustomize.go:430"
INFO[0013] creating RoleBinding/ml-pipeline              filename="kustomize/kustomize.go:430"
INFO[0013] creating Service/ml-pipeline                  filename="kustomize/kustomize.go:430"
INFO[0013] creating Deployment/ml-pipeline               filename="kustomize/kustomize.go:430"
INFO[0013] creating ConfigMap/pipeline-minio-parameters  filename="kustomize/kustomize.go:430"
INFO[0013] creating Secret/mlpipeline-minio-artifact     filename="kustomize/kustomize.go:430"
INFO[0013] creating Service/minio-service                filename="kustomize/kustomize.go:430"
INFO[0014] creating Deployment/minio                     filename="kustomize/kustomize.go:430"
INFO[0014] creating PersistentVolumeClaim/minio-pv-claim  filename="kustomize/kustomize.go:430"
INFO[0014] creating ConfigMap/pipeline-mysql-parameters  filename="kustomize/kustomize.go:430"
INFO[0014] creating Service/mysql                        filename="kustomize/kustomize.go:430"
INFO[0014] creating Deployment/mysql                     filename="kustomize/kustomize.go:430"
INFO[0015] creating PersistentVolumeClaim/mysql-pv-claim  filename="kustomize/kustomize.go:430"
INFO[0015] creating ServiceAccount/ml-pipeline-persistenceagent  filename="kustomize/kustomize.go:430"
INFO[0015] creating ClusterRole/ml-pipeline-persistenceagent  filename="kustomize/kustomize.go:430"
INFO[0015] creating ClusterRoleBinding/ml-pipeline-persistenceagent  filename="kustomize/kustomize.go:430"
INFO[0015] creating Deployment/ml-pipeline-persistenceagent  filename="kustomize/kustomize.go:430"
INFO[0016] creating ServiceAccount/pipeline-runner       filename="kustomize/kustomize.go:430"
INFO[0016] creating ClusterRole/pipeline-runner          filename="kustomize/kustomize.go:430"
INFO[0016] creating ClusterRoleBinding/pipeline-runner   filename="kustomize/kustomize.go:430"
INFO[0017] creating ServiceAccount/ml-pipeline-ui        filename="kustomize/kustomize.go:430"
INFO[0017] creating Role/ml-pipeline-ui                  filename="kustomize/kustomize.go:430"
INFO[0017] creating RoleBinding/ml-pipeline-ui           filename="kustomize/kustomize.go:430"
INFO[0017] creating ConfigMap/ui-parameters-hb792fcf5d   filename="kustomize/kustomize.go:430"
INFO[0017] creating Service/ml-pipeline-tensorboard-ui   filename="kustomize/kustomize.go:430"
INFO[0017] creating Service/ml-pipeline-ui               filename="kustomize/kustomize.go:430"
INFO[0017] creating Deployment/ml-pipeline-ui            filename="kustomize/kustomize.go:430"
INFO[0017] creating VirtualService/ml-pipeline-tensorboard-ui  filename="kustomize/kustomize.go:430"
INFO[0017] creating VirtualService/ml-pipeline-ui        filename="kustomize/kustomize.go:430"
INFO[0017] creating CustomResourceDefinition/viewers.kubeflow.org  filename="kustomize/kustomize.go:430"
INFO[0017] creating ServiceAccount/ml-pipeline-viewer-crd-service-account  filename="kustomize/kustomize.go:430"
INFO[0017] creating ClusterRole/ml-pipeline-viewer-controller-role  filename="kustomize/kustomize.go:430"
INFO[0017] creating ClusterRoleBinding/ml-pipeline-viewer-crd-role-binding  filename="kustomize/kustomize.go:430"
INFO[0018] creating Deployment/ml-pipeline-viewer-controller-deployment  filename="kustomize/kustomize.go:430"
E0810 20:09:32.999888   31811 memcache.go:135] couldn't get resource list for kubeflow.org/v1beta1: the server could not find the requested resource
INFO[0018] creating CustomResourceDefinition/scheduledworkflows.kubeflow.org  filename="kustomize/kustomize.go:430"
INFO[0018] creating ServiceAccount/ml-pipeline-scheduledworkflow  filename="kustomize/kustomize.go:430"
INFO[0018] creating Role/ml-pipeline-scheduledworkflow   filename="kustomize/kustomize.go:430"
INFO[0018] creating ClusterRoleBinding/ml-pipeline-scheduledworkflow  filename="kustomize/kustomize.go:430"
INFO[0019] creating Deployment/ml-pipeline-scheduledworkflow  filename="kustomize/kustomize.go:430"
E0810 20:09:34.026474   31811 memcache.go:135] couldn't get resource list for kubeflow.org/v1beta1: the server could not find the requested resource
INFO[0019] creating CustomResourceDefinition/profiles.kubeflow.org  filename="kustomize/kustomize.go:430"
INFO[0019] creating ServiceAccount/profiles-controller-service-account  filename="kustomize/kustomize.go:430"
INFO[0019] creating ServiceAccount/profiles-default-service-account  filename="kustomize/kustomize.go:430"
INFO[0019] creating Role/profiles-default-role           filename="kustomize/kustomize.go:430"
INFO[0019] creating RoleBinding/profiles-default-role-binding  filename="kustomize/kustomize.go:430"
INFO[0019] creating ClusterRoleBinding/profiles-cluster-role-binding  filename="kustomize/kustomize.go:430"
INFO[0019] creating ConfigMap/profiles-profiles-parameters-t4hd69826h  filename="kustomize/kustomize.go:430"
INFO[0020] creating Service/profiles-kfam                filename="kustomize/kustomize.go:430"
INFO[0020] creating Deployment/profiles-deployment       filename="kustomize/kustomize.go:430"
INFO[0020] creating VirtualService/kfam                  filename="kustomize/kustomize.go:430"
INFO[0021] creating CustomResourceDefinition/seldondeployments.machinelearning.seldon.io  filename="kustomize/kustomize.go:430"
INFO[0021] creating ServiceAccount/seldon-manager        filename="kustomize/kustomize.go:430"
INFO[0021] creating ClusterRole/seldon-operator-manager-role  filename="kustomize/kustomize.go:430"
INFO[0021] creating ClusterRoleBinding/seldon-operator-manager-rolebinding  filename="kustomize/kustomize.go:430"
INFO[0021] creating Secret/seldon-operator-webhook-server-secret  filename="kustomize/kustomize.go:430"
INFO[0021] creating Service/seldon-operator-controller-manager-service  filename="kustomize/kustomize.go:430"
INFO[0021] creating StatefulSet/seldon-operator-controller-manager  filename="kustomize/kustomize.go:430"
INFO[0021] creating Application/seldon-core-operator     filename="kustomize/kustomize.go:430"

3 Summary

Kubeflow 的部署虽然可以通过脚本一键部署，但是里面涉及很多模块，也涉及很多 K8S 的知识，所以需要对这些都有一定的了解，才容易部署成功，如果有任何问题，欢迎邮箱联系 runzhliu@163.com。

Kubeflow 还是很多值得探索的地方的，所以后面还会有更多的分析文章。

runzhliu
关注
专栏目录
kubeflow安装包
12-05
kubeflow的安装包
谷歌发布Kubeflow 0.1版本，基于Kubernetes的机器学习工具包
Docker的专栏
05-06 1506
自从 Google 发布开源容器编排工具——Kubernetes 以来，我们已经见证了其以各种方式遍地开花的景象。随着 Kubernetes 越来越受欢迎，许多辅助项目也...
Kubeflow 国内一键安装文件教程
最新发布
gitblog_00765的博客
09-25 810
Kubeflow 国内一键安装文件教程 kubeflow-manifests kubeflow国内一键安装文件 项目地址: https://gitcode.com/gh_mirrors/ku/kubeflow-manifests ...
kubeflow实践（1.5.1版本）
verastart的博客
12-21 1160
这个方法主要是让临时修改的配置生效，但也存在部分deploy在修改后出现了两个replica，有对应的两个pod被启动，因此不推荐。建议找到对应pod的yaml，delete之后，加入selector使得pod运行的节点可控，之后再apply。如果集群内添加了存储服务器，已经部署了对应的PVC，本次安装kubeflow时，只需要将对应的sc设为default，然后让pvc重启创建，自动关联就可以了（重启创建pvc需要先delete对应的yaml）出现错误：ImagePullBackOff。
kubeflow0.6.2版本搭建
Tilyp的博客
10-14 2107
目录 基础环境： 安装kubeflow 基础环境： 系统：centos7.6 kubernetes：1.14 内存需求： kubernetes > 1.11 cpu > 4 storage > 50G memory > 12G 安装ku...
k8s1.15部署kubeflow0.6
yezhizhen1996的博客
08-09 552
再部署完成k8s之后，确定k8s集群节点都能正常运行，便可进行kubeflow的配置。https://www.kubeflow.org/docs/started/k8s/overview/#minimum-system-requirements（查询k8s与kubeflow版本兼容问题）。 由于kubeflow使用的镜像较多，以及一些国内镜像库的封闭，导致镜像不是很容易拉取齐全，所以本次实验使用了的一键安装项目。 一.安装kubeflow $ sudo apt get install git
H-JTAG-V0.6.0.rar_H-JTAG V0.6.0
09-22
《H-JTAG V0.6.0：嵌入式开发利器》 在IT行业中，嵌入式系统扮演着至关重要的角色，特别是在各种智能设备和工业控制系统中。H-JTAG V0.6.0是一个专为Windows CE（Windows Embedded Compact）操作系统设计的编译器和...
K8S部署metrics-server-v0.6.0资源清单及镜像文件
01-24
原文链接：https://blog.csdn.net/m0_37814112/article/details/122666995 说明： 1、支持支持的Kubernetes版本1.19+ 2、包含yaml文件和镜像，直接apply即可部署完成
ShadowEditor 3D场景编辑器 v0.6.0.zip
04-10
《ShadowEditor 3D场景编辑器 v0.6.0：开启三维创作之旅》 ShadowEditor，这是一款专为3D场景设计者打造的专业编辑工具，版本号v0.6.0，它提供了丰富的功能，旨在帮助用户高效地创建、编辑和管理他们的3D场景。在...
1_ifcopenshell-python-37-v0.6.0-ed98983-win64_python读取ifc_Python
09-29
标题中的“1_ifcopenshell-python-37-v0.6.0-ed98983-win64_python读取ifc_Python”表明这是一个基于Python的IFC（Industry Foundation Classes）解析工具，名为ifcopenshell，适用于Python 3.7版本，且此版本号为v...
.net混淆加壳confuserex v0.6.0汉化版
01-06
ConfuserEx 支持 .NET 框架 2.0 - 4.5 和 Mono (还有其他 .NET 框架)： Symbol 重命名 WPF/BAML 重命名 Control flow obfuscation Method reference hiding Anti debuggers/profilers Anti memory dumping ...
Debian部署kubeflow v0.6.0-rc.1
urmsone
07-23 827
环境说明 kubernetes v1.10.11 kubeflow v0.6.0.rc.1 版本介绍： 1）较之v0.5.1,v0.6.0使用了istio组件 2）包管理器也从ksonet转成了kustomize 3）还需要提供要给类似于公有云服务器上面的负载均衡器 依赖： 1）kubernetes 2）kfctl 3）负载均衡器 使用开源的负载均衡器，部署流程如下： 执行 kubectl app...
kubeflow v1.8 离线安装
qq836825331的专栏
03-19 2788
Kubeflow 项目致力于使 Kubernetes 上的机器学习 (ML) 工作流程的部署变得简单、可移植且可扩展。Kubeflow包含了一系列机器学习相关组件，并帮助在k8s上开发、部署机器学习应用。下图显示了主要的 Kubeflow 组件，涵盖了 Kubernetes 上的 ML 生命周期的每个步骤。
Kubeflow（1.6.1）部署
丰耳的博客
02-17 4074
基于K8s-v1.22.8的kubeflow-1.6.1部署
kubernetes集群使用GPU及安装kubeflow1.0.RC
mockmockdone的博客
07-27 1410
kubernetes集群使用GPU及安装kubeflow1.0.RC操作步骤 安装显卡驱动 安装CUDA sudo yum-config-manager --add-repo http://developer.download.nvidia.com/compute/cuda/repos/rhel7/x86_64/cuda-rhel7.repo sudo yum clean all sudo yum...
katib独立部署外部访问方式配置
burger1221的博客
03-04 620
前提： 1、已有 k8s 集群环境，我的k8s版本是1.14.2 2、按照官网的非kubeflow方式部署了katib： 执行端口转发命令后只能本机访问 kubectl port-forward svc/katib-ui -n kubeflow 8080:80 & 配置外网可访问： 1、查询katib-ui服务 [root@master ~]# kubectl get...
机器学习平台kubeflow搭建
piaobeizu2011的专栏
11-17 3641
机器学习平台kubeflow搭建 文章目录机器学习平台kubeflow搭建前言一、搭建流程1. k0s 构建k8s平台2. 准备pv3. 安装kubeflow二、问题总结 前言 首先来一段官网的介绍：Kubeflow项目致力于使Kubernetes上机器学习(ML)工作流的部署变得简单、可移植和可扩展。我们的目标不是重新创建其他服务，而是提供一种直接的方式，为ML部署最佳的开源系统到不同的基础设施。无论你在哪里运行Kubernetes，你都应该能够运行Kubeflow。(谷歌翻译) kubefl
kubeflow简介及安装
赵英超的博客
03-05 7219
1 背景 Kubernetes 本来是一个用来管理无状态应用的容器平台，但是在近两年，有越来越多的公司用它来运行各种各样的工作负载，尤其是机器学习炼丹。各种 AI 公司或者互联网公司的 AI 部门都会尝试在 Kubernetes 上运行 TensorFlow，Caffe，MXNet 等等分布式学习的任务，这为 Kubernetes 带来了新的挑战。 首先，分布式的机器学习任务一般会涉及参数服务...
kubeflow_Kubeflow简介
cumj63710的博客
07-06 2460
kubeflow 模型构建和训练只是支持机器学习（ML）工作流的一小部分。 您需要解决的其他问题包括将数据移植到可访问的格式和位置； 数据清理和特征工程； 分析您训练有素的模型； 管理模型版本控制； 可伸缩地为您训练有素的模型服务； 并避免训练/服务偏斜。 当工作流需要可移植且始终如一地可重复并且需要集成许多活动部件时，尤其如此。 此外，大多数这些活动会在多个工作流中重复发生，也许只是使用不...
H-JTAG V0.6.0：Wince嵌入式编译器的应用与驱动开发
资源摘要信息:"H-JTAG V0.6.0是一款专为wince嵌入式系统设计的编译器，它提供了一个强大的环境，用于编写和调试按键、串口等驱动程序。这款工具对于嵌入式开发者来说是一个极其重要的资源，因为它可以极大地简化和...
评论 1
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值