2023-03-01T02:52:07 info Start to create a new cluster: master [172.16.13.215], worker [], registry 172.16.13.215
2023-03-01T02:52:07 info Executing pipeline Check in CreateProcessor.
2023-03-01T02:52:07 info checker:hostname [172.16.13.215:22]
2023-03-01T02:52:07 info checker:timeSync [172.16.13.215:22]
2023-03-01T02:52:07 info Executing pipeline PreProcess in CreateProcessor.
Resolving "labring/kubernetes" using unqualified-search registries (/etc/containers/registries.conf)
Trying to pull docker.io/labring/kubernetes:v1.25.0...
Getting image source signatures
Copying blob b57fa1fe3ade done
Copying blob 88af23a6a8b4 done
Copying blob 4013845ba3fe done
Copying blob 0ad330619635 done
Copying config e910e0d38d done
Writing manifest to image destination
Storing signatures
Resolving "labring/helm" using unqualified-search registries (/etc/containers/registries.conf)
Trying to pull docker.io/labring/helm:v3.8.2...
Getting image source signatures
Copying blob 53a6eade9e7e done
Copying config 1123e8b4b4 done
Writing manifest to image destination
Storing signatures
Resolving "labring/calico" using unqualified-search registries (/etc/containers/registries.conf)
Trying to pull docker.io/labring/calico:v3.24.1...
Getting image source signatures
Copying blob f9de59270f64 done
Copying config e2122fc58f done
Writing manifest to image destination
Storing signatures
2023-03-01T02:54:18 info Executing pipeline RunConfig in CreateProcessor.
2023-03-01T02:54:18 info Executing pipeline MountRootfs in CreateProcessor.
2023-03-01T02:54:36 info Executing pipeline MirrorRegistry in CreateProcessor.
2023-03-01T02:54:36 info Executing pipeline Bootstrap in CreateProcessor
/var/lib/sealos/data/default/rootfs /var/lib/sealos/data/default/rootfs/scripts
/var/lib/sealos/data/default/rootfs/scripts
which: no docker in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin)
WARN [2023-03-01 02:54:37] >> Replace disable_apparmor = false to disable_apparmor = true
INFO [2023-03-01 02:54:37] >> check root,port,cri success
Created symlink from /etc/systemd/system/multi-user.target.wants/containerd.service to /etc/systemd/system/containerd.service.
INFO [2023-03-01 02:54:40] >> Health check containerd!
INFO [2023-03-01 02:54:40] >> containerd is running
INFO [2023-03-01 02:54:40] >> init containerd success
Created symlink from /etc/systemd/system/multi-user.target.wants/image-cri-shim.service to /etc/systemd/system/image-cri-shim.service.
INFO [2023-03-01 02:54:40] >> Health check image-cri-shim!
INFO [2023-03-01 02:54:40] >> image-cri-shim is running
INFO [2023-03-01 02:54:40] >> init shim success
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
* Applying /usr/lib/sysctl.d/00-system.conf ...
net.bridge.bridge-nf-call-ip6tables = 0
net.bridge.bridge-nf-call-iptables = 0
net.bridge.bridge-nf-call-arptables = 0
* Applying /usr/lib/sysctl.d/10-default-yama-scope.conf ...
kernel.yama.ptrace_scope = 0
* Applying /usr/lib/sysctl.d/50-default.conf ...
kernel.sysrq = 16
kernel.core_uses_pid = 1
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.default.promote_secondaries = 1
net.ipv4.conf.all.promote_secondaries = 1
fs.protected_hardlinks = 1
fs.protected_symlinks = 1
* Applying /etc/sysctl.d/99-sysctl.conf ...
* Applying /etc/sysctl.d/k8s.conf ...
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.conf.all.rp_filter = 0
* Applying /etc/sysctl.conf ...
net.ipv4.ip_forward = 1
SELINUX=enforcing
Created symlink from /etc/systemd/system/multi-user.target.wants/kubelet.service to /etc/systemd/system/kubelet.service.
INFO [2023-03-01 02:54:40] >> init kubelet success
INFO [2023-03-01 02:54:40] >> init rootfs success
Created symlink from /etc/systemd/system/multi-user.target.wants/registry.service to /etc/systemd/system/registry.service.
INFO [2023-03-01 02:54:42] >> Health check registry!
INFO [2023-03-01 02:54:42] >> registry is running
INFO [2023-03-01 02:54:42] >> init registry success
2023-03-01T02:54:42 info Executing pipeline Init in CreateProcessor.
2023-03-01T02:54:42 info start to copy kubeadm config to master0
2023-03-01T02:54:43 info start to generate cert and kubeConfig...
2023-03-01T02:54:43 info start to generator cert and copy to masters...
2023-03-01T02:54:43 info apiserver altNames : {map[apiserver.cluster.local:apiserver.cluster.local kubernetes:kubernetes kubernetes.default:kubernetes.default kubernetes.default.svc:kubernetnetes.default.svc.cluster.local:kubernetes.default.svc.cluster.local localhost:localhost localhost.localdomain:localhost.localdomain] map[10.103.97.2:10.103.97.2 10.96.0.1:10.96.0.1 127.0.0.3.215:172.16.13.215]}
2023-03-01T02:54:43 info Etcd altnames : {map[localhost:localhost localhost.localdomain:localhost.localdomain] map[127.0.0.1:127.0.0.1 172.16.13.215:172.16.13.215 ::1:::1]}, commonName : loc
2023-03-01T02:54:45 info start to copy etc pki files to masters
2023-03-01T02:54:45 info start to copy etc pki files to masters
2023-03-01T02:54:45 info start to create kubeconfig...
2023-03-01T02:54:46 info start to copy kubeconfig files to masters
2023-03-01T02:54:46 info start to copy static files to masters
2023-03-01T02:54:46 info start to init master0...
2023-03-01T02:54:46 info registry auth in node 172.16.13.215:22
2023-03-01T02:54:46 info domain sealos.hub:172.16.13.215 append success
2023-03-01T02:54:46 info domain apiserver.cluster.local:172.16.13.215 append success
W0301 02:54:46.364307 16781 initconfiguration.go:119] Usage of CRI endpoints without URL scheme is deprecated and can cause kubelet errors in the future. Automatically prepending scheme "uet" with value "/run/containerd/containerd.sock". Please update your configuration!
W0301 02:54:46.364369 16781 utils.go:69] The recommended value for "healthzBindAddress" in "KubeletConfiguration" is: 127.0.0.1; the provided value is: 0.0.0.0
[init] Using Kubernetes version: v1.25.0
[preflight] Running pre-flight checks
[WARNING FileExisting-socat]: socat not found in system path
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Using existing kubeconfig file: "/etc/kubernetes/admin.conf"
[kubeconfig] Using existing kubeconfig file: "/etc/kubernetes/kubelet.conf"
W0301 02:55:26.116550 16781 kubeconfig.go:249] a kubeconfig file "/etc/kubernetes/controller-manager.conf" exists already but has an unexpected API Server URL: expected: https://172.16.13.://apiserver.cluster.local:6443
[kubeconfig] Using existing kubeconfig file: "/etc/kubernetes/controller-manager.conf"
W0301 02:55:26.187415 16781 kubeconfig.go:249] a kubeconfig file "/etc/kubernetes/scheduler.conf" exists already but has an unexpected API Server URL: expected: https://172.16.13.215:6443,ver.cluster.local:6443
[kubeconfig] Using existing kubeconfig file: "/etc/kubernetes/scheduler.conf"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[apiclient] All control plane components are healthy after 8.013978 seconds
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config" in namespace kube-system with the configuration for the kubelets in the cluster
[upload-certs] Skipping phase. Please see --upload-certs
[mark-control-plane] Marking the node localhost.localdomain as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balanc
[mark-control-plane] Marking the node localhost.localdomain as control-plane by adding the taints [node-role.kubernetes.io/control-plane:NoSchedule]
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
[bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes
[bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
[kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy
Your Kubernetes control-plane has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
Alternatively, if you are the root user, you can run:
export KUBECONFIG=/etc/kubernetes/admin.conf
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
https://kubernetes.io/docs/concepts/cluster-administration/addons/
You can now join any number of control-plane nodes by copying certificate authorities
and service account keys on each node and then running the following as root:
kubeadm join apiserver.cluster.local:6443 --token <value withheld> \
--discovery-token-ca-cert-hash sha256:18f9c4d86b98ba5737da911c23200b01644167521f8e526d00c8fb6b2e3491c7 \
--control-plane --certificate-key <value withheld>
Then you can join any number of worker nodes by running the following on each as root:
kubeadm join apiserver.cluster.local:6443 --token <value withheld> \
--discovery-token-ca-cert-hash sha256:18f9c4d86b98ba5737da911c23200b01644167521f8e526d00c8fb6b2e3491c7
2023-03-01T02:55:36 info Executing pipeline Join in CreateProcessor.
2023-03-01T02:55:36 info start to get kubernetes token...
2023-03-01T02:55:40 info Executing pipeline RunGuest in CreateProcessor.
namespace/tigera-operator created
NAME: calico
LAST DEPLOYED: Wed Mar 1 02:55:43 2023
NAMESPACE: tigera-operator
STATUS: deployed
REVISION: 1
TEST SUITE: None
2023-03-01T02:55:46 info succeeded in creating a new cluster, enjoy it!
2023-03-01T02:55:46 info
___ ___ ___ ___ ___ ___
/\ \ /\ \ /\ \ /\__\ /\ \ /\ \
/::\ \ /::\ \ /::\ \ /:/ / /::\ \ /::\ \
/:/\ \ \ /:/\:\ \ /:/\:\ \ /:/ / /:/\:\ \ /:/\ \ \
_\:\~\ \ \ /::\~\:\ \ /::\~\:\ \ /:/ / /:/ \:\ \ _\:\~\ \ \
/\ \:\ \ \__\ /:/\:\ \:\__\ /:/\:\ \:\__\ /:/__/ /:/__/ \:\__\ /\ \:\ \ \__\
\:\ \:\ \/__/ \:\~\:\ \/__/ \/__\:\/:/ / \:\ \ \:\ \ /:/ / \:\ \:\ \/__/
\:\ \:\__\ \:\ \:\__\ \::/ / \:\ \ \:\ /:/ / \:\ \:\__\
\:\/:/ / \:\ \/__/ /:/ / \:\ \ \:\/:/ / \:\/:/ /
\::/ / \:\__\ /:/ / \:\__\ \::/ / \::/ /
\/__/ \/__/ \/__/ \/__/ \/__/ \/__/
Website :https://www.sealos.io/
Address :github.com/labring/sealos
BuildVersion: 4.1.4-bf121904