karmada项目地址
Karmada 是对 Kubefed 项目的延续,它继承了来自 kubefed 中的一些概念,目前也处于积极开发和维护中,这也是目前社区中比较活跃和成熟的集群联邦项目。
架构图
Karmada 本身与 Kubernetes 架构非常相似
背景:
已有两个k8s集群
default
remote
安装:
安装kubectl plugin
wget https://gh-proxy.du33169.workers.dev/https://github.com/karmada-io/karmada/releases/download/v1.0.2/kubectl-karmada-linux-amd64.tgz
tar -zxvf kubectl-karmada-linux-amd64.tgz
mv kubectl-karmada /bin/
kubectl karmada version
在k8s集群default上安装karmada
需要指定镜像,默认使用国外源安装会失败,初始化安装karmada,
kubectl karmada init --etcd-image registry.aliyuncs.com/google_containers/etcd:3.5.1-0 --karmada-apiserver-image registry.aliyuncs.com/google_containers/kube-apiserver:v1.21.7 --karmada-kube-controller-manager-image registry.aliyuncs.com/google_containers/kube-controller-manager:v1.21.7
安装完成:
安装完后按以下提示创建联邦集群
Karmada is installed successfully.
Register Kubernetes cluster to Karmada control plane.
Register cluster with 'Push' mode
Step 1: Use kubectl karmada join to register the cluster to Karmada control panel. --cluster-kubeconfig is members kubeconfig.
(In karmada)~# MEMBER_CLUSTER_NAME=`cat ~/.kube/config | grep current-context | sed 's/: /\n/g'| sed '1d'`
(In karmada)~# kubectl karmada --kubeconfig /etc/karmada/karmada-apiserver.config join ${MEMBER_CLUSTER_NAME} --cluster-kubeconfig=$HOME/.kube/config
Step 2: Show members of karmada
(In karmada)~# kubectl --kubeconfig /etc/karmada/karmada-apiserver.config get clusters
Register cluster with 'Pull' mode
Step 1: Send karmada kubeconfig and karmada-agent.yaml to member kubernetes
(In karmada)~# scp /etc/karmada/karmada-apiserver.config /etc/karmada/karmada-agent.yaml {member kubernetes}:~
Step 2: Create karmada kubeconfig secret
Notice:
Cross-network, need to change the config server address.
(In member kubernetes)~# kubectl create ns karmada-system
(In member kubernetes)~# kubectl create secret generic karmada-kubeconfig --from-file=karmada-kubeconfig=/root/karmada-apiserver.config -n karmada-system
Step 3: Create karmada agent
(In member kubernetes)~# MEMBER_CLUSTER_NAME="demo"
(In member kubernetes)~# sed -i "s/{member_cluster_name}/${MEMBER_CLUSTER_NAME}/g" karmada-agent.yaml
(In member kubernetes)~# kubectl create -f karmada-agent.yaml
Step 4: Show members of karmada
(In karmada)~# kubectl --kubeconfig /etc/karmada/karmada-apiserver.config get clusters
本地集群加入联邦cluster
MEMBER_CLUSTER_NAME=`cat ~/.kube/config | grep current-context | sed 's/: /\n/g'| sed '1d'`
kubectl karmada --kubeconfig /etc/karmada/karmada-apiserver.config join ${MEMBER_CLUSTER_NAME} --cluster-kubeconfig=$HOME/.kube/config
另一集群注册到联邦
拷贝文件到另一集群
scp /etc/karmada/karmada-apiserver.config /etc/karmada/karmada-agent.yaml 172.31.0.8:/root
#172.31.0.8执行以下命令,(remote cluster)
kubectl create ns karmada-system
kubectl create secret generic karmada-kubeconfig --from-file=karmada-kubeconfig=/root/karmada-apiserver.config -n karmada-system
MEMBER_CLUSTER_NAME="remote"
sed -i "s/{member_cluster_name}/${MEMBER_CLUSTER_NAME}/g" karmada-agent.yaml
kubectl create -f karmada-agent.yaml
kubectl --kubeconfig /etc/karmada/karmada-apiserver.config get clusters
push模式: 由karamda控制端监听各个子集群状态,并将组合的资源分发到各个子集群
pull模式: 在子集群部署karmada-agent,用于监听与本集群相关的资源,并维护资源的生命周期
添加别名
alias karmadactl='kubectl --kubeconfig /etc/karmada/karmada-apiserver.config'
测试:
创建常规资源,创建分发规则
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginxtest
labels:
app: nginx
spec:
replicas: 2
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- image: nginx
name: nginx
---
apiVersion: policy.karmada.io/v1alpha1
kind: PropagationPolicy
metadata:
name: nginx-propagation
spec:
resourceSelectors:
- apiVersion: apps/v1
kind: Deployment
name: nginxtest
placement:
clusterAffinity:
clusterNames:
- default
- remote
replicaScheduling:
replicaDivisionPreference: Weighted
replicaSchedulingType: Divided
weightPreference:
staticWeightList:
- targetCluster:
clusterNames:
- default
weight: 1
- targetCluster:
clusterNames:
- remote
weight: 1
---
apiVersion: v1
kind: Service
metadata:
labels:
app: nginx
name: nginxtest
spec:
type: ClusterIP
ports:
- port: 80
targetPort: 80
name: http
selector:
app: nginx
---
apiVersion: policy.karmada.io/v1alpha1
kind: PropagationPolicy
metadata:
name: nginx-svc-propagation
spec:
resourceSelectors:
- apiVersion: v1
kind: Service
name: nginxtest
placement:
clusterAffinity:
clusterNames:
- default
- remote
replicaScheduling:
replicaDivisionPreference: Weighted
replicaSchedulingType: Divided
weightPreference:
staticWeightList:
- targetCluster:
clusterNames:
- default
weight: 1
- targetCluster:
clusterNames:
- remote
weight: 1
停止集群remote的kube-api,验证故障转移,状态为False后,default上的deployments nginxtest扩容为两个副本
迁移现有资源
karmada控制平台创建namespace
执行kubectl karmada promote
创建分发规则
例子:
xxdeployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: xxnginx
namespace: xx
labels:
app: nginx
spec:
replicas: 2
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- image: nginx
name: nginx
default 集群操作:
kubectl apply -f xxdeployment.yaml
karmadactl create ns xx
Promote resources from legacy clusters to karmada control plane
kubectl karmada --kubeconfig /etc/karmada/karmada-apiserver.config promote deployment xxnginx -n xx -c default
修改分发规则
apiVersion: policy.karmada.io/v1alpha1
kind: PropagationPolicy
metadata:
name: nginx-propagation
spec:
resourceSelectors:
- apiVersion: apps/v1
kind: Deployment
name: xxnginx
namespace: xx
placement:
clusterAffinity:
clusterNames:
- default
- remote
replicaScheduling:
replicaDivisionPreference: Weighted
replicaSchedulingType: Divided
weightPreference:
staticWeightList:
- targetCluster:
clusterNames:
- default
weight: 1
- targetCluster:
clusterNames:
- remote
weight: 1