k8s联邦 karmada 安装

karmada项目地址
Karmada 是对 Kubefed 项目的延续，它继承了来自 kubefed 中的一些概念，目前也处于积极开发和维护中，这也是目前社区中比较活跃和成熟的集群联邦项目。

架构图

Karmada 本身与 Kubernetes 架构非常相似

背景：

已有两个k8s集群
default
remote

安装：

安装kubectl plugin

wget https://gh-proxy.du33169.workers.dev/https://github.com/karmada-io/karmada/releases/download/v1.0.2/kubectl-karmada-linux-amd64.tgz
tar -zxvf kubectl-karmada-linux-amd64.tgz
mv kubectl-karmada /bin/
kubectl karmada version

在k8s集群default上安装karmada
需要指定镜像，默认使用国外源安装会失败，初始化安装karmada，

kubectl karmada init --etcd-image registry.aliyuncs.com/google_containers/etcd:3.5.1-0 --karmada-apiserver-image registry.aliyuncs.com/google_containers/kube-apiserver:v1.21.7 --karmada-kube-controller-manager-image registry.aliyuncs.com/google_containers/kube-controller-manager:v1.21.7

安装完成：

安装完后按以下提示创建联邦集群

Karmada is installed successfully.

Register Kubernetes cluster to Karmada control plane.

Register cluster with 'Push' mode
                                                                                                                                                                             
Step 1: Use kubectl karmada join to register the cluster to Karmada control panel. --cluster-kubeconfig is members kubeconfig.
(In karmada)~# MEMBER_CLUSTER_NAME=`cat ~/.kube/config  | grep current-context | sed 's/: /\n/g'| sed '1d'`
(In karmada)~# kubectl karmada --kubeconfig /etc/karmada/karmada-apiserver.config  join ${MEMBER_CLUSTER_NAME} --cluster-kubeconfig=$HOME/.kube/config

Step 2: Show members of karmada
(In karmada)~# kubectl  --kubeconfig /etc/karmada/karmada-apiserver.config get clusters


Register cluster with 'Pull' mode

Step 1:  Send karmada kubeconfig and karmada-agent.yaml to member kubernetes
(In karmada)~# scp /etc/karmada/karmada-apiserver.config /etc/karmada/karmada-agent.yaml {member kubernetes}:~
                                                                                                                                                                             
Step 2:  Create karmada kubeconfig secret
 Notice:
   Cross-network, need to change the config server address.
(In member kubernetes)~#  kubectl create ns karmada-system
(In member kubernetes)~#  kubectl create secret generic karmada-kubeconfig --from-file=karmada-kubeconfig=/root/karmada-apiserver.config  -n karmada-system                  

Step 3: Create karmada agent
(In member kubernetes)~#  MEMBER_CLUSTER_NAME="demo"
(In member kubernetes)~#  sed -i "s/{member_cluster_name}/${MEMBER_CLUSTER_NAME}/g" karmada-agent.yaml
(In member kubernetes)~#  kubectl create -f karmada-agent.yaml
                                                                                                                                                                             
Step 4: Show members of karmada                                                                                                                                              
(In karmada)~# kubectl  --kubeconfig /etc/karmada/karmada-apiserver.config get clusters

本地集群加入联邦cluster

MEMBER_CLUSTER_NAME=`cat ~/.kube/config  | grep current-context | sed 's/: /\n/g'| sed '1d'`
kubectl karmada --kubeconfig /etc/karmada/karmada-apiserver.config  join ${MEMBER_CLUSTER_NAME} --cluster-kubeconfig=$HOME/.kube/config

另一集群注册到联邦
拷贝文件到另一集群

scp /etc/karmada/karmada-apiserver.config /etc/karmada/karmada-agent.yaml 172.31.0.8:/root

#172.31.0.8执行以下命令，（remote cluster）

kubectl create ns karmada-system
kubectl create secret generic karmada-kubeconfig --from-file=karmada-kubeconfig=/root/karmada-apiserver.config  -n karmada-system
MEMBER_CLUSTER_NAME="remote"
sed -i "s/{member_cluster_name}/${MEMBER_CLUSTER_NAME}/g" karmada-agent.yaml
kubectl create -f karmada-agent.yaml

kubectl  --kubeconfig /etc/karmada/karmada-apiserver.config get clusters

push模式： 由karamda控制端监听各个子集群状态，并将组合的资源分发到各个子集群
pull模式： 在子集群部署karmada-agent,用于监听与本集群相关的资源，并维护资源的生命周期

添加别名

alias karmadactl='kubectl  --kubeconfig /etc/karmada/karmada-apiserver.config'

测试：
创建常规资源，创建分发规则

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginxtest
  labels:
    app: nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - image: nginx
        name: nginx
---        
apiVersion: policy.karmada.io/v1alpha1
kind: PropagationPolicy
metadata:
  name: nginx-propagation
spec:
  resourceSelectors:
    - apiVersion: apps/v1
      kind: Deployment
      name: nginxtest
  placement:
    clusterAffinity:
      clusterNames:
        - default
        - remote
    replicaScheduling:
      replicaDivisionPreference: Weighted
      replicaSchedulingType: Divided
      weightPreference:
        staticWeightList:
          - targetCluster:
              clusterNames:
                - default
            weight: 1
          - targetCluster:
              clusterNames:
                - remote
            weight: 1
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: nginx
  name: nginxtest
spec:
  type: ClusterIP
  ports:
  - port: 80
    targetPort: 80
    name: http
  selector:
    app: nginx
---
apiVersion: policy.karmada.io/v1alpha1
kind: PropagationPolicy
metadata:
  name: nginx-svc-propagation
spec:
  resourceSelectors:
    - apiVersion: v1
      kind: Service
      name: nginxtest
  placement:
    clusterAffinity:
      clusterNames:
        - default
        - remote
    replicaScheduling:
      replicaDivisionPreference: Weighted
      replicaSchedulingType: Divided
      weightPreference:
        staticWeightList:
          - targetCluster:
              clusterNames:
                - default
            weight: 1
          - targetCluster:
              clusterNames:
                - remote
            weight: 1

停止集群remote的kube-api,验证故障转移，状态为False后，default上的deployments nginxtest扩容为两个副本

迁移现有资源

karmada控制平台创建namespace
执行kubectl karmada promote
创建分发规则
例子：
xxdeployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: xxnginx
  namespace: xx
  labels:
    app: nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - image: nginx
        name: nginx

default 集群操作：

kubectl apply -f xxdeployment.yaml

karmadactl create ns xx

---

Promote resources from legacy clusters to karmada control plane

kubectl karmada --kubeconfig /etc/karmada/karmada-apiserver.config promote deployment xxnginx  -n  xx -c default

修改分发规则

apiVersion: policy.karmada.io/v1alpha1
kind: PropagationPolicy
metadata:
  name: nginx-propagation
spec:
  resourceSelectors:
    - apiVersion: apps/v1
      kind: Deployment
      name: xxnginx
      namespace: xx
  placement:
    clusterAffinity:
      clusterNames:
        - default
        - remote
    replicaScheduling:
      replicaDivisionPreference: Weighted
      replicaSchedulingType: Divided
      weightPreference:
        staticWeightList:
          - targetCluster:
              clusterNames:
                - default
            weight: 1
          - targetCluster:
              clusterNames:
                - remote
            weight: 1