kubernetes federation搭建
前置要求
软件以及版本需求
软件 | 版本 |
---|
kubernetes | >= v1.13 |
minikube(可用于替代kubernetes) | 只要安装的kubernetes版本同上即可(装最新的肯定没问题) |
helm | >= 2.10 |
kubefedctl | 这个kubefed版本选择即可 |
docker | 看kubernetes需要的版本 |
软件环境搭建
docker环境搭建
- 海外版本
apt-get update
apt-get install -y docker.io
systemctl start docker
- 国内版本
apt-get update
apt-get install -y apt-transport-https
apt-get install -y ca-certificates
apt-get install -y curl
apt-get install -y software-properties-common
curl -fsSL https://mirrors.aliyun.com/docker-ce/linux/ubuntu/gpg | sudo apt-key add -
add-apt-repository "deb [arch=amd64] https://mirrors.aliyun.com/docker-ce/linux/ubuntu $(lsb_release -cs) stable"
sudo apt-get update
apt-get install -y docker-ce
systemctl start docker
minikube搭建 (ubuntu 18.04)
海外版本(可以直接访问google的版本)
- 安装kubectl
- 直接下载二进制
步骤 | 命令 | example |
---|
下载kubectl | curl -LO https://storage.googleapis.com/kubernetes-release/release/curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt /bin/linux/amd64/kubectl | curl -LO https://storage.googleapis.com/kubernetes-release/release/v1.18.0/bin/linux/amd64/kubectl |
给kubectl添加可执行属性 | chmod +x ./kubectl | |
将kubectl移动到本地PATH中 | sudo mv ./kubectl /usr/local/bin/kubectl | |
测试kubectl的可用性 | kubectl version --client | |
- apt安装
sudo apt-get update && sudo apt-get install -y apt-transport-https gnupg2
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
echo "deb https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee -a /etc/apt/sources.list.d/kubernetes.list
sudo apt-get update
sudo apt-get install -y kubectl
- yum 安装
cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
EOF
yum install -y kubectl
- 下载minikube可执行二进制文件
curl -Lo minikube https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64 && chmod +x minikube
- 将minikube 二进制文件放入到本地PATH
sudo mkdir -p /usr/local/bin/
sudo install minikube /usr/local/bin/
- 启动minikube
sudo apt-get install -y conntrack
minikube start --driver=none
国内版本(通过阿里云安装)
curl -Lo minikube https://kubernetes.oss-cn-hangzhou.aliyuncs.com/minikube/releases/v1.11.0/minikube-linux-amd64 && chmod +x minikube && sudo mv minikube /usr/local/bin/
sudo apt-get install -y conntrack
minikube start --driver=none
helm安装
- 下载helm离线安装包
# 下载地址
# https://github.com/helm/helm/releases
wget https://get.helm.sh/helm-v2.16.9-linux-amd64.tar.gz
# 解压
tar -zxvf helm-v2.16.9-linux-amd64.tar.gz
# 将helm二进制可执行文件移动到本地path中
mv linux-amd64/helm /usr/local/bin/helm
- 下载tiller image
# 版本请自行修改,与helm的版本有关系,init的时候就知道要下载哪个了
docker pull gcr.io/kubernetes-helm/tiller:v2.16.9
- 安装socat
apt-get install -y socat
- helm init
helm init --service-account tiller
kubefed安装
helm安装方法
过程 | 命令 |
---|
安装repo | helm repo add kubefed-charts https://raw.githubusercontent.com/kubernetes-sigs/kubefed/master/charts |
查看已安装的repo | helm repo list |
查询kubefed | helm search kubefed --devel |
安装kubefed | helm install kubefed-charts/kubefed --name kubefed --version=<x.x.x> --namespace kube-federation-system --devel |
卸载 | kubectl -n kube-federation-system delete FederatedTypeConfig --all |
| kubectl delete crd $(kubectl get crd |
| helm delete --purge kubefed |
查看安装后的api资源 | kubectl apiresources |
离线安装
- 下载kubefed的源代码包
git clone https://github.com/kubernetes-sigs/kubefed.git
- 进入到kubefed的chart目录下
cd charts/
- 使用helm离线安装
helm install --name kubefed --namespace kube-federation-system ./kubefed/
# 注意,此步骤还是需要通过途径获取到镜像quay.io/kubernetes-multicluster/kubefed:canary
# 另外注意,如果通过helm delete --purge的方法删除了kubefed,其安装的FederatedDeployment等api resource是无法被删除的,目前还没有找到解决方案
kubefedctl 安装
- 下载网址
https://github.com/kubernetes-sigs/kubefed/releases
- example
wget https://github.com/kubernetes-sigs/kubefed/releases/download/v0.3.0/kubefedctl-0.3.0-linux-amd64.tgz
tar -zxvf kubefedctl-0.3.0-linux-amd64.tgz
mv linux-amd64/kubefedctl /usr/local/bin/
kubefed的使用指南
配置kubeconfig
需要在安装了kubefed的设备上配置到管理的集群的context
kubeconfig的路径和结构
- 路径
/root/.kube/config
- 文件结构
配置kubeconfig
- 将kubefed之外的其他集群的以下几个文件下载到本地
- 文件列表(以minikube举例):
文件作用 | 文件路径 |
---|
kubernetes API server的ca证书 | /root/.minikube/ca.crt |
kubernetes 用户的crt | /root/.minikube/profiles/minikube/client.crt |
kubernetes 用户的key | /root/.minikube/profiles/minikube/client.key |
- 举例
- 下载来自另一台设备的对应文件
# 创建文件夹
mkdir -p /root/.cluster1/
# 下载文件
scp ${host}:/root/.minikube/ca.crt /root/.cluster1/
scp ${host}:/root/.minikube/profiles/minikube/client.crt /root/.cluster1/
scp ${host}:/root/.minikube/profiles/minikube/client.key /root/.cluster1/
# 配置kubeconfig
# 可以直接改/root/.kube/config,按照上面的文件结构加上cluster、context、user就行
# 另一种命令方式如下
## 添加cluster
kubectl config set-cluster cluster1 --server=https://${host}:8443 --certificate-authority=/root/.cluster1/ca.crt
## 添加user
kubectl config set-credentials cluster1 --client-certificate=/root/.cluster1/client.crt --client-key=/root/.cluster1/client.key
## 添加context
kubectl config set-context cluster1 --cluster cluster1 --user=cluster1
- 配置完成后,使用如下命令做测试
kubectl config use-context cluster1
kubectl get namespaces
将集群纳入到fed集群的管辖范围内
# kubefedctl join ${hostname} --cluster-context ${hostname's k8s context} --host-cluster-context ${host machine's context} --v=2
kubefedctl join cluster1 --cluster-context cluster1 --host-cluster-context minikube --v=2
查看集群加入情况
kubectl get kubefedclusters -n kube-federation-system
在fed集群上创建一个namespace
kubectl create namespace federated-namespace
- 这一步必须做, namespace名可以自定义,只要保证后面步骤中的namespace和当前的namespace保持一致
创建FederatedNamespace
apiVersion: types.kubefed.io/v1beta1
kind: FederatedNamespace
metadata:
name: federated-namespace
namespace: federated-namespace
spec:
placement:
clusters:
- name: cluster1
- name: cluster2
查看namespace创建情况
# 查看cluster1上是否创建了federated-namespace
kubectl config use-context cluster1
kubectl get namespaces
# 查看cluster2上是否创建了federated-namespace
kubectl config use-context cluster2
kubectl get namespaces
创建FederatedDeployment
kind: FederatedDeployment
metadata:
name: federated-nginx
namespace: federated-namespace
spec:
template:
metadata:
labels:
app: nginx
spec:
replicas: 3
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- image: nginx
name: nginx
placement:
clusters:
- name: cluster2
- name: cluster1
overrides:
- clusterName: cluster2
clusterOverrides:
- path: "/spec/replicas"
value: 5
- path: "/spec/template/spec/containers/0/image"
value: "nginx:1.17.0-alpine"
- path: "/metadata/annotations"
op: "add"
value:
foo: bar
查看deployment创建情况
# 查看cluster1上是否创建了federated-nginx
kubectl config use-context cluster1
kubectl get deployments -n federated-namespace
kubectl get pods -n federated-namespace
# 查看cluster2上是否创建了federated-nginx
kubectl config use-context cluster2
kubectl get deployments -n federated-namespace
kubectl get pods -n federated-namespace