本指南介绍如何配置Cilium和kube-router,使kube-router使用BGP协议进行对等路由传播,以及如何配置Cilium进行策略执行和负载平衡。
该功能尚在测试版。如果您遇到任何问题,请提供反馈并提交GitHub问题。
部署kube-router
下载kube-router DaemonSet模板
curl -LO https://raw.githubusercontent.com/cloudnativelabs/kube-router/v1.2/daemonset/generic-kuberouter-only-advertise-routes.yaml
打开generic-kuberouter-only-advertise-routes.yaml
并编辑args:
部分,需要为下面的这些参数设置特定值:
- "--run-router=true"
- "--run-firewall=false"
- "--run-service-proxy=false"
- "--enable-cni=false"
- "--enable-pod-egress=false"
以下参数是可选的,可以根据需要设置。为了简化本指南,使用了以下值,这些值在集群中需要最少的准备。有关更多信息,请参阅 [kube-router user guide](https://github.com/cloudnativelabs/kube-router/blob/master/docs/user-guide.md)。
- "--enable-ibgp=true"
- "--enable-overlay=true"
- "--advertise-cluster-ip=true"
- "--advertise-external-ip=true"
- "--advertise-loadbalancer-ip=true"
以下参数是可选的。如果希望BGP与外部路由器对等,则应设置这些参数。与外部路由器互为BGP peer的目的是为了实现外部可路由的Kubernetes Pod和服务IP。注意:此处使用的值应更改为外部路由器上配置的IP和ASNs。
- "--cluster-asn=65001"
- "--peer-router-ips=10.0.0.1,10.0.2"
- "--peer-router-asns=65000,65000"
应用DaemonSet以部署kube-router,并通过下面命令验证是否正确安装。
$ kubectl apply -f generic-kuberouter-only-advertise-routes.yaml
$ kubectl -n kube-system get pods -l k8s-app=kube-router
NAME READY STATUS RESTARTS AGE
kube-router-n6fv8 1/1 Running 0 10m
kube-router-nj4vs 1/1 Running 0 10m
kube-router-xqqwc 1/1 Running 0 10m
kube-router-xsmd4 1/1 Running 0 10m
部署Cilium
为了将路由委托给kube-router,必须禁用tunneling/encapsulation功能。在ConfigMap cilium-config
配置中设置tunnel=disabled
,或者通过调整agent程序启动选项--tunnel=disabled
来禁用隧道功能。此外,在同一ConfigMap中,我们必须显式设置ipam: kubernetes
,因为kube-router直接从K8s中提取pod CIDR:
# Encapsulation mode for communication between nodes
# Possible values:
# - disabled
# - vxlan (default)
# - geneve
tunnel: "disabled"
ipam: "kubernetes"
然后,您可以根据cilium安装要求一节中的说明安装Cilium。
确认cilium成功安装并正常运行:
$ kubectl -n kube-system get pods -l k8s-app=cilium
NAME READY STATUS RESTARTS AGE
cilium-fhpk2 1/1 Running 0 45m
cilium-jh6kc 1/1 Running 0 44m
cilium-rlx6n 1/1 Running 0 44m
cilium-x5x9z 1/1 Running 0 45m
验证安装
验证kube-router成功安装了路由规则:
$ kubectl -n kube-system exec ds/cilium -- ip route list scope global
default via 172.0.32.1 dev eth0 proto dhcp src 172.0.50.227 metric 1024
10.2.0.0/24 via 10.2.0.172 dev cilium_host src 10.2.0.172
10.2.1.0/24 via 172.0.51.175 dev eth0 proto 17
10.2.2.0/24 dev tun-172011760 proto 17 src 172.0.50.227
10.2.3.0/24 dev tun-1720186231 proto 17 src 172.0.50.227
在上面的例子中,有三类路由规则:
- Local PodCIDR: 该路由指向主机上运行的所有pod,并使这些pod可用于*
10.2.0.0/24 via 10.2.0.172 dev cilium_host src 10.2.0.172
- BGP route: 如果kube-router确定可以通过本地主机已知的路由器访问远程PodCIDR,则安装这种类型的路由。它将指示Pod到Pod的流量转发到该路由器,不需要任何封装。*
10.2.1.0/24 via 172.0.51.175 dev eth0 proto 17
- PIP tunnel route: 如果不存在直接路由路径,kube-router将返回使用overlay,并在节点之间建立IPIP隧道。*
10.2.2.0/24 dev tun-172011760 proto 17 src 172.0.50.227
*10.2.3.0/24 dev tun-1720186231 proto 17 src 172.0.50.227
验证安装
安装最新版本的Cilium CLI。Cilium CLI可用于安装Cilium、检查Cilium安装状态以及启用/禁用各种功能(如clustermesh、Hubble)。
CILIUM_CLI_VERSION=$(curl -s https://raw.githubusercontent.com/cilium/cilium-cli/master/stable.txt)
CLI_ARCH=amd64
if [ "$(uname -m)" = "aarch64" ]; then CLI_ARCH=arm64; fi
curl -L --fail --remote-name-all https://github.com/cilium/cilium-cli/releases/download/${CILIUM_CLI_VERSION}/cilium-linux-${CLI_ARCH}.tar.gz{,.sha256sum}
sha256sum --check cilium-linux-${CLI_ARCH}.tar.gz.sha256sum
sudo tar xzvfC cilium-linux-${CLI_ARCH}.tar.gz /usr/local/bin
rm cilium-linux-${CLI_ARCH}.tar.gz{,.sha256sum}
要验证Cilium是否已正确安装,可以运行
$ cilium status --wait
/¯¯\
/¯¯\__/¯¯\ Cilium: OK
\__/¯¯\__/ Operator: OK
/¯¯\__/¯¯\ Hubble: disabled
\__/¯¯\__/ ClusterMesh: disabled
\__/
DaemonSet cilium Desired: 2, Ready: 2/2, Available: 2/2
Deployment cilium-operator Desired: 2, Ready: 2/2, Available: 2/2
Containers: cilium-operator Running: 2
cilium Running: 2
Image versions cilium quay.io/cilium/cilium:v1.9.5: 2
cilium-operator quay.io/cilium/operator-generic:v1.9.5: 2
运行以下测试用例以验证群集是否具有正确的网络连接:
$ cilium connectivity test
ℹ️ Monitor aggregation detected, will skip some flow validation steps
✨ [k8s-cluster] Creating namespace for connectivity check...
(...)
---------------------------------------------------------------------------------------------------------------------
📋 Test Report
---------------------------------------------------------------------------------------------------------------------
✅ 69/69 tests successful (0 warnings)
测试用例全部通过,恭喜你成功安装了带有cilium且网络功能完备的kubernetes集群。