Kubernetes(k8s) v1.30.1 本地集群部署 安装metallb 支持LoadBalancer 生产环境 推荐 BGP模式部署

1 metallb 安装参考:Kubernetes(k8s) v1.30.1 本地集群部署 默认不支持LoadBalancer metallb来解决-CSDN博客

2  删除 Layer 2 模式 配置

kubectl delete -f IPAddressPool.yaml
kubectl delete -f L2Advertisement.yaml
kubectl delete -f discuz-srv.yaml

3 配置 k8s MetalLB BGP

#路由器配置,参考:OpenWrt 安装Quagga 支持ospf Bgp等动态路由协议 软路由实测 系列四-CSDN博客

#k8s master11 配置ip地址池等

[root@master11 soft]# cat bgpippool.yaml
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: first-pool
  namespace: metallb-system
spec:
  addresses:
   - 192.168.60.2-192.168.60.100

[root@master11 soft]# cat bgpad1.yaml 
apiVersion: metallb.io/v1beta1
kind: BGPAdvertisement
metadata:
  name: local
  namespace: metallb-system
spec:
  ipAddressPools:
  - first-pool
  aggregationLength: 32
  localPref: 100

[root@master11 soft]# cat bgppeer.yaml 
apiVersion: metallb.io/v1beta2
kind: BGPPeer
metadata:
  name: local
  namespace: metallb-system
spec:
  myASN: 65108
  peerASN: 65100
  peerAddress: 192.168.50.1
  peerPort: 179
#
kubectl apply -f  bgpippool.yaml
kubectl apply -f  bgpad1.yaml
kubectl apply -f  bgppeer.yaml

查看speaker日志,可以看到BGP连接
{"caller":"native.go:109","event":"sessionUp","level":"info","localASN":65108,"msg":"BGP session established","peer":"192.168.50.1:179","peerASN":65100,"ts":"2024-05-25T14:48:25Z"}

#安装discuz loadbalancer 应用 

kubectl apply -f  discuz-srv.yaml

#查看controller日志,看到IP 192.168.60.2 已经分配
{"caller":"service.go:150","event":"ipAllocated","ip":["192.168.60.2"],"level":"info","msg":"IP address assigned by controller","ts":"2024-05-25T15:11:31Z"}
#查看speaker日志,看到BGP路由信息
{"caller":"bgp_controller.go:309","event":"updatedAdvertisements","ips":["192.168.60.2"],"level":"info","msg":"making advertisements using BGP","numAds":1,"pool":"first-pool","protocol":"bgp","ts":"2024-05-25T15:11:31Z"}
{"caller":"main.go:409","event":"serviceAnnounced","ips":["192.168.60.2"],"level":"info","msg":"service has IP, announcing","pool":"first-pool","protocol":"bgp","ts":"2024-05-25T15:11:31Z"}

 4 k8s 查看

root@master11 ~]# kubectl get node,po,svc -A -owide|grep discuz
default                pod/discuz-859bc9f964-5wlwp                                 1/1     Running   7 (5h9m ago)    3d2h   10.244.199.25    slave12    <none>           <none>
default                pod/discuz-859bc9f964-8jpct                                 1/1     Running   7 (5h9m ago)    3d2h   10.244.199.27    slave12    <none>           <none>
default                pod/discuz-859bc9f964-qmx8z                                 1/1     Running   7 (5h9m ago)    3d2h   10.244.199.39    slave12    <none>           <none>
default                service/discuz-service                         LoadBalancer   10.111.38.239    192.168.60.2   88:30036/TCP                    5m11s   app=discuz

5 路由器检查

#看看BGP状态
show ip bgp  summary 
BGP router identifier 192.168.50.1, local AS number 65100
RIB entries 1, using 112 bytes of memory
Peers 3, using 27 KiB of memory

Neighbor        V         AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
192.168.50.11   4 65108     656    2244        0    0    0 00:10:31        0
192.168.50.12   4 65108     661    2251        0    0    0 00:10:31        1
192.168.50.13   4 65108     664    2252        0    0    0 00:10:30        1

Total number of neighbors 3

Total num. Established sessions 3
Total num. of routes received     2
#查看路由表

 show ip route 
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, P - PIM, A - Babel, N - NHRP,
       > - selected route, * - FIB route

K>* 0.0.0.0/0 via 192.168.1.1, eth1
C>* 127.0.0.0/8 is directly connected, lo
C>* 192.168.1.0/24 is directly connected, eth1
C>* 192.168.50.0/24 is directly connected, br-lan
B>* 192.168.60.2/32 [20/0] via 192.168.50.12, br-lan, 00:10:20
#
show ip bgp  
BGP table version is 0, local router ID is 192.168.50.1
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
              i internal, r RIB-failure, S Stale, R Removed
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 192.168.60.2/32  192.168.50.12                          0 65108 i
*                   192.168.50.13                          0 65108 i

Displayed  1 out of 2 total prefixes

6  pc端浏览器访问

在正式环境中，如果条件满足，推荐使用K8S BGP模式 .