如果把K8S搞在公有云上,可以跟云厂商买它的负载均衡服务,就用这个负载均衡服务提供的公网IP,把你的域名映射到这个公网IP上,然后配置这个云厂商提供的负载均衡服务,让它往后端的ECS主机上转发
但是呢,如果是自己的物理服务器,并不是假设在公有云上的K8S集群,那么很显然是没有云厂商预先提供的负载均衡公网IP地址的,此时怎么办呢?
在K8S里面,如果把Service暴露到集群外,可以供客户端访问的话,那么要通过NodePort、LoadBalancer或者Ingress这三种方式才可以的,具体是啥意思呢?
- NodePort,就是在每台K8S的worker节点上都通过kube-proxy进行端口映射,打个比方Service对外的端口是8080,那么通过NodePort方式就要求在每台物理主机上都要通过kube-proxy进程进行数据转发,而且要让kube-proxy监听某个特定的端口比如33000,客户端访问的时候是访问NodeIp:33000的方式。
kube-proxy进程在33000端口收到TCP报文后,就会转发给这个Service背后对应的Pod,它会查询kube-apiserver,看看这个Service背后的Pod当前都部署在哪些物理主机上,然后往这些Pod进行转发。
因为kube-proxy进程和这些pod容器都是在集群内的,所以kube-proxy进程是可以访问到这些Pod在集群内的IP地址的,换句话说,kube-proxy进程直接向Pod的IP地址发起请求,在IP层模式通的
- LoadBalancer,这个是让阿里云、华为云的SLB提供IP地址,这个简单,直接配置即可,但是要求k8s集群部署在公有云上,这个也有点扯,并不是每次都是部署在公有云上的
- Ingress,是七层代理,一般是http代理,可以基于url做转发,比kube-proxy的TCP四层代理更高层,k8s的Ingress实际上是包装了nginx,基于nginx做了一个Pod,当在K8S上面部署一个Ingress资源的时候,实际上是启动了一个nginx的Pod,然后把nginx的nginx.conf文件做了修改,这样就可以让nginx做http转发了。
在使用k8s里面的Ingress资源对象的时候,k8s本质上是把Ingress资源对象的yaml文件转换为nginx.conf文件,然后让Pod里面的Nginx容器中的Nginx进程reload一下配置文件,就是执行nginx -s reload,这样就实现了转发规则的动态调整,比如下面的rules部分,其实就是会被转换为nginx.conf文件
apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata: name: example-ingress spec: rules: # 指定主机 - host: example.ctnrs.com http: paths: # 指定路径 - path: / backend: # 通过kubectl get svc查询到的servicename serviceName: web # 通过kubectl get svc查询到的前面的端口 servicePort: 80
我研究了一下https://porterlb.io/zh/,它应对的场景是自己搭建的K8S集群,比如给你10台物理主机你去搞一个k8s集群,然后怎么对外暴露你的Service呢?
这个porterlb是KubeSphere提供的LB,说是支持BGP路由方式,我一直觉得很奇怪,怎么会和BGP路由扯上关系?BGP是用于AS之间高效路由的路由协议啊,比如支持BGP的数据中心,你去租赁一条线路,可以避免南北访问的速度问题,而这里说的是K8S的Service对外提供服务的问题,怎么会和BGP扯上关系?
K8S的Service虽然有IP地址,但是说到底,这个IP地址并不是网管员分配的IP地址,也不是运营商分配的公网IP,用户是无法访问的,这个Service的IP地址是在k8s集群内的私有IP地址,根本不对外公开的,所以如果没有一定的机制,是无法让客户端访问到这个对外暴露的Service的。
比如应用部署在K8S上,客户可以通过app、小程序、pc网站访问,那你的Service必须得有一个公网IP,你直接给它一个Service的IP地址,客户端是无法路由的,路由不可达!
PorterLB产品支持2种对外暴露Service的机制,L2和BGP,先来看看L2模式,核心是让Service的IP地址必须和路由器的IP地址在同一个网段内,这样的话,路由器192.168.0.5才能和192.168.0.91互通
porterlb的L2模式
怎么样才能让客户端的访问请求均衡到Pod1和Pod2呢?这玩意是这样玩的:
- 首先porterlb,它确确实实是一个进程,它就是用golang写的一个程序,假设你的k8s集群有10台物理机组成,找台物理机安装一下porterlb即可
- 现在按照老样子,先创建一个k8s的Service,搞个yaml就可以创建一个k8s的service资源对象,背后有2个pod组成,分别在2台物理机上,如上图在Worker1和Worker2上
- 然后我们肯定想要让client的数据流量均衡到达这2个物理机,怎么搞呢?
- 核心在这个中间的路由器,你想,Service是有一个IP地址的,客户端访问这个IP地址,只要路由器能够把IP报文路由出去,不就OK了吗,在这里Service的IP是0.91,但是现在并没有哪一台物理主机的IP地址是0.91,所以路由器肯定是不知道怎么把数据报文路由出去的!
- 此时魔法开始出现,我们一旦创建了一个k8s的服务后,就让这个porter进程去和路由器通信,所以写porter的人要懂怎么和路由器通信,我们让porterlb进程发一个ARP报文给路由器,意思是,唉,兄弟,告诉你,你不是不知道0.91这个IP对应的MAC地址吗,我告诉你啊,IP地址是192.168.0.91的机器,它对应的MAC地址是52:54:22:3a:e6:64,也就是Worker1节点
- 这样一搞的话,本来Worker1的IP地址是192.168.0.3,结果现在它又莫名其妙多了一个IP地址192.168.0.91,路由器后续就知道了,它一旦收到客户端发给0.91的报文,就会直接发给Worker1机器,因为路由器已经通过porterlb进程主动发给它的ARP报文知道了0.91对应的MAC地址
- 然后呢,Worker1机器上不是跑着kube-proxy进程嘛,它就是一个类似nginx的反向代理,数据报文达到Worker1上的kube-proxy进程后,它会转发给Pod1或者Pod2
- 如果Worker1服务器挂了的话,porterlb进程又开始谎报军情,它由主动给路由器发了一个ARP报文说:IP地址是192.168.0.91的机器,现在变了啊,它对应的MAC地址是52:54:22:37:6c:7b,这样路由器再收到客户端发给0.91的数据,就会转发给worker2这台机器了。
通过这种方式,相当于是多了一跳,先从路由器发给Worker1,然后由Worker1上的kube-proxy再发给Worker1上的Pod1或者Worker2上的Pod2,多了一跳,同时也有缺点,任何时候,从路由器过来的数据报文流量都只能发给Worker1或者Worker2,不能同时发给它们,这就导致Worker1的网卡成为流量瓶颈,本来如果效果更好的话,让路由器同时发给Worker1和Worker2,带宽不就翻倍了嘛,这完全就是一个谎报军情的L2模式嘛,呵呵
Porterlb的BGP模式
这个BGP模式也很有意思,竟然让Porter进程给BGP路由器进行tcp通信,发送BGP路由报文,搞这个要懂BGP路由的协议格式。安装在Kubernetes群集中的PorterLB进程与BGP路由器建立BGP连接(这是一个TCP连接),构造一条路由规则发给BGP路由器,什么规则呢,如图右上角,发往172.22.0.2的报文,下一跳的地址是0.3、0.4,这样就实现了负载均衡,牛逼!
这种方式,BGP路由器会把流量均衡的发给0.3、0.4这2台物理机,比上面的L2模式好,與此同時也就意味着,如果Pod1发生销毁重建,那么PorterLB必须得到通知,假设Pod1现在漂移到了192.168.0.10上面,那么PorterLB必须重新给BGP路由器发新的路由报文,否则BGP路由器还把报文发给0.3,这就不对了。
所以开发PorterLB这个产品,需要对BGP路由比较懂,同时还要知道怎么给路由器发ARP报文,同时还要懂K8S相关的源码级别的API才好做二次开发。
这种BGP模式,不需要Service的IP地址和路由器在同一个网段内,比如上面Service的IP是172.22.0.2,但是路由器的IP是192.168.0.5,根本不是一个网段。
这种方式下,客户端可以直接请求Service的IP地址172.22.0.0,此时会达到BGP路由器,BGP路由器会发给Worker1和Worker2,BGP路由器为啥会发给它俩?因为Porterlb进程给它发了BGP路由规则,所以它才知道。
看这个:https://github.com/osrg/gobgp github上真的是可供参考的资料太多了,竟然还有直接和BGP路由器在TCP层面直接交换数据的库,porterlb就是用这个库和BGP路由器直接通信,让BGP路由器知道如何路由。
再来一个这个:https://github.com/mdlayher/arp
这玩意实现了arp协议,可以用来发arp报文,golang写的,PorterLB是用它来给路由器发ARP报文的,可以看到porterlb依赖的第三方库。
https://github.com/coreos/go-iptables,还用到了这个第三方库,直接操作linux上的iptables,go-iptables wraps invocation of iptables utility with functions to append and delete rules; create, clear and delete chains.
https://github.com/mdlayher/ethernet,这个,Package ethernet implements marshaling and unmarshaling of IEEE 802.3 Ethernet II frames and IEEE 802.1Q VLAN tags. MIT Licensed,可以对带VLAN的以太网数据报文进行解析。
apiVersion: v1
kind: Namespace
metadata:
name: porter-system
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.4.0
creationTimestamp: null
name: bgpconfs.network.kubesphere.io
spec:
group: network.kubesphere.io
names:
kind: BgpConf
listKind: BgpConfList
plural: bgpconfs
singular: bgpconf
scope: Cluster
versions:
- name: v1alpha1
schema:
openAPIV3Schema:
description: BgpConf is the Schema for the bgpconfs API
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: struct for container bgp:config. Configuration parameters relating to the global BGP router.
properties:
as:
description: original -> bgp:as bgp:as's original type is inet:as-number. Local autonomous system number of the router. Uses the 32-bit as-number type from the model in RFC 6991.
format: int32
type: integer
port:
description: original -> gobgp:port
format: int32
maximum: 65535
minimum: 1
type: integer
routerID:
description: original -> bgp:router-id bgp:router-id's original type is inet:ipv4-address. Router id of the router, expressed as an 32-bit value, IPv4 address.
pattern: ^([0-9]{1,3}\.){3}[0-9]{1,3}$
type: string
required:
- as
- port
- routerID
type: object
status:
description: BgpConfStatus defines the observed state of BgpConf
type: object
type: object
served: true
storage: false
- name: v1alpha2
schema:
openAPIV3Schema:
description: BgpConf is the Schema for the bgpconfs API
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: Configuration parameters relating to the global BGP router.
properties:
as:
format: int32
type: integer
asPerRack:
additionalProperties:
format: int32
type: integer
type: object
families:
items:
format: int32
type: integer
type: array
gracefulRestart:
properties:
deferralTime:
format: int32
type: integer
enabled:
type: boolean
helperOnly:
type: boolean
localRestarting:
type: boolean
longlivedEnabled:
type: boolean
mode:
type: string
notificationEnabled:
type: boolean
peerRestartTime:
format: int32
type: integer
peerRestarting:
type: boolean
restartTime:
format: int32
type: integer
staleRoutesTime:
format: int32
type: integer
type: object
listenAddresses:
items:
type: string
type: array
listenPort:
format: int32
type: integer
routerId:
type: string
useMultiplePaths:
type: boolean
type: object
status:
description: BgpConfStatus defines the observed state of BgpConf
properties:
nodesConfStatus:
additionalProperties:
properties:
as:
format: int32
type: integer
routerId:
type: string
type: object
type: object
type: object
type: object
served: true
storage: true
subresources:
status: {}
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.4.0
creationTimestamp: null
name: bgppeers.network.kubesphere.io
spec:
group: network.kubesphere.io
names:
kind: BgpPeer
listKind: BgpPeerList
plural: bgppeers
singular: bgppeer
scope: Cluster
versions:
- name: v1alpha1
schema:
openAPIV3Schema:
description: BgpPeer is the Schema for the bgppeers API
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
properties:
addPaths:
description: original -> bgp:add-paths Parameters relating to the advertisement and receipt of multiple paths for a single NLRI (add-paths).
properties:
sendMax:
description: original -> bgp:send-max The maximum number of paths to advertise to neighbors for a single NLRI.
type: integer
type: object
config:
description: original -> bgp:neighbor-address original -> bgp:neighbor-config Configuration parameters relating to the BGP neighbor or group.
properties:
neighborAddress:
description: original -> bgp:neighbor-address bgp:neighbor-address's original type is inet:ip-address. Address of the BGP peer, either in IPv4 or IPv6.
pattern: ^([0-9]{1,3}\.){3}[0-9]{1,3}$
type: string
peerAs:
description: original -> bgp:peer-as bgp:peer-as's original type is inet:as-number. AS number of the peer.
format: int32
type: integer
required:
- neighborAddress
- peerAs
type: object
transport:
description: original -> bgp:transport Transport session parameters for the BGP neighbor or group.
properties:
passiveMode:
description: original -> bgp:passive-mode bgp:passive-mode's original type is boolean. Wait for peers to issue requests to open a BGP session, rather than initiating sessions from the local router.
type: boolean
remotePort:
description: original -> gobgp:remote-port gobgp:remote-port's original type is inet:port-number.
maximum: 65535
minimum: 1
type: integer
type: object
usingPortForward:
type: boolean
type: object
status:
description: BgpPeerStatus defines the observed state of BgpPeer
type: object
type: object
served: true
storage: false
- name: v1alpha2
schema:
openAPIV3Schema:
description: BgpPeer is the Schema for the bgppeers API
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
properties:
afiSafis:
items:
properties:
addPaths:
properties:
config:
properties:
receive:
type: boolean
sendMax:
format: int32
type: integer
type: object
type: object
config:
properties:
enabled:
type: boolean
family:
properties:
afi:
type: string
safi:
type: string
type: object
type: object
mpGracefulRestart:
properties:
config:
properties:
enabled:
type: boolean
type: object
type: object
type: object
type: array
conf:
properties:
adminDown:
type: boolean
allowOwnAs:
format: int32
type: integer
authPassword:
type: string
description:
type: string
localAs:
format: int32
type: integer
neighborAddress:
type: string
neighborInterface:
type: string
peerAs:
format: int32
type: integer
peerGroup:
type: string
peerType:
format: int32
type: integer
removePrivateAs:
type: string
replacePeerAs:
type: boolean
routeFlapDamping:
type: boolean
sendCommunity:
format: int32
type: integer
vrf:
type: string
type: object
ebgpMultihop:
properties:
enabled:
type: boolean
multihopTtl:
format: int32
type: integer
type: object
gracefulRestart:
properties:
deferralTime:
format: int32
type: integer
enabled:
type: boolean
helperOnly:
type: boolean
localRestarting:
type: boolean
longlivedEnabled:
type: boolean
mode:
type: string
notificationEnabled:
type: boolean
peerRestartTime:
format: int32
type: integer
peerRestarting:
type: boolean
restartTime:
format: int32
type: integer
staleRoutesTime:
format: int32
type: integer
type: object
nodeSelector:
description: A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.
properties:
matchExpressions:
description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
items:
description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
properties:
key:
description: key is the label key that the selector applies to.
type: string
operator:
description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
type: string
values:
description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
timers:
properties:
config:
description: https://stackoverflow.com/questions/21151765/cannot-unmarshal-string-into-go-value-of-type-int64
properties:
connectRetry:
type: string
holdTime:
type: string
keepaliveInterval:
type: string
minimumAdvertisementInterval:
type: string
type: object
type: object
transport:
properties:
mtuDiscovery:
type: boolean
passiveMode:
type: boolean
remoteAddress:
type: string
remotePort:
format: int32
type: integer
tcpMss:
format: int32
type: integer
type: object
type: object
status:
description: BgpPeerStatus defines the observed state of BgpPeer
properties:
nodesPeerStatus:
additionalProperties:
properties:
peerState:
properties:
adminState:
type: string
authPassword:
type: string
description:
type: string
flops:
format: int32
type: integer
localAs:
format: int32
type: integer
messages:
properties:
received:
properties:
discarded:
type: string
keepalive:
type: string
notification:
type: string
open:
type: string
refresh:
type: string
total:
type: string
update:
type: string
withdrawPrefix:
type: string
withdrawUpdate:
type: string
type: object
sent:
properties:
discarded:
type: string
keepalive:
type: string
notification:
type: string
open:
type: string
refresh:
type: string
total:
type: string
update:
type: string
withdrawPrefix:
type: string
withdrawUpdate:
type: string
type: object
type: object
neighborAddress:
type: string
outQ:
format: int32
type: integer
peerAs:
format: int32
type: integer
peerGroup:
type: string
peerType:
format: int32
type: integer
queues:
properties:
input:
format: int32
type: integer
output:
format: int32
type: integer
type: object
removePrivateAs:
format: int32
type: integer
routeFlapDamping:
type: boolean
routerId:
type: string
sendCommunity:
format: int32
type: integer
sessionState:
type: string
type: object
timersState:
properties:
connectRetry:
type: string
downtime:
type: string
holdTime:
type: string
keepaliveInterval:
type: string
minimumAdvertisementInterval:
type: string
negotiatedHoldTime:
type: string
uptime:
type: string
type: object
type: object
description: 'INSERT ADDITIONAL STATUS FIELD - define observed state of cluster Important: Run "make" to regenerate code after modifying this file'
type: object
type: object
type: object
served: true
storage: true
subresources:
status: {}
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.4.0
creationTimestamp: null
name: eips.network.kubesphere.io
spec:
group: network.kubesphere.io
names:
categories:
- networking
kind: Eip
listKind: EipList
plural: eips
singular: eip
scope: Cluster
versions:
- additionalPrinterColumns:
- jsonPath: .spec.address
name: cidr
type: string
- jsonPath: .status.usage
name: usage
type: integer
- jsonPath: .status.poolSize
name: total
type: integer
name: v1alpha1
schema:
openAPIV3Schema:
description: Eip is the Schema for the eips API
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: EipSpec defines the desired state of EIP
properties:
address:
pattern: ^([0-9]{1,3}\.){3}[0-9]{1,3}((\/([0-9]|[1-2][0-9]|3[0-2]))|(\-([0-9]{1,3}\.){3}[0-9]{1,3}))?$
type: string
disable:
type: boolean
protocol:
enum:
- bgp
- layer2
type: string
usingKnownIPs:
type: boolean
required:
- address
type: object
status:
description: EipStatus defines the observed state of EIP
properties:
occupied:
type: boolean
poolSize:
type: integer
usage:
type: integer
type: object
type: object
served: true
storage: false
subresources: {}
- additionalPrinterColumns:
- jsonPath: .spec.address
name: cidr
type: string
- jsonPath: .status.usage
name: usage
type: integer
- jsonPath: .status.poolSize
name: total
type: integer
name: v1alpha2
schema:
openAPIV3Schema:
description: Eip is the Schema for the eips API
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: EipSpec defines the desired state of EIP
properties:
address:
type: string
disable:
type: boolean
interface:
type: string
protocol:
enum:
- bgp
- layer2
type: string
usingKnownIPs:
type: boolean
required:
- address
type: object
status:
description: EipStatus defines the observed state of EIP
properties:
firstIP:
type: string
lastIP:
type: string
occupied:
type: boolean
poolSize:
type: integer
ready:
type: boolean
usage:
type: integer
used:
additionalProperties:
type: string
type: object
v4:
type: boolean
type: object
type: object
served: true
storage: true
subresources:
status: {}
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: porter-admission
namespace: porter-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: leader-election-role
namespace: porter-system
rules:
- apiGroups:
- coordination.k8s.io
resources:
- leases
verbs:
- get
- list
- watch
- create
- update
- patch
- delete
- apiGroups:
- coordination.k8s.io
resources:
- leases/status
verbs:
- get
- update
- patch
- apiGroups:
- ""
resources:
- events
verbs:
- create
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: porter-admission
namespace: porter-system
rules:
- apiGroups:
- ""
resources:
- secrets
verbs:
- get
- create
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: porter-admission
rules:
- apiGroups:
- admissionregistration.k8s.io
resources:
- validatingwebhookconfigurations
verbs:
- get
- update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
creationTimestamp: null
name: porter-manager-role
rules:
- apiGroups:
- ""
resources:
- endpoints
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- events
verbs:
- create
- patch
- update
- apiGroups:
- ""
resources:
- nodes
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- services
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- ""
resources:
- services/status
verbs:
- get
- patch
- update
- apiGroups:
- network.kubesphere.io
resources:
- bgpconfs
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- network.kubesphere.io
resources:
- bgpconfs/status
verbs:
- get
- patch
- update
- apiGroups:
- network.kubesphere.io
resources:
- bgppeers
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- network.kubesphere.io
resources:
- bgppeers/status
verbs:
- get
- patch
- update
- apiGroups:
- network.kubesphere.io
resources:
- eips
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- network.kubesphere.io
resources:
- eips/status
verbs:
- get
- patch
- update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: leader-election-rolebinding
namespace: porter-system
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: leader-election-role
subjects:
- kind: ServiceAccount
name: default
namespace: porter-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: porter-admission
namespace: porter-system
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: porter-admission
subjects:
- kind: ServiceAccount
name: porter-admission
namespace: porter-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: manager-rolebinding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: porter-manager-role
subjects:
- kind: ServiceAccount
name: default
namespace: porter-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: porter-admission
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: porter-admission
subjects:
- kind: ServiceAccount
name: porter-admission
namespace: porter-system
---
apiVersion: v1
kind: Service
metadata:
name: porter-admission
namespace: porter-system
spec:
ports:
- name: https-webhook
port: 443
targetPort: webhook
selector:
app: porter-manager
control-plane: porter-manager
type: ClusterIP
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: porter-manager
control-plane: porter-manager
name: porter-manager
namespace: porter-system
spec:
selector:
matchLabels:
app: porter-manager
control-plane: porter-manager
strategy:
rollingUpdate:
maxUnavailable: 1
type: RollingUpdate
template:
metadata:
labels:
app: porter-manager
control-plane: porter-manager
spec:
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: app
operator: In
values:
- porter-manager
topologyKey: kubernetes.io/hostname
containers:
- args:
- --api-hosts=:50051
- --webhook-port=443
command:
- porter-manager
env:
- name: PORTER_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
image: kubesphere/porter:v0.4.1
imagePullPolicy: IfNotPresent
name: porter-manager
ports:
- containerPort: 443
name: webhook
protocol: TCP
readinessProbe:
exec:
command:
- sh
- -c
- |
gobgp -p 50051 global
failureThreshold: 3
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
resources:
limits:
cpu: 100m
memory: 300Mi
requests:
cpu: 100m
memory: 100Mi
securityContext:
capabilities:
add:
- NET_ADMIN
- SYS_TIME
volumeMounts:
- mountPath: /tmp/k8s-webhook-server/serving-certs/
name: webhook-cert
readOnly: true
hostNetwork: true
nodeSelector:
kubernetes.io/os: linux
terminationGracePeriodSeconds: 10
tolerations:
- key: CriticalAddonsOnly
operator: Exists
- effect: NoSchedule
key: node-role.kubernetes.io/master
volumes:
- name: webhook-cert
secret:
items:
- key: key
path: tls.key
- key: cert
path: tls.crt
secretName: porter-admission
---
apiVersion: batch/v1
kind: Job
metadata:
name: porter-admission-create
namespace: porter-system
spec:
template:
metadata:
name: porter-admission-create
spec:
containers:
- args:
- create
- --host=porter-admission,porter-admission.$(POD_NAMESPACE).svc
- --namespace=$(POD_NAMESPACE)
- --secret-name=porter-admission
env:
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
image: docker.io/jettech/kube-webhook-certgen:v1.5.0
imagePullPolicy: IfNotPresent
name: create
restartPolicy: OnFailure
securityContext:
runAsNonRoot: true
runAsUser: 2000
serviceAccountName: porter-admission
---
apiVersion: batch/v1
kind: Job
metadata:
name: porter-admission-patch
namespace: porter-system
spec:
template:
metadata:
name: porter-admission-patch
spec:
containers:
- args:
- patch
- --webhook-name=porter-admission
- --namespace=$(POD_NAMESPACE)
- --patch-mutating=false
- --secret-name=porter-admission
- --patch-failure-policy=Fail
env:
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
image: docker.io/jettech/kube-webhook-certgen:v1.5.0
imagePullPolicy: IfNotPresent
name: patch
restartPolicy: OnFailure
securityContext:
runAsNonRoot: true
runAsUser: 2000
serviceAccountName: porter-admission
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
name: porter-admission
webhooks:
- admissionReviewVersions:
- v1beta1
- v1
clientConfig:
service:
name: porter-admission
namespace: porter-system
path: /validate-network-kubesphere-io-v1alpha2-eip
failurePolicy: Fail
matchPolicy: Equivalent
name: validate.eip.network.kubesphere.io
rules:
- apiGroups:
- network.kubesphere.io
apiVersions:
- v1alpha2
operations:
- CREATE
- UPDATE
resources:
- eips
sideEffects: None