F5实现k8s环境下应用自动发布

Kubernetes设计角度灵活的利于了clusterip实现了集群内部的服务互访，然而针对面向互联网的应用，需要实现基于集群外部的发布。满足DevOps的趋势，需要实现k8s环境下应用的自动发布。

Container Connector包含f5-k8s-controller和用户定义的“F5资源”。 f5-k8s-controller是一个可以在Kubernetes Pod中运行的Docker容器。
“F5资源”是Kubernetes ConfigMap资源，它将编码数据传递给f5-k8s-controller。这些资源告诉f5-k8s-controller：
•在BIG-IP上配置哪些对象
•BIG-IP对象所属的Kubernetes服务（分别是ConfigMap中的前端和后端属性）。
f5-k8s-controller监视Kubernetes中F5资源的创建和修改。当它发现更改时，它会相应地修改BIG-IP。例如，对于F5 virtualServer资源，
CC-Kubernetes执行以下操作：
•创建对象以表示指定分区中BIG-IP上的VS;
•使用Kubernetes分配给服务端口的NodePort为Kubernetes集群中的每个节点创建pool member;
•监控F5资源和链接的Kubernetes resources for changes并重新配置BIG-IP。
•然后，BIG-IP在指定的虚拟地址上处理服务的流量，并对集群中的所有节点进行负载平衡。
•在群集内，分配的NodePort负载均衡到服务的所有Pod。

下面讲演示通过F5 CC（Container Connector）与F5 BIGIP联动实现业务自动的对外发布。

1. 在BIGIP设备上创建新的partition，命名为kubernetes

2. 创建serviceaccount

kubectl create serviceaccount bigip-ctlr -n kube-system

3. 创建RBAC policy f5-k8s-sample-rbac.yaml，并apply

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: bigip-ctlr-clusterrole
rules:

• apiGroups:
  • “”
  • “extensions”
    resources:
  • nodes
  • services
  • endpoints
  • namespaces
  • ingresses
  • secrets
    verbs:
  • get
  • list
  • watch
• apiGroups:
  • “”
  • “extensions”
    resources:
  • configmaps
  • events
  • ingresses/status
    verbs:
  • get
  • list
  • watch
  • update
  • create
  • patch

---

kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: bigip-ctlr-clusterrole-binding
namespace: kube-system
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: bigip-ctlr-clusterrole
subjects:

• kind: ServiceAccount
  name: bigip-ctlr
  namespace: kube-system

kubectl create -f f5-k8s-sample-rbac.yaml

4. 创建secrect保存admin用户和密码

kubectl create secret generic bigip-login --namespace kube-system --from-literal=username=admin --from-literal=password=admin

5. 创建f5-cc-deployment.yaml，并apply

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: k8s-bigip-ctlr-deployment
namespace: kube-system
spec:
replicas: 1
template:
metadata:
name: k8s-bigip-ctlr
labels:
app: k8s-bigip-ctlr
spec:
serviceAccountName: bigip-ctlr
containers:
- name: k8s-bigip-ctlr
# replace the version as needed
image: “f5networks/k8s-bigip-ctlr:1.7.1”
env:
- name: BIGIP_USERNAME
valueFrom:
secretKeyRef:
name: bigip-login
key: username
- name: BIGIP_PASSWORD
valueFrom:
secretKeyRef:
name: bigip-login
key: password
command: ["/app/bin/k8s-bigip-ctlr"]
args: [
“–bigip-username=$(BIGI{P}_{U}SERNAME)","--bigip-password=$(BIGIP_PASSWORD)”,
“–bigip-url=10.1.10.176”,
“–bigip-partition=kubernetes”,
# The Controller can use local DNS to resolve hostnames;
# defaults to LOOKUP; can be replaced with custom DNS server IP
# or left blank (introduced in v1.3.0)
#"–resolve-ingress-names=LOOKUP",
#"–pool-member-type=cluster",
#"–flannel-name=flannel_vxlan"
# The Controller can access Secrets by default;
# set to “false” if you only want to use preconfigured
# BIG-IP SSL profiles
#"–use-secrets=false",
# The Controller watches all namespaces by default.
# To manage a single namespace, or multiple namespaces, provide a
# single entry for each. For example:
# “–namespace=test”,
# “–namespace=prod”
]
imagePullSecrets:
- name: f5-docker-images
- name: bigip-login

kubectl create -f f5-cc-deployment.yaml

创建完成后通过kubectl get pod -n kube-system | grep bigip 命令确认CC状态

kubectl get pod -n kube-system | grep bigip
k8s-bigip-ctlr-deployment-66977b7bc6-x9rbf 1/1 Running 0 41m

使用命令kubectl logs k8s-bigip-ctlr-deployment-66977b7bc6-x9rbf -n kube-system确认CC处于监听状态

kubectl logs k8s-bigip-ctlr-deployment-66977b7bc6-x9rbf -n kube-system
2019/04/29 07:18:46 [INFO] Starting: Version: v1.7.1, BuildInfo: n1279-465125010
2019/04/29 07:18:46 [INFO] ConfigWriter started: 0xc000445a10
2019/04/29 07:18:46 [INFO] Started config driver sub-process at pid: 12
2019/04/29 07:18:46 [INFO] NodePoller (0xc0003b4900) registering new listener: 0x1829650
2019/04/29 07:18:46 [INFO] NodePoller started: (0xc0003b4900)
2019/04/29 07:18:46 [INFO] Watching ConfigMap resources.
2019/04/29 07:18:46 [INFO] Handling ConfigMap resource events.
2019/04/29 07:18:46 [INFO] Registered BigIP Metrics
2019/04/29 07:18:46 [INFO] Wrote 0 Virtual Server and 0 IApp configs
2019/04/29 07:18:47 [INFO] [2019-04-29 07:18:47,333 main INFO] entering inotify loop to watch /tmp/k8s-bigip-ctlr.config215984094/config.json

5. 在k8s上创建应用，共包含三个yaml 文件my-frontend-deployment.yaml，my-frontend-configmap.yaml，my-frontend-service.yaml。

my-frontend-deployment.yaml

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: my-frontend
spec:
replicas: 2
template:
metadata:
labels:
run: my-frontend
spec:
containers:
- image: “chen23/f5-demo-app”
env:
- name: F5DEMO_APP
value: “frontend”
- name: F5DEMO_BACKEND_URL
value: “http://my-backend/”
imagePullPolicy: IfNotPresent
name: my-frontend
ports:
- containerPort: 80
protocol: TCP

my-frontend-configmap.yaml

kind: ConfigMap
apiVersion: v1
metadata:
name: my-frontend
namespace: default
labels:
f5type: virtual-server
data:
schema: “f5schemadb://bigip-virtual-server_v0.1.7.json”
data: |-
{
“virtualServer”: {
“frontend”: {
“balance”: “round-robin”,
“mode”: “http”,
“partition”: “kubernetes”,
“virtualAddress”: {
“bindAddr”: “10.1.10.81”,
“port”: 80
}
},
“backend”: {
“serviceName”: “my-frontend”,
“servicePort”: 80,
“healthMonitors”: [{
“interval”: 30,
“protocol”: “http”,
“send”: “HEAD / HTTP/1.0\r\n\r\n”,
“timeout”: 91
}]
}
}
}

my-frontend-service.yaml

apiVersion: v1
kind: Service
metadata:
name: my-frontend
labels:
run: my-frontend
spec:
ports:

• port: 80
  protocol: TCP
  targetPort: 80
  type: NodePort
  selector:
  run: my-frontend

kubectl create -f my-frontend-deployment.yaml
kubectl create -f my-frontend-configmap.yaml
kubectl create -f my-frontend-service.yaml

创建完成后查看service，该服务监听在31752端口

kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 443/TCP 125d
my-frontend NodePort 10.109.185.196 80:31752/TCP 47m

6. 检查BIGIP上配置，worker node 10.1.1.175的端口31752自动添加到BIGIP配置中。

通过浏览器访问可以打开页面：