Istio 金丝雀发布

转载：备考ICA-Istio 金丝雀实验4

 

环境清理

kubectl delete gw/helloworld-gateway vs/helloworld dr/helloworld-destination

#测试
kubectl get svc,pods
for i in {1..10};do curl $(kubectl get svc helloworld|grep helloworld|awk '{print $3":"$5}'|awk -F"/" '{print $1"/hello"}');sleep .5 ;done
kubectl get gw,vs,dr

恢复成上述这样就可以实验通过helloword的svc将流量随机分配到v1和v2上。

如果实验环境有问题，可以重新部署hello。

kubectl delete -f istio/samples/helloworld/helloworld.yaml
kubectl apple -f istio/samples/helloworld/helloworld.yaml

所有流量转发到v1

模拟存在一个版本

canary/helloworld-canary-all-v1.yaml

apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  name: helloworld-gateway
spec:
  selector:
    istio: ingressgateway # use istio default controller
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "*"
---
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: helloworld-destination
spec:
  host: helloworld
  subsets:
  - name: v1
    labels:
      version: v1
  - name: v2
    labels:
      version: v2
---
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: helloworld
spec:
  hosts:
  - "*"
  gateways:
  - helloworld-gateway
  http:
  - match:
    - uri:
        exact: /hello
    route:
    - destination:
        host: helloworld
        port:
          number: 5000
        subset: v1
      weight: 100

部署gw,vs,dr

kubectl apply -f canary/helloworld-canary-all-v1.yaml

验证测试结果(流量应该都交给v1进行响应)

for i in {1..10};do curl http://192.168.126.220/hello;sleep .5;done

 90%流量v1,10%流量v2

此时v2版本应用已经上线,将10%流量给v2,其余流量仍由v1进行应答

配置流量分发比例

canary/helloworld-canary-allin1-10v2.yaml

apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  name: helloworld-gateway
spec:
  selector:
    istio: ingressgateway # use istio default controller
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "*"
---
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: helloworld-destination
spec:
  host: helloworld
  subsets:
  - name: v1
    labels:
      version: v1
  - name: v2
    labels:
      version: v2
---
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: helloworld
spec:
  hosts:
  - "*"
  gateways:
  - helloworld-gateway
  http:
  - match:
    - uri:
        exact: /hello
    route:
    - destination:
        host: helloworld
        port:
          number: 5000
        subset: v1
      weight: 90
    - destination:
        host: helloworld
        port:
          number: 5000
        subset: v2
      weight: 10

部署gw,vs,dr

kubectl apply -f canary/helloworld-canary-allin1-10v2.yaml

测试效果

可以看到10个请求中有1个由v2应答,其他仍由v1进行响应

 

加入HPA

---
apiVersion: autoscaling/v1
kind: HorizontalPodAutoscaler
metadata:
  name: hpa-helloworld-v1
spec:
  maxReplicas: 20
  minReplicas: 1
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: helloworld-v1
  targetCPUUtilizationPercentage: 50
---
apiVersion: autoscaling/v1
kind: HorizontalPodAutoscaler
metadata:
  name: hpa-helloworld-v2
spec:
  maxReplicas: 20
  minReplicas: 1
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: helloworld-v2
  targetCPUUtilizationPercentage: 50

部署hpa

kubectl apply -f canary/hpa.yaml

压测

while true;do curl http://192.168.126.220/hello;done

产生大量请求

此时v1,v2因访问量大触发hpa扩容,直到v1到达上线16个pod,v2到达3个

50%流量v1，50流量v2

配置流量分发比例

修改vs的weight：

---
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: helloworld
spec:
  hosts:
  - "*"
  gateways:
  - helloworld-gateway
  http:
  - match:
    - uri:
        exact: /hello
    route:
    - destination:
        host: helloworld
        port:
          number: 5000
        subset: v1
      weight: 50
    - destination:
        host: helloworld
        port:
          number: 5000
        subset: v2
      weight: 50

kubectl apply -f canary/helloworld-canary-vs-50v2.yaml

 

压测

流量以1:1分发给v1和v2

再观测hpa的情况会发现v2的cpu逐渐升高,v1的cpu逐渐降低,v2开始扩容,v1开始缩容,逐渐扩缩容到10:10

所有流量转发至v2

---
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: helloworld
spec:
  hosts:
  - "*"
  gateways:
  - helloworld-gateway
  http:
  - match:
    - uri:
        exact: /hello
    route:
    - destination:
        host: helloworld
        port:
          number: 5000
        subset: v2
      weight: 100

kubectl apply -f canary/helloworld-canary-all-v2.yaml

压测

while true;do curl http://192.168.126.220/hello;done

拓展Canary +AB测试

canary + ab配置

进行Canary测试时，平衡普通用户和测试人员之间的流量分配，例如普通用户流量1:1分配到两个版本，测试人员分配v2版本上。

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: helloworld
spec:
  hosts:
  - "*"
  gateways:
  - helloworld-gateway
  http:
  - match:
    - headers:
        user:
          exact: trump
      uri:
        exact: /hello
    route:
    - destination:
        host: helloworld
        port:
          number: 5000
        subset: v2
      weight: 100
  - route:
    - destination:
        host: helloworld
        port:
          number: 5000
        subset: v1
      weight: 50
    - destination:
        host: helloworld
        port:
          number: 5000
        subset: v2
      weight: 50

kubectl apply -f canary/canary-ab-vs.yaml

 

测试

普通用户(流量1:1)

for i in {1..20};do curl http://192.168.126.220/hello;done

测试人员(流量至v2)

for i in {1..20};do curl -H "user:trump" http://192.168.126.220/hello;done