kubernetes -- 03
服务与卷
服务基础
创建后端
[root@master config]# vim apache-example.yaml --- apiVersion: apps/v1 kind: Deployment metadata: name: apache-example spec: selector: matchLabels: app: myapp-apache replicas: 2 template: metadata: labels: app: myapp-apache spec: containers: - name: apache image: 192.168.1.100:5000/myos:httpd ports: - protocol: TCP containerPort: 80 restartPolicy: Always [root@master config]# kubectl apply -f apache-example.yaml [root@master config]# kubectl get pod NAME READY STATUS RESTARTS AGE apache-example-65fb568b4c-p6mrl 1/1 Running 0 4m6s
创建服务
[root@master config]# vim service-example.yaml --- apiVersion: v1 kind: Service metadata: name: apache-service spec: ports: - protocol: TCP port: 80 targetPort: 80 selector: app: myapp-apache type: ClusterIP [root@master config]# kubectl apply -f service-example.yaml [root@master config]# kubectl get service NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE apache-service ClusterIP 10.254.78.148 <none> 80/TCP 9m46s
访问服务
服务只有在集群内部才可以访问,创建 Pod,在Pod 中访问服务
[root@master config]# vim pod-example.yaml --- apiVersion: v1 kind: Pod metadata: name: pod-example labels: app: myapp spec: containers: - name: myos image: 192.168.1.100:5000/myos:v1804 stdin: true tty: true restartPolicy: Always [root@master config]# kubectl apply -f pod-example.yaml [root@master config]# kubectl exec -it pod-example -- /bin/bash [root@pod-example /]# curl http://10.254.78.148/info.php <pre> Array ( [REMOTE_ADDR] => 10.244.3.12 [REQUEST_METHOD] => GET [HTTP_USER_AGENT] => curl/7.29.0 [REQUEST_URI] => /info.php ) php_host: apache-example-65fb568b4c-thks8 1229
扩容集群节点,服务自动扩展
# 在master上执行扩容节点 [root@master ~]# kubectl scale deployment apache-example --replicas=2 # 服务本质是LVS规则 [root@master ~]# ipvsadm -L -n TCP 10.254.78.148:80 rr -> 10.244.4.66:80 Masq 1 0 0 -> 10.244.5.11:80 Masq 1 0 0 ----------------------------------------------------------------------------------------- # 在pod里访问 [root@pod-example /]# curl http://10.254.78.148/info.php ... ... php_host: apache-example-65fb568b4c-p6mrl ... ... php_host: apache-example-65fb568b4c-thks8
headless服务
[root@master ~]# vim headless-service.yaml --- apiVersion: v1 kind: Service metadata: name: apache-headless spec: ports: - protocol: TCP port: 80 targetPort: 80 selector: app: myapp-apache type: ClusterIP clusterIP: None [root@master ~]# kubectl apply -f headless-service.yaml service/apache-headless created [root@master ~]# kubectl get service NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE apache-headless ClusterIP None <none> 80/TCP 5s apache-service ClusterIP 10.254.78.148 <none> 80/TCP 74m kubernetes ClusterIP 10.254.0.1 <none> 443/TCP 2d20h #-----------------------------------进入pod查看解析结果------------------------------------ [root@master ~]# kubectl exec -it pod-example -- /bin/bash [root@pod-example /]# yum install -y bind-utils [root@pod-example /]# host apache-headless.default.svc.cluster.local apache-headless.default.svc.cluster.local has address 10.244.5.11 apache-headless.default.svc.cluster.local has address 10.244.4.66
nodeport服务
[root@master ~]# vim nodeport-example.yaml --- apiVersion: v1 kind: Service metadata: name: apache-nodeport spec: ports: - protocol: TCP port: 80 targetPort: 80 selector: app: myapp-apache type: NodePort [root@master ~]# kubectl apply -f nodeport-example.yaml [root@master ~]# kubectl get service NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE apache-nodeport NodePort 10.254.24.100 <none> 80:31410/TCP 7m10s #---------------------------所有node节点31410端口均可访问----------------------------------- # 在跳板机上访问服务 [root@ecs-proxy ~]# curl http://192.168.1.31:31410/info.php [root@ecs-proxy ~]# curl http://192.168.1.32:31410/info.php [root@ecs-proxy ~]# curl http://192.168.1.33:31410/info.php
ingress控制器
[root@master ingress]# docker load -i ingress-nginx.tar.gz [root@master ingress]# docker tag quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.30.0 192.168.1.100:5000/nginx-ingress-controller:0.30.0 [root@master ingress]# docker push 192.168.1.100:5000/nginx-ingress-controller:0.30.0 [root@master ingress]# curl http://192.168.1.100:5000/v2/nginx-ingress-controller/tags/list {"name":"nginx-ingress-controller","tags":["0.30.0"]}
安装控制器
[root@master ~]# vim ingress/mandatory.yaml 221: image: 192.168.1.100:5000/nginx-ingress-controller:0.30.0 [root@master ~]# kubectl apply -f ingress/mandatory.yaml [root@master ~]# kubectl -n ingress-nginx get pod NAME READY STATUS RESTARTS AGE nginx-ingress-controller-fc6766d7-ptppp 1/1 Running 0 47s [root@master ~]# vim ingress/ingress-service.yaml --- kind: Service apiVersion: v1 metadata: name: ingress-nginx namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx spec: externalTrafficPolicy: Local type: LoadBalancer externalIPs: - 192.168.1.101 selector: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx ports: - name: http port: 80 protocol: TCP targetPort: http - name: https port: 443 protocol: TCP targetPort: https [root@master ~]# kubectl apply -f ingress/ingress-service.yaml service/ingress-nginx created [root@master ~]# kubectl -n ingress-nginx get service NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) ingress-nginx LoadBalancer 10.254.152.162 192.168.1.101 80:30143/TCP,443:31025/TCP [root@master ingress]# vim ingress-example.yaml --- apiVersion: extensions/v1beta1 kind: Ingress metadata: name: my-app annotations: kubernetes.io/ingress.class: "nginx" spec: backend: serviceName: apache-service servicePort: 80 [root@master ingress]# kubectl apply -f ingress-example.yaml [root@master ingress]# kubectl get ingresses NAME HOSTS ADDRESS PORTS AGE my-app * 192.168.1.33 80 3m2s #----------------------- 在跳板机访问测试 ------------------------------------------------- [root@ecs-proxy ~]# curl http://192.168.1.33/info.php <pre> Array ( [REMOTE_ADDR] => 10.244.3.0 [REQUEST_METHOD] => GET [HTTP_USER_AGENT] => curl/7.29.0 [REQUEST_URI] => /info.php ) php_host: apache-example-65fb568b4c-p6mrl 1229
存储卷
configmap配置
获取 nginx 配置文件,参考运维课程添加动静分离,并创建 configmap
[root@master configmap]# vim nginx.conf ... ... location ~ \.php$ { root html; fastcgi_pass 127.0.0.1:9000; fastcgi_index index.php; include fastcgi.conf; } ... ... [root@master configmap]# kubectl create configmap nginx-conf --from-file=nginx.conf configmap/nginx-conf created [root@master configmap]# kubectl get configmaps NAME DATA AGE nginx-conf 1 8s
如果需要修改配置文件
1、删除 configmap
编辑新的配置文件,重新创建configmap
2、使用 kubectl edit configmap 名称直接修改(注意格式)
在容器中使用configmap
由于 apache 与 nginx 都使用 80 端口,把之前的实验容器全部删除
[root@master configmap]# kubectl get service NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE apache-service ClusterIP 10.254.78.148 <none> 80/TCP 5h43m kubernetes ClusterIP 10.254.0.1 <none> 443/TCP 3d [root@master configmap]# kubectl delete service apache-service service "apache-service" deleted [root@master configmap]# kubectl get deployments NAME READY UP-TO-DATE AVAILABLE AGE apache-example 2/2 2 2 5h48m [root@master configmap]# kubectl delete deployments apache-example deployment.apps "apache-example" deleted
创建 nginx + php 容器,调用 configmap
[root@master configmap]# vim nginx-example.yaml --- apiVersion: apps/v1 kind: Deployment metadata: name: nginx-example spec: selector: matchLabels: app: myapp-nginx replicas: 1 template: metadata: labels: app: myapp-nginx spec: volumes: - name: nginx-php configMap: name: nginx-conf containers: - name: nginx image: 192.168.1.100:5000/myos:nginx ports: - protocol: TCP containerPort: 80 volumeMounts: - name: nginx-php subPath: nginx.conf mountPath: /usr/local/nginx/conf/nginx.conf - name: php image: 192.168.1.100:5000/myos:php-fpm restartPolicy: Always [root@master configmap]# kubectl apply -f nginx-example.yaml [root@master configmap]# kubectl get pod -o wide NAME READY STATUS RESTARTS AGE IP nginx-example-bbb8ddf7b-kxrdf 2/2 Running 0 38s 10.244.4.67 [root@master configmap]# curl http://10.244.4.67/info.html <html> <marquee behavior="alternate"> <font size="12px" color=#00ff00>Hello World</font> </marquee> </html> [root@master configmap]# curl http://10.244.4.67/info.php <pre> Array ( [REMOTE_ADDR] => 10.244.0.0 [REQUEST_METHOD] => GET [HTTP_USER_AGENT] => curl/7.29.0 [REQUEST_URI] => /info.php ) php_host: nginx-example-bbb8ddf7b-kxrdf 1229
持久化存储卷
在 registry 上搭建NFS服务器
[root@registry ~]# yum install -y nfs-utils [root@registry ~]# mkdir -m 777 /var/webroot [root@registry ~]# vim /etc/exports /var/webroot *(rw) [root@registry ~]# systemctl enable --now nfs #-------------------------------下面在任意其他节点测试------------------------------ [root@master ~]# yum install -y nfs-utils [root@master ~]# showmount -e 192.168.1.100 Export list for 192.168.1.100: /var/webroot *
创建 PV
[root@master configmap]# vim pv-example.yaml --- apiVersion: v1 kind: PersistentVolume metadata: name: pv-nfs labels: app: web-nfs spec: volumeMode: Filesystem capacity: storage: 10Gi accessModes: - ReadWriteMany persistentVolumeReclaimPolicy: Retain nfs: path: /var/webroot server: 192.168.1.100 [root@master configmap]# kubectl apply -f pv-example.yaml persistentvolume/pv-nfs created [root@master configmap]# kubectl get pv NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS pv-nfs 10Gi RWX Retain Available
创建pvc,并绑定pv
[root@master configmap]# vim pvc-example.yaml --- apiVersion: v1 kind: PersistentVolumeClaim metadata: name: pvc-nfs spec: volumeMode: Filesystem accessModes: - ReadWriteMany resources: requests: storage: 8Gi selector: matchLabels: app: web-nfs [root@master configmap]# kubectl apply -f pvc-example.yaml [root@master configmap]# kubectl get pv NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM pv-nfs 10Gi RWX Retain Bound default/pvc-nfs [root@master configmap]# kubectl get pvc NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE pvc-nfs Bound pv-nfs 10Gi RWX 27s
为资源文件 pod 添加持久卷访问支持,并在所有节点安装 nfs-utils 软件工具包
[root@node-0001 ~]# yum install -y nfs-utils ----------------------------------------------------------------------------------------- [root@node-0002 ~]# yum install -y nfs-utils ----------------------------------------------------------------------------------------- [root@node-0003 ~]# yum install -y nfs-utils ----------------------------------------------------------------------------------------- [root@master configmap]# kubectl delete -f nginx-example.yaml deployment.apps "nginx-example" deleted [root@master configmap]# vim nginx-example.yaml --- apiVersion: apps/v1 kind: Deployment metadata: name: nginx-example spec: selector: matchLabels: app: myapp-nginx replicas: 3 template: metadata: labels: app: myapp-nginx spec: volumes: - name: nginx-php configMap: name: nginx-conf - name: site-data persistentVolumeClaim: claimName: pvc-nfs containers: - name: nginx image: 192.168.1.100:5000/myos:nginx ports: - protocol: TCP containerPort: 80 volumeMounts: - name: nginx-php subPath: nginx.conf mountPath: /usr/local/nginx/conf/nginx.conf - name: site-data mountPath: /usr/local/nginx/html - name: php image: 192.168.1.100:5000/myos:php-fpm volumeMounts: - name: site-data mountPath: /usr/local/nginx/html restartPolicy: Always [root@master configmap]# kubectl apply -f nginx-example.yaml
添加 service 服务,并使用 ingress 发布到集群外部
[root@master configmap]# vim nginx-service.yaml --- apiVersion: v1 kind: Service metadata: name: nginx-service spec: ports: - protocol: TCP port: 80 targetPort: 80 selector: app: myapp-nginx type: ClusterIP --- apiVersion: extensions/v1beta1 kind: Ingress metadata: name: my-app annotations: kubernetes.io/ingress.class: "nginx" spec: backend: serviceName: nginx-service servicePort: 80 [root@master configmap]# kubectl apply -f nginx-service.yaml
在registry上添加网页文件,在跳板机上完成访问测试
拷贝 info.php 到 nfs server 的 /var/webroot/ 目录下
[root@ecs-proxy ~]# watch -n 1 'curl -s http://192.168.1.33/info.php'