kubernetes练习（一）

1、查看pod日志，并将日志中Error的行记录到指定文件
pod名称：web
文件：/opt/web_error.log

[root@k8s-master ~]# kubectl logs web-nginx-b9c949c47-mgx9k | grep error
2021/03/25 08:41:05 [error] 6#6: *2 open() "/usr/share/nginx/html/favicon.ico" failed (2: No such file or directory), client: 10.244.190.0, server: localhost, request: "GET /favicon.ico HTTP/1.1", host: "10.10.20.215:31736", referrer: "http://10.10.20.215:31736/"
2021/03/25 08:41:22 [error] 6#6: *4 open() "/usr/share/nginx/html/favicon.ico" failed (2: No such file or directory), client: 10.10.20.214, server: localhost, request: "GET /favicon.ico HTTP/1.1", host: "10.10.20.214:31736", referrer: "http://10.10.20.214:31736/"
2021/03/31 07:57:11 [error] 6#6: *8 open() "/usr/share/nginx/html/favicon.ico" failed (2: No such file or directory), client: 10.244.135.64, server: localhost, request: "GET /favicon.ico HTTP/1.1", host: "10.10.20.213:31736", referrer: "http://10.10.20.213:31736/"
2021/03/31 07:57:42 [error] 6#6: *9 open() "/usr/share/nginx/html/favicon.ico" failed (2: No such file or directory), client: 10.244.135.64, server: localhost, request: "GET /favicon.ico HTTP/1.1", host: "10.10.20.213:31736", referrer: "http://10.10.20.213:31736/"
[root@k8s-master ~]# kubectl logs web-nginx-b9c949c47-mgx9k | grep error >>/opt/web_error.log
[root@k8s-master ~]# cat /opt/web_error.log
2021/03/25 08:41:05 [error] 6#6: *2 open() "/usr/share/nginx/html/favicon.ico" failed (2: No such file or directory), client: 10.244.190.0, server: localhost, request: "GET /favicon.ico HTTP/1.1", host: "10.10.20.215:31736", referrer: "http://10.10.20.215:31736/"
2021/03/25 08:41:22 [error] 6#6: *4 open() "/usr/share/nginx/html/favicon.ico" failed (2: No such file or directory), client: 10.10.20.214, server: localhost, request: "GET /favicon.ico HTTP/1.1", host: "10.10.20.214:31736", referrer: "http://10.10.20.214:31736/"
2021/03/31 07:57:11 [error] 6#6: *8 open() "/usr/share/nginx/html/favicon.ico" failed (2: No such file or directory), client: 10.244.135.64, server: localhost, request: "GET /favicon.ico HTTP/1.1", host: "10.10.20.213:31736", referrer: "http://10.10.20.213:31736/"
2021/03/31 07:57:42 [error] 6#6: *9 open() "/usr/share/nginx/html/favicon.ico" failed (2: No such file or directory), client: 10.244.135.64, server: localhost, request: "GET /favicon.ico HTTP/1.1", host: "10.10.20.213:31736", referrer: "http://10.10.20.213:31736/"

扩展：
把pod中的容器日志输出到宿主机之上，这里以nginx容器为例，使用宿主机挂在卷

[root@k8s-master manifest]# cat web-dep.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  creationTimestamp: null
  labels:
    app: web-dep
  name: web-dep
spec:
  replicas: 1
  selector:
    matchLabels:
      app: web-dep
  strategy: {}
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: web-dep
    spec:
      containers:
      - image: nginx:1.14.2
        name: nginx1-14-2
        resources: {}
        volumeMounts:
          - mountPath: /var/log/nginx/
            name: test-volume
      volumes:
        - name: test-volume
          hostPath:
            path: /opt/logs
            type: DirectoryOrCreate  ##如果在给定路径上什么都不存在，那么将根据需要创建空目录。

查看pod的运行节点，可以看出在node2节点上

[root@k8s-master manifest]# kubectl get pod -o wide
NAME                        READY   STATUS    RESTARTS   AGE     IP             NODE            NOMINATED NODE   READINESS GATES
web-dep-5f46d86b7-7522m     1/1     Running   0          8m30s   10.244.190.4   k8s-node2.com   <none>           <none>

访问测试

[root@k8s-master ~]# for i in {1..5};do curl 10.244.190.4;done

到node2节点上查看，输出的日志信息

[root@k8s-node2 ~]# cd /opt/logs
[root@k8s-node2 logs]# ls
access.log  error.log
[root@k8s-node2 logs]# cat access.log 
10.244.135.64 - - [31/Mar/2021:06:51:31 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.29.0" "-"
10.244.135.64 - - [31/Mar/2021:06:52:17 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.29.0" "-"
10.244.135.64 - - [31/Mar/2021:06:52:17 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.29.0" "-"
10.244.135.64 - - [31/Mar/2021:06:52:17 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.29.0" "-"
10.244.135.64 - - [31/Mar/2021:06:52:17 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.29.0" "-"
10.244.135.64 - - [31/Mar/2021:06:52:17 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.29.0" "-"
10.244.135.64 - - [31/Mar/2021:07:50:51 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.29.0" "-"
10.244.135.64 - - [31/Mar/2021:07:50:51 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.29.0" "-"
10.244.135.64 - - [31/Mar/2021:07:50:51 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.29.0" "-"
10.244.135.64 - - [31/Mar/2021:07:50:51 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.29.0" "-"
10.244.135.64 - - [31/Mar/2021:07:50:51 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.29.0" "-"

在pod对象内部查看nginx的日志信息，发现日志内容完全一样

[root@k8s-master ~]# kubectl exec -it web-dep-5f46d86b7-7522m bash
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
root@web-dep-5f46d86b7-7522m:/# tail /var/log/nginx/access.log 
10.244.135.64 - - [31/Mar/2021:06:52:17 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.29.0" "-"
10.244.135.64 - - [31/Mar/2021:06:52:17 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.29.0" "-"
10.244.135.64 - - [31/Mar/2021:06:52:17 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.29.0" "-"
10.244.135.64 - - [31/Mar/2021:06:52:17 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.29.0" "-"
10.244.135.64 - - [31/Mar/2021:06:52:17 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.29.0" "-"
10.244.135.64 - - [31/Mar/2021:07:50:51 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.29.0" "-"
10.244.135.64 - - [31/Mar/2021:07:50:51 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.29.0" "-"
10.244.135.64 - - [31/Mar/2021:07:50:51 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.29.0" "-"
10.244.135.64 - - [31/Mar/2021:07:50:51 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.29.0" "-"
10.244.135.64 - - [31/Mar/2021:07:50:51 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.29.0" "-"

2、查看指定标签使用cpu最高的pod，并记录到到指定文件
标签：app=web-nginx
文件：/opt/cpu
查看监控指标需要在集群中安装Metrics Server
yaml文件地址： https://github.com/kubernetes-sigs/metrics-server/releases/download/v0.3.7/components.yaml
下载好yaml文件后，还需要编辑一下此文件，因为默认的镜像下载地址在国内是无法访问的，这里需要需改一下。

[root@k8s-master manifest]# vim components.yaml 
...
      containers:
      - name: metrics-server
        #image: k8s.gcr.io/metrics-server/metrics-server:v0.3.7
        image: lizhenliang/metrics-server:v0.3.7  ##修改镜像下载地址
        imagePullPolicy: IfNotPresent
        args:
          - --cert-dir=/tmp
          - --secure-port=4443
          - --kubelet-insecure-tls                                             #加上此项，不验证kubelet提供的https证书
          - --kubelet-preferred-address-types=InternalIP        #加上此项，使用节点IP连接kubelet
 ...

[root@k8s-master manifest]# kubectl apply -f components.yaml 
clusterrole.rbac.authorization.k8s.io/system:aggregated-metrics-reader created
clusterrolebinding.rbac.authorization.k8s.io/metrics-server:system:auth-delegator created
rolebinding.rbac.authorization.k8s.io/metrics-server-auth-reader created
Warning: apiregistration.k8s.io/v1beta1 APIService is deprecated in v1.19+, unavailable in v1.22+; use apiregistration.k8s.io/v1 APIService
apiservice.apiregistration.k8s.io/v1beta1.metrics.k8s.io created
serviceaccount/metrics-server created
deployment.apps/metrics-server created
service/metrics-server created
clusterrole.rbac.authorization.k8s.io/system:metrics-server created
clusterrolebinding.rbac.authorization.k8s.io/system:metrics-server created
##验证
[root@k8s-master manifest]# kubectl get pods -n kube-system | grep metrics
metrics-server-85f5cb5f84-5rjxx            1/1     Running   0          4m30s

查看Node资源消耗：

[root@k8s-master manifest]# kubectl top node k8s-node1.com
NAME            CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%   
k8s-node1.com   87m          4%     901Mi           51%       
[root@k8s-master manifest]# kubectl top node k8s-node2.com
NAME            CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%   
k8s-node2.com   86m          4%     877Mi           50%       
[root@k8s-master manifest]# kubectl top node k8s-master.com
NAME             CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%   
k8s-master.com   158m         7%     1128Mi          64% 

查看Pod资源消耗

[root@k8s-master manifest]# kubectl top pod web-dep-5f46d86b7-7522m
NAME                      CPU(cores)   MEMORY(bytes)   
web-dep-5f46d86b7-7522m   0m           2Mi 

按照指定条件查找并排序

[root@k8s-master manifest]# kubectl top pod -l app  --sort-by cpu
NAME                        CPU(cores)   MEMORY(bytes)   
web-nginx-b9c949c47-mgx9k   0m           2Mi             
web-dep-5f46d86b7-7522m     0m           2Mi             
[root@k8s-master manifest]# kubectl top pod -l app  --sort-by memory
NAME                        CPU(cores)   MEMORY(bytes)   
web-dep-5f46d86b7-7522m     0m           2Mi             
web-nginx-b9c949c47-mgx9k   0m           2Mi 
##输出到指定文件
[root@k8s-master manifest]# kubectl top pod -l app  --sort-by cpu | head -n2 | tail -n1 >>/opt/pod

3、Pod里创建一个边车容器读取业务容器日志

apiVersion: v1
kind: Pod
metadata:
  name: counter
spec:
  containers:
  - name: count
    image: busybox
    args:
    - /bin/sh
    - -c
    - >
      i=0;
      while true;
      do
        echo "$i: $(date)" >> /var/log/1.log;
        echo "$(date) INFO $i" >> /var/log/2.log;
        i=$((i+1));
        sleep 1;
      done
    volumeMounts:
    - name: varlog
      mountPath: /var/log
  - name: count-log-1
    image: busybox
    args: [/bin/sh, -c, 'tail -n+1 -f /var/log/1.log']
    volumeMounts:
    - name: varlog
      mountPath: /var/log
  - name: count-log-2
    image: busybox
    args: [/bin/sh, -c, 'tail -n+1 -f /var/log/2.log']
    volumeMounts:
    - name: varlog
      mountPath: /var/log
  volumes:
  - name: varlog
    emptyDir: {}

4、pod配置多容器，pod中分别运行nginx /redis /memcached

apiVersion: v1
kind: Pod
metadata:
  name: kucc
spec:
  containers:
  - name: nginx
    image: nginx:1.14.2
  - name: redis
    image: redis
  - name: memcached
    image: memcached

5、创建PV，容量为2G

[root@k8s-master manifest]# vim app-data-pv.yaml
apiVersion: v1
kind: PersistentVolume
metadata:
  name: app-data
spec:
  capacity:
    storage: 2Gi
  volumeMode: Filesystem
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  hostPath:
    path: "/srv/app-data"

查看

[root@k8s-master manifest]# kubectl apply -f app-data-pv.yaml 
persistentvolume/app-data created
[root@k8s-master manifest]# kubectl get pv
NAME       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM   STORAGECLASS   REASON   AGE
app-data   2Gi        RWO            Recycle          Available                                   7s

pod通过PVC来关联上面创建的PV

##创建pvc与pod
[root@k8s-master manifest]# vim pvc-pod.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
 name: pod-pvc

spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 2Gi

---
apiVersion: v1
kind: Pod
metadata:
  name: pod-use-pv

spec:
  containers:
    - name: nginx-pv
      image: nginx:1.14.2
      volumeMounts:
        - name: pv-demo
          mountPath: /var/log/nginx/

  volumes:
    - name: pv-demo
      persistentVolumeClaim:
        claimName: pod-pvc
        readOnly: false

##查看pvc与pv绑定状态
##pvc处于bound状态
[root@k8s-master manifest]# kubectl get pvc
NAME      STATUS   VOLUME     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
pod-pvc   Bound    app-data   2Gi        RWO                           3m42s

##pv也属于bound状态，且和pod-pvc这个pvc绑定
[root@k8s-master manifest]# kubectl get pv
NAME       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM             STORAGECLASS   REASON   AGE
app-data   2Gi        RWO            Recycle          Bound    default/pod-pvc                           5d19h

6、网络策略
创建两个命名空间my-app、my-dev，并在my-app命名空间中创建网络策略。策略规则为：
允许来自my-dev命名空间中的流量访问此命名空间pod的80端口
此命名空间内的pod相互之间只允许访问80端口

##创建命名空间
[root@k8s-master ~]# kubectl create namespace my-app
namespace/my-app created
[root@k8s-master ~]# kubectl create namespace my-dev
namespace/my-dev created
[root@k8s-master ~]# kubectl get ns
NAME              STATUS   AGE
default           Active   18d
kube-node-lease   Active   18d
kube-public       Active   18d
kube-system       Active   18d
my-app            Active   9s
my-dev            Active   9s

##在my-app命名空间运行一个Nginx容器的pod，默认监听的是80端口
[root@k8s-master ~]# kubectl run nginx-web --image=nginx:1.14.2 -n my-app
pod/nginx-web created
[root@k8s-master ~]# kubectl get pod -n my-app
NAME        READY   STATUS    RESTARTS   AGE
nginx-web   1/1     Running   0          31s
##在集群节点上访问此端口，可以正常访问
[root@k8s-master ~]# kubectl get pod -n my-app -o wide
NAME        READY   STATUS    RESTARTS   AGE   IP             NODE            NOMINATED NODE   READINESS GATES
nginx-web   1/1     Running   0          23m   10.244.89.72   k8s-node1.com   <none>           <none>
[root@k8s-master ~]# curl 10.244.89.72
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>
[root@k8s-master ~]# curl -I  10.244.89.72
HTTP/1.1 200 OK
Server: nginx/1.14.2
Date: Tue, 13 Apr 2021 05:34:52 GMT
Content-Type: text/html
Content-Length: 612
Last-Modified: Tue, 04 Dec 2018 14:44:49 GMT
Connection: keep-alive
ETag: "5c0692e1-264"
Accept-Ranges: bytes
##创建网络策略
[root@k8s-master manifest]# vim ingress-networkpolicy.yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: ingress-network-policy
  namespace: my-app  ##定义NetworkPolicy生效的命名空间
spec:
  podSelector: {}   ##表示匹配my-app命名空间下所有的pod对象，因此该命名空间下的所有pod对象的此网络策略都生效
  policyTypes:
  - Ingress         ##定义ingress规则生效

  ingress:
  - from:
    - namespaceSelector:
        matchLabels:
          name: my-dev  ##通过标签来匹配命名空间，因此在my-dev命名空间得拥有对应的标签
    - podSelector: {}    ##匹配my-app命名空间下的所有pod对象（若没有此条规则，则my-app命名空间中的pod都不可以互相访问）
    ports:
    - protocol: TCP
      port: 80

###podSelector前面‘“-”表示此选择器将在与 NetworkPolicy 相同的命名空间中选择特定的 Pod，没有“-”则表示此选择器将在与上面namespaceSelector字段匹配的命名空间中选择特定的 Pod
##也可以理解为有“-”是平级的关系，没有是附属关系，上面的规则中是有两个列表元素

[root@k8s-master manifest]# kubectl apply -f ingress-networkpolicy.yaml 
networkpolicy.networking.k8s.io/ingress-network-policy created
[root@k8s-master manifest]# kubectl get networkpolicy -n my-app
NAME                     POD-SELECTOR   AGE
ingress-network-policy   <none>         25s

##测试
##1、现在集群节点中访问，一直卡住不动，说明网络策略生效
[root@k8s-master manifest]# curl 10.244.89.72
curl: (7) Failed connect to 10.244.89.72:80; Connection timed out

##在my-dev命名空间中访问
##先给my-dev命名空间打标
[root@k8s-master manifest]# kubectl label ns my-dev name=my-dev
namespace/my-dev labeled
##先在my-dev中运行一个busybox容器的pod对象
[root@k8s-master manifest]# kubectl run busybox --image=busybox:stable --command sleep 3600 -n my-dev
pod/busybox created
[root@k8s-master manifest]# kubectl get pod -n my-dev
NAME      READY   STATUS    RESTARTS   AGE
busybox   1/1     Running   0          32s
##在my-dev命名空间中可以正常访问
[root@k8s-master manifest]# kubectl exec busybox -n my-dev -it -- /bin/sh
/ # wget 10.244.89.72
Connecting to 10.244.89.72 (10.244.89.72:80)
saving to 'index.html'
index.html           100% |***********************************************************************************************************************|   612  0:00:00 ETA
'index.html' saved
/ # 


##2、在my-app命名空间中也创建一个busybox容器pod对象
[root@k8s-master ~]# kubectl run busybox --image=busybox:stable --command sleep 3600 -n my-app
pod/busybox created
[root@k8s-master ~]# kubectl get pod -n my-app
NAME        READY   STATUS              RESTARTS   AGE
busybox     0/1     ContainerCreating   0          3s
nginx-wed   1/1     Running             0          2m36s
##访问测试
[root@k8s-master manifest]# kubectl exec busybox -it -n my-app -- /bin/sh
/ # wget 10.244.89.77
Connecting to 10.244.89.77 (10.244.89.77:80)
saving to 'index.html'
index.html           100% |***********************************************************************************************************************|   612  0:00:00 ETA
'index.html' saved
/ # 

7、创建ingress资源
在my-dev命名空间下创建一个ingress资源，具体要求如下：
允许集群外部流量通过七层模式访问my-dev命名空间下service为nginx-web的80端口，URL路径为http://ingress-controller的IP/nginx
注意：配置ingress之前先安装ingress controller，这里使用Nginx ingress controller

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-ingress
  namespace: my-dev
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - http:
      paths:
      - path: /nginx
        pathType: Prefix
        backend:
          service:
            name: nginx-web ##通过service辅助挑选出后端的pod资源，但是流量不经过service资源
            port:
              number: 80