云计算之kubernetes系列——配置本地dns服务并关联kube-dns

一，背景
    因为k8s集群以及web服务测试，每次使用/etc/hosts配置或者写ip都不太好，所以决定搭建dns服务器做域名解析，后期可以做k8s-kube-dns的父dns服务器。
二，搭建
    1. 服务器
        dnsserver   192.168.89.128
    2. 安装dns软件包
        yum -y install bind bind-chroot bind-utils
    3. 修改dns配置
        vim /etc/named.conf
            options {
                /*此处改成any*/
                listen-on port 53 { any; };
                listen-on-v6 port 53 { ::1; };
                directory  "/var/named";
                dump-file  "/var/named/data/cache_dump.db";
                statistics-file "/var/named/data/named_stats.txt";
                memstatistics-file "/var/named/data/named_mem_stats.txt";
                recursing-file  "/var/named/data/named.recursing";
                secroots-file   "/var/named/data/named.secroots";
                /*此处改成any*/
                allow-query     { any; };

                recursion yes;

                dnssec-enable yes;
                dnssec-validation yes;

                /* Path to ISC DLV key */
                bindkeys-file "/etc/named.root.key";

                managed-keys-directory "/var/named/dynamic";

                pid-file "/run/named/named.pid";
                session-keyfile "/run/named/session.key";
            };

            logging {
                    channel default_debug {
                            file "data/named.run";
                            severity dynamic;
                    };
            };

            zone "." IN {
                type hint;
                file "named.ca";
            };
            /*此zone做测试域名解析使用*/
            zone "ktz.com" IN {
                type master;
                file "ktz.com.zone";
            };
            /*此zone做web网站域名解析使用*/
            zone "web.com" IN {
                type master;
                file "web.com.zone";
            };
            /*此zone做k8s集群域名解析使用*/
            zone "k8s.com" IN {
                type master;
                file "k8s.com.zone";
            };

            include "/etc/named.rfc1912.zones";
            include "/etc/named.root.key";
        cp -a /var/named/named.localhost /var/named/ktz.com.zone
        cp -a /var/named/named.localhost /var/named/web.com.zone
        cp -a /var/named/named.localhost /var/named/k8s.com.zone
            $TTL 1D
            @  IN SOA @ k8s.com. (
                                0  ; serial
                                1D ; refresh
                                1H ; retry
                                1W ; expire
                                3H )   ; minimum
                NS @
                A  192.168.89.132
            k8smaster  A  192.168.89.132
            k8snode01  A  192.168.89.133
            k8snode02  A  192.168.89.134
    4. 重启并开机自启，配置/etc/resolv.conf
        systemctl restart named
        systemctl enabled named
        vim /etc/resolv.conf(修改相关服务器dns)
            nameserver  192.168.89.128
    5. 测试验证
        k8smaster:
            yum -y install bind-utils(提供nslookup命令做域名解析)
            nslookup k8snode01.k8s.com
                Server:       192.168.89.128
                Address:   192.168.89.128#53

                Name:  k8snode01.k8s.com
                Address: 192.168.89.133
            可以正确的解析到对应的服务器，操作完成
三， 启动k8s集群服务/kube-dns，并关联本地dns服务器
    1. 启动k8s集群服务, 并查看状态
        k8smaster
            systemctl restart docker kube-controller-manager kube-scheduler kube-apiserver
        k8snode01
            systemctl restart docker flanneld kubelet kube-proxy
        k8snode02
            systemctl restart docker flanneld kubelet kube-proxy
        k8smaster
            kubectl get pods -n kube-system
                NAME                        READY   STATUS    RESTARTS   AGE
                kube-dns-85bdb85857-bjvbf   3/3     Running   0          43s
                kube-dns服务仍然在正常运行
    2. 将kube-dns与本地dns服务器关联
        1). 测试一下kube-dns
            kubectl run busybox --image=192.168.89.132:5000/busybox  --command -- sleep 3600
                deployment.apps/busybox created
            kubectl get pods
                NAME                        READY   STATUS    RESTARTS   AGE
                busybox-54584f87db-4prnh    1/1     Running   0          23s
            kubectl get svc
                mysql-service   NodePort    169.169.188.11    <none>        3306:64298/TCP   12d
            kubectl exec -it busybox-54584f87db-4prnh -- nslookup mysql-service
                Server:       169.169.0.10
                Address:   169.169.0.10:53

                Name:  mysql-service.default.svc.k8s.com
                Address: 169.169.188.11

                *** Can't find mysql-service.svc.k8s.com: No answer
                *** Can't find mysql-service.k8s.com: No answer
                *** Can't find mysql-service.localdomain: No answer
                *** Can't find mysql-service.default.svc.k8s.com: No answer
                *** Can't find mysql-service.svc.k8s.com: No answer
                *** Can't find mysql-service.k8s.com: No answer
                *** Can't find mysql-service.localdomain: No answer
            kubectl exec busybox-54584f87db-4prnh --  ping kubernetes.default.svc.k8s.com
                PING kubernetes.default.svc.k8s.com (169.169.0.1): 56 data bytes
                64 bytes from 169.169.0.1: seq=0 ttl=64 time=0.028 ms
            kubectl exec -it busybox-54584f87db-4prnh /bin/sh
                cat /etc/resolv.conf
                    nameserver 169.169.0.10
                    search default.svc.k8s.com svc.k8s.com k8s.com localdomain
                    options ndots:5
                    可知集群dns的ip地址未169.169.0.10
                ping www.baidu.com
                    PING www.baidu.com (112.80.248.75): 56 data bytes
                    64 bytes from 112.80.248.75: seq=0 ttl=127 time=2.616 ms
                ping k8smaster.k8s.com
                    ping: bad address 'k8smaster.k8s.com'
                ping公网的可以，ping本地的一个服务不行，则需要关联本地dns
        2). 将本地dns服务器配置成kube-dns上游dns服务器
            a. 修改 kube-dns.yaml文件中ConfigMap部分, 添加了data部分
                apiVersion: v1
                kind: ConfigMap
                metadata:
                  name: kube-dns
                  namespace: kube-system
                  labels:
                    addonmanager.kubernetes.io/mode: EnsureExists
                data:
                  # 配置上游dns服务器
                  upstreamNameservers: |
                    ["192.168.89.128"]
            b. 删除之前的kube-dns，重新创建
                kubectl delete -f kube-dns.yaml
                kubectl get pods -n kube-system
                kubectl create -f kube-dns.yaml
                kubectl get pods -n kube-system
                    NAME                        READY   STATUS    RESTARTS   AGE
                    kube-dns-85bdb85857-z62nj   3/3     Running   0          9s
            c. 验证
                kubectl exec -it busybox-54584f87db-d97qk  /bin/sh
                ping k8smaster.web.com 无效 ，可以说明配置上游dns服务器无效。
                继续修改kube-dns.yaml
                vim kube-dns.yaml
                    ---
                    apiVersion: v1
                    kind: ConfigMap
                    metadata:
                      name: kube-dns
                      namespace: kube-system
                      labels:
                        addonmanager.kubernetes.io/mode: EnsureExists
                    data:
                      stubDomains: |
                        {"web.com": ["192.168.89.128"]}
                    ---
                kubectl exec -it busybox-54584f87db-d97qk  /bin/sh
                    ping todolist.web.com
                        PING todolist.web.com (192.168.89.132): 56 data bytes
                        64 bytes from 192.168.89.132: seq=0 ttl=63 time=0.234 ms
                    可以ping通，测试通过，可以和k8s集群外的本地服务通信了
四， 总结
    使用本地域名前缀ktz.com进行验证
    vim /var/named/ktz.com.zone
        $TTL 1D
        @  IN SOA @ ktz.com. (
                            0  ; serial
                            1D ; refresh
                            1H ; retry
                            1W ; expire
                            3H )   ; minimum
            NS @
            A  192.168.89.133
        www    A  192.168.89.133
    systemctl restart named
    kubectl exec -it busybox-54584f87db-d97qk  /bin/sh
        ping www.ktz.com
            PING www.ktz.com (192.168.89.133): 56 data bytes
            64 bytes from 192.168.89.133: seq=0 ttl=64 time=0.037 ms
    当本地再新加服务，需要域名解析，且服务不在k8s集群中，但是集群中要使用时，则只需要配置本地域名解析即可。