【k8s】坑

节点规划

	ip
master.example.com	192.168.122.66	kubeadm kubelet kubectl
node1.example.com	192.168.122.68	kubeadm kubelet kubectl
node2.example.com	192.168.122.82	kubeadm kubelet kubectl

所有节点调整内核参数

vim /etc/sysctl.d/k8s.conf
==================k8s.conf内容=================
#解决在iptables bypass情况下引发的路由转发问题
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
#关闭使用swap
vm.swappiness = 0
#开启转发
net.ipv4.ip_forward = 1
==============================================
modprobe netfilter
sysctl -p /etc/sysctl.d/k8s.conf

设置K8S的yum源

##注意这里由于ipv6原因，用到了代理，代理点是我笔记本上的翻墙软件，当然要在xshell中开启一下ssh隧道才能进行代理
[kubernetes]
name=Kubernetes
proxy=http://127.0.0.1:7070
#proxy_username=user17181
#proxy_password=fuck_evil_GFW123
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg

master节点安装kubeadm kubelet kubectl

yum install -y kubeadm kubelet kubectl

进行kubeadm 初始化，拉取镜像

若初始化有问题用kubeadm reset还原
=================
kubeadm init --pod-network-cidr=10.244.0.0/16

=====初始化后，输出得最后部分有如下=======
Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.122.66:6443 --token 8bfdl0.wiguk8up1ro6l02r \
    --discovery-token-ca-cert-hash sha256:b688f576b762bd715277d90f04b64aa59a0c2c010ae1dc8bb7b91cd374eae691

mkdir -p ~/.kube
cp /etc/kubernetes/admin.conf ~/.kube/config
chown $(id -u):$(id -g) $HOME/.kube/config

flannel安装

kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/62e44c867a2846fefb68bd5f178daf4da3095ccb/Documentation/kube-flannel.yml

查验pod

kubectl get pods --all-namespaces
输出如下
NAMESPACE     NAME                                         READY   STATUS    RESTARTS   AGE
kube-system   coredns-7688bbffb9-4rd6j                     1/1     Running   0          36s
kube-system   coredns-7688bbffb9-fc572                     1/1     Running   0          36s
kube-system   etcd-master.example.com                      1/1     Running   0          43m
kube-system   kube-apiserver-master.example.com            1/1     Running   0          43m
kube-system   kube-controller-manager-master.example.com   1/1     Running   0          43m
kube-system   kube-flannel-ds-amd64-pnlnd                  1/1     Running   0          38m
kube-system   kube-proxy-wwrrb                             1/1     Running   0          44m
kube-system   kube-scheduler-master.example.com            1/1     Running   0          43m
====================================================
若status有error或CrashLoopBackOff，请查看官方解决方案：
https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/troubleshooting-kubeadm/#coredns-pods-have-crashloopbackoff-or-error-state

===================node节点操作================

加入集群的命令是

kubeadm join --token <token> <master-ip>:<master-port> --discovery-token-ca-cert-hash sha256:<hash>

由于node节点加入集群需要token，其实在master节点上进行kube init XXX操作时候，输出的最后几行就给出了加入集群需要的讯息，但是没记住也不要紧，可按如下操作继续k8s旅程

#在master节点上
kubeadm token list
TOKEN                     TTL       EXPIRES                     USAGES                   DESCRIPTION                                                EXTRA GROUPS
3un299.k6l81vt00wqrghh9   23h       2019-08-01T21:20:22+08:00   authentication,signing   The default bootstrap token generated by 'kubeadm init'.   system:bootstrappers:kubeadm:default-node-token

同样的若不知道加入集群命令的<hash>部分填写什么，可按如下操作

openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //'
==========
输出为:5631762e784629af0be9248e89c72be07e75c13beb9eb965d98e816ad0c90856

现在再执行node加入集群命令

kubeadm join --token 3un299.k6l81vt00wqrghh9 192.168.122.66:6443 --discovery-token-ca-cert-hash sha256:5631762e784629af0be9248e89c72be07e75c13beb9eb965d98e816ad0c90856

在master上查看是否加入集群

kubectl get node
结果
NAME                 STATUS     ROLES    AGE     VERSION
master.example.com   Ready      master   65m     v1.15.1
node1.example.com    Ready      <none>   4m21s   v1.15.1

上面标明节点已经加入集群

 

删除node1节点操作

master上
kubectl drain node1.example.com --delete-local-data --force --ignore-daemonsets
kubeadm delete node node1.example.com 

node1上
kubeset reset

===========================dashboard

访问有坑，老版装完的dashboard所在namespace和新版不一样

官方有说明如下

https://kubernetes.io/docs/tasks/access-application-cluster/web-ui-dashboard/

 

创建用户并授权

====创建admin-user.yml====
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: admin
  annotations:
     rbac.authorization.kubernetes.io/autoupdate: "true"
roleRef:
  kind: ClusterRole
  name: cluster-admin
  apiGroup: rbac.authorization.k8s.io
subjects:
- kind: ServiceAccount
  name: admin
  namespace: kube-system
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: admin
  namespace: kube-system
  labels:
     kubernetes.io/cluster-service: "true"
     addonmanager.kubernetes.io/mode: Reconcile
====执行一下===========
kubectl apply -f admin-user.yml
=======================
kubectl get secret -n kube-system | grep admin
输出：
    admin-token-q4vdz                                kubernetes.io/service-account-token   3      6m29s
=======================
kubectl describe secret admin-token-q4vdz -n kube-system
输出：
Name:         admin-token-q4vdz
Namespace:    kube-system
Labels:       <none>
Annotations:  kubernetes.io/service-account.name: admin
              kubernetes.io/service-account.uid: d9cdc5ae-2231-435d-a3ff-2010606d1148

Type:  kubernetes.io/service-account-token

Data
====
ca.crt:     1025 bytes
namespace:  11 bytes
token:      eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJhZG1pbi10b2tlbi1xNHZkeiIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJhZG1pbiIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6ImQ5Y2RjNWFlLTIyMzEtNDM1ZC1hM2ZmLTIwMTA2MDZkMTE0OCIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDprdWJlLXN5c3RlbTphZG1pbiJ9.ir8Hw5ehqaQSuyDR0-lJ9rtI6Fktz2jSRfPxAbllCfZy7t-FGoMVpRIdFg8Qcxh98EsYTuAotjQSVG0TKzmATf2qr8SRkdrqZIFeloXze5xbOG-gzVaUIjg7qQB6Rr9RgL57zJVki0K6iYyZy45FhpveY1b9_7WDYjmYFUyXmYy6lrxz8s76ppESXvEhCABvN7XLs5g3kztd3vNB1wrDQWLGZZpbjucRtru78ew0ab9ycmAeXj0idnTFuGeu6HnvW5GB7n5eo0b7UJzPV8IevyNSARrX0ku7GDWMrhxI4nkmaqUHOu7EJoE6gMUyqmu77EQGWMvXq7Zxa9iw6PJmAQ

 

 

访问http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/#/login