k8s集群搭建（docker & k8s）

安装docker

通过apt的docker官方源安装最新的Docker CE(Community Edition)，即Docker社区版，是开发人员和小型团队的理想选择，可以指定安装版本。

删除原有docker

sudo apt-get remove docker docker-engine docker-ce docker.io

更新apt库使apt可以通过HTTPS使用存储库

sudo apt-get install -y apt-transport-https ca-certificates curl software-properties-common
sudo apt-get update

添加Docker官方的GPG密钥 设置stable存储库

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
sudo apt-get update

查询需要的docker版本

apt-cache madison docker-ce

根据需要的docker版本进行安装

sudo apt-get install docker-ce=$version

查看docker服务状态

systemctl status docker

启动遇到问题

docker网络未建立导致启动失败

sudo ip link add name docker0 type bridge
sudo ip addr add dev docker0 172.17.42.1/16   

写入改动

sudo systemctl daemon-reload

重启docker服务

sudo systemctl restart docker

环境其他准备

禁掉swap分区

你可以用sudo swapoff -a,要永久禁用swap分区的话,需要sudo vim /etc/fstab,注释掉swap那一行

k8s安装

阿里云镜像源安装kubeadm

sudo apt-get update && sudo apt-get install -y apt-transport-https
sudo curl https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | apt-key add - 
sudo cat <<EOF >/etc/apt/sources.list.d/kubernetes.list
deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main
EOF
sudo apt-get update
（安装最新）sudo apt-get install -y kubelet kubeadm kubectl
（指定版本）sudo apt-get install -y kubeadm=1.15.2-00 kubelet=1.15.2-00 kubectl=1.15.2-00

下载k8s所需镜像，使用阿里云镜像，kubeadm config images list查看k8s所需镜像列表

for i in `kubeadm config images list`; do 
  imageName=${i#k8s.gcr.io/}
  sudo docker pull registry.aliyuncs.com/google_containers/$imageName
  sudo docker tag registry.aliyuncs.com/google_containers/$imageName k8s.gcr.io/$imageName
  sudo docker rmi registry.aliyuncs.com/google_containers/$imageName
done;

初始化集群

kubeadm init --apiserver-advertise-address=$MASTERTIP --pod-network-cidr=10.244.0.0/16

正确初始化,会看到字样

Your Kubernetes master has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id-u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
…

执行命令：

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

查看集群状态：

$ kubectl get pod --all-namespaces

flannel插件安装

直接安装

kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/62e44c867a2846fefb68bd5f178daf4da3095ccb/Documentation/kube-flannel.yml

cni编译安装：

git clone https://github.com/containernetworking/plugins.git
./build_linux.sh
mkdir -pv /opt/cni/bin
cp ./bin/* /opt/cni/bin
ls -l /opt/cni/bin

flannel.yaml

apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"extensions/v1beta1","kind":"DaemonSet","metadata":{"annotations":{},"labels":{"app":"flannel","tier":"node"},"name":"kube-flannel-ds-amd64","namespace":"kube-system"},"spec":{"template":{"metadata":{"labels":{"app":"flannel","tier":"node"}},"spec":{"containers":[{"args":["--ip-masq","--kube-subnet-mgr"],"command":["/opt/bin/flanneld"],"env":[{"name":"POD_NAME","valueFrom":{"fieldRef":{"fieldPath":"metadata.name"}}},{"name":"POD_NAMESPACE","valueFrom":{"fieldRef":{"fieldPath":"metadata.namespace"}}}],"image":"quay.io/coreos/flannel:v0.11.0-amd64","name":"kube-flannel","resources":{"limits":{"cpu":"100m","memory":"50Mi"},"requests":{"cpu":"100m","memory":"50Mi"}},"securityContext":{"privileged":true},"volumeMounts":[{"mountPath":"/run","name":"run"},{"mountPath":"/etc/kube-flannel/","name":"flannel-cfg"}]}],"hostNetwork":true,"initContainers":[{"args":["-f","/etc/kube-flannel/cni-conf.json","/etc/cni/net.d/10-flannel.conflist"],"command":["cp"],"image":"quay.io/coreos/flannel:v0.11.0-amd64","name":"install-cni","volumeMounts":[{"mountPath":"/etc/cni/net.d","name":"cni"},{"mountPath":"/etc/kube-flannel/","name":"flannel-cfg"}]}],"nodeSelector":{"beta.kubernetes.io/arch":"amd64"},"serviceAccountName":"flannel","tolerations":[{"effect":"NoSchedule","operator":"Exists"}],"volumes":[{"hostPath":{"path":"/run"},"name":"run"},{"hostPath":{"path":"/etc/cni/net.d"},"name":"cni"},{"configMap":{"name":"kube-flannel-cfg"},"name":"flannel-cfg"}]}}}}
  generation: 1
  labels:
    app: flannel
    tier: node
  name: kube-flannel-ds-amd64
  namespace: kube-system
spec:
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app: flannel
      tier: node
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: flannel
        tier: node
    spec:
      containers:
      - args:
        - --ip-masq
        - --kube-subnet-mgr
        command:
        - /opt/bin/flanneld
        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: metadata.namespace
        image: quay.io/coreos/flannel:v0.11.0-amd64
        imagePullPolicy: IfNotPresent
        name: kube-flannel
        resources:
          limits:
            cpu: 100m
            memory: 50Mi
          requests:
            cpu: 100m
            memory: 50Mi
        securityContext:
          privileged: true
          procMount: Default
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /run
          name: run
        - mountPath: /etc/kube-flannel/
          name: flannel-cfg
      dnsPolicy: ClusterFirst
      hostNetwork: true
      initContainers:
      - args:
        - -f
        - /etc/kube-flannel/cni-conf.json
        - /etc/cni/net.d/10-flannel.conflist
        command:
        - cp
        image: quay.io/coreos/flannel:v0.11.0-amd64
        imagePullPolicy: IfNotPresent
        name: install-cni
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /etc/cni/net.d
          name: cni
        - mountPath: /etc/kube-flannel/
          name: flannel-cfg
      nodeSelector:
        beta.kubernetes.io/arch: amd64
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      serviceAccount: flannel
      serviceAccountName: flannel
      terminationGracePeriodSeconds: 30
      tolerations:
      - effect: NoSchedule
        operator: Exists
      volumes:
      - hostPath:
          path: /run
          type: ""
        name: run
      - hostPath:
          path: /etc/cni/net.d
          type: ""
        name: cni
      - configMap:
          defaultMode: 420
          name: kube-flannel-cfg
        name: flannel-cfg
  templateGeneration: 1
  updateStrategy:
    type: OnDelete
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: flannel
rules:
  - apiGroups:
      - ""   
    resources:
      - pods   
    verbs:      
      - get
  - apiGroups:
      - ""   
    resources:
      - nodes   
    verbs:
      - list
      - watch  
  - apiGroups:
      - ""   
    resources:
      - nodes/status   
    verbs:
      - patch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: flannel
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: flannel
subjects:
- kind: ServiceAccount
  name: flannel
  namespace: kube-system
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: flannel
  namespace: kube-system
---
kind: ConfigMap
apiVersion: v1
metadata:
  name: kube-flannel-cfg
  namespace: kube-system
  labels:   
    tier: node   
    app: flannel
data:
  cni-conf.json: |   
    {
      "name": "cbr0",
      "type": "flannel",
      "delegate": {
        "isDefaultGateway": true
      }
    }
  net-conf.json: |
    {
      "Network": "10.244.0.0/16",
      "Backend": {
        "Type": "vxlan"
      }
    }

kubectl create -f flannel.yaml

修改cni配置文件：

vi /etc/cni/net.d/10-flannel.conflist

{
  "name":"cni0",
  "cniVersion":"0.3.1",
  "plugins":[
    {
      "type":"flannel",
      "delegate":{
        "forceAddress":true,
        "isDefaultGateway":true
      }
    },
    {
      "type":"portmap",
      "capabilities":{
        "portMappings":true
      }
    }
  ]
}

$ kubectl get pod -n kube-system | grep flannel

helm插件安装

参考文章：https://www.cnblogs.com/ericnie/p/8463127.html

下载指定版本的tar.gz包

https://github.com/helm/helm/releases
解压放到PATH目录下

tar -xvzf  $HELM.tar.gz
mv linux-amd64/helm /usr/local/bin/helm

初始化tiller

$ helm init --service-account tiller --upgrade -i registry.cn-hangzhou.aliyuncs.com/google_containers/tiller:v2.12.3  --skip-refresh
$ kubectl create serviceaccount --namespace kube-system tiller
$ kubectl create clusterrolebinding tiller-cluster-rule --clusterrole=cluster-admin --serviceaccount=kube-system:tiller
$ kubectl patch deploy --namespace kube-system tiller-deploy -p '{"spec":{"template":{"spec":{"serviceAccount":"tiller"}}}}'

一些操作指令

强制删除pod

kubectl delete pod podName -n NAMESPACE --force --grace-period=0

使服务运行在master上

kubectl taint nodes --all node-role.kubernetes.io/master-

设置默认的namespace

kubectl config set-context $(kubectl config current-context) --namespace=ivdai

master生成token，node加入集群

kubeadm token create --print-join-command
kubeadm join $MASTERTIP --token kn2h8w.o6bs4ltxsgycqj72 \
    --discovery-token-ca-cert-hash sha256:5ff95cb03074490344263d391cc09474349e32c67e39e98137a8c2660a4c6752

给node打标签

kubectl label nodes $NODENAME key=value
kubectl get nodes --show-labels

进入子容器

kubectl exec -it my-pod --container main-app bash

end