kubeadm 在virtualbox ubuntu18.04上安装kubernetes 1.18.20

时间同步

把master和所有节点，先做时间同步。

virtualbox操作-------------------------------------

下载virtualbox

https://download.virtualbox.org/virtualbox/6.1.18/VirtualBox-6.1.18-142142-Win.exe

下载增强

https://download.virtualbox.org/virtualbox/6.1.18/Oracle_VM_VirtualBox_Extension_Pack-6.1.18.vbox-extpack

安装virtualbox和增强

安装路径不要有中文名，避免麻烦。
选择NAT 和host-only网络。

安装ubuntu 18.04虚拟机

虚拟机设置------------------------------------------

备份源

sudo cp /etc/apt/sources.list /etc/apt/sources.list.back

改为阿里源

sudo cat << EOF >/etc/apt/sources.list
deb http://mirrors.aliyun.com/ubuntu/ bionic main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic main restricted universe multiverse
 
deb http://mirrors.aliyun.com/ubuntu/ bionic-security main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-security main restricted universe multiverse
 
deb http://mirrors.aliyun.com/ubuntu/ bionic-updates main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-updates main restricted universe multiverse
 
deb http://mirrors.aliyun.com/ubuntu/ bionic-proposed main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-proposed main restricted universe multiverse
 
deb http://mirrors.aliyun.com/ubuntu/ bionic-backports main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-backports main restricted universe multiverse
EOF

sudo apt update

安装增强功能（可选，图形化界面最大化，挂载等）

sudo apt install -y gcc make perl

然后“设备” -> “安装增强功能”。

关闭防火墙

sudo ufw disable
sudo ufw status

关闭交换内存

临时关闭

sudo swapoff -a
free -m

永久关闭

 sudo vim /etc/fstab 

注释掉交换内存

允许 iptables 检查桥接流量

cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
br_netfilter
EOF

cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sudo sysctl --system

设置master可访问NodePort

cat <<EOF | sudo tee /etc/sysctl.d/99-ipv4_forward.conf
net.ipv4.ip_forward=1
EOF
sudo sysctl --load=/etc/sysctl.d/99-ipv4_forward.conf

安装docker----------------------------------------------------

安装必要的一些系统工具

sudo apt update
sudo apt install -y apt-transport-https ca-certificates curl software-properties-common

安装GPG证书

curl -fsSL http://mirrors.aliyun.com/docker-ce/linux/ubuntu/gpg | sudo apt-key add -

写入软件源信息

sudo add-apt-repository "deb [arch=amd64] http://mirrors.aliyun.com/docker-ce/linux/ubuntu $(lsb_release -cs) stable"

Step 4: 更新

sudo apt update

安装指定版本的Docker-CE:

查找Docker-CE的版本:

apt-cache madison docker-ce

安装指定版本的Docker-CE:

sudo apt-get -y install docker-ce=5:19.03.15~3-0~ubuntu-bionic

锁定版本，以免apt upgrade时自动升级，出现版本不兼容的情况：

sudo echo "docker-ce hold" | sudo dpkg --set-selections

更新docker配置文件

sudo mkdir -p /etc/docker

cat <<EOF | sudo tee /etc/docker/daemon.json
{
  "registry-mirrors": ["https://registry.docker-cn.com"],
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "storage-driver": "overlay2"
}
EOF

sudo service docker restart

安装kubeadm,kubelet,kubectl----------所有节点执行

软件源

sudo apt update &&  sudo apt install -y apt-transport-https

curl https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | sudo apt-key add -

cat <<EOF | sudo tee /etc/apt/sources.list.d/kubernetes.list
deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main
EOF

sudo apt update

查找kubeadm的版本

apt-cache madison kubeadm

安装Kubernetes：

sudo apt install -y kubeadm=1.18.20-00 kubectl=1.18.20-00 kubelet=1.18.20-00

锁定版本

以免apt upgrade时自动升级，出现版本不兼容的情况：

sudo echo "kubeadm hold" | sudo dpkg --set-selections
sudo echo "kubectl hold" | sudo dpkg --set-selections
sudo echo "kubelet hold" | sudo dpkg --set-selections

查看该版本用到的docker镜像

kubeadm config images list

拉取镜像

MY_REGISTRY=registry.cn-hangzhou.aliyuncs.com/google_containers
 
sudo docker pull ${MY_REGISTRY}/kube-apiserver:v1.18.20
sudo docker pull ${MY_REGISTRY}/kube-controller-manager:v1.18.20
sudo docker pull ${MY_REGISTRY}/kube-scheduler:v1.18.20
sudo docker pull ${MY_REGISTRY}/kube-proxy:v1.18.20
sudo docker pull ${MY_REGISTRY}/pause:3.2
sudo docker pull ${MY_REGISTRY}/etcd:3.4.3-0
sudo docker pull ${MY_REGISTRY}/coredns:1.6.7

添加Tag

默认使用的是 k8s.gcr.io

sudo docker tag ${MY_REGISTRY}/kube-apiserver:v1.18.20 k8s.gcr.io/kube-apiserver:v1.18.20
sudo docker tag ${MY_REGISTRY}/kube-scheduler:v1.18.20 k8s.gcr.io/kube-scheduler:v1.18.20
sudo docker tag ${MY_REGISTRY}/kube-controller-manager:v1.18.20 k8s.gcr.io/kube-controller-manager:v1.18.20
sudo docker tag ${MY_REGISTRY}/kube-proxy:v1.18.20 k8s.gcr.io/kube-proxy:v1.18.20
sudo docker tag ${MY_REGISTRY}/etcd:3.4.3-0 k8s.gcr.io/etcd:3.4.3-0
sudo docker tag ${MY_REGISTRY}/pause:3.2 k8s.gcr.io/pause:3.2
sudo docker tag ${MY_REGISTRY}/coredns:1.6.7 k8s.gcr.io/coredns:1.6.7

# 删除不需要的
sudo docker rmi ${MY_REGISTRY}/kube-apiserver:v1.18.20
sudo docker rmi ${MY_REGISTRY}/kube-controller-manager:v1.18.20
sudo docker rmi ${MY_REGISTRY}/kube-scheduler:v1.18.20
sudo docker rmi ${MY_REGISTRY}/kube-proxy:v1.18.20
sudo docker rmi ${MY_REGISTRY}/pause:3.2
sudo docker rmi ${MY_REGISTRY}/etcd:3.4.3-0
sudo docker rmi ${MY_REGISTRY}/coredns:1.6.7

创建master节点-----------------------

初始化master节点

192.168.3.156 是master ip地址。

kubeadm init --node-name=192.168.3.156 --kubernetes-version=v1.18.20 --apiserver-advertise-address=192.168.3.156 --pod-network-cidr=10.244.0.0/16

输出下面信息，表示初始化成功

kubectl 配置文件

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

安装网络插件flannel

mkdir -p ~/k8s
cd ~/k8s

# 内容https://github.com/flannel-io/flannel/blob/master/Documentation/kube-flannel.yml
# 1.有多个网卡，需添加参数 --iface=xxx, 指定网卡。
# 2.增加了 "VNI": 4096 和 "Port": 4789。以便在使用vxlan时，添加windows节点。

cat > kube-flannel.yml <<EOF
---
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: psp.flannel.unprivileged
  annotations:
    seccomp.security.alpha.kubernetes.io/allowedProfileNames: docker/default
    seccomp.security.alpha.kubernetes.io/defaultProfileName: docker/default
    apparmor.security.beta.kubernetes.io/allowedProfileNames: runtime/default
    apparmor.security.beta.kubernetes.io/defaultProfileName: runtime/default
spec:
  privileged: false
  volumes:
  - configMap
  - secret
  - emptyDir
  - hostPath
  allowedHostPaths:
  - pathPrefix: "/etc/cni/net.d"
  - pathPrefix: "/etc/kube-flannel"
  - pathPrefix: "/run/flannel"
  readOnlyRootFilesystem: false
  # Users and groups
  runAsUser:
    rule: RunAsAny
  supplementalGroups:
    rule: RunAsAny
  fsGroup:
    rule: RunAsAny
  # Privilege Escalation
  allowPrivilegeEscalation: false
  defaultAllowPrivilegeEscalation: false
  # Capabilities
  allowedCapabilities: ['NET_ADMIN', 'NET_RAW']
  defaultAddCapabilities: []
  requiredDropCapabilities: []
  # Host namespaces
  hostPID: false
  hostIPC: false
  hostNetwork: true
  hostPorts:
  - min: 0
    max: 65535
  # SELinux
  seLinux:
    # SELinux is unused in CaaSP
    rule: 'RunAsAny'
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: flannel
rules:
- apiGroups: ['extensions']
  resources: ['podsecuritypolicies']
  verbs: ['use']
  resourceNames: ['psp.flannel.unprivileged']
- apiGroups:
  - ""
  resources:
  - pods
  verbs:
  - get
- apiGroups:
  - ""
  resources:
  - nodes
  verbs:
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - nodes/status
  verbs:
  - patch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: flannel
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: flannel
subjects:
- kind: ServiceAccount
  name: flannel
  namespace: kube-system
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: flannel
  namespace: kube-system
---
kind: ConfigMap
apiVersion: v1
metadata:
  name: kube-flannel-cfg
  namespace: kube-system
  labels:
    tier: node
    app: flannel
data:
  cni-conf.json: |
    {
      "name": "cbr0",
      "cniVersion": "0.3.1",
      "plugins": [
        {
          "type": "flannel",
          "delegate": {
            "hairpinMode": true,
            "isDefaultGateway": true
          }
        },
        {
          "type": "portmap",
          "capabilities": {
            "portMappings": true
          }
        }
      ]
    }
  net-conf.json: |
    {
      "Network": "10.244.0.0/16",
      "Backend": {
        "Type": "vxlan",
        "VNI": 4096,
        "Port": 4789
      }
    }
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: kube-flannel-ds
  namespace: kube-system
  labels:
    tier: node
    app: flannel
spec:
  selector:
    matchLabels:
      app: flannel
  template:
    metadata:
      labels:
        tier: node
        app: flannel
    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: kubernetes.io/os
                operator: In
                values:
                - linux
      hostNetwork: true
      priorityClassName: system-node-critical
      tolerations:
      - operator: Exists
        effect: NoSchedule
      serviceAccountName: flannel
      initContainers:
      - name: install-cni
        image: quay.io/coreos/flannel:v0.14.0
        command:
        - cp
        args:
        - -f
        - /etc/kube-flannel/cni-conf.json
        - /etc/cni/net.d/10-flannel.conflist
        volumeMounts:
        - name: cni
          mountPath: /etc/cni/net.d
        - name: flannel-cfg
          mountPath: /etc/kube-flannel/
      containers:
      - name: kube-flannel
        image: quay.io/coreos/flannel:v0.14.0
        command:
        - /opt/bin/flanneld
        args:
        - --ip-masq
        - --kube-subnet-mgr
        - --iface=enp0s3
        resources:
          requests:
            cpu: "100m"
            memory: "50Mi"
          limits:
            cpu: "100m"
            memory: "50Mi"
        securityContext:
          privileged: false
          capabilities:
            add: ["NET_ADMIN", "NET_RAW"]
        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        volumeMounts:
        - name: run
          mountPath: /run/flannel
        - name: flannel-cfg
          mountPath: /etc/kube-flannel/
      volumes:
      - name: run
        hostPath:
          path: /run/flannel
      - name: cni
        hostPath:
          path: /etc/cni/net.d
      - name: flannel-cfg
        configMap:
          name: kube-flannel-cfg
EOF

kubectl apply -f kube-flannel.yml

验证master

#查看节点
kubectl get nodes -o wide

#查看kubernetes 系统组件
kubectl get pods -o wide -n kube-system

创建slave节点----------------------------------------------------

把slave节点加入到集群

192.168.3.157 是slave ip, 192.168.3.156 是 master ip

  kubeadm --node-name=192.168.3.157 join 192.168.3.156:6443 --token bsi3ya.yfsh4w5pp3buvesg     --discovery-token-ca-cert-hash sha256:aa6dc4112dcbb70b4c1c1aaeefeed1eee0c54886071d00296ab01c7b78b38e68 

输出如下信息，表示成功

启用主节点同时作为工作节点（实验环境）--------------

kubectl taint nodes --all node-role.kubernetes.io/master-

参考------------------------------------------------

k8s官网:
https://kubernetes.io/zh/docs/setup/production-environment/tools/kubeadm/install-kubeadm/
ubuntu18.04 安装k8s:
https://www.jianshu.com/p/e09df10565fc
master访问NodePort:
https://www.it610.com/article/1282424429049036800.htm