利用Oracle VM VirtualBox搭建单master节点的k8s集群

这个步骤是我一边搭建，一边查资料，一边记录的，希望会对正在查找资料的你有帮助。

目录

一.搭建环境：Oracle VM VirtualBox

二.系统初始化

三.安装Docker/kubeadm/kubelet

四.部署 Kubernetes Master

五.安装 Pod 网络插件（CNI）

六.加入 Kubernetes Node

七.测试k8s集群

---

---

一.搭建环境：Oracle VM VirtualBox

创建3个虚机，1个充当master节点，2个充当node节点

操作系统镜像：CentOS-7-x86_64-DVD-2009.iso  (centos官网下载即可)

硬件配置：2GB +，2 个 CPU +

集群中所有机器之间网络互通

可以访问外网，需要拉取镜像

禁止 swap 分区 （在第二部分系统初始化中有设置）

其他设置都默认就可以

需要注意：为在后期方便在本地通过xshell ssh到虚机内部（Oracle VM VirtualBox太难用了），需要设置下：

1.虚机网络--我这里选择的是桥接网卡,因为比较方便。

2.要进行的一步则是开放22端口，需要进行防火墙的设置

# 防火墙配置，开放22端口
sudo firewall-cmd --zone=public --add-port=22/tcp --permanent
# 重启防火墙
sudo systemctl restart firewalld.service

3.编辑这个文件 /etc/sysconfig/network-scripts/ifcfg-enp0s3 (你的虚机可能叫类似的别的名字)

4.文件里设置ONBOOT=yes

5.重启网络服务

systemctl restart network.service

6.在虚拟机中执行 ip addr 命令，找到ip地址，直接ssh 远程连接

二.系统初始化

1.安装wget

yum install wget

2.关闭防火墙

systemctl stop firewalld
systemctl disable firewalld

3.永久关闭selinux

sed -i 's/enforcing/disabled/' /etc/selinux/config

4.永久关闭swap

sed -ri 's/.*swap.*/#&/' /etc/fstab

5.改主机名，把三个虚机名改成master，node1，node2，容易操作

可通过hostname命令查看结果

hostnamectl set-hostname <hostname>

6 在 master 节点添加 hosts： ip改成你自己的ip

$ vi/etc/hosts 
192.168.31.61 master 
192.168.31.62 node1 
192.168.31.63 node2

7. 将桥接的 IPv4 流量传递到 iptables 的链：

vi /etc/sysctl.d/k8s.conf 
net.bridge.bridge-nf-call-ip6tables = 1 
net.bridge.bridge-nf-call-iptables = 1 

sysctl --system # 生效

8. 时间同步：

yum install ntpdate -y ntpdate time.windows.com

三.安装Docker/kubeadm/kubelet

1.Kubernetes 默认 CRI（容器运行时）为 Docker，因此先安装 Docker。

$ wget https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo -O 
/etc/yum.repos.d/docker-ce.repo 
$ yum -y install docker-ce-18.06.1.ce-3.el7 
$ systemctl enable docker && systemctl start docker 
$ docker --version

2.添加阿里云 YUM 软件源

设置仓库地址 
# vi /etc/docker/daemon.json
{ 
"registry-mirrors": ["https://b9pmyelo.mirror.aliyuncs.com"] 
}

3.添加 yum 源

# vi /etc/yum.repos.d/kubernetes.repo
[kubernetes] 
name=Kubernetesbaseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64 
enabled=1 
gpgcheck=0 

4.安装 kubeadm，kubelet 和 kubectl

$ yum install -y kubelet kubeadm kubectl 
$ systemctl enable kubelet

四.部署 Kubernetes Master

1.Master节点执行

注意：--apiserver-advertise-address=172.27.3.238是你master节点的ip地址

其他和我保持一致就好

kubeadm init --apiserver-advertise-address=172.27.3.238 --image-repository registry.aliyuncs.com/google_containers --kubernetes-version v1.17.0 --service-cidr=10.96.0.0/12 --pod-network-cidr=10.244.0.0/16

注意记录下方信息

2.使用 kubectl 工具：

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

 3.检查下集群的node情况，现在是无node的情况，等下面的步骤执行完，再次运行，会发现被添加到集群中的node

$ kubectl get nodes

五.安装 Pod 网络插件（CNI）

$ kubectl apply -f kube-flannel.yml

kube-flannel.yml文件的内容是：

---
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: psp.flannel.unprivileged
  annotations:
    seccomp.security.alpha.kubernetes.io/allowedProfileNames: docker/default
    seccomp.security.alpha.kubernetes.io/defaultProfileName: docker/default
    apparmor.security.beta.kubernetes.io/allowedProfileNames: runtime/default
    apparmor.security.beta.kubernetes.io/defaultProfileName: runtime/default
spec:
  privileged: false
  volumes:
  - configMap
  - secret
  - emptyDir
  - hostPath
  allowedHostPaths:
  - pathPrefix: "/etc/cni/net.d"
  - pathPrefix: "/etc/kube-flannel"
  - pathPrefix: "/run/flannel"
  readOnlyRootFilesystem: false
  # Users and groups
  runAsUser:
    rule: RunAsAny
  supplementalGroups:
    rule: RunAsAny
  fsGroup:
    rule: RunAsAny
  # Privilege Escalation
  allowPrivilegeEscalation: false
  defaultAllowPrivilegeEscalation: false
  # Capabilities
  allowedCapabilities: ['NET_ADMIN', 'NET_RAW']
  defaultAddCapabilities: []
  requiredDropCapabilities: []
  # Host namespaces
  hostPID: false
  hostIPC: false
  hostNetwork: true
  hostPorts:
  - min: 0
    max: 65535
  # SELinux
  seLinux:
    # SELinux is unused in CaaSP
    rule: 'RunAsAny'
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: flannel
rules:
- apiGroups: ['extensions']
  resources: ['podsecuritypolicies']
  verbs: ['use']
  resourceNames: ['psp.flannel.unprivileged']
- apiGroups:
  - ""
  resources:
  - pods
  verbs:
  - get
- apiGroups:
  - ""
  resources:
  - nodes
  verbs:
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - nodes/status
  verbs:
  - patch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: flannel
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: flannel
subjects:
- kind: ServiceAccount
  name: flannel
  namespace: kube-system
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: flannel
  namespace: kube-system
---
kind: ConfigMap
apiVersion: v1
metadata:
  name: kube-flannel-cfg
  namespace: kube-system
  labels:
    tier: node
    app: flannel
data:
  cni-conf.json: |
    {
      "name": "cbr0",
      "cniVersion": "0.3.1",
      "plugins": [
        {
          "type": "flannel",
          "delegate": {
            "hairpinMode": true,
            "isDefaultGateway": true
          }
        },
        {
          "type": "portmap",
          "capabilities": {
            "portMappings": true
          }
        }
      ]
    }
  net-conf.json: |
    {
      "Network": "10.244.0.0/16",
      "Backend": {
        "Type": "vxlan"
      }
    }
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: kube-flannel-ds
  namespace: kube-system
  labels:
    tier: node
    app: flannel
spec:
  selector:
    matchLabels:
      app: flannel
  template:
    metadata:
      labels:
        tier: node
        app: flannel
    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: kubernetes.io/os
                operator: In
                values:
                - linux
      hostNetwork: true
      priorityClassName: system-node-critical
      tolerations:
      - operator: Exists
        effect: NoSchedule
      serviceAccountName: flannel
      initContainers:
      - name: install-cni-plugin
        image: rancher/mirrored-flannelcni-flannel-cni-plugin:v1.0.0
        command:
        - cp
        args:
        - -f
        - /flannel
        - /opt/cni/bin/flannel
        volumeMounts:
        - name: cni-plugin
          mountPath: /opt/cni/bin
      - name: install-cni
        image: quay.io/coreos/flannel:v0.15.1
        command:
        - cp
        args:
        - -f
        - /etc/kube-flannel/cni-conf.json
        - /etc/cni/net.d/10-flannel.conflist
        volumeMounts:
        - name: cni
          mountPath: /etc/cni/net.d
        - name: flannel-cfg
          mountPath: /etc/kube-flannel/
      containers:
      - name: kube-flannel
        image: quay.io/coreos/flannel:v0.15.1
        command:
        - /opt/bin/flanneld
        args:
        - --ip-masq
        - --kube-subnet-mgr
        resources:
          requests:
            cpu: "100m"
            memory: "50Mi"
          limits:
            cpu: "100m"
            memory: "50Mi"
        securityContext:
          privileged: false
          capabilities:
            add: ["NET_ADMIN", "NET_RAW"]
        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        volumeMounts:
        - name: run
          mountPath: /run/flannel
        - name: flannel-cfg
          mountPath: /etc/kube-flannel/
      volumes:
      - name: run
        hostPath:
          path: /run/flannel
      - name: cni-plugin
        hostPath:
          path: /opt/cni/bin
      - name: cni
        hostPath:
          path: /etc/cni/net.d
      - name: flannel-cfg
        configMap:
          name: kube-flannel-cfg

使用的镜像可能被墙  替换为国内源 【lizhenliang/flannel:v0.11.0-amd64】

六.加入 Kubernetes Node

（1）在 Node上执行 向集群添加新节点，执行在 kubeadm init 输出的 kubeadm join 命令：

kubeadm join 172.27.3.238:6443 --token v0p5w5.3lejfnewe7qrmai8 \ --discovery-token-ca-cert-hash sha256:590e48120afd80adad159f1a27ab57e6011bb07df8d8a0f85ea04c6e1305e19c

七.测试k8s集群

在集群中创建第一个pod

$ kubectl create deployment nginx --image=nginx

$ kubectl expose deployment nginx --port=80 --type=NodePort

$ kubectl get pod,svc

如果版本版本不对的话会报错：可以先卸载，在安装1.17.0，亲测可以

yum remove kubeadm.x86_64 kubectl.x86_64 kubelet.x86_64

yum install -y kubelet-1.17.0 kubeadm-1.17.0 kubectl-1.17.0

其他问题：

yum makecache的时候 报错 Cannot find a valid baseurl for repo: base/7/x86_64

解决：vi /etc/resolv.conf

如果ping不通baidu域名：

systemctl restart network
 

遇到错误：

error execution phase preflight: [preflight] Some fatal errors occurred:
    [ERROR FileContent--proc-sys-net-ipv4-ip_forward]: /proc/sys/net/ipv4/ip_forward contents are not set to 1
解决：

 sysctl -w net.ipv4.ip_forward=1

如果node加入不了集群：

1.回到kubernees-maser  依次输入列命令
systemctl stop kubelet
systemctl stop docker
iptables --flush
iptables -tnat --flush
systemctl start kubelet
systemctl start docker

  2.重新生成新token
  在kubernetse-master重新生成token：

# kubeadm token create

  424mp7.nkxx07p940mkl2nd

# openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //'

  d88fb55cb1bd659023b11e61052b39bbfe99842b0636574a16c76df186fd5e0d

   3.在kubernetes-slave中执行此命令 join就成功了

#  kubeadm join 192.168.169.21:6443 –token 424mp7.nkxx07p940mkl2nd \ --discovery-token-ca-cert-hash sha256:d88fb55cb1bd659023b11e61052b39bbfe99842b0636574a16c76df186fd5e0d