基于Vagrant+K8S+Harbor+gitlab的CI/CD(1）

初始化虚拟机

首先在windows上安装virtualbox（https://www.virtualbox.org/wiki/Downloads），Vagrant（https://www.vagrantup.com/downloads.html)。virtualbox的虚拟硬盘路径设置到非系统盘里。

下载centos 7镜像 https://mirrors.ustc.edu.cn/centos-cloud/centos/7/vagrant/x86_64/images/CentOS-7.box

添加镜像: vagrant box add k8s ./centos-7.0-x86_64.box 

生成Vagrantfile文件: vagrant init k8s

修改Vagrantfile文件,为了资源足够，这里使用了6G内存，2CPU

Vagrant.configure("2") do |config|
  config.vm.box = "k8s"
  config.vm.provider "virtualbox" do |vb|
    vb.memory = "6144"
    vb.cpus = 2
  end
  config.vm.define :master do |cfg|
    cfg.vm.hostname = "master"
    cfg.vm.network :private_network, ip: "172.28.128.10"
  end
 end

启动虚拟机  vagrant up

登录虚拟机  vagrant ssh

修改linux的设置

# 关闭selinux
sed -i 's/^SELINUX=enforcing$/SELINUX=disabled/' /etc/selinux/config
# 内核参数修改
cat <<EOF >  /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sysctl --system
# 关闭防火墙
service firewalld stop
systemctl disable firewalld

安装Docker

yum install -y yum-utils \
  device-mapper-persistent-data \
  lvm2
yum-config-manager \
    --add-repo \
    https://download.docker.com/linux/centos/docker-ce.repo
yum install -y docker-ce-19.03.14 docker-ce-cli-19.03.14 
systemctl enable docker
service docker start

安装 kubeadm

cat <<EOF > /etc/yum.repos.d/k8s.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

cat >/etc/sysconfig/kubelet<<EOF
KUBELET_EXTRA_ARGS="--pod-infra-container-image=registry.cn-hangzhou.aliyuncs.com/google_containers/pause-amd64:3.1"
EOF
yum install -y kubelet-1.20.2 kubectl-1.20.2 kubeadm-1.20.2
systemctl enable kubelet

使用eth1网卡初始化k8s

# /etc/sysconfig/kubelet
KUBELET_EXTRA_ARGS="--node-ip=172.28.128.10 --pod-infra-container-image=registry.cn-hangzhou.aliyuncs.com/google_containers/pause-amd64:3.1  --fail-swap-on=false"

# 重启kubelet
service kubelet restart

kubeadm 预下载基础镜像

kubeadm config images pull --image-repository registry.cn-hangzhou.aliyuncs.com/google_containers

kubeadm 初始化(强制apiserver-advertise-address=eth1)

kubeadm init --pod-network-cidr 10.244.0.0/16 --apiserver-advertise-address 172.28.128.10 --image-repository registry.cn-hangzhou.aliyuncs.com/google_containers --ignore-preflight-errors=Swap

使用kubectl 前 需要export KUBECONFIG=/etc/kubernetes/admin.conf

安装flannel网络组件

使master也能部署pod

kubectl taint nodes --all node-role.kubernetes.io/master- 

下载

https://hub.fastgit.org/flannel-io/flannel/releases/download/v0.15.1/flanneld-v0.15.1-arm.docker

安装

docker load < flanneld-v0.15.1-arm.docker

下载

https://hub.fastgit.org/flannel-io/flannel/blob/master/Documentation/kube-flannel.yml

编辑修改kube-flannel.yml（强制指定eth1网卡初始化，只需要找到amd64这段修改即可）

# kube-flannel.yml
...
      containers:
      - name: kube-flannel
        image: quay.io/coreos/flannel:v0.11.0-amd64
        command:
        - /opt/bin/flanneld
        args:
        - --ip-masq
        - --kube-subnet-mgr
        - --iface=eth1
...

看一下容器情况，确认都是 Running

[root@master vagrant]# kubectl get pods -n kube-system -o wide
NAME                             READY   STATUS    RESTARTS   AGE   IP              NODE     NOMINATED NODE   READINESS GATES
coredns-6967fb4995-dvmdq         1/1     Running   0          14m   10.244.1.2      node     <none>           <none>
coredns-6967fb4995-zgpwh         1/1     Running   0          14m   10.244.0.2      master   <none>           <none>
etcd-master                      1/1     Running   0          13m   172.28.128.10   master   <none>           <none>
kube-apiserver-master            1/1     Running   0          13m   172.28.128.10   master   <none>           <none>
kube-controller-manager-master   1/1     Running   0          13m   172.28.128.10   master   <none>           <none>
kube-flannel-ds-amd64-q8phh      1/1     Running   0          33s   172.28.128.11   node     <none>           <none>
kube-flannel-ds-amd64-zv2rw      1/1     Running   0          33s   172.28.128.10   master   <none>           <none>
kube-proxy-drzzq                 1/1     Running   0          12m   172.28.128.11   node     <none>           <none>
kube-proxy-vlfvk                 1/1     Running   0          14m   172.28.128.10   master   <none>           <none>
kube-scheduler-master            1/1     Running   0          13m   172.28.128.10   master   <none>           <none>
 

这样K8S在Vagrant上部署成功。