k8sv1.11.0 集群搭建

Ubuntu18.04_x64_TLS & k8s_v1.11.0 集群搭建

• 1.cluster 集群
• 2.Master 主控
• 3.Node 节点
• 4.Pod 资源对象 kubernetes以pod最为最小单位来进行调度、扩展、共享资源、管理生命周期；

---

k8s搭建方法：

1. kubeadm(官方给出集群版搭建方式)：

• https://github.com/kubernetes/kubeadm

2. 二进制方式：

• https://github.com/kelseyhightower/kubernetes-the-hard-way
• https://www.qikqiak.com/post/manual-install-high-available-kubernetes-cluster/

---

Ubuntu18.04_x64_TLS & k8s_v1.11.0 集群搭建记录

1. 准备环境并配置hosts

服务器	操作系统版本	角色
10.151	Ubuntu1804x64	master
10.152	Ubuntu1804x64	node1
10.153	Ubuntu1804x64	node2

cat >> /etc/hosts << EOF
192.168.10.151 u1804x64-master01
192.168.10.152 u1804x64-node1
192.168.10.153 u1804x64-node2
EOF

1.1. 安装依赖（针对所有节点）

#在三台服务器同时执行命令：
apt-get install apt-transport-https ca-certificates curl software-properties-common lrzsz -y 

#PS: 若apt lrzsz安装不成功换用源码安装：
mkdir -p /server/tools; \
cd /server/tools; \
wget https://www.ohse.de/uwe/releases/lrzsz-0.12.20.tar.gz; \
tar -xzf lrzsz-0.12.20.tar.gz; \
cd lrzsz-0.12.20; \
./configure --prefix=/usr/local/lrzsz; \
sudo make; \
sudo make install; \
cd /usr/bin;sudo ln -s /usr/local/lrzsz/bin/lrz rz;sudo ln -s /usr/local/lrzsz/bin/lsz sz;

1.2. 禁用SWAP

swapoff -a;
sed -i '/ swap / s/^/#/' /etc/fstab

1.3. 更新软件

apt-get update;apt-get upgrade;

1.4. 关闭防火墙

sudo ufw disable;

1.5. 更改hostname

sudo hostnamectl set-hostname u1804x64-master01
vi /etc/hostname;
vi /etc/hosts;
127.0.0.1 u1804x64-master01
vi /etc/cloud/cloud.cfg;
preserve_hostname: true

1.6. 安装docker_v18.03.1

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository \
   "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
   $(lsb_release -cs) \
   stable"
sudo apt-get update
sudo apt-get install docker-ce=18.03.1~ce~3-0~ubuntu

#静态源码使用方法
wget https://download.docker.com/linux/static/stable/x86_64/docker-18.03.1-ce.tgz;
tar xvzf docker-18.03.1-ce.tgz;cd docker*/;
sudo cp docker/* /usr/bin/;
sudo dockerd &

1.7. 配置加速器

curl -sSl https://get.daocloud.io/daotools/setmirror.sh | sh -s http://f1361db2.m.daocloud.io

2. master主机操作

2.1. 搭建镜像仓库

#获取仓库镜像
docker pull registry
#启动仓库容器
docker run --restart=always --name=registry -d -p 5000:5000 registry

2.2. 进行仓库配置

2.2.1. 编辑docker配置文件

sudo vim /etc/default/docker
DOCKER_OPTS="--insecure-registry 192.168.10.151:5000"

2.2.2. 创建服务依赖文件

sudo mkdir -p /etc/systemd/system/docker.service.d/
sudo vim /etc/systemd/system/docker.service.d/Using_Environment_File.conf
[service]
EnvironmentFile=-/etc/default/docker
ExecStart=
ExecStart=/usr/bin/dockerd -H fd:// $DOCKER_OPTS

2.2.3. 重启加载配置文件

systemctl daemon-reload;systemctl restart docker;

3. 安装kubeadm

3.1.配置kubernetes阿里源

sudo curl https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | apt-key add - 
sudo cat <<EOF >/etc/apt/sources.list.d/kubernetes.list
deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main
EOF  
sudo apt-get update 

3.2. 开始安装kubelet、kubeadm、kubectl

sudo apt-get install -y kubelet=1.11.1-00 kubeadm=1.11.1-00 kubectl=1.11.1-00

3.3. 如遇缺少kubenetes-cni-0.6.0依赖,则手动安装

cd /root/;
wget https://mirrors.aliyun.com/kubernetes/apt/pool/kubernetes-cni_0.6.0-00_amd64_43460dd3c97073851f84b32f5e8eebdc84fadedb5d5a00d1fc6872f30a4dd42c.deb;
dpkg -i kubernetes-cni_0.6.0-00_amd64_43460dd3c97073851f84b32f5e8eebdc84fadedb5d5a00d1fc6872f30a4dd42c.deb

4. 部署master节点

4.1. 禁用交换分区

sudo swapoff -a

4.2. 从其他镜像源下载后，修改tag

#!/bin/bash
images=(kube-proxy-amd64:v1.11.0 kube-scheduler-amd64:v1.11.0 kube-controller-manager-amd64:v1.11.0 kube-apiserver-amd64:v1.11.0
etcd-amd64:3.2.18 coredns:1.1.3 pause-amd64:3.1 kubernetes-dashboard-amd64:v1.8.3 k8s-dns-sidecar-amd64:1.14.9 k8s-dns-kube-dns-amd64:1.14.9
k8s-dns-dnsmasq-nanny-amd64:1.14.9 )
for imageName in ${images[@]} ; do
  docker pull registry.cn-hangzhou.aliyuncs.com/k8sth/$imageName
  docker tag registry.cn-hangzhou.aliyuncs.com/k8sth/$imageName k8s.gcr.io/$imageName
  #docker rmi registry.cn-hangzhou.aliyuncs.com/k8sth/$imageName
done
docker tag da86e6ba6ca1 k8s.gcr.io/pause:3.1

4.3. 节点初始化

kubeadm init --kubernetes-version=v1.11.0 --service-cidr=10.1.0.0/16 --pod-network-cidr=10.244.0.0/16 --apiserver-advertise-address=192.168.10.151 
#定义POD的网段为: 10.244.0.0/16， api server地址就是master本机IP地址

#安装成功现象如下：
Your Kubernetes master has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

You can now join any number of machines by running the following on each node
as root:

  kubeadm join 192.168.10.151:6443 --token ugymoh.hwq3q3968c0gnsq6 --discovery-token-ca-cert-hash sha256:43ffefbcf196d73fe639d902faa134638405e4c66289dd9eb45189bfe8f0ddd8

4.4. 安装pod网络插件flannel

4.4.1. 安装插件命令

kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/a70459be0084506e4ec919aa1c114638878db11b/Documentation/kube-flannel.yml

4.4.2. 查看flannel pod插件

kubectl get pod -n kube-system

4.4.3. 解决raw.githubusercontent.com连接被拒

echo "151.101.76.133 raw.githubusercontent.com" >> /etc/hosts

5. 部署node

5.0. 禁用SWAP

swapoff -a;sed -i '/ swap / s/^/#/' /etc/fstab

5.1.配置kubernetes阿里源

sudo curl https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | apt-key add - 
echo "deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main" >> /etc/apt/sources.list.d/kubernetes.list
sudo apt-get update 

5.2. 开始安装kubelet、kubeadm、kubectl

cd /root/;wget https://mirrors.aliyun.com/kubernetes/apt/pool/kubernetes-cni_0.6.0-00_amd64_43460dd3c97073851f84b32f5e8eebdc84fadedb5d5a00d1fc6872f30a4dd42c.deb;dpkg -i kubernetes-cni_0.6.0-00_amd64_43460dd3c97073851f84b32f5e8eebdc84fadedb5d5a00d1fc6872f30a4dd42c.deb
sudo apt-get install -y kubelet=1.11.1-00 kubeadm=1.11.1-00 kubectl=1.11.1-00

5.3. 加入集群前权限配置

mkdir -p $HOME/.kube;
sudo scp root@u1804x64-master01:/etc/kubernetes/admin.conf $HOME/.kube/config;
sudo chown $(id -u):$(id -g) $HOME/.kube/config;

5.4. 加入cluster集群

#以下命令是部署master自动生成！！！
kubeadm join 192.168.10.151:6443 --token ugymoh.hwq3q3968c0gnsq6 --discovery-token-ca-cert-hash sha256:43ffefbcf196d73fe639d902faa134638405e4c66289dd9eb45189bfe8f0ddd8

#24小时后token失效，需要手动生成token
kubeadm token create;
openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //' ;

5.5. 安装pod网络插件flannel

echo "151.101.76.133 raw.githubusercontent.com" >> /etc/hosts;
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/a70459be0084506e4ec919aa1c114638878db11b/Documentation/kube-flannel.yml;
kubectl get pod -n kube-system;

5.6. 从集群删除节点

kubectl delete node;kubectl get node;
#node上在执行
kubeadm reset
rm -rf $HOME/.kube

5.7. 节点不在线,"Unable to update cni config"报错,解决办法

mkdir -p /etc/cni/net.d/;
vi /etc/cni/net.d/10-flannel.conflist 
{
  "name": "cbr0",
  "plugins": [
    {
      "type": "flannel",
      "delegate": {
        "hairpinMode": true,
        "isDefaultGateway": true
      }
    },
    {
      "type": "portmap",
      "capabilities": {
        "portMappings": true
      }
    }
  ]
}
systemctl restart kubelet