镜像源站点:
https://mirror.tuna.tsinghua.edu.cn/help
阿里云下载源:
https://mirrors.aliyun.com/kubernetes/apt/
服务器环境准备:
注:1、关闭swap分区:swapoff -a(临时) 修改/etc/fstab,注释swap挂载 那一行(永久)
2、禁用ipv6
3、加载ipvs相关模块
配置源 /etc/apt/sources.list
deb [arch=amd64] https://download.docker.com/linux/ubuntu bionic stable
deb [arch=amd64] http://mirrors.aliyun.com/docker-ce/linux/ubuntu bionic stable
如果你过去安装过 docker,先删掉:
sudo apt-get remove docker docker-engine docker.io
# 信任 Docker 的 GPG 公钥(执行完后显示ok即可):
# curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
# 安装docker:
# apt-cache madison docker-ce
# apt-get update && apt-get install -y docker-ce=$(apt-cache madison docker-ce | grep 19.03.6 | head -1 | awk '{print $3}')
对于 amd64 架构的计算机,添加软件仓库:
echo "deb [arch=armhf] https://mirrors.tuna.tsinghua.edu.cn/docker-ce/linux/ubuntu \
$(lsb_release -cs) stable" | \
sudo tee /etc/apt/sources.list.d/docker.list
使用ustc的镜像源
需要获取公钥
https://mirror.tuna.tsinghua.edu.cn/help/docker-ce/
apt-get install -y apt-transport-https ca-certificates curl software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
add-apt-repository \
"deb [arch=amd64] https://mirrors.ustc.edu.cn/docker-ce/linux/$(. /etc/os-release; echo "$ID") \
$(lsb_release -cs) \
stable"
master节点安装kubelet,kubeadm,kubectl
1.添加gpg公钥(国内):
wget https://raw.githubusercontent.com/EagleChen/kubernetes_init/master/kube_apt_key.gpg
cat kube_apt_key.gpg | sudo apt-key add -
显示OK即可
2.添加镜像源
echo "deb [arch=amd64] https://mirrors.ustc.edu.cn/kubernetes/apt kubernetes-xenial main" | sudo tee -a /etc/apt/sources.list
apt-get update
3.安装kubectl kubelet kubeadm
apt-get install kubelet kubeadm kubectl
kubelet此时状态还未ok因为没有提供config file "/var/lib/kubelet/config.yaml",先忽略,后续安装初始化k8s时会自动生成
4.加载内核模块,kubernetes内部路由转发使用ipvs,内核默认未加载,需要手动加载
cat > /etc/sysconfig/modules/ipvs.modules
#!/bin/bash
modprobe -- br_netfilter
modprobe -- ip_vs
modprobe -- ip_vs_rr
modprobe -- ip_vs_wrr
modprobe -- ip_vs_sh
modprobe -- nf_conntrack_ipv4
chmod 755 /etc/sysconfig/modules/ipvs.modules && \
bash /etc/sysconfig/modules/ipvs.modules && \
验证如下显示正常:
lsmod | grep -E "ip_vs|nf_conntrack_ipv4"
ip_vs_sh 16384 0
ip_vs_wrr 16384 0
ip_vs_rr 16384 0
ip_vs 151552 6 ip_vs_rr,ip_vs_sh,ip_vs_wrr
nf_defrag_ipv6 20480 1 ip_vs
nf_conntrack_ipv4 16384 3
nf_defrag_ipv4 16384 1 nf_conntrack_ipv4
nf_conntrack 131072 12 xt_conntrack,nf_nat_masquerade_ipv4,nf_conntrack_ipv4,nf_nat,nf_conntrack_pptp,ipt_MASQUERADE,nf_nat_ipv4,xt_nat,nf_nat_pptp,nf_conntrack_netlink,nf_conntrack_proto_gre,ip_vs
libcrc32c 16384 5 nf_conntrack,nf_nat,raid456,ip_vs,sctp
5.确保系统配置允许路由转发
cat <<EOF > /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-arptables = 1
net.bridge.bridge-nf-call-iptables = 1
#禁用ipv6
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1
EOF
或修改grub禁用ipv6
/etc/default/grub
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash ipv6.disable=1"
GRUB_CMDLINE_LINUX="ipv6.disable=1"
update-grub
6.生成kubeadm配置文件kubeadm-config.yaml
kubeadm config print init-defaults > kubeadm-config.yaml
修改几处配置:
advertiseAddress: 192.168.1.219 --->改为本机ip,多网卡,随机选择一个,注:此ip一旦绑定,不能修改,否则k8s集群将会异常
imageRepository: registry.aliyuncs.com/google_containers --->修改能拉取镜像的站点仓库
serviceSubnet: 10.96.0.0/12 --->可使用默认的,也可自定义
7.初始化主节点
kubeadm init --config kubeadm-config.yaml
若报错:The connection to the server localhost:8080 was refused - did you specify the right host or port?
需要如下操作
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
8.此时集群还未真正ready,需要安装网络插件
wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
选择适合本地操作系统的ds
9.添加新的节点
master上执行:
kubeadm token create --print-join-command --->执行结果作为node节点加入集群的命令
kubeadm init phase upload-certs --experimental-upload-certs --->执行结果作为新master加入集群中的证书
新的node节点上执行:
kubeadm join 192.168.1.219:6443 --token dxe4nm.wwa70dvhh7zdr6as\
--discovery-token-ca-cert-hash sha256:82f4a72b4959f67b67b34276650ab7b53fda753c5d52ad4843c2c85313667719
新的master节点执行:
kubeadm join 192.168.1.219:6443 --token dxe4nm.wwa70dvhh7zdr6as\
--discovery-token-ca-cert-hash sha256:82f4a72b4959f67b67b34276650ab7b53fda753c5d52ad4843c2c85313667719\
--experimental-control-plane --certificate-key $(kubeadm init phase upload-certs --experimental-upload-certs)
亦可尝试使用另一种方法(不建议):
新node节点上执行:
kubeadm config print join-defaults > kubeadm-config.yaml
修改生成的配置文件
kubeadm join --config kubeadm-config.yaml
10.重置kubernetes服务,重置网络。删除网络配置,link(慎用!!!)
kubeadm reset
systemctl stop kubelet
systemctl stop docker
rm -rf /var/lib/cni/
rm -rf /var/lib/kubelet/*
rm -rf /etc/cni/
ifconfig cni0 down
ifconfig flannel.1 down
ifconfig docker0 down
ip link delete cni0
ip link delete flannel.1
systemctl start docker
笔记:
允许普通用户使用kubectl查看集群状况--->将 /etc/kubernetes/admin.conf 拷贝到 普通用户的家目录下 ~/.kube/config文件
dashboard:
生成kubeconfig文件
DASH_TOCKEN=$(kubectl get secret -n kube-system dashboard-admin-token-xxxx -o jsonpath={.data.token}|base64 -d)
kubectl config set-cluster kubernetes --server=xxx.xxx.xxx.xxx:6443 --kubeconfig=/root/dashbord-admin.conf
kubectl config set-credentials dashboard-admin --token=$DASH_TOCKEN --kubeconfig=/root/dashbord-admin.conf
kubectl config set-context dashboard-admin@kubernetes --cluster=kubernetes --user=dashboard-admin --kubeconfig=/root/dashbord-admin.conf
kubectl config user-context dashboard-admin@kubernets --kubeconfig=/root/dashbord-admin.conf
生成的dashbord-admin.conf即可用于登录dashboard