k8s 基础学习
ubuntu 16.04安装 k8s
ubuntu 16.04 系统初始化 (所有节点都安装)
step 0 系统初始化
# 准备基础包, ubuntu 必备包
sudo apt-get install wget net-tools telnet tree nmap sysstat lrzsz dos2unix vim less selinux-utils ntpdate unzip zip -y
# 安装显卡驱动
sudo apt-get --purge remove nvidia*
sudo apt install nvidia-430
setenforce 0
swapoff -a
free -m
# 关闭防火墙 (临时关闭)
setenforce 0 (0 表示关闭, 1表示启动)
# 关闭swap
swapoff -a
# 查看是否关闭成功 ( 0 0 0 表示Ok)
free -m
total used free shared buff/cache available
Mem: 32123 200 30854 145 1068 31384
Swap: 0 0 0
# 永久关闭
cp -p /etc/fstab /etc/fstab.bak$(date '+%Y%m%d%H%M%S')
swapoff -a && sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab
# 配置主机名
hostnamectl set-hostname k8s-master
# 在 master 添加 hosts
cat >> /etc/hosts << EOF
192.168.1.133 k8s-master1
192.168.1.114 k8s-master2
192.168.2.158 k8s-node1
EOF
# 将桥接的 IPv4 流量传递到 iptables 的链
# 执行 docker info 时出现警告
# WARNING: bridge-nf-call-iptables is disabled
# WARNING: bridge-nf-call-ip6tables is disabled
# 解决方案如下
cat > /etc/sysctl.d/k8s.conf << EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sysctl --system # 生效
# 时间同步
ntpdate time.windows.com
系统初始化完毕
安装k8s (所有节点都安装)
step.1 准备安装k8s
apt-get update && apt-get install -y apt-transport-https
step.2
curl https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | apt-key add -
step.3
apt-add-repository "deb http://apt.kubernetes.io/ kubernetes-xenial main"
# 使用阿里源
apt-add-repository "deb https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial main"
step.4
apt-get update
step.5 安装指定版本
apt-get install -y kubelet=1.14.0-00 kubeadm=1.14.0-00 kubectl=1.14.0-00 kubernetes-cni=0.7.5-00
# 查看
kubeadm config images list
step.6 锁定版本不被更新
apt-mark hold kubelet kubeadm kubectl kubernetes-cni
在所有节点,都执行以上安装配置,注意IP和主机名设置,事先规划好.
部署 k8s (只在master 上安装)
step 1 在k8s-master1 (192.168.1.133 )执行
kubeadm init --image-repository=registry.aliyuncs.com/google_containers --pod-network-cidr=10.244.0.0/16 --apiserver-advertise-address=192.168.1.133 --kubernetes-version=v1.14.0 --node-name=k8s-master1 --ignore-preflight-errors=all
# 指定了 阿里源
step 1.1 保存token,后面加入节点的时候用到
# 加入节点时 24小时内有效 使用的token:
kubeadm join 192.168.1.133:6443 --token gbmlio.v0ngqeydw80th547 \
--discovery-token-ca-cert-hash sha256:530db7991d181c1e6cd410f7c3db88231be639cf563521c82e1af6c1388e8458
step 1.2 如果token过期了,执行以下操作
kubeadm token generate
#输出:pmiad9.1g4qrc4yvkqxggse
kubeadm token create pmiad9.1g4qrc4yvkqxggse --print-join-command --ttl=0
step 2 配置kubectl 工具 (如果需要在node上使用, 也需要配置)
mkdir -p $HOME/.kube
cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
chown $(id -u):$(id -g) $HOME/.kube/config
kubectl get nodes
# 显示 NotReady,等下安装CNI就 Ready了
NAME STATUS ROLES AGE VERSION
k8s-master1 NotReady master 7m11s v1.14.0
# 本次生效
export KUBECONFIG=/etc/kubernetes/admin.conf
# 永久
vim /etc/profile
export KUBECONFIG=/etc/kubernetes/admin.conf
step 3 安装 Pod 网络插件( CNI)
# kube-flannel.yml 在gitlab上
kubectl apply -f tools/dev-ops/k8s/flannel/kube-flannel.yml
# 查看 等1min
kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master1 Ready master 25m v1.14.0
master 节点准备完毕
部署 Node节点
step 1 kubeadm join
在k8s-node1(192.168.1.114)机器上安装k8s. 然后添加到master上
# 在k8s-node1(192.168.1.114)上执行
kubeadm join 192.168.1.133:6443 --token gbmlio.v0ngqeydw80th547 \
--discovery-token-ca-cert-hash sha256:530db7991d181c1e6cd410f7c3db88231be639cf563521c82e1af6c1388e8458
step 2 查看
# 在master机器上执行
# 可以看到k8s-node1已经添加到了集群中
kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master1 Ready master 3h56m v1.14.0
k8s-node1 Ready <none> 5m36s v1.14.0
测试集群
在 Kubernetes 集群中创建一个 pod, 验证是否正常运行
kubectl create deployment nginx --image=nginx
kubectl expose deployment nginx --port=80 --type=NodePort
kubectl get pod,svc
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod/nginx-65f88748fd-xvk2q 1/1 Running 0 36m 10.244.1.2 k8s-node1 <none> <none>
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 4h35m <none>
service/nginx NodePort 10.103.126.238 <none> 80:31241/TCP 35m app=nginx
可以看到 nginx 已经在 k8s-node1上 执行起来了.
访问地址: http://NodeIP:Port ( http://192.168.1.133:31241 )
Docker 安装
卸载旧版本
step 1 删除某软件,及其安装时自动安装的所有包
apt-get autoremove docker docker-ce docker-engine docker.io containerd runc
step 2 删除docker
dpkg -l | grep docker
dpkg -l |grep ^rc|awk '{print $2}' |sudo xargs dpkg -P # 删除无用的相关的配置文件
step 3 卸载没有删除的docker相关插件
apt-get autoremove docker-ce-*
step 4 删除docker的相关配置&目录
rm -rf /etc/systemd/system/docker.service.d
rm -rf /var/lib/docker
安装 Docker 19.03.9
包安装
# 19.03.11
apt install docker.io
二进制安装
step 1 下载
wget https://download.docker.com/linux/static/stable/x86_64/docker-
19.03.9.tgz
step 2 解压二进制包
tar zxvf docker-19.03.9.tgz
mv docker/* /usr/bin
step 3
docker version
step 4 systemd 管理 docker
mkdir /usr/lib/systemd/system/
cat > /usr/lib/systemd/system/docker.service << EOF
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network-online.target firewalld.service
Wants=network-online.target
[Service]
Type=notify
ExecStart=/usr/bin/dockerd
ExecReload=/bin/kill -s HUP $MAINPID
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
TimeoutStartSec=0
Delegate=yes
KillMode=process
Restart=on-failure
StartLimitBurst=3
StartLimitInterval=60s
[Install]
WantedBy=multi-user.target
EOF
step 5 配置阿里云
mkdir /etc/docker
cat > /etc/docker/daemon.json << EOF
{
"registry-mirrors": ["https://b9pmyelo.mirror.aliyuncs.com"]
}
EOF
例子:
cat > /etc/docker/daemon.json << EOF
{
"insecure-registries":["prd-ai-registry:5000"]
}
EOF
cat > /etc/docker/daemon.json << EOF
{
"insecure-registries":["prd-ai-registry:5000"],
"default-runtime": "nvidia",
"runtimes": {
"nvidia": {
"path": "/usr/bin/nvidia-container-runtime",
"runtimeArgs": []
}
}
}
EOF
step 6 设置开机启动
systemctl daemon-reload
systemctl start docker
systemctl enable docker
step 7
docker version
Docker 安装完成
笔记
- k8s配置ingress
https://www.jianshu.com/p/c726ed03562a
- k8s–pod的状态为evicted
kubectl get pods -n dev
发现很多pod的状态为evicted.
解决方案:
kubectl get pods -n dev | grep Evicted | awk '{print $1}' | xargs kubectl delete pod -n dev
- k8s 删除 service
kubectl delete svc/svc-name -n namespace
OR
kubectl delete -f svc.yaml
例子:
kubectl delete svc/svc-b-ai-predict-50000498 -n dev
参考
- 使用Kubeadm创建k8s集群之节点部署(三十一)
- k8s学习笔记
- https://www.yuque.com/duduniao/k8s
- https://github.com/fanjianhai/K8S