k8s 部署指南 (国内版)

全新 CentOS 允许 root ssh 连接

echo "PermitRootLogin yes" >> /etc/ssh/sshd_config
systemctl restart sshd.service

全面切换到 Aliyun 镜像

OPSX

yum 源

# 备份
mv /etc/yum.repos.d/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo.backup
# 拉取配置文件
curl -o /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo
# 去掉无效地址(Aliyun 内网的一个源)
sed -i -e '/mirrors.cloud.aliyuncs.com/d' -e '/mirrors.aliyuncs.com/d' /etc/yum.repos.d/CentOS-Base.repo
# 构建缓存
yum makecache

docker 源 (带安装)

yum install -y yum-utils device-mapper-persistent-data lvm2
yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
yum makecache fast

# 安装 (已安装则忽略)
yum -y install docker-ce
systemctl enable docker.service && service docker start

kubernetes (带安装)

cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

# 安装 (已安装则忽略)
yum install -y kubelet kubeadm kubectl
systemctl enable kubelet && systemctl start kubelet

环境预备

关闭 swap

# 临时关闭
swapoff -a
# 永久关闭
sed -i "/swap/ s/^/#/" /etc/fstab

关闭 selinux

# 临时关闭
setenforce 0
# 永久关闭
echo "SELINUXTYPE=targeted" >>/etc/sysconfig/selinux
echo "SELINUX=disabled" >/etc/sysconfig/selinux

关闭防火墙

# 临时关闭
systemctl stop firewalld.service
# 永久关闭
systemctl disable firewalld.service

docker set to system instead of cgroup

mkdir /etc/docker

cat > /etc/docker/daemon.json <<EOF
{
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "storage-driver": "overlay2",
  "storage-opts": [
    "overlay2.override_kernel_check=true"
  ]
}
EOF

mkdir -p /etc/systemd/system/docker.service.d
systemctl daemon-reload
systemctl restart docker

set to one

cat > /etc/sysctl.d/k8s.conf << EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF

sysctl --system

镜像预备

由于在国内无法直连谷歌的服务器, 所以需要先下载好镜像然后再开始初始化

查看需要的镜像

kubeadm config images list

准备脚本

根据所需镜像版本准备 shell 处理脚本, 原理为自行拉取镜像, 然后修改成 k8s 的 tag 来让 kubeadm 误以为镜像已经拉取完毕

images=(
    kube-apiserver:v1.15.3
    kube-controller-manager:v1.15.3
    kube-scheduler:v1.15.3
    kube-proxy:v1.15.3
    pause:3.1
    etcd:3.3.10
    coredns:1.3.1
        pause-amd64:3.1
)

for imageName in ${images[@]} ; do
    docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/$imageName
    docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/$imageName k8s.gcr.io/$imageName
    docker rmi registry.cn-hangzhou.aliyuncs.com/google_containers/$imageName
done

执行脚本

kubeadm 初始化

kubeadm init --pod-network-cidr=192.168.0.0/16

安装网络插件 - Calico

使用 Azure 镜像下载

images=(
    node:v3.1.7
    cni:v3.1.7
    typha:v3.1.7
)

for imageName in ${images[@]} ; do
    docker pull quay.azk8s.cn/calico/$imageName
    docker tag quay.azk8s.cn/calico/$imageName quay.io/calico/$imageName
    docker rmi quay.azk8s.cn/calico/$imageName
done

安装

kubectl apply -f calico.yaml && kubectl apply -f rbac-kdd.yaml

Dashboard UI

images=(
    kubernetes-dashboard-amd64:v1.10.1
)

for imageName in ${images[@]} ; do
    docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/$imageName
    docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/$imageName k8s.gcr.io/$imageName
        docker rmi registry.cn-hangzhou.aliyuncs.com/google_containers/$imageName
done

问题跟踪

The connection to the server localhost:8080 was refused - did you specify the right host or port?

mkdir -p $HOME/.kube
cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
chown $(id -u):$(id -g) $HOME/.kube/config

附件

# calico.yaml
# Calico Version v3.1.