@[TOC]国内环境Kubernetes v1.12.3的安装与配置
0、环境
主机名 IP
master1 172.16.105.21
master2 172.16.105.22
master3 172.16.105.23
node1 172.16.105.21
node2 172.16.105.22
node2 172.16.105.23
ansible-client 172.16.105.21
1、安装ansible和依赖
在172.16.105.21安装ansible
升级内环版本(用于cephfs文件系统):
rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrepo.org
rpm -Uvh https://www.elrepo.org/elrepo-release-7.0-3.el7.elrepo.noarch.rpm
yum -y --enablerepo=elrepo-kernel install kernel-ml-devel kernel-ml
grep menuentry /boot/grub2/grub.cfg
grub2-set-default ‘CentOS Linux (4.20.3-1.el7.elrepo.x86_64) 7 (Core)’
modprobe nf_conntrack
安装 python 及 epel
yum install -y epel-release python-pip python34 python34-pip
安装 ansible
yum install -y ansible
pip install netaddr
pip install --upgrade jinja2
2、建立公私钥,分发各服务器
在ansible-clinet机器生成免密密钥对
ssh-keygen -t rsa -P ‘’
将生成的公钥(id_rsa.pub)传到其他节点,这样ansible-client可以免密登陆其他机器
cat id_rsa.pub >> ~/.ssh/authorized_keys
3、下载源码
git clone https://github.com/kubernetes-sigs/kubespray
git checkout release-2.8
4.安装指定docker
yum list docker-ce --showduplicates | sort -r #要安装特定版本的Docker CE,请列出repo中的可用版本
yum install docker-ce- #安装特定版本
yum install docker-ce-18.06.1.ce-3.el7
yum install kubectl-1.12.5-0
yum install kubelet-1.12.5
yum install kubeadm-1.12.5-0
5.生成cluster配置文件
Copy inventory/sample
as inventory/mycluster
cp -rfp inventory/sample inventory/mycluster
Update Ansible inventory file with inventory builder
declare -a IPS=(10.10.1.3 10.10.1.4 10.10.1.5)
CONFIG_FILE=inventory/mycluster/hosts.ini python3 contrib/inventory_builder/inventory.py ${IPS[@]}
6.替换镜像
find . -name ‘.yml’ | xargs -n1 -I{} sed -i ‘s/gcr.io/google-containers//mirrorgooglecontainers//’ {}
find . -name '.yml’ | xargs -n1 -I{} sed -i ‘s/k8s.gcr.io//mirrorgooglecontainers//’ {}
7.运行命令
ansible-playbook -i inventory/mycluster/hosts.ini --become --become-user=root cluster.yml -vvv
8.启用addons
编辑inventory/testcluster/group_vars/k8s-cluster/addons.yml,将需要的addon设为true,并配置对应的参数。比如我这里启用了dashboard、heml、local_volume_provisioner、cephfs_provisioner、ingress_nginx和cert_manager:
dashboard_enabled: true
helm_enabled: true
registry_enabled: false
metrics_server_enabled: true
local_volume_provisioner_enabled: true
cephfs_provisioner_enabled: true
cephfs_provisioner_namespace: "cephfs-provisioner"
cephfs_provisioner_cluster: ceph
cephfs_provisioner_monitors: "10.32.3.70:6789,10.32.3.71:6789,10.32.3.72:6789"
cephfs_provisioner_admin_id: k8s
cephfs_provisioner_secret: AQBCk+tbHeLjORAAHiUMFIeu8f76JWBWlCWfbg==
cephfs_provisioner_storage_class: cephfs
cephfs_provisioner_reclaim_policy: Delete
cephfs_provisioner_claim_root: /k8s_volumes
cephfs_provisioner_deterministic_names: true
ingress_nginx_enabled: true
cert_manager_enabled: true
7、安装失败如何清理
rm -rf /etc/kubernetes/
rm -rf /etc/ssl/etcd/
rm -rf /var/lib/kubelet
rm -rf /var/lib/etcd
rm -rf /usr/local/bin/kubectl
rm -rf /etc/systemd/system/calico-node.service
rm -rf /etc/systemd/system/kubelet.service
systemctl stop etcd.service
systemctl disable etcd.service
systemctl stop calico-node.service
systemctl disable calico-node.service
docker stop $(docker ps -q)
docker rm $(docker ps -a -q)
systemctl restart docker
访问dashboard
创建dashboard管理用户
kubectl create serviceaccount dashboard-admin -n kube-system
绑定用户为集群管理用户
kubectl create clusterrolebinding dashboard-cluster-admin --clusterrole=cluster-admin --serviceaccount=kube-system:dashboard-admin
获取tocken
kubectl describe secret -n kube-system dashboard-admin-token-
cephfs:
git clone https://github.com/kubernetes-incubator/external-storage.git
cd external-storage/ceph/cephfs/deploy/rbac/
vi clusterrole.yaml
加入配置项:
- apiGroups: [""]
resources: [“secrets”]
verbs: [“get”, “create”, “delete”]