前言:官方提供Kubernetes部署3种方式
minikube
Minikube是一个工具,可以在本地快速运行一个单点的Kubernetes,尝试Kubernetes或日常开发的用户使用。不能用于生产环境。
官方文档:https://kubernetes.io/docs/setup/minikube/
kubeadm
kubeadm可帮助你快速部署一套kubernetes集群。kubeadm设计目的为新用户开始尝试kubernetes提供一种简单的方法。目前是Beta版。
官方文档:https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm/
https://kubernetes.io/docs/setup/independent/install-kubeadm/
二进制包
从官方下载发行版的二进制包,手动部署每个组件,组成Kubernetes集群。目前企业生产环境中主要使用该方式。
下载地址:https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.11.md#v1113
1. 安装要求
操作系统
Ubuntu 16.04+、Debian 9、CentOS 7、RHEL 7、Fedora 25/26 (best-effort)、其他
内存2GB + ,2核CPU +
集群节点之间可以通信
每个节点唯一主机名,MAC地址和product_uuid
检查MAC地址:使用ip link或者ifconfig -a
检查product_uuid:cat /sys/class/dmi/id/product_uuid
禁止swap分区。这样才能使kubelet正常工作
2. 准备环境
以下内容(master和node都需要执行)
关闭防火墙:
systemctl stop firewalld
systemctl disable firewalld
关闭selinux:
sed -i 's/enforcing/disabled/' /etc/selinux/config
setenforce 0
关闭swap:
swapoff -a # 临时
vim /etc/fstab # 永久
添加主机名与IP对应关系:
cat /etc/hosts
192.168.0.246 k8s-master
192.168.0.247 k8s-node1
192.168.0.248 k8s-node2
同步时间:
yum install ntpdate -y
ntpdate ntp.api.bz
kube-proxy开启ipvs:
创建/etc/sysctl.d/k8s.conf文件,添加如下内容:
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
执行命令使修改生效。
modprobe br_netfilter
sysctl -p /etc/sysctl.d/k8s.conf
由于ipvs已经加入到了内核的主干,所以为kube-proxy开启ipvs的前提需要加载以下的内核模块:
在所有的Kubernetes所有node节点和master上执行以下脚本:
cat > /etc/sysconfig/modules/ipvs.modules <<EOF
#!/bin/bash
modprobe -- ip_vs
modprobe -- ip_vs_rr
modprobe -- ip_vs_wrr
modprobe -- ip_vs_sh
modprobe -- nf_conntrack_ipv4
EOF
chmod 755 /etc/sysconfig/modules/ipvs.modules && bash /etc/sysconfig/modules/ipvs.modules && lsmod | grep -e ip_vs -e nf_conntrack_ipv4
3. 安装Docker
Docker与Kubernetes关系图:
设置阿里云docker(master和node都需要执行)
yum remove docker docker-common docker-selinux docker-engine –y
yum install -y yum-utils device-mapper-persistent-data lvm2
yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
yum list docker-ce --showduplicates | sort –r
yum makecache fast
yum install docker-ce-18.06.1.ce -y
或者可以设置官方docker
yum install -y yum-utils device-mapper-persistent-data lvm2
yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
yum install docker-ce-18.06.1.ce -y #目前kubeadm最大支持docker-ce-17.03,所以要指定该版本安装
systemctl enable docker && systemctl start docker
如果提示container-selinux依赖问题,先安装ce-18.06匹配版本:yum localinstall https://download.docker.com/linux/centos/7/x86_64/stable/Packages/docker-ce-selinux-17.03.3.ce-1.el7.noarch.rpm
最新版本是用一下内容:
wget https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo -O /etc/yum.repos.d/docker-ce.repo
yum -y install docker-ce-18.06.1.ce-3.el7
systemctl enable docker && systemctl start docker
docker --version
4. 安装kubeadm,kubelet和kubectl
kubeadm: 引导集群的命令
kubelet:集群中运行任务的代理程序
kubectl:命令行管理工具
4.1 添加阿里云YUM软件源
cat << EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF
4.2 安装kubeadm,kubelet和kubectl
yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes
systemctl enable kubelet && systemctl start kubelet
指定版本操作如下:
yum install -y kubeadm-1.13.3 kubectl-1.13.3
yum install -y kubelet-1.13.3
systemctl enable kubelet.service
注意:修改/etc/sysctl.d/k8s.conf添加下面一行:
vm.swappiness=0
然后执行以下命令
systemctl daemon-reload
systemctl restart kubelet
5. 使用kubeadm创建单个Master集群
5.1 (手动下载)默认下载镜像地址在国外无法访问,先从准备好所需镜像
保存到脚本之间运行:
K8S_VERSION=v1.13.3
ETCD_VERSION=3.2.18
DASHBOARD_VERSION=v1.8.3
FLANNEL_VERSION=v0.10.0-amd64
DNS_VERSION=1.1.3
PAUSE_VERSION=3.1
基本组件
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-apiserver-amd64:$K8S_VERSION
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-controller-manager-amd64:$K8S_VERSION
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-scheduler-amd64:$K8S_VERSION
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-proxy-amd64:$K8S_VERSION
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/etcd-amd64:$ETCD_VERSION
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/pause:$PAUSE_VERSION
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/coredns:$DNS_VERSION
网络组件docker pull quay.io/coreos/flannel:$FLANNEL_VERSION
修改tag
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-apiserver-amd64:$K8S_VERSION k8s.gcr.io/kube-apiserver-amd64:$K8S_VERSION
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-controller-manager-amd64:$K8S_VERSION k8s.gcr.io/kube-controller-manager-amd64:$K8S_VERSION
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-scheduler-amd64:$K8S_VERSION k8s.gcr.io/kube-scheduler-amd64:$K8S_VERSION
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-proxy-amd64:$K8S_VERSION k8s.gcr.io/kube-proxy-amd64:$K8S_VERSION
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/etcd-amd64:$ETCD_VERSION k8s.gcr.io/etcd-amd64:$ETCD_VERSION
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/pause:$PAUSE_VERSION k8s.gcr.io/pause:$PAUSE_VERSION
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/coredns:$DNS_VERSION k8s.gcr.io/coredns:$DNS_VERSION
5.2 初始化Master
初始化master之前确认修改/etc/sysconfig/kubelet;
加入:
KUBELET_EXTRA_ARGS=--fail-swap-on=false
kubeadm init \
--apiserver-advertise-address=192.168.0.246 \
--image-repository registry.aliyuncs.com/google_containers \
--kubernetes-version v1.13.3 \
--service-cidr=10.1.0.0/16 \
--pod-network-cidr=10.244.0.0/16
...
Your Kubernetes master has initialized successfully!
To start using your cluster, you need to run (as a regular user):
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the addon options listed at:
http://kubernetes.io/docs/admin/addons/
You can now join any number of machines by running the following on each node
as root:
kubeadm join --token <token> <master-ip>:<master-port> --discovery-token-ca-cert-hash sha256:<hash>
执行一下命令
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
5.3 安装Pod网络 - 插件
wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
kubectl apply -f kube-flannel.yml
或使用一下方法kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/v0.10.0/Documentation/kube-flannel.yml
配置完flannel.yml后执行一下命令查看是否运行:
kubectl get pod --all-namespaces -o wide
途中显示镜像下载出现问题:可以手动进行下载,但是需要×××:
docker pull quay.io/coreos/flannel:v0.11.0-amd64
5.4 加入工作节点
在Node节点切换到root账号执行:kubeadm join 192.168.0.11:6443 --token 6hk68y.0rdz1wdjyh85ntkr --discovery-token-ca-cert-hash sha256:d1d3f59ae37fbd632707cbeb9b095d0d0b19af535078091993c4bc4d9d2a7782
格式:kubeadm join --token <token> <master-ip>:<master-port> --discovery-token-ca-cert-hash sha256:<hash>
5.5.测试kubernetes集群
kubectl create deployment nginx --image=nginx
kubectl expose deployment nginx --port=80 --type=NodePort
kubectl get pod,svc
查看nginx分配到那个节点等信息Kubectl get pod,svc –o wide
kubernetes dashboard
先将yaml文件下载下来,修改里面镜像地址和Service NodePort类型。wget https://raw.githubusercontent.com/kubernetes/dashboard/v1.10.1/src/deploy/recommended/kubernetes-dashboard.yaml
通过一下地址确认dashboard版本列表
https://github.com/kubernetes/dashboard
修改镜像地址:(Dashboard Deployment 的位置)
将 image: k8s.gcr.io/kubernetes-dashboard-amd64:v1.10.1
修改为:
registry.cn-hangzhou.aliyuncs.com/google_containers/kubernetes-dashboard-amd64:v1.10.1
或者直接使用:或者直接:docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kubernetes-dashboard-amd64:v1.10.1
修改Service:(注意因为yaml文件不支持使用tab键)
kind: Service
apiVersion: v1
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard
namespace: kube-system
spec:
type: NodePort
ports:
port: 443
targetPort: 8443
nodePort: 30001
selector:
k8s-app: kubernetes-dashboard
应用kubernetes-dashboard.yamlkubectl apply -f kubernetes-dashboard.yaml
登陆方法1:创建一个管理员角色:
apiVersion: v1
kind: ServiceAccount
metadata:
name: dashboard-admin
namespace: kube-system
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: dashboard-admin
subjects:
kind: ServiceAccount
name: dashboard-admin
namespace: kube-system
roleRef:
kind: ClusterRole
name: cluster-admin
apiGroup: rbac.authorization.k8s.io
kubectl apply -f k8s-admin.yaml
使用上述创建账号的token登录Kubernetes Dashboard:
kubectl get secret -n kube-system
kubectl describe secret dashboard-admin-token-bwdjj -n kube-system
...
token: eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJkYXNoYm9hcmQtYWRtaW4tdG9rZW4tYndkamoiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoiZGFzaGJvYXJkLWFkbWluIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiNzIyOTRmNTUtYjc1OC0xMWU4LThkY2UtMDAwYzI5ZGUyNWVhIiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50Omt1YmUtc3lzdGVtOmRhc2hib2FyZC1hZG1pbiJ9.0hQU5Di_P1OX1DcnW2AYzjDAED66EOrqhF5iupv39wvB8wE-aLRSQyp0twX2M8u1KMZ67n6LxbH17VwEQkMDRVXs7ZlUCyAAD6kHDz3k-f7PAzH5vcuyO4veQ9ooVjk3DKjrP4zXQChHllBB1wmD_oyLjoWxK3Z5MBTlVGzSixVwuQNpFPbuS6Z7iLGwUOgjI0cGZ9Tt6cXzcK81KfAEpDIP_CtFV_Jw4s98EgBex9mZh6vq1dcxr03qfuK--udd_8GWZctu_p_P15hZZLoKEm5GCbs6JGvKL2aao_DEHfLp3XYEnApojI91vU4qAqdkvMZ2qWQNGYv4KNi2yPOOJQ
登录方法2:创建sa并绑定默认的cluster-admin管理员集群角色
kubectl create serviceaccount dashboard-admin -n kube-system
kubectl create clusterrolebinding dashboard-admin --clusterrole=cluster-admin --serviceaccount=kube-system:dashboard-admin
使用上述创建账号的token登录Kubernetes Dashboard:
kubectl get secret -n kube-system
kubectl describe secret dashboard-admin-token-bwdjj -n kube-system
注意:查看kubernetes-dashboard 命令kubectl --namespace=kube-system get service kubernetes-dashboard
7解决Google浏览器不能打开kubernetes dashboard方法
mkdir key && cd key
生成证书
openssl genrsa -out dashboard.key 2048
openssl req -new -out dashboard.csr -key dashboard.key -subj '/CN=192.168.246.200'
openssl x509 -req -in dashboard.csr -signkey dashboard.key -out dashboard.crt
删除原有的证书secret
kubectl delete secret kubernetes-dashboard-certs -n kube-system
创建新的证书secret
kubectl create secret generic kubernetes-dashboard-certs --from-file=dashboard.key --from-file=dashboard.crt -n kube-system
查看pod
kubectl get pod -n kube-system
重启pod
kubectl delete pod <pod name> -n kube-system (pad name 为kubernetes-dashboard的名称)