一、概述
Rancher是一套容器管理平台,它可以帮助组织在生产环境中轻松快捷的部署和管理容器。 Rancher可以轻松地管理各种环境的Kubernetes,满足IT需求并为DevOps团队提供支持
1、架构设计
二、安装环境
1、硬件环境
节点 | ip | OS | 角色 | 配置 |
---|---|---|---|---|
node0 | 192.168.1.100 | centos7.4+ | nginx | 2Ccpu、4G内存、50G磁盘 |
node1 | 192.168.1.101 | centos7.4+ | master,etcd,worker | 4Ccpu、16G内存、200G磁盘 |
node2 | 192.168.1.102 | centos7.4+ | master,etcd,worker | 4Ccpu、16G内存、200G磁盘 |
node3 | 192.168.1.103 | centos7.4+ | master,etcd,worker | 4Ccpu、16G内存、200G磁盘 |
2、软件环境
os:centos7.4+
rancher:1.6+
docker:17.03.2-ce , overlay2存储,extfs 文件系统
三、架构
四、安装前的准备
1、主机环境配置
1.1、 关闭防火墙
sudo systemctl stop firewalld.service && systemctl disable firewalld.service
1.2、关闭setlinx
sudo sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config
1.3、关闭swap
注释或删除swap交换分区:vi /etc/fstab
#/dev/mapper/centos-swap swap swap defaults 0 0
1.4、配置主机时间、时区、系统语言
sudo ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
sudo echo 'LANG="en_US.UTF-8"' >> /etc/profile;source /etc/profile
1.5、Kernel性能调优
cat >> /etc/sysctl.conf<<EOF net.ipv4.ip_forward=1 net.bridge.bridge-nf-call-iptables=1 net.ipv4.neigh.default.gc_thresh1=4096 net.ipv4.neigh.default.gc_thresh2=6144 net.ipv4.neigh.default.gc_thresh3=8192 EOF
2、安装docker-ce并配置
2.1安装docker-ce
# 添加用户(可选)
sudo adduser user
# 为新用户设置密码
sudo passwd user
# 为新用户添加sudo权限
sudo echo 'user ALL=(ALL) ALL' >> /etc/sudoers
# 卸载旧版本Docker软件
sudo yum remove docker docker-client docker-client-latest docker-common docker-latest docker-latest-logrotate docker-logrotate docker-selinux docker-engine-selinux docker-engine container*
# 定义安装版本
export docker_version=17.03.2
# step 1: 安装必要的一些系统工具
sudo yum update -y sudo yum install -y yum-utils device-mapper-persistent-data lvm2 bash-completion
# Step 2: 添加软件源信息
sudo yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
# Step 3: 更新并安装 Docker-CE
sudo yum makecache all
version=$(yum list docker-ce.x86_64 --showduplicates | sort -r|grep ${docker_version}|awk '{print $2}')
sudo yum -y install --setopt=obsoletes=0 docker-ce-${version} docker-ce-selinux-${version}
# 如果已经安装高版本Docker,可进行降级安装(可选)
yum downgrade --setopt=obsoletes=0 -y docker-ce-${version} docker-ce-selinux-${version}
# 把当前用户加入docker组
sudo usermod -aG docker user
# 设置开机启动 sudo systemctl enable docker
2.2、docker配置
sudo vi /etc/docker/daemon.sjon
{
,"log-driver": "json-file"
,"log-opts": {
"max-size": "100m",
"max-file": "3"
}
,"max-concurrent-downloads": 10
,"max-concurrent-uploads": 10
,"storage-driver": "overlay2"
,"storage-opts": ["overlay2.override_kernel_check=true"]
}
3、下载命令行工具
kubectl:https://www.cnrancher.com/download/kubectl/kubectl_amd64-linux
rke:https://www.cnrancher.com/download/rke/rke_linux-amd64
helm:https://www.cnrancher.com/download/helm/helm-linux.tar.gz
cd /usr/local/bin
sudo wget https://www.cnrancher.com/download/kubectl/kubectl_amd64-linux
sudo wget https://www.cnrancher.com/download/rke/rke_linux-amd64
sudo wget https://www.cnrancher.com/download/helm/helm-linux.tar.gz
sudo mv kubectl_amd64-linux kubectl
sudo mv rke_linux-amd64 rke
解压helm-linux.tar.gz并将解压包helm复制到usr/local/bin
sudo tar -zxvf helm-linux.tar.gz
sudo cp linux-amd64/helm .
赋可执行权限
sudo chmod +x .
修改profile
sudo vi /etc/profile
追加:
export PATH="/usr/local/bin:$PATH"
保存退出,使之生效
sudo source /etc/profile
4、免密登录
在192.168.1.100用user用户执行:
ssh-keygen -t rsa
ssh-copy-id -i ~/.ssh/id_rsa.pub user@192.168.1.101
ssh-copy-id -i ~/.ssh/id_rsa.pub user@192.168.1.102
ssh-copy-id -i ~/.ssh/id_rsa.pub user@192.168.1.103
5、安装nginx并配置
5.1、安装ngin
sudo rpm -Uvh http://nginx.org/packages/centos/7/noarch/RPMS/nginx-release-centos-7-0.el7.ngx.noarch.rpm
yum install nginx -y
sudo systemctl enable nginx.service
5.2 配置nginx
sudo vi /etc/nginx/nginx.conf
user nginx;
worker_processes 4;
error_log /var/log/nginx/error.log warn;
worker_rlimit_nofile 40000;
events {
worker_connections 8192;
}
http {
server {
listen 80;
return 301 https://$host$request_uri;
}
}
stream {
upstream rancher_servers {
least_conn;
server 192.168.1.101:443 max_fails=3 fail_timeout=5s;
server 192.168.1.102:443 max_fails=3 fail_timeout=5s;
server 192.168.1.103:443 max_fails=3 fail_timeout=5s;
}
server {
listen 443;
proxy_pass rancher_servers;
}
}
5、rke安装k8s
5.1、配置rke 创建rancher-cluster.yml
文件
nodes:
- address: 192.168.1.101
user: user
role: [controlplane,worker,etcd]
- address: 192.168.1.102
user: user
role: [controlplane,worker,etcd]
- address: 192.168.1.103
user: user
role: [controlplane,worker,etcd]
services:
etcd:
snapshot: true
creation: 6h
retention: 24h
具体rke配置参数 可参考 https://www.cnrancher.com/docs/rke/v0.1.x/en/example-yamls/
可定制配置参数部分用例:
cluster_name: mycluster
# 集群版本,必顺在 rancher/types defaults map中定义 : https://github.com/rancher/types/blob/master/apis/management.cattle.io/v3/k8s_defaults.go#L14
# 如果k8s版本和是像在 system_images被定义, system_images中定义的k8s版本将优先于k8s版本.
kubernetes_version: v1.10.3-rancher2
services:
etcd:
kube-api:
# 集群service集群ip范转
service_cluster_ip_range: 10.43.0.0/16
# nodeport service 端口区间
service_node_port_range: 30000-32767
pod_security_policy: false
# Add additional arguments to the kubernetes API server # This WILL OVERRIDE any existing defaults
extra_args: # Enable audit log to stdout
audit-log-path: "-"
# Increase number of delete workers
delete-collection-workers: 3
# Set the level of log output to debug-level
v: 4
# Note for Rancher 2 users: If you are configuring Cluster Options using a Config File when creating Rancher Launched Kubernetes, the names of services should contain underscores only: `kube_controller`. This only applies to Rancher v2.0.5 and v2.0.6.
kube-controller:
# 集群podip范围
cluster_cidr: 10.42.0.0/16
#集群service集群ip范转
#应和kube-api中service_cluster_ip_range保持一致
service_cluster_ip_range: 10.43.0.0/16
kubelet:
# Base domain for the cluster
cluster_domain: cluster.local
# IP address for the DNS service endpoint
cluster_dns_server: 10.43.0.10
# Fail if swap is on
fail_swap_on: false
# 设置最大pod数量为250代替默认110
extra_args: max-pods: 250
5.2、运行RKE命令
rke up --config rancher-cluster.yml
完成后,它应显示:Finished building Kubernetes cluster successfully。
在本目录会生成config文件:kube_config_rancher-cluster.yml
如若操作失败,重新安装需要清理数据:
su - root
rm -rf /var/lib/rancher/etcd/*
rm -rf /etc/kubernetes/*
su - rancher
rke remove --config ./rancher-cluster.yml
5.3、测试集群
mkdir ~/.kube
cp kube_config_rancher-cluster.yml ~/.kube/config
kubectl get cs
NAME STATUS MESSAGE ERROR
controller-manager Healthy ok
scheduler Healthy ok
etcd-0 Healthy {"health": "true"}
etcd-2 Healthy {"health": "true"}
etcd-1 Healthy {"health": "true"}
kubectl get node
NAME STATUS ROLES AGE VERSION
192.168.1.101 Ready controlplane,etcd,worker 20d v1.13.4
192.168.1.102 Ready controlplane,etcd,worker 20d v1.13.4
192.168.1.103 Ready controlplane,etcd,worker 20d v1.13.4
6、安装配置Helm
6.1、创建helm客户端访问权限
kubectl -n kube-system create serviceaccount tiller
ubectl create clusterrolebinding tiller --clusterrole cluster-admin --serviceaccount=kube-system:tiller
6.2、安装helm服务端tiller
helm init --service-account tiller --tiller-image registry.cn-hangzhou.aliyuncs.com/google_containers/tiller:v2.13.0 --stable-repo-url https://kubernetes.oss-cn-hangzhou.aliyuncs.com/charts
7、helm安装rancher
7.1、添加Chart仓库地址
helm repo add rancher-stable https://releases.rancher.com/server-charts/stable
7.2、安装证书管理器
helm install stable/cert-manager \
--name cert-manager \
--namespace kube-system
7.3、安装rancher
helm install rancher-stable/rancher \
--name rancher \
--namespace cattle-system \
--set hostname=k8s-100
--version v2.1.6 (可选版本)
k8s-100就是后面访问rancher的域名,需要在/etc/hosts文件中添加关联(所有主机) 192.168.1.100 k8s-100
由于我们通过hosts文件来添加映射,所以需要为Agent Pod添加主机别名(/etc/hosts):
kubectl -n cattle-system patch deployments cattle-cluster-agent --patch '{
"spec": {
"template": {
"spec": {
"hostAliases": [
{
"hostnames":
[
"k8s-100"
],
"ip": "192.168.1.100"
}
]
}
}
}
}'
kubectl -n cattle-system patch daemonsets cattle-node-agent --patch '{
"spec": {
"template": {
"spec": {
"hostAliases": [
{
"hostnames":
[
"k8s-100"
],
"ip": "192.168.1.100"
}
]
}
}
}
}'
8、登录rancher管理端
https://k8s-100
输入:admin/admin,设置用户密码。
9、rancher版本升级
helm安装后会在用户目录下创建本地库目录.helm
更新库:
helm repo update
查找版本:
helm search rancher
升级:
helm upgrade rancher rancher-stable/rancher --version v2.1.7
10、rancher版本回滚
#相看历史版本
helm history rancher
#回滚到历史版本
helm rollback rancher 1