二进制安装k8s1.19.8及prometheus监控

目录

机器规划（CentOS 7.6  内核3.10+）

一、二进制安装kubernetes

1、环境准备（每台主机都执行）

2、安装docker（每台主机都执行）

3、搭建etcd集群（master1节点执行）

4、kubernetes组件部署

5、 部署kubelet

6、部署kube-proxy

7、配置网络组件calico

 8、部署coredns

9、安装K8S遇到的问题

 二、基于k8s安装Prometheus

1、下载Prometheus文件（prometheus-operator）

2、修改配置文件

 3、创建monitoring 名称空间并安装prometheus-operator

---

机器规划（CentOS 7.6  内核3.10+）

k8s网络环境规划：

• k8s版本：v1.19.8

• Pod网段：10.0.0.0/16

• Service网段：10.255.0.0/16

docker版本：19.03.9

hostname	ip	k8s节点	机器配置
k8s-master-01	192.168.10.10	master	2C4G
k8s-master-02	192.168.10.11	master	2C4G
k8s-master-03	192.168.10.12	master	2C4G
k8s-node-01	192.168.10.13	node	2C4G
k8s-node-02	192.168.10.14	node	2C4G
负载均衡VIP	192.168.10.100

一、二进制安装kubernetes

1、环境准备（每台主机都执行）

1）修改host主机名

hostname master1
hostname master2
hostname master3
hostname node1
hostname node2

2）配置hosts文件

cat >>/etc/hosts <<EOF
192.168.10.10 master1
192.168.10.11 master2
192.168.10.12 master3
192.168.10.13 node1
192.168.10.13 node2
EOF

3）关闭防火墙和selinux

systemctl stop firewalld && systemctl disable firewalld
setenforce 0
sed -i 's/^SELINUX=enforcing/SELINUX=disabled/' /etc/selinux/config

4）关闭交换分区

swapoff -a
sed -i 's/^[^#].*swap*/#&/g' /etc/fstab

5）时间同步

yum install -y chrony
systemctl start chronyd
systemctl enable chronyd
chronyc sources

6）加载ipvs模块

modprobe -- ip_vs
modprobe -- ip_vs_rr
modprobe -- ip_vs_wrr
modprobe -- ip_vs_sh
modprobe -- nf_conntrack_ipv4
lsmod | grep ip_vs
lsmod | grep nf_conntrack_ipv4
yum install -y ipvsadm

7）修改内核参数

# 加载br_netfilter模块

modprobe br_netfilter

#验证模块是否加载成功

lsmod |grep br_netfilter

#修改内核参数

cat > /etc/sysctl.d/k8s.conf <<EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
EOF

#使刚才修改的内核参数生效

sysctl -p /etc/sysctl.d/k8s.conf

#设置文件描述符

echo '*               -       nofile          65535' >> /etc/security/limits.conf

2、安装docker（每台主机都执行）

sudo yum install -y yum-utils
sudo yum-config-manager \
    --add-repo \
    https://download.docker.com/linux/centos/docker-ce.repo
sudo yum install docker-ce-19.03.9 docker-ce-cli-19.03.9 containerd.io

1）命令自动补全

yum -y install bash-completion
source /etc/profile.d/bash_completion.sh

systemctl start docker && systemctl enable docker && systemctl status docker

2）设置仓库源和驱动（下述源为国内加速源）

cat > /etc/docker/daemon.json << EOF
{
    "exec-opts": ["native.cgroupdriver=systemd"],
    "registry-mirrors": [
        "https://1nj0zren.mirror.aliyuncs.com",
        "https://kfwkfulq.mirror.aliyuncs.com",
        "https://2lqq34jg.mirror.aliyuncs.com",
        "https://pee6w651.mirror.aliyuncs.com",
        "http://hub-mirror.c.163.com",
        "https://docker.mirrors.ustc.edu.cn",
        "http://f1361db2.m.daocloud.io",
        "https://registry.docker-cn.com"
    ]
}
EOF

3）下载镜像

#阿里云版本
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.2
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.2 k8s.gcr.io/pause:3.2
docker rmi registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.2
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/coredns:1.7.0
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/coredns:1.7.0 k8s.gcr.io/coredns:1.7.0
docker rmi registry.cn-hangzhou.aliyuncs.com/google_containers/coredns:1.7.0

#官方版本
docker pull coredns/coredns:1.8.4
docker pull k8s.gcr.io/pause:3.2

 

3、搭建etcd集群（master1节点执行）

1）配置工具目录及免密登录

mkdir -p /data/work
ssh-keygen -t rsa -b 2048
ssh-copy-id master-02
ssh-copy-id master-03
ssh-copy-id node-01
ssh-copy-id node-02

#配置etcd工作目录
mkdir -p /var/lib/etcd/default.etcd
mkdir -p /etc/etcd/ssl

2）创建etcd证书 

cfssl证书Github网址：https://github.com/cloudflare/cfssl

wget https://hub.fastgit.org/cloudflare/cfssl/releases/download/v1.6.0/cfssl-certinfo_1.6.0_linux_amd64
wget https://hub.fastgit.org/cloudflare/cfssl/releases/download/v1.6.0/cfssljson_1.6.0_linux_amd64
wget https://hub.fastgit.org/cloudflare/cfssl/releases/download/v1.6.0/cfssl_1.6.0_linux_amd64

chmod +x cfssl*
mv cfssl_1.6.0_linux_amd64 /usr/local/bin/cfssl
mv cfssljson_1.6.0_linux_amd64 /usr/local/bin/cfssljson
mv cfssl-certinfo_1.6.0_linux_amd64 /usr/local/bin/cfssl-certinfo

#创建ca请求证书 

cat > ca-csr.json <<EOF
{
  "CN": "kubernetes",
  "key": {
      "algo": "rsa",
      "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "Hubei",
      "L": "Wuhan",
      "O": "k8s",
      "OU": "system"
    }
  ],
  "ca": {
          "expiry": "87600h"
  }
}
EOF

cfssl gencert -initca ca-csr.json  | cfssljson -bare ca

#配置ca证书策略
cat > ca-config.json <<EOF
{
  "signing": {
      "default": {
          "expiry": "87600h"
        },
      "profiles": {
          "kubernetes": {
              "usages": [
                  "signing",
                  "key encipherment",
                  "server auth",
                  "client auth"
              ],
              "expiry": "87600h"
          }
      }
  }
}
EOF

#创建etcd请求csr文件 

cat > etcd-csr.json <<EOF
{
  "CN": "etcd",
  "hosts": [
    "127.0.0.1",
    "192.168.10.10",
    "192.168.10.11",
    "192.168.10.12"
  ],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [{
    "C": "CN",
    "ST": "Hubei",
    "L": "Wuhan",
    "O": "k8s",
    "OU": "system"
  }]
}
EOF

cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes etcd-csr.json | cfssljson  -bare etcd
ls etcd*.pem

3）部署etcd集群

etcd Github网址：https://github.com/etcd-io/etcd/releases/

wget https://hub.fastgit.org/etcd-io/etcd/releases/download/v3.4.13/etcd-v3.4.13-linux-amd64.tar.gz
tar -xf etcd-v3.4.13-linux-amd64.tar.gz
cp -p etcd-v3.4.13-linux-amd64/etcd* /usr/local/bin/
scp etcd-v3.4.13-linux-amd64/etcd* master2:/usr/local/bin/
scp etcd-v3.4.13-linux-amd64/etcd* master3:/usr/local/bin/

cat > etcd.conf <<EOF
#[Member]
ETCD_NAME="etcd1"
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="https://192.168.10.10:2380"
ETCD_LISTEN_CLIENT_URLS="https://192.168.10.10:2379,http://127.0.0.1:2379"

#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.10.10:2380"
ETCD_ADVERTISE_CLIENT_URLS="https://192.168.10.10:2379"
ETCD_INITIAL_CLUSTER="etcd1=https://192.168.10.10:2380,etcd2=https://192.168.10.11:2380,etcd3=https://192.168.10.12:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"
EOF

cat > etcd.service <<EOF
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target

[Service]
Type=notify
EnvironmentFile=-/etc/etcd/etcd.conf
WorkingDirectory=/var/lib/etcd/
ExecStart=/usr/local/bin/etcd \
  --cert-file=/etc/etcd/ssl/etcd.pem \
  --key-file=/etc/etcd/ssl/etcd-key.pem \
  --trusted-ca-file=/etc/etcd/ssl/ca.pem \
  --peer-cert-file=/etc/etcd/ssl/etcd.pem \
  --peer-key-file=/etc/etcd/ssl/etcd-key.pem \
  --peer-trusted-ca-file=/etc/etcd/ssl/ca.pem \
  --peer-client-cert-auth \
  --client-cert-auth
Restart=on-failure
RestartSec=5
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target
EOF

#同步相关文件到各个节点
cp ca*.pem /etc/etcd/ssl/
cp etcd*.pem /etc/etcd/ssl/
cp etcd.conf /etc/etcd/
cp etcd.service /usr/lib/systemd/system/
for i in master2 master3
do 
	ssh -o StrictHostKeyChecking=no root@$i "mkdir -p /var/lib/etcd/default.etcd && mkdir -p /etc/etcd/ssl"
	scp etcd.conf $i:/etc/etcd/
  scp etcd*.pem ca*.pem $i:/etc/etcd/ssl/
  scp etcd.service $i:/usr