【笔记1】Kubernetes 1.28.2安装部署

断尘丶轻舞

已于 2023-12-01 17:23:28 修改

阅读量1.3k

点赞数 22

分类专栏： kubernetes学习笔记文章标签：笔记 kubernetes 容器

于 2023-11-22 18:03:11 首次发布

本文链接：https://blog.csdn.net/a309446547/article/details/134555347

版权

kubernetes学习笔记专栏收录该内容

13 篇文章 0 订阅

订阅专栏

kubernetes概念介绍

k8s的概念基本围绕资源对象来说的，通常情况下，资源对象分为以下两类：

某种资源对象：例如节点、服务、存储卷等
与资源对象相关的事物或动作：例如标签、注释、命名空间等
可以采用yaml或json格式创建或修改一个资源对象，并使用kubectl工具进行资源对象的增、删、改、查等操作。

k8s中数量众多的资源对象分为四类，分别是集群类、应用类、存储类及安全类。

1、集群类

集群是指一个由master（控制节点）和node（工作节点）组成的k8s集群。

master是集群的控制节点，每个集群有一个或一组控制节点，负责整个集群的管理控制，运行了以下进程：

kube-apiserver：提供了资源操作的唯一入口，并提供认证、授权、访问控制、API注册和发现等机制
kube-controller-manager：负责维护集群的状态，比如故障检测、自动扩展、滚动更新等
kube-scheduler：负责资源的调度，按照预定的调度策略将Pod调度到相应的机器上
etcd：保存整个集群的状态信息（可独立部署）

其他节点为node，主要承担由master分配的工作负载，当node宕机时，其上的工作负载会被master转移到其他node上，主要运行了以下进程：

kubelet：负责工作负载的生命周期管理
kube-proxy：负责为Service提供cluster内部的服务发现和负载均衡
容器运行时：负责本机的容器创建和管理

集群的一个重要概念是namespace（名称空间），它在很多情况下用于多租户的逻辑隔离，典型的思路是在集群中创建多个namespace，并分给不同的租户，不同namespace之间的资源对象是独立的（虽然运行在同一个集群但从逻辑上相互隔离）。

2、应用类

应用类的资源对象主要是围绕pod和service这两个核心展开的，主要包括以下资源对象：

Pod：工作负载的最小管理单元
Service：服务发现以及内部访问入口
Label（标签）和Label Selector（标签选择器）：用于多个资源对象之间关联
Deployment：负责维护工作负载使其始终符合预期（用于无状态服务）
Ingress：提供了一个集群外部访问入口
Statefulset：负责维护工作负载使其始终符合预期（用于有状态服务）
Job和CronJob：用于运行单次批处理任务和定时批处理任务
ConfigMap和Secret：用于存储程序的配置文件和敏感信息
HorizontalPodAutoscaler：可以根据定义的指标进行自动的pod扩容及缩减
VerticalPodAutoscaler：可实现垂直自动扩缩容（目前为测试特性，不稳定）

3、存储类

存储类的资源包括如下几类：

Volume：是一个可以被pod中的多个容器访问的共享目录
PersistentVolume：是集群中已由管理员配置的一段网络存储
PersistentVolumeClaim：用户存储的请求
StorageClass：用于描述和定义某类存储系统的特征

4、安全类

Role和ClusterRole：定义一组权限规则，局限于命名空间内使用Role定义，作用于整个集群使用ClusterRole定义
RoleBinding和ClusterRoleBinding：将定义好的权限规则绑定到对应的用户实体上（相当于给用户授权）
NetworkPolicy：定义Pod间及Pod与外部服务通信的安全规则

Kubernetes 1.28.2安装部署

主机配置

主机名	IP地址	操作系统	内核版本
master	192.168.129.128	openEuler 22.03 (LTS-SP1)	5.10.0-136.52.0.131.oe2203sp1.x86_64
node1	192.168.129.129	openEuler 22.03 (LTS-SP1)	5.10.0-136.52.0.131.oe2203sp1.x86_64
node2	192.168.129.130	openEuler 22.03 (LTS-SP1)	5.10.0-136.52.0.131.oe2203sp1.x86_64

初始化操作

时间同步

ntpdate ntp.aliyun.com

关闭SeLinux、Firewalld

sed -i s/SELINUX=enforcing/SELINUX=disabled/g /etc/selinux/config && setenforce 0 
systemctl stop firewalld ; systemctl disable firewalld

关闭Swap分区

swapoff -a
mv /etc/fstab /etc/fstab_bak
cat /etc/fstab_bak |grep -v swap > /etc/fstab
echo "vm.swappiness = 0">> /etc/sysctl.conf

修改主机名

hostnamectl set-hostname master
hostnamectl set-hostname node1
hostnamectl set-hostname node2

修改内核参数

cat > /etc/sysctl.d/k8s.conf <<EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
EOF

加载内核参数

modprobe br_netfilter
sysctl -p /etc/sysctl.d/k8s.conf

加载IPVS模块

cat > /etc/sysconfig/modules/ipvs.modules <<EOF
#!/bin/bash
modprobe -- ip_vs
modprobe -- ip_vs_rr
modprobe -- ip_vs_wrr
modprobe -- ip_vs_sh
modprobe -- nf_conntrack_ipv4
EOF
chmod 755 /etc/sysconfig/modules/ipvs.modules && bash /etc/sysconfig/modules/ipvs.modules && lsmod | grep -e ip_vs -e nf_conntrack_ipv4

安装containerd

下载containerd

wget https://github.com/containerd/containerd/releases/download/v1.6.24/containerd-1.6.24-linux-amd64.tar.gz
tar zxvf containerd-1.6.24-linux-amd64.tar.gz && mv bin/* /usr/local/

添加配置文件

cat > /etc/containerd/config.toml <<EOF
disabled_plugins = []
imports = []
oom_score = 0
plugin_dir = ""
required_plugins = []
root = "/var/lib/containerd"
state = "/run/containerd"
temp = ""
version = 2

[cgroup]
  path = ""

[debug]
  address = ""
  format = ""
  gid = 0
  level = ""
  uid = 0

[grpc]
  address = "/run/containerd/containerd.sock"
  gid = 0
  max_recv_message_size = 16777216
  max_send_message_size = 16777216
  tcp_address = ""
  tcp_tls_ca = ""
  tcp_tls_cert = ""
  tcp_tls_key = ""
  uid = 0

[metrics]
  address = ""
  grpc_histogram = false

[plugins]

  [plugins."io.containerd.gc.v1.scheduler"]
    deletion_threshold = 0
    mutation_threshold = 100
    pause_threshold = 0.02
    schedule_delay = "0s"
    startup_delay = "100ms"

  [plugins."io.containerd.grpc.v1.cri"]
    device_ownership_from_security_context = false
    disable_apparmor = false
    disable_cgroup = false
    disable_hugetlb_controller = true
    disable_proc_mount = false
    disable_tcp_service = true
    enable_selinux = false
    enable_tls_streaming = false
    enable_unprivileged_icmp = false
    enable_unprivileged_ports = false
    ignore_image_defined_volumes = false
    max_concurrent_downloads = 3
    max_container_log_line_size = 16384
    netns_mounts_under_state_dir = false
    restrict_oom_score_adj = false
    sandbox_image = "registry.aliyuncs.com/google_containers/pause:3.6"
    selinux_category_range = 1024
    stats_collect_period = 10
    stream_idle_timeout = "4h0m0s"
    stream_server_address = "127.0.0.1"
    stream_server_port = "0"
    systemd_cgroup = false
    tolerate_missing_hugetlb_controller = true
    unset_seccomp_profile = ""

    [plugins."io.containerd.grpc.v1.cri".cni]
      bin_dir = "/opt/cni/bin"
      conf_dir = "/etc/cni/net.d"
      conf_template = ""
      ip_pref = ""
      max_conf_num = 1

    [plugins."io.containerd.grpc.v1.cri".containerd]
      default_runtime_name = "runc"
      disable_snapshot_annotations = true
      discard_unpacked_layers = false
      ignore_rdt_not_enabled_errors = false
      no_pivot = false
      snapshotter = "overlayfs"

      [plugins."io.containerd.grpc.v1.cri".containerd.default_runtime]
        base_runtime_spec = ""
        cni_conf_dir = ""
        cni_max_conf_num = 0
        container_annotations = []
        pod_annotations = []
        privileged_without_host_devices = false
        runtime_engine = ""
        runtime_path = ""
        runtime_root = ""
        runtime_type = ""

        [plugins."io.containerd.grpc.v1.cri".containerd.default_runtime.options]

      [plugins."io.containerd.grpc.v1.cri".containerd.runtimes]

        [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
          base_runtime_spec = ""
          cni_conf_dir = ""
          cni_max_conf_num = 0
          container_annotations = []
          pod_annotations = []
          privileged_without_host_devices = false
          runtime_engine = ""
          runtime_path = ""
          runtime_root = ""
          runtime_type = "io.containerd.runc.v2"

          [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
            BinaryName = ""
            CriuImagePath = ""
            CriuPath = ""
            CriuWorkPath = ""
            IoGid = 0
            IoUid = 0
            NoNewKeyring = false
            NoPivotRoot = false
            Root = ""
            ShimCgroup = ""
            SystemdCgroup = true

      [plugins."io.containerd.grpc.v1.cri".containerd.untrusted_workload_runtime]
        base_runtime_spec = ""
        cni_conf_dir = ""
        cni_max_conf_num = 0
        container_annotations = []
        pod_annotations = []
        privileged_without_host_devices = false
        runtime_engine = ""
        runtime_path = ""
        runtime_root = ""
        runtime_type = ""

        [plugins."io.containerd.grpc.v1.cri".containerd.untrusted_workload_runtime.options]

    [plugins."io.containerd.grpc.v1.cri".image_decryption]
      key_model = "node"

    [plugins."io.containerd.grpc.v1.cri".registry]
      config_path = ""

      [plugins."io.containerd.grpc.v1.cri".registry.auths]

      [plugins."io.containerd.grpc.v1.cri".registry.configs]

      [plugins."io.containerd.grpc.v1.cri".registry.headers]

      [plugins."io.containerd.grpc.v1.cri".registry.mirrors]
        [plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
          endpoint = ["http://mirrors.ustc.edu.cn"]
        [plugins."io.containerd.grpc.v1.cri".registry.mirrors."*"]
          endpoint = ["http://hub-mirror.c.163.com"]

    [plugins."io.containerd.grpc.v1.cri".x509_key_pair_streaming]
      tls_cert_file = ""
      tls_key_file = ""

  [plugins."io.containerd.internal.v1.opt"]
    path = "/opt/containerd"

  [plugins."io.containerd.internal.v1.restart"]
    interval = "10s"

  [plugins."io.containerd.internal.v1.tracing"]
    sampling_ratio = 1.0
    service_name = "containerd"

  [plugins."io.containerd.metadata.v1.bolt"]
    content_sharing_policy = "shared"

  [plugins."io.containerd.monitor.v1.cgroups"]
    no_prometheus = false

  [plugins."io.containerd.runtime.v1.linux"]
    no_shim = false
    runtime = "runc"
    runtime_root = ""
    shim = "containerd-shim"
    shim_debug = false

  [plugins."io.containerd.runtime.v2.task"]
    platforms = ["linux/amd64"]
    sched_core = false

  [plugins."io.containerd.service.v1.diff-service"]
    default = ["walking"]

  [plugins."io.containerd.service.v1.tasks-service"]
    rdt_config_file = ""

  [plugins."io.containerd.snapshotter.v1.aufs"]
    root_path = ""

  [plugins."io.containerd.snapshotter.v1.btrfs"]
    root_path = ""

  [plugins."io.containerd.snapshotter.v1.devmapper"]
    async_remove = false
    base_image_size = ""
    discard_blocks = false
    fs_options = ""
    fs_type = ""
    pool_name = ""
    root_path = ""

  [plugins."io.containerd.snapshotter.v1.native"]
    root_path = ""

  [plugins."io.containerd.snapshotter.v1.overlayfs"]
    mount_options = []
    root_path = ""
    sync_remove = false
    upperdir_label = false

  [plugins."io.containerd.snapshotter.v1.zfs"]
    root_path = ""

  [plugins."io.containerd.tracing.processor.v1.otlp"]
    endpoint = ""
    insecure = false
    protocol = ""

[proxy_plugins]

[stream_processors]

  [stream_processors."io.containerd.ocicrypt.decoder.v1.tar"]
    accepts = ["application/vnd.oci.image.layer.v1.tar+encrypted"]
    args = ["--decryption-keys-path", "/etc/containerd/ocicrypt/keys"]
    env = ["OCICRYPT_KEYPROVIDER_CONFIG=/etc/containerd/ocicrypt/ocicrypt_keyprovider.conf"]
    path = "ctd-decoder"
    returns = "application/vnd.oci.image.layer.v1.tar"

  [stream_processors."io.containerd.ocicrypt.decoder.v1.tar.gzip"]
    accepts = ["application/vnd.oci.image.layer.v1.tar+gzip+encrypted"]
    args = ["--decryption-keys-path", "/etc/containerd/ocicrypt/keys"]
    env = ["OCICRYPT_KEYPROVIDER_CONFIG=/etc/containerd/ocicrypt/ocicrypt_keyprovider.conf"]
    path = "ctd-decoder"
    returns = "application/vnd.oci.image.layer.v1.tar+gzip"

[timeouts]
  "io.containerd.timeout.bolt.open" = "0s"
  "io.containerd.timeout.shim.cleanup" = "5s"
  "io.containerd.timeout.shim.load" = "5s"
  "io.containerd.timeout.shim.shutdown" = "3s"
  "io.containerd.timeout.task.state" = "2s"

[ttrpc]
  address = ""
  gid = 0
  uid = 0
EOF

配置service

cat > /etc/systemd/system/containerd.service <<EOF
# Copyright The containerd Authors.
# 
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

[Unit]
Description=containerd container runtime
Documentation=https://containerd.io
After=network.target local-fs.target

[Service]
ExecStartPre=-/sbin/modprobe overlay
ExecStart=/usr/local/bin/containerd

Type=notify
Delegate=yes
KillMode=process
Restart=always
RestartSec=5
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNPROC=infinity
LimitCORE=infinity
LimitNOFILE=infinity
# Comment TasksMax if your systemd version does not supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
OOMScoreAdjust=-999

[Install]
WantedBy=multi-user.target
EOF

启动containerd

systemctl daemon-reload
systemctl enable --now containerd

安装runc

wget https://github.com/opencontainers/runc/releases/download/v1.1.10/runc.amd64
install -m 755 runc.amd64 /usr/local/sbin/runc

安装cni插件

wget https://github.com/containernetworking/plugins/releases/download/v1.3.0/cni-plugins-linux-amd64-v1.3.0.tgz
mkdir -p /opt/cni/bin
tar Cxzvf /opt/cni/bin cni-plugins-linux-amd64-v1.3.0.tgz

安装kubeadm

配置阿里云仓库

cat > /etc/yum.repos.d/kubernetes.repo <<EOF
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

安装对应版本的kubeadm、kubectl、kubelet

yum install -y kubeadm-1.28.2 kubectl-1.28.2 kubelet-1.28.2

编写kubeadm安装配置文件

cat > kubeadm-config.yaml <<EOF
apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
localAPIEndpoint:
  advertiseAddress: 192.168.129.128
  bindPort: 6443
nodeRegistration:
  criSocket: unix:///run/containerd/containerd.sock
  imagePullPolicy: IfNotPresent
  taints: null
---
apiServer:
  timeoutForControlPlane: 4m0s
  extraVolumes:
 - name: localtime
    hostPath: /etc/localtime
    mountPath: /etc/localtime
    readOnly: true
    pathType: File
apiVersion: kubeadm.k8s.io/v1beta3
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controllerManager: 
  extraVolumes:
 - hostPath: /etc/localtime
    mountPath: /etc/localtime
    name: localtime
    readOnly: true
    pathType: File
dns: {}
etcd:
  local:
    dataDir: /var/lib/etcd
imageRepository: registry.aliyuncs.com/google_containers
kind: ClusterConfiguration
kubernetesVersion: 1.28.2
networking:
  dnsDomain: cluster.local
  serviceSubnet: "10.65.0.0/16"
  podSubnet: "10.64.0.0/16"
scheduler: 
  extraVolumes:
 - hostPath: /etc/localtime
    mountPath: /etc/localtime
    name: localtime
    readOnly: true
    pathType: File
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
mode: "ipvs"
EOF

执行安装命令

kubeadm init --config kubeadm-config.yaml

创建kubectl配置文件

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

在其他节点执行join

kubeadm join 192.168.129.128:6443 --token abcdef.0123456789abcdef --discovery-token-ca-cert-hash sha256:ebe505f8d355c6d13afd39ae8057a97e7e0f1b86f130cfe6a2288d676123b283

部署网络插件
检查集群状态

[root@master ~]# kubectl get node -o wide
NAME     STATUS   ROLES           AGE   VERSION   INTERNAL-IP       EXTERNAL-IP   OS-IMAGE                    KERNEL-VERSION                         CONTAINER-RUNTIME
master   Ready    control-plane   22d   v1.28.2   192.168.129.128   <none>        openEuler 22.03 (LTS-SP1)   5.10.0-136.52.0.131.oe2203sp1.x86_64   containerd://1.6.24
node1    Ready    <none>          22d   v1.28.2   192.168.129.129   <none>        openEuler 22.03 (LTS-SP1)   5.10.0-136.52.0.131.oe2203sp1.x86_64   containerd://1.6.24
node2    Ready    <none>          22d   v1.28.2   192.168.129.130   <none>        openEuler 22.03 (LTS-SP1)   5.10.0-136.52.0.131.oe2203sp1.x86_64   containerd://1.6.24