K8s 集群资源监控、高可用搭建、部署java项目 (下)

最新推荐文章于 2024-04-30 21:59:16 发布
最新推荐文章于 2024-04-30 21:59:16 发布
阅读量460 收藏 1
点赞数 1
分类专栏： K8S kubernetes 文章标签： kubernetes docker K8S
原文链接：https://blog.csdn.net/weixin_44350137/article/details/126930494
版权
K8S 同时被 2 个专栏收录
104 篇文章 130 订阅
订阅专栏
kubernetes
32 篇文章 6 订阅
订阅专栏

目录

集群资源监控

监控指标和搭建方案

  • 集群监控
    • 节点资源利用率
    • 节点数
    • 运行的pod
  • Pod监控
    • 容器指标
    • 应用程序
  • 搭建方案（prometheus+ Grafana）
    • prometheus
      • 开源的
      • 监控、报警、数据库
      • 以http协议周期性抓取被监控组件的状态
      • 不需要复杂的集成过程，使用http接口接入
    • Grafana
      • 开源的数据分析和可视化工具
      • 支持多种数据源

搭建监控平台（k8s部署）

部署守护进程

下面的命名空间都是用 kube-system 的，也可以自己指定
这些yaml可以在官网上进行查看参数进行对应修改

vim node-exporter.yaml

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: node-exporter
  namespace: kube-system
  labels:
    k8s-app: node-exporter
spec:
  selector:
    matchLabels:
      k8s-app: node-exporter
  template:
    metadata:
      labels:
        k8s-app: node-exporter
    spec:
      containers:
      - image: prom/node-exporter
        name: node-exporter
        ports:
        - containerPort: 9100
          protocol: TCP
          name: http
---
apiVersion: v1
kind: Service
metadata:
  labels:
    k8s-app: node-exporter
  name: node-exporter
  namespace: kube-system
spec:
  ports:
  - name: http
    port: 9100
    nodePort: 31672
    protocol: TCP
  type: NodePort
  selector:
    k8s-app: node-exporter

# 执行
kubectl apply -f node-exporter.yaml

部署prometheus

  1. 创建目录
mkdir prometheus
cd prometheus
  1. 创建rbac
vim rbac-setup.yaml

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: prometheus
rules:
- apiGroups: [""]
  resources:
  - nodes
  - nodes/proxy
  - services
  - endpoints
  - pods
  verbs: ["get", "list", "watch"]
- apiGroups:
  - extensions
  resources:
  - ingresses
  verbs: ["get", "list", "watch"]
- nonResourceURLs: ["/metrics"]
  verbs: ["get"]
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: prometheus
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: prometheus
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: prometheus
subjects:
- kind: ServiceAccount
  name: prometheus
  namespace: kube-system

# 创建rbac
kubectl create -f rbac-setup.yaml
  1. 创建configmap
vim configmap.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: prometheus-config
  namespace: kube-system
data:
apiVersion: v1
kind: ConfigMap
metadata:
  name: prometheus-config
  namespace: kube-system
data:
  prometheus.yml: |
    global:
      scrape_interval:     15s
      evaluation_interval: 15s
    scrape_configs:

    - job_name: 'kubernetes-apiservers'
      kubernetes_sd_configs:
      - role: endpoints
      scheme: https
      tls_config:
        ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
      bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
      relabel_configs:
      - source_labels: [__meta_kubernetes_namespace, __meta_kubernetes_service_name, __meta_kubernetes_endpoint_port_name]
        action: keep
        regex: default;kubernetes;https

    - job_name: 'kubernetes-nodes'
      kubernetes_sd_configs:
      - role: node
      scheme: https
      tls_config:
        ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
      bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
      relabel_configs:
      - action: labelmap
        regex: __meta_kubernetes_node_label_(.+)
      - target_label: __address__
        replacement: kubernetes.default.svc:443
      - source_labels: [__meta_kubernetes_node_name]
        regex: (.+)
        target_label: __metrics_path__
        replacement: /api/v1/nodes/${1}/proxy/metrics

    - job_name: 'kubernetes-cadvisor'
      kubernetes_sd_configs:
      - role: node
      scheme: https
      tls_config:
        ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
      bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
      relabel_configs:
      - action: labelmap
        regex: __meta_kubernetes_node_label_(.+)
      - target_label: __address__
        replacement: kubernetes.default.svc:443
      - source_labels: [__meta_kubernetes_node_name]
        regex: (.+)
        target_label: __metrics_path__
        replacement: /api/v1/nodes/${1}/proxy/metrics/cadvisor

    - job_name: 'kubernetes-service-endpoints'
      kubernetes_sd_configs:
      - role: endpoints
apiVersion: v1
kind: ConfigMap
metadata:
  name: prometheus-config
  namespace: kube-system
data:
  prometheus.yml: |
    global:
      scrape_interval:     15s
      evaluation_interval: 15s
    scrape_configs:

    - job_name: 'kubernetes-apiservers'
      kubernetes_sd_configs:
      - role: endpoints
      scheme: https
      tls_config:
        ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
      bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
      relabel_configs:
      - source_labels: [__meta_kubernetes_namespace, __meta_kubernetes_service_name, __meta_kubernetes_endpoint_port_name]
        action: keep
        regex: default;kubernetes;https

    - job_name: 'kubernetes-nodes'
      kubernetes_sd_configs:
      - role: node
      scheme: https
      tls_config:
        ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
      bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
apiVersion: v1
kind: ConfigMap
metadata:
  name: prometheus-config
  namespace: kube-system
data:
  prometheus.yml: |
    global:
      scrape_interval:     15s
      evaluation_interval: 15s
    scrape_configs:

    - job_name: 'kubernetes-apiservers'
      kubernetes_sd_configs:
      - role: endpoints
      scheme: https
      tls_config:
        ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
      bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
      relabel_configs:
      - source_labels: [__meta_kubernetes_namespace, __meta_kubernetes_service_name, __meta_kubernetes_endpoint_port_name]
        action: keep
        regex: default;kubernetes;https

    - job_name: 'kubernetes-nodes'
      kubernetes_sd_configs:
      - role: node
      scheme: https
      tls_config:
        ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
      bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
      relabel_configs:
      - action: labelmap
        regex: __meta_kubernetes_node_label_(.+)
      - target_label: __address__
        replacement: kubernetes.default.svc:443
      - source_labels: [__meta_kubernetes_node_name]
        regex: (.+)
        target_label: __metrics_path__
        replacement: /api/v1/nodes/${1}/proxy/metrics

    - job_name: 'kubernetes-cadvisor'
      kubernetes_sd_configs:
      - role: node
      scheme: https
      tls_config:
        ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
      bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
      relabel_configs:
      - action: labelmap
        regex: __meta_kubernetes_node_label_(.+)
      - target_label: __address__
        replacement: kubernetes.default.svc:443
      - source_labels: [__meta_kubernetes_node_name]
        regex: (.+)
        target_label: __metrics_path__
        replacement: /api/v1/nodes/${1}/proxy/metrics/cadvisor

    - job_name: 'kubernetes-service-endpoints'
      kubernetes_sd_configs:
      - role: endpoints
      relabel_configs:
      - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scrape]
        action: keep
        regex: true
      - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scheme]
        action: replace
        target_label: __scheme__
        regex: (https?)
      - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_path]
        action: replace
        target_label: __metrics_path__
        regex: (.+)
      - source_labels: [__address__, __meta_kubernetes_service_annotation_prometheus_io_port]
        action: replace
        target_label: __address__
        regex: ([^:]+)(?::\d+)?;(\d+)
        replacement: $1:$2
      - action: labelmap
        regex: __meta_kubernetes_service_label_(.+)
      - source_labels: [__meta_kubernetes_namespace]
        action: replace
        target_label: kubernetes_namespace
      - source_labels: [__meta_kubernetes_service_name]
        action: replace
        target_label: kubernetes_name

    - job_name: 'kubernetes-services'
      kubernetes_sd_configs:
      - role: service
      metrics_path: /probe
      params:
        module: [http_2xx]
      relabel_configs:
      - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_probe]
        action: keep
        regex: true
      - source_labels: [__address__]
        target_label: __param_target
      - target_label: __address__
        replacement: blackbox-exporter.example.com:9115
      - source_labels: [__param_target]
        target_label: instance
      - action: labelmap
        regex: __meta_kubernetes_service_label_(.+)
      - source_labels: [__meta_kubernetes_namespace]
        target_label: kubernetes_namespace
      - source_labels: [__meta_kubernetes_service_name]
        target_label: kubernetes_name

    - job_name: 'kubernetes-ingresses'
      kubernetes_sd_configs:
      - role: ingress
      relabel_configs:
      - source_labels: [__meta_kubernetes_ingress_annotation_prometheus_io_probe]
        action: keep
        regex: true
      - source_labels: [__meta_kubernetes_ingress_scheme,__address__,__meta_kubernetes_ingress_path]
        regex: (.+);(.+);(.+)
        replacement: ${1}://${2}${3}
        target_label: __param_target
      - target_label: __address__
        replacement: blackbox-exporter.example.com:9115
      - source_labels: [__param_target]
        target_label: instance
      - action: labelmap
        regex: __meta_kubernetes_ingress_label_(.+)
      - source_labels: [__meta_kubernetes_namespace]
        target_label: kubernetes_namespace
      - source_labels: [__meta_kubernetes_ingress_name]
        target_label: kubernetes_name

    - job_name: 'kubernetes-pods'
      kubernetes_sd_configs:
      - role: pod
      relabel_configs:
      - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
        action: keep
        regex: true
      - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]
        action: replace
        target_label: __metrics_path__
        regex: (.+)
      - source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]
        action: replace
        regex: ([^:]+)(?::\d+)?;(\d+)
        replacement: $1:$2
        target_label: __address__
      - action: labelmap
        regex: __meta_kubernetes_pod_label_(.+)
      - source_labels: [__meta_kubernetes_namespace]
        action: replace
        target_label: kubernetes_namespace
      - source_labels: [__meta_kubernetes_pod_name]
        action: replace
        target_label: kubernetes_pod_name

# 执行
kubectl apply -f configmap.yaml
  1. 创建deployment文件
vim prometheus.deploy.yml

---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    name: prometheus-deployment
  name: prometheus
  namespace: kube-system
spec:
  replicas: 1
  selector:
    matchLabels:
      app: prometheus
  template:
    metadata:
      labels:
        app: prometheus
    spec:
      containers:
      - image: prom/prometheus:v2.0.0
        name: prometheus
        command:
        - "/bin/prometheus"
        args:
        - "--config.file=/etc/prometheus/prometheus.yml"
        - "--storage.tsdb.path=/prometheus"
        - "--storage.tsdb.retention=24h"
        ports:
        - containerPort: 9090
          protocol: TCP
        volumeMounts:
        - mountPath: "/prometheus"
          name: data
        - mountPath: "/etc/prometheus"
          name: config-volume
        resources:
          requests:
            cpu: 100m
            memory: 100Mi
          limits:
            cpu: 500m
            memory: 2500Mi
      serviceAccountName: prometheus
      volumes:
      - name: data
        emptyDir: {}
      - name: config-volume
        configMap:
          name: prometheus-config

# 执行
kubectl apply -f prometheus.deploy.yml
  1. 创建service文件
vim prometheus.svc.yml

kind: Service
apiVersion: v1
metadata:
  labels:
    app: prometheus
  name: prometheus
  namespace: kube-system
spec:
  type: NodePort
  ports:
  - port: 9090
    targetPort: 9090
    nodePort: 30003
  selector:
    app: prometheus

# 执行
kubectl apply -f prometheus.svc.yml
  1. 查看
# 查看
kubectl get pods -n kube-system

部署Grafana

  1. 创建deployment.yaml文件
vim grafana-deploy.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: grafana-core
  namespace: kube-system
  labels:
    app: grafana
    component: core
spec:
  replicas: 1
  selector:
    matchLabels:
        app: grafana
        component: core
  template:
    metadata:
      labels:
        app: grafana
        component: core
    spec:
      containers:
      - image: grafana/grafana:4.2.0
        name: grafana-core
        imagePullPolicy: IfNotPresent
        # env:
        resources:
          # keep request = limit to keep this container in guaranteed class
          limits:
            cpu: 100m
            memory: 100Mi
          requests:
            cpu: 100m
            memory: 100Mi
        env:
          # The following env variables set up basic auth twith the default admin user and admin password.
          - name: GF_AUTH_BASIC_ENABLED
            value: "true"
          - name: GF_AUTH_ANONYMOUS_ENABLED
            value: "false"
          # - name: GF_AUTH_ANONYMOUS_ORG_ROLE
          #   value: Admin
          # does not really work, because of template variables in exported dashboards:
          # - name: GF_DASHBOARDS_JSON_ENABLED
          #   value: "true"
        readinessProbe:
          httpGet:
            path: /login
            port: 3000
          # initialDelaySeconds: 30
          # timeoutSeconds: 1
        volumeMounts:
        - name: grafana-persistent-storage
          mountPath: /var
      volumes:
      - name: grafana-persistent-storage
        emptyDir: {}

# 执行
kubectl apply -f grafana-deploy.yaml
  1. 创建service文件
vim grafana-svc.yaml

apiVersion: v1
kind: Service
metadata:
  name: grafana
  namespace: kube-system
  labels:
    app: grafana
    component: core
spec:
  type: NodePort
  ports:
    - port: 3000
  selector:
    app: grafana
    component: core

# 执行
kubectl apply -f grafana-svc.yaml
  1. 创建对外暴露端口文件
vim grafana-ing.yaml

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
   name: grafana
   namespace: kube-system
spec:
   rules:
   - host: k8s.grafana
     http:
       paths:
       - path: /
         backend:
          serviceName: grafana
          servicePort: 3000

# 执行
kubectl apply -f grafana-ing.yaml
  1. 查看
# 查看
kubectl get pod -n kube-system
kubectl get svc -n kube-system

# 登录
http://192.168.10.102:30426/

# 账号密码一样
admin

在这里插入图片描述

  1. 配置数据源

在这里插入图片描述
这里的url地址是对应CLUSTER-IP的地址
在这里插入图片描述
6. 设置显示数据模板
在这里插入图片描述
在这里插入图片描述

在这里插入图片描述
在这里插入图片描述

高可用k8s集群

系统初始化

#禁用防火墙
systemctl stop firewalld 
# 开机不启动防火墙
systemctl disable firewalld 

# 关闭selinux
sed -i 's/enforcing/disabled/' /etc/selinux/config  # 永久
setenforce 0  # 临时

# 关闭swap
swapoff -a  # 临时
sed -ri 's/.*swap.*/#&/' /etc/fstab    # 永久

# 根据规划设置主机名
hostnamectl set-hostname <hostname>

#设置hosts 注意：这里的master01.k8s.io是访问名
cat >> /etc/hosts << EOF
192.168.10.101 master01.k8s.io master1
192.168.10.111 master02.k8s.io master2
192.168.10.102 node01.k8s.io slave1
192.168.10.103 node02.k8s.io slave2
192.168.10.104 master.k8s.io k8s-vip
EOF

# 将桥接的IPv4流量传递到iptables的链
cat > /etc/sysctl.d/k8s.conf << EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sysctl --system  # 生效

# 时间同步
yum install ntpdate -y
ntpdate time.windows.com

master节点部署keepalived

  1. 安装相关包和keepalived
yum install -y conntrack-tools libseccomp libtool-ltdl
yum install -y keepalived
  1. 配置master节点
    在master1 上
# 注意ip是vip的ip
cat > /etc/keepalived/keepalived.conf <<EOF 
! Configuration File for keepalived

global_defs {
   router_id k8s
}

vrrp_script check_haproxy {
    script "killall -0 haproxy"
    interval 3
    weight -2
    fall 10
    rise 2
}

vrrp_instance VI_1 {
    state MASTER 
    interface ens33 
    virtual_router_id 51
    priority 250
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass ceb1b3ec013d66163d6ab
    }
    virtual_ipaddress {
        192.168.10.104
    }
    track_script {
        check_haproxy
    }

}
EOF

在master2 上

# 这里只需修改状态和优先级，也可以根据自己需求不修改
cat > /etc/keepalived/keepalived.conf <<EOF 
! Configuration File for keepalived

global_defs {
   router_id k8s
}

vrrp_script check_haproxy {
    script "killall -0 haproxy"
    interval 3
    weight -2
    fall 10
    rise 2
}

vrrp_instance VI_1 {
    state BACKUP 
    interface ens33 
    virtual_router_id 51
    priority 200
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass ceb1b3ec013d66163d6ab
    }
    virtual_ipaddress {
        192.168.10.104
    }
    track_script {
        check_haproxy
    }

}
EOF
  1. 设置开机启动并启动
# 启动keepalived
systemctl start keepalived.service
设置开机启动
systemctl enable keepalived.service
# 查看启动状态
systemctl status keepalived.service

查看master1 上网卡信息是否有2个ip

ip a s ens33

master节点上部署haproxy

  1. 下载haproxy
yum install -y haproxy
  1. 添加配置文件
# 需修改ip
# 指定了haproxy运行的端口为16443等，因此16443端口为集群的入口
cat > /etc/haproxy/haproxy.cfg << EOF
#---------------------------------------------------------------------
# Global settings
#---------------------------------------------------------------------
global
    # to have these messages end up in /var/log/haproxy.log you will
    # need to:
    # 1) configure syslog to accept network log events.  This is done
    #    by adding the '-r' option to the SYSLOGD_OPTIONS in
    #    /etc/sysconfig/syslog
    # 2) configure local2 events to go to the /var/log/haproxy.log
    #   file. A line like the following can be added to
    #   /etc/sysconfig/syslog
    #
    #    local2.*                       /var/log/haproxy.log
    #
    log         127.0.0.1 local2
    
    chroot      /var/lib/haproxy
    pidfile     /var/run/haproxy.pid
    maxconn     4000
    user        haproxy
    group       haproxy
    daemon 
       
    # turn on stats unix socket
    stats socket /var/lib/haproxy/stats
#---------------------------------------------------------------------
# common defaults that all the 'listen' and 'backend' sections will
# use if not designated in their block
#---------------------------------------------------------------------  
defaults
    mode                    http
    log                     global
    option                  httplog
    option                  dontlognull
    option http-server-close
    option forwardfor       except 127.0.0.0/8
    option                  redispatch
    retries                 3
    timeout http-request    10s
    timeout queue           1m
    timeout connect         10s
    timeout client          1m
    timeout server          1m
    timeout http-keep-alive 10s
    timeout check           10s
    maxconn                 3000
#---------------------------------------------------------------------
# kubernetes apiserver frontend which proxys to the backends
#--------------------------------------------------------------------- 
frontend kubernetes-apiserver
    mode                 tcp
    bind                 *:16443
    option               tcplog
    default_backend      kubernetes-apiserver    
#---------------------------------------------------------------------
# round robin balancing between the various backends
#---------------------------------------------------------------------
backend kubernetes-apiserver
    mode        tcp
    balance     roundrobin
    server      master01.k8s.io   192.168.10.101:6443 check
    server      master02.k8s.io   192.168.10.111:6443 check
#---------------------------------------------------------------------
# collection haproxy statistics message
#---------------------------------------------------------------------
listen stats
    bind                 *:1080
    stats auth           admin:awesomePassword
    stats refresh        5s
    stats realm          HAProxy\ Statistics
    stats uri            /admin?stats
EOF
  1. 启动
# 设置开机启动
systemctl enable haproxy
# 开启haproxy
systemctl start haproxy
# 查看启动状态
systemctl status haproxy
  1. 查看端口
# 是否监听上面指定的16443端口
netstat -lntup|grep haproxy

所有节点部署docker、kubeadm、kubelect、kubectl

  1. 安装docker
# 下载docker
wget https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo -O /etc/yum.repos.d/docker-ce.repo
# 安装docker
yum -y install docker-ce-18.06.1.ce-3.el7
# 设置开机启动并启动docker
systemctl enable docker && systemctl start docker
# 查看版本
docker --version
  1. 换docker仓库
cat > /etc/docker/daemon.json << EOF
{
  "registry-mirrors": ["https://b9pmyelo.mirror.aliyuncs.com"]
}
EOF
  1. 添加阿里云yum源
cat > /etc/yum.repos.d/kubernetes.repo << EOF
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF


# 重启docker
systemctl restart docker
  1. 安装kkubeadm、kubelet、kubectl
yum install -y kubelet-1.16.3 kubeadm-1.16.3 kubectl-1.16.3
systemctl enable kubelet

部署master

这里是在具有vip的master上操作

  1. 创建工作路径
mkdir -p /opt/soft/kubernetes/manifests/
cd /opt/soft/kubernetes/manifests/
  1. 创建kubeadm配置文件
vi kubeadm-config.yaml

apiServer:
  certSANs:
    - master1
    - master2
    - master.k8s.io
    - 192.168.10.104
    - 192.168.10.101
    - 192.168.10.111
    - 127.0.0.1
  extraArgs:
    authorization-mode: Node,RBAC
  timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta1
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controlPlaneEndpoint: "master.k8s.io:16443"
controllerManager: {}
dns: 
  type: CoreDNS
etcd:
  local:    
    dataDir: /var/lib/etcd
imageRepository: registry.aliyuncs.com/google_containers
kind: ClusterConfiguration
kubernetesVersion: v1.16.3
networking: 
  dnsDomain: cluster.local  
  podSubnet: 10.244.0.0/16
  serviceSubnet: 10.1.0.0/16
scheduler: {}
  1. 初始化
kubeadm init --config kubeadm-config.yaml

如果报一下错误，执行下面命令
在这里插入图片描述

sysctl -w net.ipv4.ip_forward=1

在这里插入图片描述
按照提示执行

 mkdir -p $HOME/.kube
 sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
 sudo chown $(id -u):$(id -g) $HOME/.kube/config

下面这个是其他节点加入时需要，提前保存

kubeadm join master.k8s.io:16443 --token neh38n.6fumb7axtu19l99e \
    --discovery-token-ca-cert-hash sha256:911fef375a62712a82a67199c528f7c9ce5e7237855be0377f6f6ccdd0820596 \
    --control-plane


# 表示把master2节点加入控制节点
  --control-plane
  1. 查看集群状态
kubectl get cs
kubectl get nodes
kubectl get pods -n kube-system

安装集群网络

  1. 创建工作目录
mkdir -p /opt/soft/flannel
cd /opt/soft/flannel
  1. 下载并安装
# 下载
wget -c https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
# 注意： 需修改yaml文件，把命名空间名称改成kube-system
# 安装
kubectl apply -f kube-flannel.yml

如果按照上面的flannel执行不起来把yaml文件换成下面的

---
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: psp.flannel.unprivileged
  annotations:
    seccomp.security.alpha.kubernetes.io/allowedProfileNames: docker/default
    seccomp.security.alpha.kubernetes.io/defaultProfileName: docker/default
    apparmor.security.beta.kubernetes.io/allowedProfileNames: runtime/default
    apparmor.security.beta.kubernetes.io/defaultProfileName: runtime/default
spec:
  privileged: false
  volumes:
    - configMap
    - secret
    - emptyDir
    - hostPath
  allowedHostPaths:
    - pathPrefix: "/etc/cni/net.d"
    - pathPrefix: "/etc/kube-flannel"
    - pathPrefix: "/run/flannel"
  readOnlyRootFilesystem: false
  # Users and groups
  runAsUser:
    rule: RunAsAny
  supplementalGroups:
    rule: RunAsAny
  fsGroup:
    rule: RunAsAny
  # Privilege Escalation
  allowPrivilegeEscalation: false
  defaultAllowPrivilegeEscalation: false
  # Capabilities
  allowedCapabilities: ['NET_ADMIN']
  defaultAddCapabilities: []
  requiredDropCapabilities: []
  # Host namespaces
  hostPID: false
  hostIPC: false
  hostNetwork: true
  hostPorts:
  - min: 0
    max: 65535
  # SELinux
  seLinux:
    # SELinux is unsed in CaaSP
    rule: 'RunAsAny'
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: flannel
rules:
  - apiGroups: ['extensions']
    resources: ['podsecuritypolicies']
    verbs: ['use']
    resourceNames: ['psp.flannel.unprivileged']
  - apiGroups:
      - ""
    resources:
      - pods
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - nodes
    verbs:
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - nodes/status
    verbs:
      - patch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: flannel
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: flannel
subjects:
- kind: ServiceAccount
  name: flannel
  namespace: kube-system
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: flannel
  namespace: kube-system
---
kind: ConfigMap
apiVersion: v1
metadata:
  name: kube-flannel-cfg
  namespace: kube-system
  labels:
    tier: node
    app: flannel
data:
  cni-conf.json: |
    {
      "cniVersion": "0.2.0",
      "name": "cbr0",
      "plugins": [
        {
          "type": "flannel",
          "delegate": {
            "hairpinMode": true,
            "isDefaultGateway": true
          }
        },
        {
          "type": "portmap",
          "capabilities": {
            "portMappings": true
          }
        }
      ]
    }
  net-conf.json: |
    {
      "Network": "10.244.0.0/16",
      "Backend": {
        "Type": "vxlan"
      }
    }
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: kube-flannel-ds-amd64
  namespace: kube-system
  labels:
    tier: node
    app: flannel
spec:
  selector:
    matchLabels:
      app: flannel
  template:
    metadata:
      labels:
        tier: node
        app: flannel
    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
              - matchExpressions:
                  - key: beta.kubernetes.io/os
                    operator: In
                    values:
                      - linux
                  - key: beta.kubernetes.io/arch
                    operator: In
                    values:
                      - amd64
      hostNetwork: true
      tolerations:
      - operator: Exists
        effect: NoSchedule
      serviceAccountName: flannel
      initContainers:
      - name: install-cni
        image: lizhenliang/flannel:v0.11.0-amd64 
        command:
        - cp
        args:
        - -f
        - /etc/kube-flannel/cni-conf.json
        - /etc/cni/net.d/10-flannel.conflist
        volumeMounts:
        - name: cni
          mountPath: /etc/cni/net.d
        - name: flannel-cfg
          mountPath: /etc/kube-flannel/
      containers:
      - name: kube-flannel
        image: lizhenliang/flannel:v0.11.0-amd64 
        command:
        - /opt/bin/flanneld
        args:
        - --ip-masq
        - --kube-subnet-mgr
        resources:
          requests:
            cpu: "100m"
            memory: "50Mi"
          limits:
            cpu: "100m"
            memory: "50Mi"
        securityContext:
          privileged: false
          capabilities:
             add: ["NET_ADMIN"]
        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        volumeMounts:
        - name: run
          mountPath: /run/flannel
        - name: flannel-cfg
          mountPath: /etc/kube-flannel/
      volumes:
        - name: run
          hostPath:
            path: /run/flannel
        - name: cni
          hostPath:
            path: /etc/cni/net.d
        - name: flannel-cfg
          configMap:
            name: kube-flannel-cfg
  1. 检查
kubectl get pods -n kube-system

master2加入集群

  1. 在master2上创建工作路径
mkdir -p /etc/kubernetes/pki/etcd
  1. 在master1上，复制master1 文件到master2
scp /etc/kubernetes/admin.conf master2:/etc/kubernetes
scp /etc/kubernetes/pki/{ca.*,sa.*,front-proxy-ca.*} master2:/etc/kubernetes/pki
scp /etc/kubernetes/pki/etcd/ca.* master2:/etc/kubernetes/pki/etcd
  1. 加入集群
# 在master2上执行上面所保留的
kubeadm join master.k8s.io:16443 --token neh38n.6fumb7axtu19l99e \
    --discovery-token-ca-cert-hash sha256:911fef375a62712a82a67199c528f7c9ce5e7237855be0377f6f6ccdd0820596 \
    --control-plane

在这里插入图片描述

  1. 执行操作
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

部署node节点

  1. 执行加入节点命令
kubeadm join master.k8s.io:16443 --token neh38n.6fumb7axtu19l99e \
    --discovery-token-ca-cert-hash sha256:911fef375a62712a82a67199c528f7c9ce5e7237855be0377f6f6ccdd0820596
  1. 查看节点
kubectl get nodes
kubectl get pods --all-namespaces

重新部署集群网络

如果都是Ready则不需要重新部署
在这里插入图片描述

  1. 删除
kubectl delete -f kube-flannel.yml
  1. 重新添加
kubectl apply -f kube-flannel.yml

测试集群

  1. 创建一个nginx Pod
kubectl create deploy web --image=nginx
  1. 对外暴露端口
kubectl expose deploy web --port=80 --target-port=80 --type=NodePort
  1. 访问集群

在这里插入图片描述

部署java项目

容器交付

  • 开发代码阶段
    • 编码
    • 测试容器化
    • 编写DockerFile
  • 集成阶段
    • 代码编译
    • 制作镜像
    • 上传镜像仓库
  • 部署阶段
    • 环境准备
    • 部署
    • 对外暴露
  • 运维
    • 监控
    • 故障排查
    • 升级优化

制作镜像

  1. 编写好java代码
  2. 打成jar包
  3. 编写DockerFile
FROM openjdk:8-jdk-alpine
VOLUME /tmp
# jar包路径
ADD ./target/demojenkins.jar demojenkins.jar
ENTRYPOINT ["java","-jar","/demojenkins.jar", "&"]
  1. 使用docke制作镜像
docker build -t java-dome-01:latest .

在这里插入图片描述

把镜像上传至镜像服务器

在这里插入图片描述
在这里插入图片描述
在这里插入图片描述

部署以及对外暴露

  1. 创建配置文件
# 通过镜像仓库创建yaml文件
kubectl create deployment javademo1 --image=registry.cn-hangzhou.aliyuncs.com/dockerdome/k8s-java-demo:1.0.0 --dry-run -o yaml > javademo1.yaml
  1. 创建pod
kubectl apply -f javademo1.yaml
  1. 对外暴露
# 进行扩容
kubectl scale deployment javademo1 --replicas=3
# 对外暴露
kubectl expose deployment javademo1 --port=8111 --target-port=8111 --type=NodePort
  1. 访问测试
    在这里插入图片描述

转载至https://blog.csdn.net/weixin_44350137/article/details/126930494

victoruu
关注
  • 1
    点赞
  • 踩
  • 1
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
专栏目录
k8s中job与cronjob使用详解
congge
03-02 7100
k8s job详细使用
使用kubeadm搭建高可用的K8s集群
11-03
使用kubeadm搭建高可用的K8s集群
K8S Pod该如何监控
B.I.T
06-17 8478
背景 由于业务容器化的推进，对K8S上运行的业务，也必须做到向之前物理机/虚拟机上一样，要有完备的监控保障。但是，毕竟K8S不是物理机/虚拟机，由于技术实现方式不同，监控方面也是有一定差别的。 Pod是K8S上调度的最小单元，本文就K8S Pod该如何监控进行说明。 CPU 在使用物理机/虚拟机时，对于CPU的监控通常关注的是CPU使用率、CPU负载等。在K8S场景下，关注的指标有所不同：CPU使用率、CPU受限（CPU Throttled）时间或占比等。 CPU使用率 K8S场景下，CPU使用率是相对CP
ks8的数据管理---PV/PVC
最美dee时光的博客
04-30 974
1、背景 所谓容器的Volume,其实就是将一个宿主机上的目录跟一个容器里的目录绑定挂载在了一起；volume 提供了非常好的数据持久化方案，不过在可管理性上还有不足: 要使用 Volume，Pod 必须事先知道Volume来自哪儿，需要提前创建； Pod 通常是由应用的开发人员维护，而 Volume 则通常是由存储系统的管理员维护。开发人员要获得上面的信息：要么询问管理员;要么自己就是管理员。...
java远程监控k8s各节点资源占用情况
yztezhl的专栏
04-28 2258
java远程监控k8s各节点资源占用情况
k8s集群通过kube-prometheus部署prometheus监控系统
weixin_45112997的博客
07-25 5232
k8s通过kube-prometheus部署prometheus监控
Prometheus+Grafana监控K8S 监控pod的解决方案（监控 java-JVM-pod）
qq_40178286的博客
06-16 9489
prometheus 监控 k8s pod 容器服务状态
【Docker 那些事儿】快速部署集群指南
Edison's 小宇宙
07-21 3142
本篇文章将带领大家搭建出企业级的Kubernetes集群，在实际的操作中感受容器编排的魅力。
K8S 中使用 Prometheus 监控JVM
mj的博客
02-07 1012
使用Prometheus 社区开发了 JMX Exporter 用于导出 JVM 的监控指标，以便使用 Prometheus 来采集监控数据。将Java 业务容器化至 Kubernetes 后，通过Prometheus收集jvm监控数据，并在grafana面便查看。编写 JMX Exporter 配置文件。下载jmx_exporter。（可自行配置，下面是示例）
k8s集群下canal-server的伪高可用实践
01-07
k8s集群下canal-server的伪高可用实践前言问题解决方案总结 前言 前面我们已经介绍了canal的admin、server、adapter三个部分的容器化以及在k8s集群下的搭建过程。在创建canal-server的时候，k8s环境下，容器重启会...
k8s HA高可用集群搭建部署
06-29
解决了无法下载镜像问题）一部分为k8s高可用集群搭建部署以及第三方dashboard的管理页面，可以实现在页面上进行管操作，第二部分为k8s在线升级以及k8s采用第三方插件helm安装部署监控。
二进制部署k8s高可用集群（二进制-V1.20）.docx
06-29
说明：二进制部署k8s高可用集群，内容真实有效！
Docker+K8S 集群环境搭建及分布式应用部署
09-29
主要介绍了Docker+K8S 集群环境搭建及分布式应用部署，文中通过示例代码介绍的非常详细，对大家的学习或者工作具有一定的参考学习价值，需要的朋友们下面随着小编来一起学习学习吧
K8s初次入门
qq_43982499的博客
04-27 893
k8s 集群主机清单事先准备所有的k8s集群主机卸载防火墙和（docker、k8s建议禁用swap)安装工具修改配置文件配置内核参数设置Tab补全功能键对master主控节安装测试系统环境主控节点初始化安装计算节点（node系列）1.先从master主机获取凭证2.对各个计算节点进行配置（是否防火墙关掉/禁用swap、安装k8s相关软件包、配置内核参数）3.逐个加入master集群通过master主机查看所有加入集群的计算节点主机。
k8s环境prometheus operator监控集群外资源
最新发布
mingqing的博客
04-30 433
参考网站 k8s环境添加其他节点 参考文档 文档一 文档二 基于prometheus operator 引入外部exporter参考文档 rabbitmq 引入外部exporter参考文档 windows exporte 监控 K8s 集群外服务的两种方式 k8s环境prometheus operator添加node-exporter 方式一：通过 ServiceMonitor 方式 创建 Service 和 ServiceMonitor 文件名为 external-node.yaml /root/test
k8s学习(三十七)centos下离线部署kubernetes1.30(高可用)
文若书生的专栏
04-27 822
在k8s-master01 init后提示的不带control-plane的命令后添加–cri-socket unix:///var/run/cri-dockerd.sock。在初始化k8s集群的使用，IP填写的vip，这样安装好k8s集群之后，kubectl客户端而言，访问的vip:16443端口，该端口是nginx监听的端口，nginx会进行反向代理到3个master节点上的6443端口。（6）添加controlPlaneEndpoint处配置，配置为VIP:16443。
K8s: 公有镜像中心和私有镜像中心的搭建
Wang的专栏
04-26 873
如果 hub.docker.com 在国内无法访问，可以在阿里云上创建 docker hub。在 hub.docker.com 上注册账号 (国内一般访问不了，原因不多说)找到 Create Repository 按钮就行仓库的创建。这样就在官方创建了一个仓库，比如地址为: xx/y-y。现在，我将本地的docker镜像，推送到这个仓库里。1 ）在 官方docker镜像中心推送。2 ）在阿里云docker镜像中心搭建。配置国内 Docker 镜像源 $为官方docker镜像中心加速。配置之后，需要重启 $
k8s：精通 Pod 操作的关键命令
weixin_43784341的博客
04-29 683
这些命令可以帮助你在 Kubernetes 中管理和操作 Pod，使你能够轻松地创建、查看、删除和调试 Pod，并进行其他相关操作。
k8s: 从私有仓库harbor获取镜像
amadeus_liu2的博客
04-27 271
二、在清单文件中使用imagePullSecrets来引入上面这个步骤中创建的secret。一、根据私有仓库的用户名密码等信息创建secret。
k8s集群部署mysql高可用
09-27
k8s集群部署mysql高可用的方法可以通过以下步骤进行实现： 1. 首先，确保已经安装好StorageClass，可以参考文章中的指南进行安装。 2. 下载并准备mysql镜像： ``` docker pull mysql:5.7 docker save -o mysql-...

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值