【星海出品】Kubernetes(K8S) 入门

引用文档
https://kubernetes.io/docs/concepts/architecture/nodes/#swap-memory
https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm-init/

最新更新版本为
K8S：v1.28.15
pause: 3.9
calico: v3.28.1

基础环境优化

swap

Ubuntu

查看SWAP

swapon --show
NAME      TYPE SIZE USED PRIO
/swap.img file   4G   0B   -2

查看文件

内置交换分区：通常具有最高的优先级，默认为-1。
交换文件：优先级通常为-2。
其他类型的交换空间：优先级可能更低，或者根据具体情况而有所不同。
最高优先级为 -20
最低优先级为 20
0不是有限的有限数值

例如:

/path/to/your/swapfile none swap sw,pri=10 0 0

sudo swapoff -a  # 关闭所有交换空间
sudo swapon -a   # 重新启用所有交换空间

sysctl -w vm.swappiness=0
sysctl -a | grep swap

sed -ri '/^[^#]*swap/s@^@#@' /etc/fstab

永久关闭swap分区
打开系统文件 /etc/fatab
注释swap所在行

ntp

参考文献

https://developer.aliyun.com/article/1265391?spm=5176.26934562.main.1.7f2e5082yFaAzA

方法一：

apt-get install chrony
systemctl status chronyd

cat /etc/chrony/chrony.conf
confdir /etc/chrony/conf.d
#pool ntp.aliyun.com iburst
#pool ntp.tencent.com iburst
keyfile /etc/chrony/chrony.keys
driftfile /var/lib/chrony/chrony.drift
ntsdumpdir /var/lib/chrony
logdir /var/log/chrony
allow 172.23.31.0/24
local stratum 10
maxupdateskew 1.0
rtcsync
makestep 1 -1
leapsectz Asia/Shanghai
bindcmdaddress 127.0.0.1

systemctl restart chrony
systemctl enable --now chrony
systemctl enable chrony

客户端：

cat /etc/chrony/chrony.conf
confdir /etc/chrony/conf.d
#pool ntp.aliyun.com iburst
#pool ntp.tencent.com iburst
pool 172.23.31.30 iburst
keyfile /etc/chrony/chrony.keys
driftfile /var/lib/chrony/chrony.drift
ntsdumpdir /var/lib/chrony
logdir /var/log/chrony
maxupdateskew 1.0
rtcsync
makestep 1 -1
leapsectz Asia/Shanghai
bindcmdaddress 127.0.0.1

方法二

参看ansible资料一
>https://blog.csdn.net/weixin_41997073/article/details/130598897?sharetype=blogdetail&sharerId=130598897&sharerefer=PC&sharesource=weixin_41997073&spm=1011.2480.3001.8118

bash run.sh "apt-get install ntpdate"
bash run.sh "/usr/sbin/ntpdate time1.aliyun.com"

ubuntu22.04升级内核方式

sudo apt-get update

sudo apt-cache search linux-image  ## 查看可用内核

apt-get install linux-image-6.5.0-45-generic

sudo apt-get install linux-headers-x.x.x-xx-generic
sudo apt-get install linux-image-x.x.x-xx-generic
sudo apt-get install linux-modules-x.x.x-xx-generic 
sudo apt-get install linux-modules-extra-x.x.x-xx-generic

sudo apt-get install linux-headers-6.x.x-xx-generic
这个命令安装内核头文件。内核头文件对于编译需要直接与内核交互的软件（如某些驱动程序或内核模块）是必需的。它们包含了内核API的声明和宏定义，允许开发者编写与当前内核版本兼容的代码。
sudo apt-get install linux-image-6.x.x-xx-generic
这个命令安装内核映像。内核映像包含了操作系统启动时加载的内核代码。它是系统能够正常运行的基础，负责管理系统资源、处理硬件中断、提供进程调度等核心功能。
sudo apt-get install linux-modules-6.x.x-xx-generic
这个命令安装与内核版本相匹配的基本内核模块。内核模块是可加载的内核扩展，它们提供了额外的功能，如文件系统支持、硬件驱动程序等。这些模块在需要时由内核动态加载。
sudo apt-get install linux-modules-extra-6.x.x-xx-generic
这个命令安装额外的内核模块。这些模块通常不是内核发布时默认包含的，但可能由社区提供或用于支持特定的硬件或功能。它们可能包括非标准的驱动程序、实验性功能等。

注意：k8s 1.29集群内核版本要大于6.3

查看更新情况
sudo dpkg --list | grep linux-image

卸载相应的内核版本
sudo apt-get purge linux-image-xxxxx    ## 相应版本号
sudo apt-get purge linux-headers-xxxxx    ## 相应版本号
sudo apt-get autoremove

更新完内核后更新grub文件
sudo update-grub

关闭内核版本自动更新
sudo apt-mark hold linux-image-generic linux-headers-generic
启动内核版本自动更新
sudo apt-mark unhold linux-image-generic linux-headers-generic

生效
reboot

k8s系统转发设置

cat /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1

设置

modprobe br_netfilter
命令作用：
modprobe是Linux中的一个命令，用于加载或卸载内核模块。
br_netfilter是一个内核模块，它允许对桥接（bridge）接口上的流量进行iptables过滤。
这对于在Linux系统上运行Kubernetes等容器化平台尤为重要，因为Kubernetes使用网络桥接来连接容器。

sysctl -p /etc/sysctl.d/k8s.conf
命令作用：
sysctl是Linux中用于读取和写入内核运行时参数的命令。
-p选项表示从指定的文件（或默认文件）中读取系统参数设置，并应用它们。
在这个例子中，/etc/sysctl.d/k8s.conf是一个包含特定系统参数设置的文件，
这些设置通常是为了优化或配置系统以更好地支持Kubernetes等容器化平台。
应用场景：k8s.conf文件可能包含调整内核参数以优化网络性能、资源限制、安全设置等配置。通过执行sysctl -p /etc/sysctl.d/k8s.conf命令，可以将这些配置应用到正在运行的系统上。

lsmod 
查看加载了哪些模块，可以通过管道符加 grep 进行筛选
ls /sys/module | grep br_netfilter

modinfo br_netfilter
查看模块的详细信息

dmesg命令会显示内核环缓冲区的内容，这包括模块加载和卸载的消息。你可以通过搜索br_netfilter来查找相关的消息。
dmesg | grep br_netfilter
该命令是Linux系统中一个强大且实用的工具，它可以帮助用户查看和分析系统启动过程中的内核消息，以及实时监视内核的运行状态。

Ubuntu22.04 ,内核版本大于 6.3.x 自动加载模块

vim /etc/modules-load.d/br_netfilter.conf
br_netfilter

三方工具使用

建议k8s-1.18以上的版本安装 ipset 和 ipvsadm

apt-get install ipset ipvsadm

软硬限制limit

查看最大进程数

 cat /proc/sys/kernel/pid_max
4194304

/etc/security/limits.conf

* soft nofile 65536
* hard nofile 131072
* soft nproc 65535
* hard nproc 655350
* soft memlock unlimited
* hard memlock unlimited

cat ipvs.modules

modprobe -- ip_vs
modprobe -- ip_vs_rr
modprobe -- ip_vs_wrr
modprobe -- ip_vs_sh
modprobe -- nf_conntrack

高内核切换了目录
/etc/modules-load.d/ipvs.conf

ip_vs
ip_vs_lc
ip_vs_wlc
ip_vs_rr
ip_vs_wrr
ip_vs_lblc
ip_vs_lblcr
ip_vs_dh
ip_vs_sh
ip_vs_fo
ip_vs_nq
ip_vs_sed
ip_vs_ftp
ip_vs_sh
nf_conntrack
ip_tables
ip_set
xt_set
ipt_set
ipt_rpfilter
ipt_REJECT
ipip

cat /etc/sysctl.d/k8s.conf
sysctl --system
不用重启就可以看到效果

net.bridge.bridge-nf-call-ip6tables = 1
意义：允许桥接流量通过ip6tables过滤。这对于在桥接网络上使用IPv6地址转换（NAT）是必要的。
net.bridge.bridge-nf-call-iptables = 1
意义：允许桥接流量通过iptables过滤。这对于在桥接网络上使用IPv4地址转换（NAT）是必要的。
net.ipv4.ip_forward = 1
意义：启用IPv4转发。这对于路由器或任何需要转发IP数据包的系统是必要的。
fs.may_detach_mounts = 1
意义：允许卸载文件系统，即使它在使用中。这个设置可能会增加系统的不稳定性，因为它允许在文件仍被使用时卸载它们。
检查：通常不推荐启用，除非有特定需求。
net.ipv4.conf.all.route_localnet = 1
意义：允许系统路由本地网络（127.0.0.0/8）的流量。这在某些特殊配置中可能有用，但通常不推荐。
通常不推荐启用，除非有特定需求。
vm.overcommit_memory = 1
意义：允许内存过度分配。这意味着系统可以分配比物理内存更多的内存，但可能会导致内存不足的情况。
vm.panic_on_oom = 0
意义：在内存耗尽时不让系统崩溃。
fs.inotify.max_user_watches = 89100
意义：设置用户级inotify实例可以使用的最大监视器数量。这对于需要监视大量文件的系统（如文件同步工具）是必要的。
fs.file-max = 52706963
意义：设置系统级别的文件描述符限制。
fs.nr_open = 52706963
意义：设置用户级别的文件描述符限制。
要与fs.file-max协调，并确保不超过系统限制。
net.netfilter.nf_conntrack_max = 2310720
意义：设置系统可以跟踪的最大并发连接数。
net.ipv4.tcp_keepalive_time = 600
意义：设置TCP连接的保活探测的初始等待时间（秒）。
net.ipv4.tcp_keepalive_probes = 3
意义：设置TCP连接在放弃之前尝试保活探测的次数。
net.ipv4.tcp_keepalive_intvl = 15
意义：设置TCP连接在连续两次保活探测之间的时间间隔（秒）。
net.ipv4.tcp_max_tw_buckets = 36000
意义：设置系统可以保持的TIME-WAIT套接字的最大数量。
net.ipv4.tcp_tw_reuse = 1
意义：允许重用TIME-WAIT状态的套接字，用于新的连接。
net.ipv4.tcp_max_orphans = 327680
意义：设置系统可以保持的孤立套接字的最大数量。
net.ipv4.tcp_syncookies = 1
意义：在SYN泛滥攻击时启用同步cookie，以保护系统。
net.ipv4.tcp_max_syn_backlog = 16384
意义：设置系统用于存储尚未完全建立的连接请求的最大队列长度。
net.ipv4.ip_conntrack_max = 65536
意义：设置系统可以跟踪的最大连接数（与nf_conntrack_max类似，但可能是旧参数）。
可使用nf_conntrack_max。
net.ipv4.tcp_timestamps = 0
意义：禁用TCP时间戳选项。这可能会影响性能，因为它会禁用TCP的某些优化。
不推荐禁用，除非有特定需求。
net.core.somaxconn = 16384
意义：设置listen()系统调用的“backlog”队列的最大长度。

Part 2

docker 24.0.7
cri-dockerd 0.3.8

安装docker参考

https://blog.csdn.net/weixin_41997073/article/details/118858439?sharetype=blogdetail&sharerId=118858439&sharerefer=PC&sharesource=weixin_41997073&spm=1011.2480.3001.8118
最下方2024/11/22

安装工具

dpkg -i cri-dockerd_0.3.2.3-0.ubuntu-jammy_amd64.deb

/usr/lib/systemd/system/cri-docker.service

[Unit]
Description=CRI Interface for Docker Application Container Engine
Documentation=https://docs.mirantis.com
After=network-online.target firewalld.service docker.service
Wants=network-online.target
Requires=cri-docker.socket

[Service]
Type=notify
ExecStart=/usr/bin/cri-dockerd --pod-infra-container-image=registry.aliyuncs.com/google_containers/pause:3.7 --container-runtime-endpoint fd://
ExecReload=/bin/kill -s HUP $MAINPID
TimeoutSec=0
RestartSec=2
Restart=always

# Note that StartLimit* options were moved from "Service" to "Unit" in systemd 229.
# Both the old, and new location are accepted by systemd 229 and up, so using the old location
# to make them work for either version of systemd.
StartLimitBurst=3

# Note that StartLimitInterval was renamed to StartLimitIntervalSec in systemd 230.
# Both the old, and new name are accepted by systemd 230 and up, so using the old name to make
# this option work for either version of systemd.
StartLimitInterval=60s

# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity

# Comment TasksMax if your systemd version does not support it.
# Only systemd 226 and above support this option.
TasksMax=infinity
Delegate=yes
KillMode=process

[Install]
WantedBy=multi-user.target

如何开启docker-cri

systemctl daemon-reload
systemctl enable --now cri-docker.service
systemctl enable --now cri-docker.socket
systemctl start cri-docker.socket
systemctl status cri-docker.socket    

寻找docker的K8S源

https://www.cnblogs.com/Unstoppable9527/p/18320631#tid-34abmC

(X)该链接中的信息已错误

{
    "registry-mirrors": [
        "https://docker.m.daocloud.io",
        "https://dockerproxy.com",
        "https://docker.mirrors.ustc.edu.cn",
        "https://docker.nju.edu.cn"
    ]
}
将该信息进行修改
#{
#  "insecure-registries": ["harbor.jiajia.com"],  （X）失效了
#  "registry-mirrors": ["https://docker.chenby.cn"],
#  "exec-opts": ["native.cgroupdriver-systemd"]
#}

systemctl daemon-reload
systemctl enable --now docker
systemctl status docker

https://developer.aliyun.com/mirror/kubernetes/

apt-get update && apt-get install -y apt-transport-https
curl -fsSL https://mirrors.aliyun.com/kubernetes-new/core/stable/v1.28/deb/Release.key |
    gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://mirrors.aliyun.com/kubernetes-new/core/stable/v1.28/deb/ /" |
    tee /etc/apt/sources.list.d/kubernetes.list
apt-get update
apt-get install -y kubelet kubeadm kubectl
or
apt-get install -y kubeadm=1.28.15-1.1

dpkg -s kubelet | grep Version
dpkg -s kubeadm | grep Version
dpkg -s kubectl | grep Version

Package: kubectl
Status: install ok installed
Priority: optional
Section: admin
Installed-Size: 48503
Maintainer: Kubernetes Authors <dev@kubernetes.io>
Architecture: amd64
Version: 1.28.15-1.1
Description: Command-line utility for interacting with a Kubernetes cluster
 Command-line utility for interacting with a Kubernetes cluster.
Homepage: https://kubernetes.io

Version: 1.28.15-1.1

Version: 1.28.15-1.1

apt-cache madison kubeadm
   kubeadm | 1.28.15-1.1 | https://mirrors.aliyun.com/kubernetes-new/core/stable/v1.28/deb  Packages
查看 kubeadm

https://blog.csdn.net/2302_78152953/article/details/139174579

Centos in /var/lib/kubelet/config.yaml
Ubuntu in /etc/default/kubelet

dpkg -s kubelet
Package: kubelet
Status: install ok installed
Priority: optional
Section: net
Installed-Size: 108140
Maintainer: Kubernetes Authors <dev@kubernetes.io>
Architecture: amd64
Version: 1.28.15-1.1
Depends: iptables (>= 1.4.21), kubernetes-cni (>= 1.2.0), iproute2, mount, conntrack, util-linux, ethtool, libc6
Conffiles:
 /etc/default/kubelet 9ba5cd2e9a1e368fa51e13f1dd6a5ec1
Description: Node agent for Kubernetes clusters
 Node agent for Kubernetes clusters.
Homepage: https://kubernetes.io

/etc/default/kubelet

KUBELET_EXTRA_ARGS="--cgroup-driver=systemd"
systemctl enable kubelet

kubelet直接启动会报错，没有如下配置文件
/var/lib/kubelet/config.yaml

root@node1:/etc/default# kubeadm config images list
可以查看关联版本的镜像

kubeadm config images pull

Found multiple CRI endpoints on the host. Please define which one do you wish to use by setting the 'criSocket' field in the kubeadm configuration file: unix:///var/run/containerd/containerd.sock, unix:///var/run/cri-dockerd.sock
To see the stack trace of this error execute with --v=5 or higher

记录以下unix端口，然后修改命令继续pull

ls /var/run/containerd/  # 下面有多个
#kubeadm config image pull --cri-socket unix:///var/run/cri-dockerd.sock

生成配置文件
kubeadm config print init-defaults > kubeadm-config.yaml

配置文件修改

apiVersion: kubeadm.k8s.io/v1beta3
bootstrapTokens:
- groups:
  - system:bootstrappers:kubeadm:default-node-token
  token: abcdef.0123456789abcdef
  ttl: 24h0m0s
  usages:
  - signing
  - authentication
kind: InitConfiguration
localAPIEndpoint:
  advertiseAddress: 172.23.41.21
  bindPort: 6443
nodeRegistration:
  criSocket: unix:///var/run/containerd/containerd.sock
  imagePullPolicy: IfNotPresent
  name: node1
  taints: null

---
apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
apiServer:
  timeoutForControlPlane: 4m0s
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controllerManager: {}
dns: {}
etcd:
  local:
    dataDir: /var/lib/etcd
imageRepository: registry.aliyuncs.com/google_containers
kubernetesVersion: 1.28.0
networking:
  dnsDomain: cluster.local
  serviceSubnet: 172.23.41.0/24
  podSubnet: 172.23.41.0.0/24
scheduler: {}

---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
cgroupDriver: systemd

查看镜像

kubeadm config images list --config kubeadm-config.yaml

拉取镜像

kubeadm config images pull --cri-socket unix:///var/run/cri-dockerd.sock  --image-repository registry.aliyuncs.com/google_containers

sandbos配置

containerd config default > /etc/containerd/config.toml
# 2. 修改1 在60多行，sandbox_image = "registry.k8s.io/pause:3.6"，改成我们kubernetes版本一致的，现在应该是3.9，后面会说怎么看。
# 改成：sandbox_image = "registry.aliyuncs.com/google_containers/pause:3.9"  《注：此处根据镜像版本修改版本号》
# 3. 修改2  在130行左右，我这127行，SystemdCgroup = false改成SystemdCgroup = true
# 4. 直接重启吧  reboot

Other

sudo systemctl stop ufw.service
sudo systemctl disable ufw.service   #开机禁用
sudo systemctl status ufw.service
ufw disable

sudo sysctl -p /etc/sysctl.d/k8s.conf 
sysctl --system
sysctl -a

初始化

docker images
docker rmi e6f181688397
rm -rf /var/lib/etcd
rm -rf /etc/cni/net.d
rm -rf  /etc/kubernetes/mainfests/*

kubeadm reset --cri-socket unix:///var/run/cri-dockerd.sock

ipvsadm --clear
rm -rf /etc/cni/net.d
rm -rf $HOME/.kube/config

# kubeadm init --config /root/new.yaml
OR
kubeadm init --config /root/k8s/kubeadm-config.yaml

# kubeadm init --kubernetes-version=v1.28.0 --pod-network-cidr=172.17.50.0/24 --apiserver-advertise-address=172.23.41.21 --cri-socket unix:///var/run/cri-dockerd.sock
会去外网

失败后修复文档，参考

https://www.jianshu.com/p/6b7e1feaa6b5

cd /
# 生成kubeadm默认配置文件
kubeadm config print init-defaults > kubeadm-config.yaml
# 2. 修改默认配置
vim kubeadm-config.yaml

ip修改为我们master节点ip：localAPIEndpoint.advertiseAddress: 10.10.1.21
修改名称：nodeRegistration.name: vm21
镜像源修改为阿里的：imageRepository: registry.aliyuncs.com/google_containers
kubernetesVersion修改：kubernetesVersion: 1.30.3
添加networking.podSubnet：networking.podSubnet: 10.244.0.0/16

文件末尾添加cgroup(有分隔符)：
---
kind: KubeletConfiguration
apiVersion: kubelet.config.k8s.io/v1beta1
cgroupDriver: systemd

kubeadm init --config kubeadm-config.yaml

apiVersion: kubeadm.k8s.io/v1beta3
bootstrapTokens:
- groups:
  - system:bootstrappers:kubeadm:default-node-token
  token: abcdef.0123456789abcdef
  ttl: 24h0m0s
  usages:
  - signing
  - authentication
kind: InitConfiguration
localAPIEndpoint:
  advertiseAddress: 172.23.41.21
  bindPort: 6443
nodeRegistration:
  criSocket: unix:///var/run/cri-dockerd.sock
  imagePullPolicy: IfNotPresent
  name: node1
  taints: null
---
apiServer:
  timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta3
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controllerManager: {}
dns: {}
etcd:
  local:
    dataDir: /var/lib/etcd
imageRepository: registry.aliyuncs.com/google_containers
kind: ClusterConfiguration
kubernetesVersion: 1.28.0
networking:
  dnsDomain: cluster.local
  serviceSubnet: 10.244.0.0/16
scheduler: {}

---
kind: KubeletConfiguration
apiVersion: kubelet.config.k8s.io/v1beta1
cgroupDriver: systemd

Env

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config
 export KUBECONFIG=/etc/kubernetes/admin.conf

加入主节点
不记得了用这个生成：
kubeadm token create --print-join-command

Node节点加入集群
kubeadm init --kubernetes-version=v1.28.0 --pod-network-cidr=

kubeadm join 172.23.41.21:6443 --token lvho8f.mekhyxjybezwhoze --discovery-token-ca-cert-hash sha256:76783d0de1fe69b7e146accef204dad3c11f671a38f7bdac22666873cda3aa67  --cri-socket unix:///var/run/cri-dockerd.sock

version: 通过docker images查看

子节点离开Master

kubectl delete node <node-name>

kubeadm reset --cri-socket unix:///var/run/cri-dockerd.sock
ipvsadm --clear
rm -rf /etc/cni/net.d

sudo rm -rf /etc/kubernetes/
sudo rm -rf /var/lib/etcd/

kubectl get nodes
kubectl get pod -A

可以查看节点情况
kubectl get nodes  

查看pod详细信息
kubectl describe -n kube-system pod kube-apiserver-node1

检查

查
kubectl get pods -A
查
kubectl describe pod kube-proxy-gkszb -n kube-system

删
kubectl delete pod kube-proxy-gkszb -n kube-system

清理环境

systemctl stop kubectl
for i in `docker ps -a | awk 'NR>1{print $1}'`; do docker stop ${i}; done
for i in `docker ps -a | awk 'NR>1{print $1}'`; do docker rm ${i}; done
for i in `docker images | awk 'NR>1{print $3}'`;do docker rmi ${i} ;done

碰到NODE NOTready是正常的
碰到coredns是pending是正常的
是因为没有部署网络插件

安装calico

官网文献
https://docs.tigera.io/calico/latest/getting-started/kubernetes/quickstart

calico每一个节点都要有，pull的话容易出现网络问题，需要手动docker load -i

https://docs.tigera.io/calico/latest/getting-started/kubernetes/requirements#kubernetes-requirements
注意版本对应的k8s的要求

kubectl create -f https://raw.githubusercontent.com/projectcalico/calico/v3.28.1/manifests/tigera-operator.yaml

wget https://projectcalico.docs.tigera.io/manifests/calico.yaml
kubectl apply -f calico.yaml
cat calico.yaml | grep image
kubectl get pods -n kube-system
kubectl get pods -A

wget https://raw.githubusercontent.com/projectcalico/calico/v3.28.1/manifests/calico.yaml
            - name: CALICO_IPV4POOL_CIDR
              value: "10.244.0.0/16"
         还要加一个选项
            - name: IP_AUTODETECTION_METHOD
              value: "interface=ens33"

https://docs.tigera.io/calico/latest/getting-started/kubernetes/quickstart
该网址左侧导航栏查找 Install 信息
第二个安装的CMD ，需要 wget yaml 文件，然后

kubectl delete -f tigera-operator.yaml

kubectl api-resources | grep calico
kubectl delete crd bgpconfigurations.crd.projectcalico.org

rm -rf /etc/cni/net.d/
reboot

检查calico

kubectl get customresourcedefinitions | grep calico
kubectl api-resources | grep calico
kubectl get ns
kubectl get pods -n tigera-operator

强制删除所有的pods

kubectl delete -n tigera-operator --grace-period=0 --force

删除calico

kubectl delete -f calico.yaml

K8S污点处理

查看污点
kubectl describe node node1 | grep Taints
删除污点
kubectl taint nodes k8s-master node-role.kubernetes.io/control-plane-

配置K8S前端控制界面

curl -O https://raw.githubusercontent.com/kubernetes/dashboard/v2.7.0/aio/deploy/recommended.yaml

kubectl apply -f recommended.yaml

https://blog.csdn.net/m0_53928179/article/details/139068769

Kubernetes(k8s)概述

开发理念
Kubernetes是一个开源的，用于管理云平台中多个主机上的容器化的应用，Kubernetes的目标是让部署容器化的应用简单并且高效（powerful）,Kubernetes提供了应用部署，规划，更新，维护的一种机制。

应用理念
k8s是一个编排容器的工具，其实也是管理应用的全生命周期的一个工具，从创建应用，应用的部署，应用提供服务，扩容缩容应用，应用更新，都非常的方便，而且可以做到故障自愈，例如一个服务器挂了，可以自动将这个服务器上的服务调度到另外一个主机上进行运行，无需进行人工干涉。

k8s可以更快的更新新版本，打包应用，更新的时候可以做到不用中断服务，服务器故障不用停机，从开发环境到测试环境到生产环境的迁移极其方便，一个配置文件搞定，一次生成image，到处运行

在k8s进行管理应用的时候，基本步骤是：创建集群，部署应用，发布应用，扩展应用，更新应用。

为什么一次生成Image可以到处运行，需要先了解Docker：
https://blog.csdn.net/weixin_41997073/article/details/118858439

K8S架构

• k8s的架构主要是由master节点和node节点构成
• 一个master节点可以对应多个node节点。

master节点不存储容器

主要由四个部分构成：
api server k8s网关，所有请求指令都必须先到这里
scheduler 调度器，控制网关过来的请求路由到哪个node
controller 控制器，主要用来维护k8s资源对象
etcd 主要用来资源对象

node节点就是存储具体容器的，但是不仅仅只有容器，主要包括：
docker 容器引擎，运行容器的基础环境
kubelet 在每个node节点都存在一份，主要来执行关于资源操作的指令，负责pod的维护。
kube-proxy 代理服务，用于负载均衡，在多个pod之间做负载均衡
fluentd 日志收集服务
pod 是k8s的最小服务单元，pod内部才是容器，k8s通过操作pod来操作容器

pod 也是一个容器，是封装容器的容器，是一个虚拟化的分组，有自己的ip和port。
pod的子容器之间的交互，就像在一台机器上的交互。具体的实现是通过pause容器来实现的。
在初始化一个pod容器的时候，就会生成一个pause容器。这个容器使得pod里面的子容器能够共享网络和存储，方便内部容器之间的调用。

一个Pod中的应用容器共享同一组资源：
PID命名空间：Pod中的不同应用程序可以看到其他应用程序的进程ID；
网络命名空间：Pod中的多个容器能够访问同一个IP和端口范围；
IPC命名空间：Pod中的多个容器能够使用SystemV IPC或POSIX消息队列进行通信；
UTS命名空间：Pod中的多个容器共享一个主机名；
Volumes（共享存储卷）：Pod中的各个容器可以访问在Pod级别定义的Volumes；

Node包含的信息：
Node地址：主机的IP地址，或Node ID。
Node的运行状态：Pending、Running、Terminated三种状态。
Node Condition：…
Node系统容量：描述Node可用的系统资源，包括CPU、内存、最大可调度Pod数量等。
其他：内核版本号、Kubernetes版本等。

Pod的生命周期通过Replication Controller来管理；
通过模板进行定义，然后分配到一个Node上运行，在Pod所包含容器运行结束后，Pod结束。

Service可以看作一组提供相同服务的Pod的对外访问接口，Service作用于哪些Pod是通过Label Selector来定义的。

• 拥有一个指定的名字（比如my-mysql-server）；
• 拥有一个虚拟IP（Cluster IP、Service IP或VIP）和端口号，销毁之前不会改变，只能内网访问；
• 能够提供某种远程服务能力；
• 被映射到了提供这种服务能力的一组容器应用上；

Volume是Pod中能够被多个容器访问的共享目录。

Label以key/value的形式附加到各种对象上，如Pod、Service、RC、Node等，以识别这些对象，管理关联关系等，如Service和Pod的关联关系。

RC中定义的Lable筛选出对应的Pod实例，并实时监控其状态和数量，如果实例数量少于定义的副本数量（Replicas），则会根据RC中定义的Pod模板来创建一个新的Pod，然后将此Pod调度到合适的Node上启动运行，直到Pod实例数量达到预定目标。

客户端流程

客户端通过Kubectl命令行工具或Kubectl
Proxy来访问Kubernetes系统，在Kubernetes集群内部的客户端可以直接使用Kuberctl命令管理集群。Kubectl
Proxy是API Server的一个反向代理，在Kubernetes集群外部的客户端可以通过Kubernetes Proxy来访问API
Server。

使用内部流程

通过Kubectl提交一个创建RC的请求，该请求通过API Server被写入etcd中

此时Controller Manager通过API Server的监听资源变化的接口监听到这个RC事件，分析之后，发现当前集群中还没有它所对应的Pod实例，于是根据RC里的Pod模板定义生成一个Pod对象, 通过API Server写入etcd

接下来，此事件被Scheduler发现，它立即执行一个复杂的调度流程，为这个新Pod选定一个落户的Node

通过API Server讲这一结果写入到etcd中，随后，目标Node上运行的Kubelet进程通过API

Server监测到这个“新生的”Pod，并按照它的定义，启动该Pod并任劳任怨地负责它的下半生，直到Pod的生命结束。

随后，我们通过Kubectl提交一个新的映射到该Pod的Service的创建请求，Controller Manager会通过Label标签查询到相关联的Pod实例，然后生成Service的Endpoints信息，并通过API Server写入到etcd中，接下来，所有Node上运行的Proxy进程通过API

Server查询并监听Service对象与其对应的Endpoints信息，建立一个软件方式的负载均衡器来实现Service访问到后端Pod的流量转发功能。

cat nginx-deployment.yaml

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: mynginx # 部署的名称
spec:
  replicas: 2  # 设置副本数量为2
  selector:
    matchLabels:
      app: mynginx1 # 用于选择匹配的Pod标签
  template:
    metadata:
      labels:
        app: mynginx1 # Pod的标签
    spec:
      containers:
      - name: nginx # 容器名称
        image: registry.cn-hangzhou.aliyuncs.com/acs-sample/nginx:latest # 镜像拉取地址，换成阿里云的，不然会拉取失败
        imagePullPolicy: IfNotPresent # 镜像拉取策略，如果本地没有就拉取
        ports:
        - containerPort: 80 # 容器内部监听的端口
---
apiVersion: v1
kind: Service
metadata:
  name: nginx-service # 服务的名称
spec:
  externalTrafficPolicy: Cluster # 外部流量策略设置为集群
  selector:
    app: mynginx1 # 用于选择匹配的Pod标签
  ports:
  - protocol: TCP # 使用TCP协议
    port: 80 # 服务暴露的端口
    targetPort: 80 # Pod中容器的端口
    nodePort: 30080 # 在每个Node上分配的端口，用于外部访问
  type: NodePort # 服务类型，使用NodePort

常用命令

Node

[root@master ~]kubectl get no    #查看node节点
[root@master ~]kubectl get no -o wide   #查看node节点
[root@master ~]kubectl describe no   
[root@master ~]kubectl get no -o yaml  #以yaml的格式输出node的详细信息
[root@master ~]kubectl get no -o json	 #以json的格式输出node的详细信息
[root@master ~]kubectl get node --selector=[label_name]   #selector 标签选择器（yaml文件中有）
[root@master ~]kubectl top node [node_name]  #查看node节点各资源的使用情况
[root@master ~]kubectl top node node1

PODS

[root@master ~]kubectl get po 				#pod的信息
[root@master ~]kubectl get po -o wide
[root@master ~]kubectl describe po
[root@master ~]kubectl get po --show-labels      #查看pod对应的标签
[root@master ~]kubectl get po -l app=nginx      	#查看标签为app: nginx 的pod
[root@master ~]kubectl get po -o yaml   				#yaml格式
[root@master ~]kubectl get po -o json					#json格式
[root@master ~]kubectl get pods -- field-selector status.phase=Running   #查看状态为Running的pod

Namespaces

[root@master ~] kubectl get ns   #查看命名空间
[root@master ~] kubectl get ns - o yaml
[root@master ~] kubectl describe ns

Deployments
是用于管理pod的抽象层

[root@master ~] kubectl get deploy
[root@master ~] kubectl describe deploy
[root@master ~] kubectl get deploy - o wide
[root@master ~] kubectl get deploy - o yaml

Service

[root@master ~] kubectl get svc
[root@master ~] kubectl describe svc
[root@master ~] kubectl get svc - o wide
[root@master ~] kubectl get svc - o yaml
[root@master ~] kubectl get svc --show-labels

更多基础命令查看
https://blog.csdn.net/baidu_38803985/article/details/106005196

安装

安装 minikube + kubectl
需要提前安装好Docker，这里就不再赘述了

#kubectl 的安装：
wget https://storage.googleapis.com/kubernetes- release/release/v1.5.1/bin/darwin/amd64/kubectl chmod +x kubectl mv kubectl /usr/local/bin/kubectl 复制代码
#minikube
curl -Lo minikube http://kubernetes.oss-cn-hangzhou.aliyuncs.com/minikube/releases/v0.30.0/minikube-darwin-amd64 && chmod +x minikube && sudo mv minikube /usr/local/bin/
复制代码

安装完毕后可以启动k8s环境了，命令行运行 minikube start
minikube ssh 进入minikube的虚拟机内部，执行 docker ps 查看所有的组件容器是否启动成功 如果没有问题退出登录，在外部执行 minikube dashboard 即可在浏览器启动控制台，到这里k8s的实验环境就已经搭建完毕了

部署应用
在k8s只要使用两条指令就可以创建一个集群
一个是kubectl init进行初始化，创建一个master节点
第二条指令就是kubectl join xxx创建一个node节点，加入这个集群

kubectl 来创建一个deployment:
第一步就是创建 Deployment ，它将负责创建和更新我们的应用实例，并且持续监控应用的状态。

kubectl run <node_name>--image=<url> --port=8080

• 执行后master节点将会选择一个合适的node来部署该应用（minikube环境下只有一个node），创建完成后可以使用 kubectl get deployment 来查看该应用：

# 开启代理
kubectl proxy
#可以先用 kubectl get pod 来查看pod名

• 代理启动后在新终端通过restful api的方式就可以通过代理访问到内部pod所提供的服务，restful的url模式如下：
  /api/v1/namespace/{namespace}/pods/{name}/proxy/{path:*}

Service
当我们需要对外提供可用的真实服务时，需要更加可靠的手段，那就是 Service

• 一个Service可以映射多个Pod，为了把多个Pod在逻辑上组合起来，k8s又引入了 Label的概念，简单的说就是每个Pod上都可以打一个Label（标签），具有相同Label的Pod就成为一个逻辑分组。Service就是通过Label Selector的方式来关联多个Pod的
  
• 创建一个Service并将它暴露到外部，通常可以有 LoadBalancer 和 NodePort
  两种手段，不过minikube只支持后者，使用 kubectl expose 命令即可创建Service:
  
• 使用 kubectl get service 我们可以看到已经创建的服务，其中 kubernetes 是k8s默认已经创建好的。可以看到，新创建的service把对应pod的8080端口暴露到了外部的32344端口，接下来通过这个端口就可以访问到hello world服务了：

• 这里的 IP 是minikube的docker-daemon的ip，可以通过 minikube docker-env 来查看

• 我们可以查看一下service和pod的详情，来看看Label是否按照我们预期的被创建：

label是 key=value 的形式被创建的，如果想要自定义一些label，使用 kubectl label 命令即可：

• 在pod和service非常多的时候，可以用作一种过滤的手段，使用-l参数即可进行筛选，就像sql的where语句一样：

• 最后，如果想要删除一个service，只需要运行 kubectl delete service [服务名]
  即可，当然服务删除后pod仍然存在并且会持续运行，只是对外的入口消失了而已。

Prometheus

kubectl create ns monitor-sa

root@node1:~/k8s# sysctl -a
abi.vsyscall32 = 1
debug.exception-trace = 1
debug.kprobes-optimization = 1
dev.cdrom.autoclose = 1
dev.cdrom.autoeject = 0
dev.cdrom.check_media = 0
dev.cdrom.debug = 0
dev.cdrom.info = CD-ROM information, Id: cdrom.c 3.20 2003/12/17
dev.cdrom.info =
dev.cdrom.info = drive name:            sr0
dev.cdrom.info = drive speed:           1
dev.cdrom.info = drive # of slots:      1
dev.cdrom.info = Can close tray:                1
dev.cdrom.info = Can open tray:         1
dev.cdrom.info = Can lock tray:         1
dev.cdrom.info = Can change speed:      1
dev.cdrom.info = Can select disk:       0
dev.cdrom.info = Can read multisession: 1
dev.cdrom.info = Can read MCN:          1
dev.cdrom.info = Reports media changed: 1
dev.cdrom.info = Can play audio:                1
dev.cdrom.info = Can write CD-R:                1
dev.cdrom.info = Can write CD-RW:       1
dev.cdrom.info = Can read DVD:          1
dev.cdrom.info = Can write DVD-R:       1
dev.cdrom.info = Can write DVD-RAM:     1
dev.cdrom.info = Can read MRW:          1
dev.cdrom.info = Can write MRW:         1
dev.cdrom.info = Can write RAM:         1
dev.cdrom.info =
dev.cdrom.info =
dev.cdrom.lock = 0
dev.hpet.max-user-freq = 64
dev.mac_hid.mouse_button2_keycode = 97
dev.mac_hid.mouse_button3_keycode = 100
dev.mac_hid.mouse_button_emulation = 0
dev.raid.speed_limit_max = 200000
dev.raid.speed_limit_min = 1000
dev.scsi.logging_level = 0
dev.tty.ldisc_autoload = 1
dev.tty.legacy_tiocsti = 1
fs.aio-max-nr = 65536
fs.aio-nr = 0
fs.binfmt_misc.status = enabled
fs.dentry-state = 46481 24395   45      0       7418    0
fs.dir-notify-enable = 1
fs.epoll.max_user_watches = 1780134
fs.fanotify.max_queued_events = 16384
fs.fanotify.max_user_groups = 128
fs.fanotify.max_user_marks = 64780
fs.file-max = 52706963
fs.file-nr = 1728       0       52706963
fs.inode-nr = 39563     467
fs.inode-state = 39563  467     0       0       0       0       0
fs.inotify.max_queued_events = 16384
fs.inotify.max_user_instances = 128
fs.inotify.max_user_watches = 89100
fs.lease-break-time = 45
fs.leases-enable = 1
fs.mount-max = 100000
fs.mqueue.msg_default = 10
fs.mqueue.msg_max = 10
fs.mqueue.msgsize_default = 8192
fs.mqueue.msgsize_max = 8192
fs.mqueue.queues_max = 256
fs.nr_open = 52706963
fs.overflowgid = 65534
fs.overflowuid = 65534
fs.pipe-max-size = 1048576
fs.pipe-user-pages-hard = 0
fs.pipe-user-pages-soft = 16384
fs.protected_fifos = 1
fs.protected_hardlinks = 1
fs.protected_regular = 2
fs.protected_symlinks = 1
fs.quota.allocated_dquots = 0
fs.quota.cache_hits = 0
fs.quota.drops = 0
fs.quota.free_dquots = 0
fs.quota.lookups = 0
fs.quota.reads = 0
fs.quota.syncs = 46
fs.quota.writes = 0
fs.suid_dumpable = 2
fs.verity.require_signatures = 0
kernel.acct = 4 2       30
kernel.acpi_video_flags = 0
kernel.apparmor_display_secid_mode = 0
kernel.apparmor_restrict_unprivileged_io_uring = 0
kernel.apparmor_restrict_unprivileged_unconfined = 0
kernel.apparmor_restrict_unprivileged_userns = 0
kernel.apparmor_restrict_unprivileged_userns_complain = 0
kernel.apparmor_restrict_unprivileged_userns_force = 0
kernel.arch = x86_64
kernel.auto_msgmni = 0
kernel.bootloader_type = 6
kernel.bootloader_version = 38
kernel.bpf_stats_enabled = 0
kernel.cad_pid = 1
kernel.cap_last_cap = 40
kernel.core_pattern = |/usr/share/apport/apport -p%p -s%s -c%c -d%d -P%P -u%u -g%g -- %E
kernel.core_pipe_limit = 10
kernel.core_uses_pid = 1
kernel.ctrl-alt-del = 0
kernel.dmesg_restrict = 1
kernel.domainname = (none)
kernel.firmware_config.force_sysfs_fallback = 0
kernel.firmware_config.ignore_sysfs_fallback = 0
kernel.ftrace_dump_on_oops = 0
kernel.ftrace_enabled = 1
kernel.hardlockup_all_cpu_backtrace = 0
kernel.hardlockup_panic = 0
kernel.hostname = node1
kernel.hotplug =
kernel.hung_task_all_cpu_backtrace = 0
kernel.hung_task_check_count = 4194304
kernel.hung_task_check_interval_secs = 0
kernel.hung_task_panic = 0
kernel.hung_task_timeout_secs = 120
kernel.hung_task_warnings = 10
kernel.io_delay_type = 1
kernel.io_uring_disabled = 0
kernel.io_uring_group = -1
kernel.kexec_load_disabled = 0
kernel.kexec_load_limit_panic = -1
kernel.kexec_load_limit_reboot = -1
kernel.keys.gc_delay = 300
kernel.keys.maxbytes = 20000
kernel.keys.maxkeys = 200
kernel.keys.persistent_keyring_expiry = 259200
kernel.keys.root_maxbytes = 25000000
kernel.keys.root_maxkeys = 1000000
kernel.kptr_restrict = 1
kernel.max_lock_depth = 1024
kernel.max_rcu_stall_to_panic = 0
kernel.modprobe = /sbin/modprobe
kernel.modules_disabled = 0
kernel.msg_next_id = -1
kernel.msgmax = 8192
kernel.msgmnb = 16384
kernel.msgmni = 32000
kernel.ngroups_max = 65536
kernel.nmi_watchdog = 1
kernel.ns_last_pid = 22854
kernel.numa_balancing = 0
kernel.numa_balancing_promote_rate_limit_MBps = 65536
kernel.oops_all_cpu_backtrace = 0
kernel.oops_limit = 10000
kernel.osrelease = 6.5.0-45-generic
kernel.ostype = Linux
kernel.overflowgid = 65534
kernel.overflowuid = 65534
kernel.panic = 10
kernel.panic_on_io_nmi = 0
kernel.panic_on_oops = 1
kernel.panic_on_rcu_stall = 0
kernel.panic_on_unrecovered_nmi = 0
kernel.panic_on_warn = 0
kernel.panic_print = 0
kernel.perf_cpu_time_max_percent = 25
kernel.perf_event_max_contexts_per_stack = 8
kernel.perf_event_max_sample_rate = 100000
kernel.perf_event_max_stack = 127
kernel.perf_event_mlock_kb = 516
kernel.perf_event_paranoid = 4
kernel.pid_max = 4194304
kernel.poweroff_cmd = /sbin/poweroff
kernel.print-fatal-signals = 0
kernel.printk = 4       4       1       7
kernel.printk_delay = 0
kernel.printk_devkmsg = on
kernel.printk_ratelimit = 5
kernel.printk_ratelimit_burst = 10
kernel.pty.max = 4096
kernel.pty.nr = 2
kernel.pty.reserve = 1024
kernel.random.boot_id = 40dcc506-cc3e-4779-90fe-c5216c6c630a
kernel.random.entropy_avail = 256
kernel.random.poolsize = 256
kernel.random.urandom_min_reseed_secs = 60
kernel.random.uuid = 25927342-6dc1-4ae6-8bb6-1d44cc118c60
kernel.random.write_wakeup_threshold = 256
kernel.randomize_va_space = 2
kernel.real-root-dev = 0
kernel.sched_autogroup_enabled = 1
kernel.sched_cfs_bandwidth_slice_us = 5000
kernel.sched_child_runs_first = 0
kernel.sched_deadline_period_max_us = 4194304
kernel.sched_deadline_period_min_us = 100
kernel.sched_energy_aware = 1
kernel.sched_rr_timeslice_ms = 100
kernel.sched_rt_period_us = 1000000
kernel.sched_rt_runtime_us = 950000
kernel.sched_schedstats = 0
kernel.sched_util_clamp_max = 1024
kernel.sched_util_clamp_min = 1024
kernel.sched_util_clamp_min_rt_default = 1024
kernel.seccomp.actions_avail = kill_process kill_thread trap errno user_notif trace log allow
kernel.seccomp.actions_logged = kill_process kill_thread trap errno user_notif trace log
kernel.sem = 32000      1024000000      500     32000
kernel.sem_next_id = -1
kernel.shm_next_id = -1
kernel.shm_rmid_forced = 0
kernel.shmall = 18446744073692774399
kernel.shmmax = 18446744073692774399
kernel.shmmni = 4096
kernel.soft_watchdog = 1
kernel.softlockup_all_cpu_backtrace = 0
kernel.softlockup_panic = 0
kernel.split_lock_mitigate = 1
kernel.stack_tracer_enabled = 0
kernel.sysctl_writes_strict = 1
kernel.sysrq = 176
kernel.tainted = 0
kernel.task_delayacct = 0
kernel.threads-max = 62474
kernel.timer_migration = 1
kernel.traceoff_on_warning = 0
kernel.tracepoint_printk = 0
kernel.unknown_nmi_panic = 0
kernel.unprivileged_bpf_disabled = 2
kernel.unprivileged_userns_apparmor_policy = 1
kernel.unprivileged_userns_clone = 1
kernel.user_events_max = 32768
kernel.usermodehelper.bset = 4294967295 511
kernel.usermodehelper.inheritable = 4294967295  511
kernel.version = #45~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Mon Jul 15 16:40:02 UTC 2
kernel.warn_limit = 0
kernel.watchdog = 1
kernel.watchdog_cpumask = 0-3
kernel.watchdog_thresh = 10
kernel.yama.ptrace_scope = 1
net.bridge.bridge-nf-call-arptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-filter-pppoe-tagged = 0
net.bridge.bridge-nf-filter-vlan-tagged = 0
net.bridge.bridge-nf-pass-vlan-input-dev = 0
net.core.bpf_jit_enable = 1
net.core.bpf_jit_harden = 0
net.core.bpf_jit_kallsyms = 1
net.core.bpf_jit_limit = 528482304
net.core.busy_poll = 0
net.core.busy_read = 0
net.core.default_qdisc = fq_codel
net.core.dev_weight = 64
net.core.dev_weight_rx_bias = 1
net.core.dev_weight_tx_bias = 1
net.core.devconf_inherit_init_net = 0
net.core.fb_tunnels_only_for_init_net = 0
net.core.flow_limit_cpu_bitmap = 0
net.core.flow_limit_table_len = 4096
net.core.gro_normal_batch = 8
net.core.high_order_alloc_disable = 0
net.core.max_skb_frags = 17
net.core.mem_pcpu_rsv = 256
net.core.message_burst = 10
net.core.message_cost = 5
net.core.netdev_budget = 300
net.core.netdev_budget_usecs = 8000
net.core.netdev_max_backlog = 1000
net.core.netdev_rss_key = 9d:cf:a8:7d:48:fb:c2:c2:c0:69:1b:91:cb:ab:91:4c:5e:6a:b0:c3:6b:e0:cf:38:22:fb:76:9d:ca:76:f5:68:92:fb:5d:be:30:e4:f2:ec:f0:85:44:f5:e7:e6:2b:e5:01:7f:3a:d2
net.core.netdev_tstamp_prequeue = 1
net.core.netdev_unregister_timeout_secs = 10
net.core.optmem_max = 20480
net.core.rmem_default = 212992
net.core.rmem_max = 212992
net.core.rps_default_mask = 0
net.core.rps_sock_flow_entries = 0
net.core.skb_defer_max = 64
net.core.somaxconn = 16384
net.core.tstamp_allow_data = 1
net.core.txrehash = 1
net.core.warnings = 0
net.core.wmem_default = 212992
net.core.wmem_max = 212992
net.core.xfrm_acq_expires = 30
net.core.xfrm_aevent_etime = 10
net.core.xfrm_aevent_rseqth = 2
net.core.xfrm_larval_drop = 1
net.fan.version = 3
net.ipv4.cipso_cache_bucket_size = 10
net.ipv4.cipso_cache_enable = 1
net.ipv4.cipso_rbm_optfmt = 0
net.ipv4.cipso_rbm_strictvalid = 1
net.ipv4.conf.all.accept_local = 0
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.all.arp_accept = 0
net.ipv4.conf.all.arp_announce = 0
net.ipv4.conf.all.arp_evict_nocarrier = 1
net.ipv4.conf.all.arp_filter = 0
net.ipv4.conf.all.arp_ignore = 0
net.ipv4.conf.all.arp_notify = 0
net.ipv4.conf.all.bc_forwarding = 0
net.ipv4.conf.all.bootp_relay = 0
net.ipv4.conf.all.disable_policy = 0
net.ipv4.conf.all.disable_xfrm = 0
net.ipv4.conf.all.drop_gratuitous_arp = 0
net.ipv4.conf.all.drop_unicast_in_l2_multicast = 0
net.ipv4.conf.all.force_igmp_version = 0
net.ipv4.conf.all.forwarding = 1
net.ipv4.conf.all.igmpv2_unsolicited_report_interval = 10000
net.ipv4.conf.all.igmpv3_unsolicited_report_interval = 1000
net.ipv4.conf.all.ignore_routes_with_linkdown = 0
net.ipv4.conf.all.log_martians = 0
net.ipv4.conf.all.mc_forwarding = 0
net.ipv4.conf.all.medium_id = 0
net.ipv4.conf.all.promote_secondaries = 0
net.ipv4.conf.all.proxy_arp = 0
net.ipv4.conf.all.proxy_arp_pvlan = 0
net.ipv4.conf.all.route_localnet = 1
net.ipv4.conf.all.rp_filter = 2
net.ipv4.conf.all.secure_redirects = 1
net.ipv4.conf.all.send_redirects = 1
net.ipv4.conf.all.shared_media = 1
net.ipv4.conf.all.src_valid_mark = 0
net.ipv4.conf.all.tag = 0
net.ipv4.conf.default.accept_local = 0
net.ipv4.conf.default.accept_redirects = 1
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.conf.default.arp_accept = 0
net.ipv4.conf.default.arp_announce = 0
net.ipv4.conf.default.arp_evict_nocarrier = 1
net.ipv4.conf.default.arp_filter = 0
net.ipv4.conf.default.arp_ignore = 0
net.ipv4.conf.default.arp_notify = 0
net.ipv4.conf.default.bc_forwarding = 0
net.ipv4.conf.default.bootp_relay = 0
net.ipv4.conf.default.disable_policy = 0
net.ipv4.conf.default.disable_xfrm = 0
net.ipv4.conf.default.drop_gratuitous_arp = 0
net.ipv4.conf.default.drop_unicast_in_l2_multicast = 0
net.ipv4.conf.default.force_igmp_version = 0
net.ipv4.conf.default.forwarding = 1
net.ipv4.conf.default.igmpv2_unsolicited_report_interval = 10000
net.ipv4.conf.default.igmpv3_unsolicited_report_interval = 1000
net.ipv4.conf.default.ignore_routes_with_linkdown = 0
net.ipv4.conf.default.log_martians = 0
net.ipv4.conf.default.mc_forwarding = 0
net.ipv4.conf.default.medium_id = 0
net.ipv4.conf.default.promote_secondaries = 1
net.ipv4.conf.default.proxy_arp = 0
net.ipv4.conf.default.proxy_arp_pvlan = 0
net.ipv4.conf.default.route_localnet = 0
net.ipv4.conf.default.rp_filter = 2
net.ipv4.conf.default.secure_redirects = 1
net.ipv4.conf.default.send_redirects = 1
net.ipv4.conf.default.shared_media = 1
net.ipv4.conf.default.src_valid_mark = 0
net.ipv4.conf.default.tag = 0
net.ipv4.conf.docker0.accept_local = 0
net.ipv4.conf.docker0.accept_redirects = 1
net.ipv4.conf.docker0.accept_source_route = 0
net.ipv4.conf.docker0.arp_accept = 0
net.ipv4.conf.docker0.arp_announce = 0
net.ipv4.conf.docker0.arp_evict_nocarrier = 1
net.ipv4.conf.docker0.arp_filter = 0
net.ipv4.conf.docker0.arp_ignore = 0
net.ipv4.conf.docker0.arp_notify = 0
net.ipv4.conf.docker0.bc_forwarding = 0
net.ipv4.conf.docker0.bootp_relay = 0
net.ipv4.conf.docker0.disable_policy = 0
net.ipv4.conf.docker0.disable_xfrm = 0
net.ipv4.conf.docker0.drop_gratuitous_arp = 0
net.ipv4.conf.docker0.drop_unicast_in_l2_multicast = 0
net.ipv4.conf.docker0.force_igmp_version = 0
net.ipv4.conf.docker0.forwarding = 1
net.ipv4.conf.docker0.igmpv2_unsolicited_report_interval = 10000
net.ipv4.conf.docker0.igmpv3_unsolicited_report_interval = 1000
net.ipv4.conf.docker0.ignore_routes_with_linkdown = 0
net.ipv4.conf.docker0.log_martians = 0
net.ipv4.conf.docker0.mc_forwarding = 0
net.ipv4.conf.docker0.medium_id = 0
net.ipv4.conf.docker0.promote_secondaries = 1
net.ipv4.conf.docker0.proxy_arp = 0
net.ipv4.conf.docker0.proxy_arp_pvlan = 0
net.ipv4.conf.docker0.route_localnet = 0
net.ipv4.conf.docker0.rp_filter = 2
net.ipv4.conf.docker0.secure_redirects = 1
net.ipv4.conf.docker0.send_redirects = 1
net.ipv4.conf.docker0.shared_media = 1
net.ipv4.conf.docker0.src_valid_mark = 0
net.ipv4.conf.docker0.tag = 0
net.ipv4.conf.ens33.accept_local = 0
net.ipv4.conf.ens33.accept_redirects = 1
net.ipv4.conf.ens33.accept_source_route = 0
net.ipv4.conf.ens33.arp_accept = 0
net.ipv4.conf.ens33.arp_announce = 0
net.ipv4.conf.ens33.arp_evict_nocarrier = 1
net.ipv4.conf.ens33.arp_filter = 0
net.ipv4.conf.ens33.arp_ignore = 0
net.ipv4.conf.ens33.arp_notify = 0
net.ipv4.conf.ens33.bc_forwarding = 0
net.ipv4.conf.ens33.bootp_relay = 0
net.ipv4.conf.ens33.disable_policy = 0
net.ipv4.conf.ens33.disable_xfrm = 0
net.ipv4.conf.ens33.drop_gratuitous_arp = 0
net.ipv4.conf.ens33.drop_unicast_in_l2_multicast = 0
net.ipv4.conf.ens33.force_igmp_version = 0
net.ipv4.conf.ens33.forwarding = 1
net.ipv4.conf.ens33.igmpv2_unsolicited_report_interval = 10000
net.ipv4.conf.ens33.igmpv3_unsolicited_report_interval = 1000
net.ipv4.conf.ens33.ignore_routes_with_linkdown = 0
net.ipv4.conf.ens33.log_martians = 0
net.ipv4.conf.ens33.mc_forwarding = 0
net.ipv4.conf.ens33.medium_id = 0
net.ipv4.conf.ens33.promote_secondaries = 1
net.ipv4.conf.ens33.proxy_arp = 0
net.ipv4.conf.ens33.proxy_arp_pvlan = 0
net.ipv4.conf.ens33.route_localnet = 0
net.ipv4.conf.ens33.rp_filter = 2
net.ipv4.conf.ens33.secure_redirects = 1
net.ipv4.conf.ens33.send_redirects = 1
net.ipv4.conf.ens33.shared_media = 1
net.ipv4.conf.ens33.src_valid_mark = 0
net.ipv4.conf.ens33.tag = 0
net.ipv4.conf.lo.accept_local = 0
net.ipv4.conf.lo.accept_redirects = 1
net.ipv4.conf.lo.accept_source_route = 0
net.ipv4.conf.lo.arp_accept = 0
net.ipv4.conf.lo.arp_announce = 0
net.ipv4.conf.lo.arp_evict_nocarrier = 1
net.ipv4.conf.lo.arp_filter = 0
net.ipv4.conf.lo.arp_ignore = 0
net.ipv4.conf.lo.arp_notify = 0
net.ipv4.conf.lo.bc_forwarding = 0
net.ipv4.conf.lo.bootp_relay = 0
net.ipv4.conf.lo.disable_policy = 1
net.ipv4.conf.lo.disable_xfrm = 1
net.ipv4.conf.lo.drop_gratuitous_arp = 0
net.ipv4.conf.lo.drop_unicast_in_l2_multicast = 0
net.ipv4.conf.lo.force_igmp_version = 0
net.ipv4.conf.lo.forwarding = 1
net.ipv4.conf.lo.igmpv2_unsolicited_report_interval = 10000
net.ipv4.conf.lo.igmpv3_unsolicited_report_interval = 1000
net.ipv4.conf.lo.ignore_routes_with_linkdown = 0
net.ipv4.conf.lo.log_martians = 0
net.ipv4.conf.lo.mc_forwarding = 0
net.ipv4.conf.lo.medium_id = 0
net.ipv4.conf.lo.promote_secondaries = 1
net.ipv4.conf.lo.proxy_arp = 0
net.ipv4.conf.lo.proxy_arp_pvlan = 0
net.ipv4.conf.lo.route_localnet = 0
net.ipv4.conf.lo.rp_filter = 2
net.ipv4.conf.lo.secure_redirects = 1
net.ipv4.conf.lo.send_redirects = 1
net.ipv4.conf.lo.shared_media = 1
net.ipv4.conf.lo.src_valid_mark = 0
net.ipv4.conf.lo.tag = 0
net.ipv4.conf.tunl0.accept_local = 0
net.ipv4.conf.tunl0.accept_redirects = 1
net.ipv4.conf.tunl0.accept_source_route = 0
net.ipv4.conf.tunl0.arp_accept = 0
net.ipv4.conf.tunl0.arp_announce = 0
net.ipv4.conf.tunl0.arp_evict_nocarrier = 1
net.ipv4.conf.tunl0.arp_filter = 0
net.ipv4.conf.tunl0.arp_ignore = 0
net.ipv4.conf.tunl0.arp_notify = 0
net.ipv4.conf.tunl0.bc_forwarding = 0
net.ipv4.conf.tunl0.bootp_relay = 0
net.ipv4.conf.tunl0.disable_policy = 0
net.ipv4.conf.tunl0.disable_xfrm = 0
net.ipv4.conf.tunl0.drop_gratuitous_arp = 0
net.ipv4.conf.tunl0.drop_unicast_in_l2_multicast = 0
net.ipv4.conf.tunl0.force_igmp_version = 0
net.ipv4.conf.tunl0.forwarding = 1
net.ipv4.conf.tunl0.igmpv2_unsolicited_report_interval = 10000
net.ipv4.conf.tunl0.igmpv3_unsolicited_report_interval = 1000
net.ipv4.conf.tunl0.ignore_routes_with_linkdown = 0
net.ipv4.conf.tunl0.log_martians = 0
net.ipv4.conf.tunl0.mc_forwarding = 0
net.ipv4.conf.tunl0.medium_id = 0
net.ipv4.conf.tunl0.promote_secondaries = 1
net.ipv4.conf.tunl0.proxy_arp = 0
net.ipv4.conf.tunl0.proxy_arp_pvlan = 0
net.ipv4.conf.tunl0.route_localnet = 0
net.ipv4.conf.tunl0.rp_filter = 2
net.ipv4.conf.tunl0.secure_redirects = 1
net.ipv4.conf.tunl0.send_redirects = 1
net.ipv4.conf.tunl0.shared_media = 1
net.ipv4.conf.tunl0.src_valid_mark = 0
net.ipv4.conf.tunl0.tag = 0
net.ipv4.fib_multipath_hash_fields = 7
net.ipv4.fib_multipath_hash_policy = 0
net.ipv4.fib_multipath_use_neigh = 0
net.ipv4.fib_notify_on_flag_change = 0
net.ipv4.fib_sync_mem = 524288
net.ipv4.fwmark_reflect = 0
net.ipv4.icmp_echo_enable_probe = 0
net.ipv4.icmp_echo_ignore_all = 0
net.ipv4.icmp_echo_ignore_broadcasts = 1
net.ipv4.icmp_errors_use_inbound_ifaddr = 0
net.ipv4.icmp_ignore_bogus_error_responses = 1
net.ipv4.icmp_msgs_burst = 50
net.ipv4.icmp_msgs_per_sec = 1000
net.ipv4.icmp_ratelimit = 1000
net.ipv4.icmp_ratemask = 6168
net.ipv4.igmp_link_local_mcast_reports = 1
net.ipv4.igmp_max_memberships = 20
net.ipv4.igmp_max_msf = 10
net.ipv4.igmp_qrv = 2
net.ipv4.inet_peer_maxttl = 600
net.ipv4.inet_peer_minttl = 120
net.ipv4.inet_peer_threshold = 65664
net.ipv4.ip_autobind_reuse = 0
net.ipv4.ip_default_ttl = 64
net.ipv4.ip_dynaddr = 0
net.ipv4.ip_early_demux = 1
net.ipv4.ip_forward = 1
net.ipv4.ip_forward_update_priority = 1
net.ipv4.ip_forward_use_pmtu = 0
net.ipv4.ip_local_port_range = 32768    60999
net.ipv4.ip_local_reserved_ports =
net.ipv4.ip_no_pmtu_disc = 0
net.ipv4.ip_nonlocal_bind = 0
net.ipv4.ip_unprivileged_port_start = 1024
net.ipv4.ipfrag_high_thresh = 4194304
net.ipv4.ipfrag_low_thresh = 3145728
net.ipv4.ipfrag_max_dist = 64
net.ipv4.ipfrag_secret_interval = 0
net.ipv4.ipfrag_time = 30
net.ipv4.neigh.default.anycast_delay = 100
net.ipv4.neigh.default.app_solicit = 0
net.ipv4.neigh.default.base_reachable_time_ms = 30000
net.ipv4.neigh.default.delay_first_probe_time = 5
net.ipv4.neigh.default.gc_interval = 30
net.ipv4.neigh.default.gc_stale_time = 60
net.ipv4.neigh.default.gc_thresh1 = 128
net.ipv4.neigh.default.gc_thresh2 = 512
net.ipv4.neigh.default.gc_thresh3 = 1024
net.ipv4.neigh.default.interval_probe_time_ms = 5000
net.ipv4.neigh.default.locktime = 100
net.ipv4.neigh.default.mcast_resolicit = 0
net.ipv4.neigh.default.mcast_solicit = 3
net.ipv4.neigh.default.proxy_delay = 80
net.ipv4.neigh.default.proxy_qlen = 64
net.ipv4.neigh.default.retrans_time_ms = 1000
net.ipv4.neigh.default.ucast_solicit = 3
net.ipv4.neigh.default.unres_qlen = 101
net.ipv4.neigh.default.unres_qlen_bytes = 212992
net.ipv4.neigh.docker0.anycast_delay = 100
net.ipv4.neigh.docker0.app_solicit = 0
net.ipv4.neigh.docker0.base_reachable_time_ms = 30000
net.ipv4.neigh.docker0.delay_first_probe_time = 5
net.ipv4.neigh.docker0.gc_stale_time = 60
net.ipv4.neigh.docker0.interval_probe_time_ms = 5000
net.ipv4.neigh.docker0.locktime = 100
net.ipv4.neigh.docker0.mcast_resolicit = 0
net.ipv4.neigh.docker0.mcast_solicit = 3
net.ipv4.neigh.docker0.proxy_delay = 80
net.ipv4.neigh.docker0.proxy_qlen = 64
net.ipv4.neigh.docker0.retrans_time_ms = 1000
net.ipv4.neigh.docker0.ucast_solicit = 3
net.ipv4.neigh.docker0.unres_qlen = 101
net.ipv4.neigh.docker0.unres_qlen_bytes = 212992
net.ipv4.neigh.ens33.anycast_delay = 100
net.ipv4.neigh.ens33.app_solicit = 0
net.ipv4.neigh.ens33.base_reachable_time_ms = 30000
net.ipv4.neigh.ens33.delay_first_probe_time = 5
net.ipv4.neigh.ens33.gc_stale_time = 60
net.ipv4.neigh.ens33.interval_probe_time_ms = 5000
net.ipv4.neigh.ens33.locktime = 100
net.ipv4.neigh.ens33.mcast_resolicit = 0
net.ipv4.neigh.ens33.mcast_solicit = 3
net.ipv4.neigh.ens33.proxy_delay = 80
net.ipv4.neigh.ens33.proxy_qlen = 64
net.ipv4.neigh.ens33.retrans_time_ms = 1000
net.ipv4.neigh.ens33.ucast_solicit = 3
net.ipv4.neigh.ens33.unres_qlen = 101
net.ipv4.neigh.ens33.unres_qlen_bytes = 212992
net.ipv4.neigh.lo.anycast_delay = 100
net.ipv4.neigh.lo.app_solicit = 0
net.ipv4.neigh.lo.base_reachable_time_ms = 30000
net.ipv4.neigh.lo.delay_first_probe_time = 5
net.ipv4.neigh.lo.gc_stale_time = 60
net.ipv4.neigh.lo.interval_probe_time_ms = 5000
net.ipv4.neigh.lo.locktime = 100
net.ipv4.neigh.lo.mcast_resolicit = 0
net.ipv4.neigh.lo.mcast_solicit = 3
net.ipv4.neigh.lo.proxy_delay = 80
net.ipv4.neigh.lo.proxy_qlen = 64
net.ipv4.neigh.lo.retrans_time_ms = 1000
net.ipv4.neigh.lo.ucast_solicit = 3
net.ipv4.neigh.lo.unres_qlen = 101
net.ipv4.neigh.lo.unres_qlen_bytes = 212992
net.ipv4.neigh.tunl0.anycast_delay = 100
net.ipv4.neigh.tunl0.app_solicit = 0
net.ipv4.neigh.tunl0.base_reachable_time_ms = 30000
net.ipv4.neigh.tunl0.delay_first_probe_time = 5
net.ipv4.neigh.tunl0.gc_stale_time = 60
net.ipv4.neigh.tunl0.interval_probe_time_ms = 5000
net.ipv4.neigh.tunl0.locktime = 100
net.ipv4.neigh.tunl0.mcast_resolicit = 0
net.ipv4.neigh.tunl0.mcast_solicit = 3
net.ipv4.neigh.tunl0.proxy_delay = 80
net.ipv4.neigh.tunl0.proxy_qlen = 64
net.ipv4.neigh.tunl0.retrans_time_ms = 1000
net.ipv4.neigh.tunl0.ucast_solicit = 3
net.ipv4.neigh.tunl0.unres_qlen = 101
net.ipv4.neigh.tunl0.unres_qlen_bytes = 212992
net.ipv4.nexthop_compat_mode = 1
net.ipv4.ping_group_range = 0   2147483647
net.ipv4.raw_l3mdev_accept = 1
net.ipv4.route.error_burst = 1250
net.ipv4.route.error_cost = 250
net.ipv4.route.gc_elasticity = 8
net.ipv4.route.gc_interval = 60
net.ipv4.route.gc_min_interval = 0
net.ipv4.route.gc_min_interval_ms = 500
net.ipv4.route.gc_thresh = -1
net.ipv4.route.gc_timeout = 300
net.ipv4.route.max_size = 2147483647
net.ipv4.route.min_adv_mss = 256
net.ipv4.route.min_pmtu = 552
net.ipv4.route.mtu_expires = 600
net.ipv4.route.redirect_load = 5
net.ipv4.route.redirect_number = 9
net.ipv4.route.redirect_silence = 5120
net.ipv4.tcp_abort_on_overflow = 0
net.ipv4.tcp_adv_win_scale = 1
net.ipv4.tcp_allowed_congestion_control = reno cubic
net.ipv4.tcp_app_win = 31
net.ipv4.tcp_autocorking = 1
net.ipv4.tcp_available_congestion_control = reno cubic
net.ipv4.tcp_available_ulp = espintcp mptcp
net.ipv4.tcp_base_mss = 1024
net.ipv4.tcp_challenge_ack_limit = 2147483647
net.ipv4.tcp_child_ehash_entries = 0
net.ipv4.tcp_comp_sack_delay_ns = 1000000
net.ipv4.tcp_comp_sack_nr = 44
net.ipv4.tcp_comp_sack_slack_ns = 100000
net.ipv4.tcp_congestion_control = cubic
net.ipv4.tcp_dsack = 1
net.ipv4.tcp_early_demux = 1
net.ipv4.tcp_early_retrans = 3
net.ipv4.tcp_ecn = 2
net.ipv4.tcp_ecn_fallback = 1
net.ipv4.tcp_ehash_entries = 65536
net.ipv4.tcp_fack = 0
net.ipv4.tcp_fastopen = 1
net.ipv4.tcp_fastopen_blackhole_timeout_sec = 0
net.ipv4.tcp_fastopen_key = e61cbd5c-7f94cdf7-ff8e851b-edf45afe
net.ipv4.tcp_fin_timeout = 60
net.ipv4.tcp_frto = 2
net.ipv4.tcp_fwmark_accept = 0
net.ipv4.tcp_invalid_ratelimit = 500
net.ipv4.tcp_keepalive_intvl = 15
net.ipv4.tcp_keepalive_probes = 3
net.ipv4.tcp_keepalive_time = 600
net.ipv4.tcp_l3mdev_accept = 0
net.ipv4.tcp_limit_output_bytes = 1048576
net.ipv4.tcp_low_latency = 0
net.ipv4.tcp_max_orphans = 327680
net.ipv4.tcp_max_reordering = 300
net.ipv4.tcp_max_syn_backlog = 16384
net.ipv4.tcp_max_tw_buckets = 36000
net.ipv4.tcp_mem = 92523        123365  185046
net.ipv4.tcp_migrate_req = 0
net.ipv4.tcp_min_rtt_wlen = 300
net.ipv4.tcp_min_snd_mss = 48
net.ipv4.tcp_min_tso_segs = 2
net.ipv4.tcp_moderate_rcvbuf = 1
net.ipv4.tcp_mtu_probe_floor = 48
net.ipv4.tcp_mtu_probing = 0
net.ipv4.tcp_no_metrics_save = 0
net.ipv4.tcp_no_ssthresh_metrics_save = 1
net.ipv4.tcp_notsent_lowat = 4294967295
net.ipv4.tcp_orphan_retries = 0
net.ipv4.tcp_pacing_ca_ratio = 120
net.ipv4.tcp_pacing_ss_ratio = 200
net.ipv4.tcp_plb_cong_thresh = 128
net.ipv4.tcp_plb_enabled = 0
net.ipv4.tcp_plb_idle_rehash_rounds = 3
net.ipv4.tcp_plb_rehash_rounds = 12
net.ipv4.tcp_plb_suspend_rto_sec = 60
net.ipv4.tcp_probe_interval = 600
net.ipv4.tcp_probe_threshold = 8
net.ipv4.tcp_recovery = 1
net.ipv4.tcp_reflect_tos = 0
net.ipv4.tcp_reordering = 3
net.ipv4.tcp_retrans_collapse = 1
net.ipv4.tcp_retries1 = 3
net.ipv4.tcp_retries2 = 15
net.ipv4.tcp_rfc1337 = 0
net.ipv4.tcp_rmem = 4096        131072  6291456
net.ipv4.tcp_sack = 1
net.ipv4.tcp_shrink_window = 0
net.ipv4.tcp_slow_start_after_idle = 1
net.ipv4.tcp_stdurg = 0
net.ipv4.tcp_syn_linear_timeouts = 4
net.ipv4.tcp_syn_retries = 6
net.ipv4.tcp_synack_retries = 5
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_thin_linear_timeouts = 0
net.ipv4.tcp_timestamps = 0
net.ipv4.tcp_tso_rtt_log = 9
net.ipv4.tcp_tso_win_divisor = 3
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_window_scaling = 1
net.ipv4.tcp_wmem = 4096        16384   4194304
net.ipv4.tcp_workaround_signed_windows = 0
net.ipv4.udp_child_hash_entries = 0
net.ipv4.udp_early_demux = 1
net.ipv4.udp_hash_entries = 4096
net.ipv4.udp_l3mdev_accept = 0
net.ipv4.udp_mem = 185046       246731  370092
net.ipv4.udp_rmem_min = 4096
net.ipv4.udp_wmem_min = 4096
net.ipv4.vs.am_droprate = 10
net.ipv4.vs.amemthresh = 1024
net.ipv4.vs.backup_only = 0
net.ipv4.vs.cache_bypass = 0
net.ipv4.vs.conn_reuse_mode = 1
net.ipv4.vs.conntrack = 0
net.ipv4.vs.drop_entry = 0
net.ipv4.vs.drop_packet = 0
net.ipv4.vs.est_cpulist = 0-3
net.ipv4.vs.est_nice = 0
net.ipv4.vs.expire_nodest_conn = 0
net.ipv4.vs.expire_quiescent_template = 0
net.ipv4.vs.ignore_tunneled = 0
net.ipv4.vs.lblc_expiration = 86400
net.ipv4.vs.lblcr_expiration = 86400
net.ipv4.vs.nat_icmp_send = 0
net.ipv4.vs.pmtu_disc = 1
net.ipv4.vs.run_estimation = 1
net.ipv4.vs.schedule_icmp = 0
net.ipv4.vs.secure_tcp = 0
net.ipv4.vs.sloppy_sctp = 0
net.ipv4.vs.sloppy_tcp = 0
net.ipv4.vs.snat_reroute = 1
net.ipv4.vs.sync_persist_mode = 0
net.ipv4.vs.sync_ports = 1
net.ipv4.vs.sync_qlen_max = 62665
net.ipv4.vs.sync_refresh_period = 0
net.ipv4.vs.sync_retries = 0
net.ipv4.vs.sync_sock_size = 0
net.ipv4.vs.sync_threshold = 3  50
net.ipv4.vs.sync_version = 1
net.ipv4.xfrm4_gc_thresh = 32768
net.ipv6.anycast_src_echo_reply = 0
net.ipv6.auto_flowlabels = 1
net.ipv6.bindv6only = 0
net.ipv6.calipso_cache_bucket_size = 10
net.ipv6.calipso_cache_enable = 1
net.ipv6.conf.all.accept_dad = 0
net.ipv6.conf.all.accept_ra = 1
net.ipv6.conf.all.accept_ra_defrtr = 1
net.ipv6.conf.all.accept_ra_from_local = 0
net.ipv6.conf.all.accept_ra_min_hop_limit = 1
net.ipv6.conf.all.accept_ra_min_lft = 0
net.ipv6.conf.all.accept_ra_mtu = 1
net.ipv6.conf.all.accept_ra_pinfo = 1
net.ipv6.conf.all.accept_ra_rt_info_max_plen = 0
net.ipv6.conf.all.accept_ra_rt_info_min_plen = 0
net.ipv6.conf.all.accept_ra_rtr_pref = 1
net.ipv6.conf.all.accept_redirects = 1
net.ipv6.conf.all.accept_source_route = 0
net.ipv6.conf.all.accept_untracked_na = 0
net.ipv6.conf.all.addr_gen_mode = 0
net.ipv6.conf.all.autoconf = 1
net.ipv6.conf.all.dad_transmits = 1
net.ipv6.conf.all.disable_ipv6 = 0
net.ipv6.conf.all.disable_policy = 0
net.ipv6.conf.all.drop_unicast_in_l2_multicast = 0
net.ipv6.conf.all.drop_unsolicited_na = 0
net.ipv6.conf.all.enhanced_dad = 1
net.ipv6.conf.all.force_mld_version = 0
net.ipv6.conf.all.force_tllao = 0
net.ipv6.conf.all.forwarding = 0
net.ipv6.conf.all.hop_limit = 64
net.ipv6.conf.all.ignore_routes_with_linkdown = 0
net.ipv6.conf.all.ioam6_enabled = 0
net.ipv6.conf.all.ioam6_id = 65535
net.ipv6.conf.all.ioam6_id_wide = 4294967295
net.ipv6.conf.all.keep_addr_on_down = 0
net.ipv6.conf.all.max_addresses = 16
net.ipv6.conf.all.max_desync_factor = 600
net.ipv6.conf.all.mc_forwarding = 0
net.ipv6.conf.all.mldv1_unsolicited_report_interval = 10000
net.ipv6.conf.all.mldv2_unsolicited_report_interval = 1000
net.ipv6.conf.all.mtu = 1280
net.ipv6.conf.all.ndisc_evict_nocarrier = 1
net.ipv6.conf.all.ndisc_notify = 0
net.ipv6.conf.all.ndisc_tclass = 0
net.ipv6.conf.all.proxy_ndp = 0
net.ipv6.conf.all.ra_defrtr_metric = 1024
net.ipv6.conf.all.regen_max_retry = 3
net.ipv6.conf.all.router_probe_interval = 60
net.ipv6.conf.all.router_solicitation_delay = 1
net.ipv6.conf.all.router_solicitation_interval = 4
net.ipv6.conf.all.router_solicitation_max_interval = 3600
net.ipv6.conf.all.router_solicitations = -1
net.ipv6.conf.all.rpl_seg_enabled = 0
net.ipv6.conf.all.seg6_enabled = 0
net.ipv6.conf.all.seg6_require_hmac = 0
net.ipv6.conf.all.suppress_frag_ndisc = 1
net.ipv6.conf.all.temp_prefered_lft = 86400
net.ipv6.conf.all.temp_valid_lft = 604800
net.ipv6.conf.all.use_oif_addrs_only = 0
net.ipv6.conf.all.use_tempaddr = 2
net.ipv6.conf.default.accept_dad = 1
net.ipv6.conf.default.accept_ra = 1
net.ipv6.conf.default.accept_ra_defrtr = 1
net.ipv6.conf.default.accept_ra_from_local = 0
net.ipv6.conf.default.accept_ra_min_hop_limit = 1
net.ipv6.conf.default.accept_ra_min_lft = 0
net.ipv6.conf.default.accept_ra_mtu = 1
net.ipv6.conf.default.accept_ra_pinfo = 1
net.ipv6.conf.default.accept_ra_rt_info_max_plen = 0
net.ipv6.conf.default.accept_ra_rt_info_min_plen = 0
net.ipv6.conf.default.accept_ra_rtr_pref = 1
net.ipv6.conf.default.accept_redirects = 1
net.ipv6.conf.default.accept_source_route = 0
net.ipv6.conf.default.accept_untracked_na = 0
net.ipv6.conf.default.addr_gen_mode = 0
net.ipv6.conf.default.autoconf = 1
net.ipv6.conf.default.dad_transmits = 1
net.ipv6.conf.default.disable_ipv6 = 0
net.ipv6.conf.default.disable_policy = 0
net.ipv6.conf.default.drop_unicast_in_l2_multicast = 0
net.ipv6.conf.default.drop_unsolicited_na = 0
net.ipv6.conf.default.enhanced_dad = 1
net.ipv6.conf.default.force_mld_version = 0
net.ipv6.conf.default.force_tllao = 0
net.ipv6.conf.default.forwarding = 0
net.ipv6.conf.default.hop_limit = 64
net.ipv6.conf.default.ignore_routes_with_linkdown = 0
net.ipv6.conf.default.ioam6_enabled = 0
net.ipv6.conf.default.ioam6_id = 65535
net.ipv6.conf.default.ioam6_id_wide = 4294967295
net.ipv6.conf.default.keep_addr_on_down = 0
net.ipv6.conf.default.max_addresses = 16
net.ipv6.conf.default.max_desync_factor = 600
net.ipv6.conf.default.mc_forwarding = 0
net.ipv6.conf.default.mldv1_unsolicited_report_interval = 10000
net.ipv6.conf.default.mldv2_unsolicited_report_interval = 1000
net.ipv6.conf.default.mtu = 1280
net.ipv6.conf.default.ndisc_evict_nocarrier = 1
net.ipv6.conf.default.ndisc_notify = 0
net.ipv6.conf.default.ndisc_tclass = 0
net.ipv6.conf.default.proxy_ndp = 0
net.ipv6.conf.default.ra_defrtr_metric = 1024
net.ipv6.conf.default.regen_max_retry = 3
net.ipv6.conf.default.router_probe_interval = 60
net.ipv6.conf.default.router_solicitation_delay = 1
net.ipv6.conf.default.router_solicitation_interval = 4
net.ipv6.conf.default.router_solicitation_max_interval = 3600
net.ipv6.conf.default.router_solicitations = -1
net.ipv6.conf.default.rpl_seg_enabled = 0
net.ipv6.conf.default.seg6_enabled = 0
net.ipv6.conf.default.seg6_require_hmac = 0
net.ipv6.conf.default.suppress_frag_ndisc = 1
net.ipv6.conf.default.temp_prefered_lft = 86400
net.ipv6.conf.default.temp_valid_lft = 604800
net.ipv6.conf.default.use_oif_addrs_only = 0
net.ipv6.conf.default.use_tempaddr = 2
net.ipv6.conf.docker0.accept_dad = 1
net.ipv6.conf.docker0.accept_ra = 0
net.ipv6.conf.docker0.accept_ra_defrtr = 1
net.ipv6.conf.docker0.accept_ra_from_local = 0
net.ipv6.conf.docker0.accept_ra_min_hop_limit = 1
net.ipv6.conf.docker0.accept_ra_min_lft = 0
net.ipv6.conf.docker0.accept_ra_mtu = 1
net.ipv6.conf.docker0.accept_ra_pinfo = 1
net.ipv6.conf.docker0.accept_ra_rt_info_max_plen = 0
net.ipv6.conf.docker0.accept_ra_rt_info_min_plen = 0
net.ipv6.conf.docker0.accept_ra_rtr_pref = 1
net.ipv6.conf.docker0.accept_redirects = 1
net.ipv6.conf.docker0.accept_source_route = 0
net.ipv6.conf.docker0.accept_untracked_na = 0
net.ipv6.conf.docker0.addr_gen_mode = 0
net.ipv6.conf.docker0.autoconf = 1
net.ipv6.conf.docker0.dad_transmits = 1
net.ipv6.conf.docker0.disable_ipv6 = 0
net.ipv6.conf.docker0.disable_policy = 0
net.ipv6.conf.docker0.drop_unicast_in_l2_multicast = 0
net.ipv6.conf.docker0.drop_unsolicited_na = 0
net.ipv6.conf.docker0.enhanced_dad = 1
net.ipv6.conf.docker0.force_mld_version = 0
net.ipv6.conf.docker0.force_tllao = 0
net.ipv6.conf.docker0.forwarding = 0
net.ipv6.conf.docker0.hop_limit = 64
net.ipv6.conf.docker0.ignore_routes_with_linkdown = 0
net.ipv6.conf.docker0.ioam6_enabled = 0
net.ipv6.conf.docker0.ioam6_id = 65535
net.ipv6.conf.docker0.ioam6_id_wide = 4294967295
net.ipv6.conf.docker0.keep_addr_on_down = 0
net.ipv6.conf.docker0.max_addresses = 16
net.ipv6.conf.docker0.max_desync_factor = 600
net.ipv6.conf.docker0.mc_forwarding = 0
net.ipv6.conf.docker0.mldv1_unsolicited_report_interval = 10000
net.ipv6.conf.docker0.mldv2_unsolicited_report_interval = 1000
net.ipv6.conf.docker0.mtu = 1500
net.ipv6.conf.docker0.ndisc_evict_nocarrier = 1
net.ipv6.conf.docker0.ndisc_notify = 0
net.ipv6.conf.docker0.ndisc_tclass = 0
net.ipv6.conf.docker0.proxy_ndp = 0
net.ipv6.conf.docker0.ra_defrtr_metric = 1024
net.ipv6.conf.docker0.regen_max_retry = 3
net.ipv6.conf.docker0.router_probe_interval = 60
net.ipv6.conf.docker0.router_solicitation_delay = 1
net.ipv6.conf.docker0.router_solicitation_interval = 4
net.ipv6.conf.docker0.router_solicitation_max_interval = 3600
net.ipv6.conf.docker0.router_solicitations = -1
net.ipv6.conf.docker0.rpl_seg_enabled = 0
net.ipv6.conf.docker0.seg6_enabled = 0
net.ipv6.conf.docker0.seg6_require_hmac = 0
net.ipv6.conf.docker0.suppress_frag_ndisc = 1
net.ipv6.conf.docker0.temp_prefered_lft = 86400
net.ipv6.conf.docker0.temp_valid_lft = 604800
net.ipv6.conf.docker0.use_oif_addrs_only = 0
net.ipv6.conf.docker0.use_tempaddr = 2
net.ipv6.conf.ens33.accept_dad = 1
net.ipv6.conf.ens33.accept_ra = 0
net.ipv6.conf.ens33.accept_ra_defrtr = 1
net.ipv6.conf.ens33.accept_ra_from_local = 0
net.ipv6.conf.ens33.accept_ra_min_hop_limit = 1
net.ipv6.conf.ens33.accept_ra_min_lft = 0
net.ipv6.conf.ens33.accept_ra_mtu = 1
net.ipv6.conf.ens33.accept_ra_pinfo = 1
net.ipv6.conf.ens33.accept_ra_rt_info_max_plen = 0
net.ipv6.conf.ens33.accept_ra_rt_info_min_plen = 0
net.ipv6.conf.ens33.accept_ra_rtr_pref = 1
net.ipv6.conf.ens33.accept_redirects = 1
net.ipv6.conf.ens33.accept_source_route = 0
net.ipv6.conf.ens33.accept_untracked_na = 0
net.ipv6.conf.ens33.addr_gen_mode = 0
net.ipv6.conf.ens33.autoconf = 1
net.ipv6.conf.ens33.dad_transmits = 1
net.ipv6.conf.ens33.disable_ipv6 = 0
net.ipv6.conf.ens33.disable_policy = 0
net.ipv6.conf.ens33.drop_unicast_in_l2_multicast = 0
net.ipv6.conf.ens33.drop_unsolicited_na = 0
net.ipv6.conf.ens33.enhanced_dad = 1
net.ipv6.conf.ens33.force_mld_version = 0
net.ipv6.conf.ens33.force_tllao = 0
net.ipv6.conf.ens33.forwarding = 0
net.ipv6.conf.ens33.hop_limit = 64
net.ipv6.conf.ens33.ignore_routes_with_linkdown = 0
net.ipv6.conf.ens33.ioam6_enabled = 0
net.ipv6.conf.ens33.ioam6_id = 65535
net.ipv6.conf.ens33.ioam6_id_wide = 4294967295
net.ipv6.conf.ens33.keep_addr_on_down = 0
net.ipv6.conf.ens33.max_addresses = 16
net.ipv6.conf.ens33.max_desync_factor = 600
net.ipv6.conf.ens33.mc_forwarding = 0
net.ipv6.conf.ens33.mldv1_unsolicited_report_interval = 10000
net.ipv6.conf.ens33.mldv2_unsolicited_report_interval = 1000
net.ipv6.conf.ens33.mtu = 1500
net.ipv6.conf.ens33.ndisc_evict_nocarrier = 1
net.ipv6.conf.ens33.ndisc_notify = 0
net.ipv6.conf.ens33.ndisc_tclass = 0
net.ipv6.conf.ens33.proxy_ndp = 0
net.ipv6.conf.ens33.ra_defrtr_metric = 1024
net.ipv6.conf.ens33.regen_max_retry = 3
net.ipv6.conf.ens33.router_probe_interval = 60
net.ipv6.conf.ens33.router_solicitation_delay = 1
net.ipv6.conf.ens33.router_solicitation_interval = 4
net.ipv6.conf.ens33.router_solicitation_max_interval = 3600
net.ipv6.conf.ens33.router_solicitations = -1
net.ipv6.conf.ens33.rpl_seg_enabled = 0
net.ipv6.conf.ens33.seg6_enabled = 0
net.ipv6.conf.ens33.seg6_require_hmac = 0
net.ipv6.conf.ens33.suppress_frag_ndisc = 1
net.ipv6.conf.ens33.temp_prefered_lft = 86400
net.ipv6.conf.ens33.temp_valid_lft = 604800
net.ipv6.conf.ens33.use_oif_addrs_only = 0
net.ipv6.conf.ens33.use_tempaddr = 0
net.ipv6.conf.lo.accept_dad = -1
net.ipv6.conf.lo.accept_ra = 1
net.ipv6.conf.lo.accept_ra_defrtr = 1
net.ipv6.conf.lo.accept_ra_from_local = 0
net.ipv6.conf.lo.accept_ra_min_hop_limit = 1
net.ipv6.conf.lo.accept_ra_min_lft = 0
net.ipv6.conf.lo.accept_ra_mtu = 1
net.ipv6.conf.lo.accept_ra_pinfo = 1
net.ipv6.conf.lo.accept_ra_rt_info_max_plen = 0
net.ipv6.conf.lo.accept_ra_rt_info_min_plen = 0
net.ipv6.conf.lo.accept_ra_rtr_pref = 1
net.ipv6.conf.lo.accept_redirects = 1
net.ipv6.conf.lo.accept_source_route = 0
net.ipv6.conf.lo.accept_untracked_na = 0
net.ipv6.conf.lo.addr_gen_mode = 0
net.ipv6.conf.lo.autoconf = 1
net.ipv6.conf.lo.dad_transmits = 1
net.ipv6.conf.lo.disable_ipv6 = 0
net.ipv6.conf.lo.disable_policy = 0
net.ipv6.conf.lo.drop_unicast_in_l2_multicast = 0
net.ipv6.conf.lo.drop_unsolicited_na = 0
net.ipv6.conf.lo.enhanced_dad = 1
net.ipv6.conf.lo.force_mld_version = 0
net.ipv6.conf.lo.force_tllao = 0
net.ipv6.conf.lo.forwarding = 0
net.ipv6.conf.lo.hop_limit = 64
net.ipv6.conf.lo.ignore_routes_with_linkdown = 0
net.ipv6.conf.lo.ioam6_enabled = 0
net.ipv6.conf.lo.ioam6_id = 65535
net.ipv6.conf.lo.ioam6_id_wide = 4294967295
net.ipv6.conf.lo.keep_addr_on_down = 0
net.ipv6.conf.lo.max_addresses = 16
net.ipv6.conf.lo.max_desync_factor = 600
net.ipv6.conf.lo.mc_forwarding = 0
net.ipv6.conf.lo.mldv1_unsolicited_report_interval = 10000
net.ipv6.conf.lo.mldv2_unsolicited_report_interval = 1000
net.ipv6.conf.lo.mtu = 65536
net.ipv6.conf.lo.ndisc_evict_nocarrier = 1
net.ipv6.conf.lo.ndisc_notify = 0
net.ipv6.conf.lo.ndisc_tclass = 0
net.ipv6.conf.lo.proxy_ndp = 0
net.ipv6.conf.lo.ra_defrtr_metric = 1024
net.ipv6.conf.lo.regen_max_retry = 3
net.ipv6.conf.lo.router_probe_interval = 60
net.ipv6.conf.lo.router_solicitation_delay = 1
net.ipv6.conf.lo.router_solicitation_interval = 4
net.ipv6.conf.lo.router_solicitation_max_interval = 3600
net.ipv6.conf.lo.router_solicitations = -1
net.ipv6.conf.lo.rpl_seg_enabled = 0
net.ipv6.conf.lo.seg6_enabled = 0
net.ipv6.conf.lo.seg6_require_hmac = 0
net.ipv6.conf.lo.suppress_frag_ndisc = 1
net.ipv6.conf.lo.temp_prefered_lft = 86400
net.ipv6.conf.lo.temp_valid_lft = 604800
net.ipv6.conf.lo.use_oif_addrs_only = 0
net.ipv6.conf.lo.use_tempaddr = -1
net.ipv6.fib_multipath_hash_fields = 7
net.ipv6.fib_multipath_hash_policy = 0
net.ipv6.fib_notify_on_flag_change = 0
net.ipv6.flowlabel_consistency = 1
net.ipv6.flowlabel_reflect = 0
net.ipv6.flowlabel_state_ranges = 0
net.ipv6.fwmark_reflect = 0
net.ipv6.icmp.echo_ignore_all = 0
net.ipv6.icmp.echo_ignore_anycast = 0
net.ipv6.icmp.echo_ignore_multicast = 0
net.ipv6.icmp.error_anycast_as_unicast = 0
net.ipv6.icmp.ratelimit = 1000
net.ipv6.icmp.ratemask = 0-1,3-127
net.ipv6.idgen_delay = 1
net.ipv6.idgen_retries = 3
net.ipv6.ioam6_id = 16777215
net.ipv6.ioam6_id_wide = 72057594037927935
net.ipv6.ip6frag_high_thresh = 4194304
net.ipv6.ip6frag_low_thresh = 3145728
net.ipv6.ip6frag_secret_interval = 0
net.ipv6.ip6frag_time = 60
net.ipv6.ip_nonlocal_bind = 0
net.ipv6.max_dst_opts_length = 2147483647
net.ipv6.max_dst_opts_number = 8
net.ipv6.max_hbh_length = 2147483647
net.ipv6.max_hbh_opts_number = 8
net.ipv6.mld_max_msf = 64
net.ipv6.mld_qrv = 2
net.ipv6.neigh.default.anycast_delay = 100
net.ipv6.neigh.default.app_solicit = 0
net.ipv6.neigh.default.base_reachable_time_ms = 30000
net.ipv6.neigh.default.delay_first_probe_time = 5
net.ipv6.neigh.default.gc_interval = 30
net.ipv6.neigh.default.gc_stale_time = 60
net.ipv6.neigh.default.gc_thresh1 = 128
net.ipv6.neigh.default.gc_thresh2 = 512
net.ipv6.neigh.default.gc_thresh3 = 1024
net.ipv6.neigh.default.interval_probe_time_ms = 5000
net.ipv6.neigh.default.locktime = 0
net.ipv6.neigh.default.mcast_resolicit = 0
net.ipv6.neigh.default.mcast_solicit = 3
net.ipv6.neigh.default.proxy_delay = 80
net.ipv6.neigh.default.proxy_qlen = 64
net.ipv6.neigh.default.retrans_time_ms = 1000
net.ipv6.neigh.default.ucast_solicit = 3
net.ipv6.neigh.default.unres_qlen = 101
net.ipv6.neigh.default.unres_qlen_bytes = 212992
net.ipv6.neigh.docker0.anycast_delay = 100
net.ipv6.neigh.docker0.app_solicit = 0
net.ipv6.neigh.docker0.base_reachable_time_ms = 30000
net.ipv6.neigh.docker0.delay_first_probe_time = 5
net.ipv6.neigh.docker0.gc_stale_time = 60
net.ipv6.neigh.docker0.interval_probe_time_ms = 5000
net.ipv6.neigh.docker0.locktime = 0
net.ipv6.neigh.docker0.mcast_resolicit = 0
net.ipv6.neigh.docker0.mcast_solicit = 3
net.ipv6.neigh.docker0.proxy_delay = 80
net.ipv6.neigh.docker0.proxy_qlen = 64
net.ipv6.neigh.docker0.retrans_time_ms = 1000
net.ipv6.neigh.docker0.ucast_solicit = 3
net.ipv6.neigh.docker0.unres_qlen = 101
net.ipv6.neigh.docker0.unres_qlen_bytes = 212992
net.ipv6.neigh.ens33.anycast_delay = 100
net.ipv6.neigh.ens33.app_solicit = 0
net.ipv6.neigh.ens33.base_reachable_time_ms = 30000
net.ipv6.neigh.ens33.delay_first_probe_time = 5
net.ipv6.neigh.ens33.gc_stale_time = 60
net.ipv6.neigh.ens33.interval_probe_time_ms = 5000
net.ipv6.neigh.ens33.locktime = 0
net.ipv6.neigh.ens33.mcast_resolicit = 0
net.ipv6.neigh.ens33.mcast_solicit = 3
net.ipv6.neigh.ens33.proxy_delay = 80
net.ipv6.neigh.ens33.proxy_qlen = 64
net.ipv6.neigh.ens33.retrans_time_ms = 1000
net.ipv6.neigh.ens33.ucast_solicit = 3
net.ipv6.neigh.ens33.unres_qlen = 101
net.ipv6.neigh.ens33.unres_qlen_bytes = 212992
net.ipv6.neigh.lo.anycast_delay = 100
net.ipv6.neigh.lo.app_solicit = 0
net.ipv6.neigh.lo.base_reachable_time_ms = 30000
net.ipv6.neigh.lo.delay_first_probe_time = 5
net.ipv6.neigh.lo.gc_stale_time = 60
net.ipv6.neigh.lo.interval_probe_time_ms = 5000
net.ipv6.neigh.lo.locktime = 0
net.ipv6.neigh.lo.mcast_resolicit = 0
net.ipv6.neigh.lo.mcast_solicit = 3
net.ipv6.neigh.lo.proxy_delay = 80
net.ipv6.neigh.lo.proxy_qlen = 64
net.ipv6.neigh.lo.retrans_time_ms = 1000
net.ipv6.neigh.lo.ucast_solicit = 3
net.ipv6.neigh.lo.unres_qlen = 101
net.ipv6.neigh.lo.unres_qlen_bytes = 212992
net.ipv6.route.gc_elasticity = 9
net.ipv6.route.gc_interval = 30
net.ipv6.route.gc_min_interval = 0
net.ipv6.route.gc_min_interval_ms = 500
net.ipv6.route.gc_thresh = 1024
net.ipv6.route.gc_timeout = 60
net.ipv6.route.max_size = 2147483647
net.ipv6.route.min_adv_mss = 1220
net.ipv6.route.mtu_expires = 600
net.ipv6.route.skip_notify_on_dev_down = 0
net.ipv6.seg6_flowlabel = 0
net.ipv6.xfrm6_gc_thresh = 32768
net.mptcp.add_addr_timeout = 120
net.mptcp.allow_join_initial_addr_port = 1
net.mptcp.checksum_enabled = 0
net.mptcp.enabled = 1
net.mptcp.pm_type = 0
net.mptcp.stale_loss_cnt = 4
net.netfilter.nf_conntrack_acct = 0
net.netfilter.nf_conntrack_buckets = 262144
net.netfilter.nf_conntrack_checksum = 1
net.netfilter.nf_conntrack_count = 100
net.netfilter.nf_conntrack_dccp_loose = 1
net.netfilter.nf_conntrack_dccp_timeout_closereq = 64
net.netfilter.nf_conntrack_dccp_timeout_closing = 64
net.netfilter.nf_conntrack_dccp_timeout_open = 43200
net.netfilter.nf_conntrack_dccp_timeout_partopen = 480
net.netfilter.nf_conntrack_dccp_timeout_request = 240
net.netfilter.nf_conntrack_dccp_timeout_respond = 480
net.netfilter.nf_conntrack_dccp_timeout_timewait = 240
net.netfilter.nf_conntrack_events = 2
net.netfilter.nf_conntrack_expect_max = 4096
net.netfilter.nf_conntrack_frag6_high_thresh = 4194304
net.netfilter.nf_conntrack_frag6_low_thresh = 3145728
net.netfilter.nf_conntrack_frag6_timeout = 60
net.netfilter.nf_conntrack_generic_timeout = 600
net.netfilter.nf_conntrack_gre_timeout = 30
net.netfilter.nf_conntrack_gre_timeout_stream = 180
net.netfilter.nf_conntrack_icmp_timeout = 30
net.netfilter.nf_conntrack_icmpv6_timeout = 30
net.netfilter.nf_conntrack_log_invalid = 0
net.netfilter.nf_conntrack_max = 2310720
net.netfilter.nf_conntrack_sctp_timeout_closed = 10
net.netfilter.nf_conntrack_sctp_timeout_cookie_echoed = 3
net.netfilter.nf_conntrack_sctp_timeout_cookie_wait = 3
net.netfilter.nf_conntrack_sctp_timeout_established = 210
net.netfilter.nf_conntrack_sctp_timeout_heartbeat_sent = 30
net.netfilter.nf_conntrack_sctp_timeout_shutdown_ack_sent = 3
net.netfilter.nf_conntrack_sctp_timeout_shutdown_recd = 3
net.netfilter.nf_conntrack_sctp_timeout_shutdown_sent = 3
net.netfilter.nf_conntrack_tcp_be_liberal = 0
net.netfilter.nf_conntrack_tcp_ignore_invalid_rst = 0
net.netfilter.nf_conntrack_tcp_loose = 1
net.netfilter.nf_conntrack_tcp_max_retrans = 3
net.netfilter.nf_conntrack_tcp_timeout_close = 10
net.netfilter.nf_conntrack_tcp_timeout_close_wait = 3600
net.netfilter.nf_conntrack_tcp_timeout_established = 432000
net.netfilter.nf_conntrack_tcp_timeout_fin_wait = 120
net.netfilter.nf_conntrack_tcp_timeout_last_ack = 30
net.netfilter.nf_conntrack_tcp_timeout_max_retrans = 300
net.netfilter.nf_conntrack_tcp_timeout_syn_recv = 60
net.netfilter.nf_conntrack_tcp_timeout_syn_sent = 120
net.netfilter.nf_conntrack_tcp_timeout_time_wait = 120
net.netfilter.nf_conntrack_tcp_timeout_unacknowledged = 300
net.netfilter.nf_conntrack_timestamp = 0
net.netfilter.nf_conntrack_udp_timeout = 30
net.netfilter.nf_conntrack_udp_timeout_stream = 120
net.netfilter.nf_flowtable_tcp_timeout = 30
net.netfilter.nf_flowtable_udp_timeout = 30
net.netfilter.nf_hooks_lwtunnel = 0
net.netfilter.nf_log.0 = NONE
net.netfilter.nf_log.1 = NONE
net.netfilter.nf_log.10 = NONE
net.netfilter.nf_log.2 = NONE
net.netfilter.nf_log.3 = NONE
net.netfilter.nf_log.4 = NONE
net.netfilter.nf_log.5 = NONE
net.netfilter.nf_log.6 = NONE
net.netfilter.nf_log.7 = NONE
net.netfilter.nf_log.8 = NONE
net.netfilter.nf_log.9 = NONE
net.netfilter.nf_log_all_netns = 0
net.nf_conntrack_max = 2310720
net.unix.max_dgram_qlen = 512
user.max_cgroup_namespaces = 31237
user.max_fanotify_groups = 128
user.max_fanotify_marks = 64780
user.max_inotify_instances = 128
user.max_inotify_watches = 89100
user.max_ipc_namespaces = 31237
user.max_mnt_namespaces = 31237
user.max_net_namespaces = 31237
user.max_pid_namespaces = 31237
user.max_time_namespaces = 31237
user.max_user_namespaces = 31237
user.max_uts_namespaces = 31237
vm.admin_reserve_kbytes = 8192
vm.compact_unevictable_allowed = 1
vm.compaction_proactiveness = 20
vm.dirty_background_bytes = 0
vm.dirty_background_ratio = 10
vm.dirty_bytes = 0
vm.dirty_expire_centisecs = 3000
vm.dirty_ratio = 20
vm.dirty_writeback_centisecs = 500
vm.dirtytime_expire_seconds = 43200
vm.extfrag_threshold = 500
vm.hugetlb_optimize_vmemmap = 0
vm.hugetlb_shm_group = 0
vm.laptop_mode = 0
vm.legacy_va_layout = 0
vm.lowmem_reserve_ratio = 256   256     32      0       0
vm.max_map_count = 65530
vm.memfd_noexec = 0
vm.memory_failure_early_kill = 0
vm.memory_failure_recovery = 1
vm.min_free_kbytes = 67584
vm.min_slab_ratio = 5
vm.min_unmapped_ratio = 1
vm.mmap_min_addr = 65536
vm.mmap_rnd_bits = 32
vm.mmap_rnd_compat_bits = 16
vm.nr_hugepages = 0
vm.nr_hugepages_mempolicy = 0
vm.nr_overcommit_hugepages = 0
vm.numa_stat = 1
vm.numa_zonelist_order = Node
vm.oom_dump_tasks = 1
vm.oom_kill_allocating_task = 0
vm.overcommit_kbytes = 0
vm.overcommit_memory = 1
vm.overcommit_ratio = 50
vm.page-cluster = 3
vm.page_lock_unfairness = 5
vm.panic_on_oom = 0
vm.percpu_pagelist_high_fraction = 0
vm.stat_interval = 1
vm.swappiness = 60
vm.unprivileged_userfaultfd = 0
vm.user_reserve_kbytes = 131072
vm.vfs_cache_pressure = 100
vm.watermark_boost_factor = 15000
vm.watermark_scale_factor = 10
vm.zone_reclaim_mode = 0