kubesphere & K8S & Kubeedge
安装配置:一键安装
kubeKey安装 kubesphere&kubernertest:
curl -sfL https://get-kk.kubesphere.io | VERSION=v2.2.1 sh -
chmod +x kk
apt-get update
apt install conntrack && apt install socat
./kk create cluster --with-kubernetes v1.22.0 --with-kubesphere v3.3.0
PS. 目前:用kk工具管理集群是最方便的;
K8S集群直接创建网络方面有问题,storageClass没有办法配置,最方便的方法还是kk管理集群
#从已有集群创建文件
./kk create config --from-cluster
./kk create config -f ~/cluster_sed.yam
#目前最优的方案是先创建集群 && add 节点
./kk create cluster --with-kubernetes v1.22.0 --with-kubesphere v3.3.0 && ./kk add nodes -f cluster_set.yaml
#检查集群初始化是否完成
kubectl logs -n kubesphere-system $(kubectl get pod -n kubesphere-system -l 'app in (ks-install, ks-installer)' -o jsonpath='{.items[0].metadata.name}') -f
#删除集群
./kk delete cluster
#!/bin/cluster_set.yaml
apiVersion: kubekey.kubesphere.io/v1alpha2
kind: Cluster
metadata:
name: sample
spec:
hosts:
##You should complete the ssh information of the hosts
- {name: 4core-01, address: 158.39.74.172, internalAddress: 158.39.74.172}
roleGroups:
etcd:
- 4core-01
master:
- 4core-01
worker:
- 4core-02
controlPlaneEndpoint:
##Internal loadbalancer for apiservers
#internalLoadbalancer: haproxy
##If the external loadbalancer was used, 'address' should be set to loadbalancer's ip.
domain: lb.kubesphere.local
address: ""
port: 6443
kubernetes:
version: v1.22.0
clusterName: cluster.local
proxyMode: ipvs
masqueradeAll: false
maxPods: 110
nodeCidrMaskSize: 24
network:
plugin: calico
kubePodsCIDR: 10.233.64.0/18
kubeServiceCIDR: 10.233.0.0/18
registry:
privateRegistry: ""
把公钥写入authoca文件中,集群会默认接通
ssh-keygen -t rsa
#生成密钥,私钥放在本地,公钥写进autho文件中
配置各种应用:
kubectl edit cc -n kubesphere-system
. opentrix
. kubeedge
. metrics-service
. dev
错误排查:(主要问题多是资源不足)
"1. check the storage configuration and storage server",
"2. make sure the DNS address in /etc/resolv.conf is available",
"3. execute 'kubectl logs -n kubesphere-system -l job-name=minio-make-bucket-job' to watch logs",
"4. execute 'helm -n kubesphere-system uninstall ks-minio && kubectl -n kubesphere-system delete job minio-make-bucket-job'",
"5. Restart the installer pod in kubesphere-system namespace"
部署kubeedge
开启kubeedge选项 -> 加入节点
端口转发规则配置:
#iptable要设置在edge host上,将本地10002 port 转发到cloudport的 :30002port上
iptables -t nat -A OUTPUT -p tcp --dport 10000 -j DNAT --to $CLOUDCOREIPS:30000
iptables -t nat -A OUTPUT -p tcp --dport 10001 -j DNAT --to $CLOUDCOREIPS:30001
iptables -t nat -A OUTPUT -p tcp --dport 10002 -j DNAT --to $CLOUDCOREIPS:30002
iptables -t nat -A OUTPUT -p tcp --dport 10350 -j DNAT --to $CLOUDCOREIPS:10003
iptables -t nat -A OUTPUT -p tcp --dport 10004 -j DNAT --to $CLOUDCOREIPS:30004
arch=$(uname -m); if [[ $arch != x86_64 ]]; then arch='arm64'; fi; curl -LO https://kubeedge.pek3b.qingstor.com/bin/v1.9.2/$arch/keadm-v1.9.2-linux-$arch.tar.gz && tar xvf keadm-v1.9.2-linux-$arch.tar.gz && chmod +x keadm && ./keadm join --kubeedge-version=1.9.2 --region=zh --cloudcore-ipport=158.39.74.211:30000 --quicport 30001 --certport 30002 --tunnelport 30004 --edgenode-name 2core-00 --edgenode-ip 158.39.201.145 --token 8746be082ce99bbc8853f7f7c3f72e0caad20d6bc170a1de607f1e92e1bff682.eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2NTk1MTU3MjR9.jhtxb6_42XhQztFgvhjudyRcWbSknAR1E2gswjDIaEI --with-edge-taint
踩坑:
- 要使得边缘节点能够被master节点监控,要修改edge节点的edgecore.service参数
注意! ipstable端口转发如果没开启规则,则要修改server端口号为30004否则监听不到
#vim /etc/kubeedge/config/edgecore.yaml
edgeStream:
enable: true #将“false”更改为“true”。
handshakeTimeout: 30
readDeadline: 15
server: xx.xxx.xxx.xxx:10004 #如果没有添加端口转发,将端口修改为30004。
tlsTunnelCAFile: /etc/kubeedge/ca/rootCA.crt
tlsTunnelCertFile: /etc/kubeedge/certs/server.crt
tlsTunnelPrivateKeyFile: /etc/kubeedge/certs/server.key
writeDeadline: 15
edged:
cgroupDriver: cgroupfs
cgroupRoot: ""
cgroupsPerQOS: true
clusterDNS: "169.254.96.16"
clusterDomain: "cluster.local"
cniBinDir: /opt/cni/bin
cniCacheDirs: /var/lib/cni/cache
cniConfDir: /etc/cni/net.d
concurrentConsumers: 5
devicePluginEnabled: false
dockerAddress: unix:///var/run/docker.sock
edgedMemoryCapacity: 7852396000
enable: true
enableMetrics: true
systemctl restart edgecore.service
- 尚不清楚 cfgroup是否会对系统构成影响
sudo tee /etc/docker/daemon.json <<EOF
{
"exec-opts": ["native.cgroupdriver=cgroupfs"],
"registry-mirrors":["http://hub-mirror.c.163.com"]
}
EOF
systemctl daemon-reload
service docker restart
docker info | grep -i cgroup
-
Edgemesh 配置
#edmesh 可以用商店部署,注意设置tolerations #agent tolerations: - key: "node-role.kubernetes.io/edge" operator: "Exists" effect: "NoSchedule"
问题记录:
edgemesh网络链接有问题,adress找不到? woker节点上的agen倒是没问题
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-iXGcKG9S-1660405035532)(https://s2.loli.net/2022/08/04/t3X2OKkq4melA9N.png)]