k8s之wsl2+kind搭建kubevela开发集群

傅里叶、

已于 2022-12-18 11:14:54 修改

阅读量1.1k

点赞数 1

分类专栏： K8S 文章标签： kind wsl2 kubevela

于 2022-03-06 16:45:04 首次发布

本文链接：https://blog.csdn.net/qq_34562093/article/details/123312786

版权

K8S 专栏收录该内容

38 篇文章 9 订阅

订阅专栏

仅作为记录，不具备参考意义；

1、win10-1903+wsl2+ubuntu18+kind+docker desktop

wsl2和unbuntu以及docker desttop不再赘述

第一步：
集群kind01
cat <<EOF | kind create cluster --image=kindest/node:v1.20.7 --kubeconfig=/mnt/d/workspace/wsl/config --name=kind01 --config=-
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
networking:
  apiServerAddress: "127.0.0.1"
  apiServerPort: 6443
nodes:
- role: control-plane
  kubeadmConfigPatches:
  - |
    kind: InitConfiguration
    nodeRegistration:
      kubeletExtraArgs:
        node-labels: "ingress-ready=true"
  extraPortMappings:
  - containerPort: 80
    hostPort: 80
    protocol: TCP
  - containerPort: 443
    hostPort: 443
    protocol: TCP
  - containerPort: 16443
    hostPort: 16443
    protocol: TCP
  - containerPort: 2379
    hostPort: 2379
    protocol: TCP
EOF

集群kind02（可能用到）
cat <<EOF | kind create cluster --image=kindest/node:v1.20.7 --name=kind02 --config=-
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
networking:
  apiServerAddress: "127.0.0.1"
  apiServerPort: 6444
nodes:
- role: control-plane
  kubeadmConfigPatches:
  - |
    kind: InitConfiguration
    nodeRegistration:
      kubeletExtraArgs:
        node-labels: "ingress-ready=true"
EOF

第二步：
kind get clusters
cp ~/.kube/config /mnt/f/vela/(不太需要这一步)
在/etc/profile加入export KUBECONFIG=/mnt/f/vela/config


第三步：
cp /mnt/f/vela/xxx/xxx/vela /usr/local/bin/
vela version

第三点五步：
docker pull oamdev/vela-core:v1.3.1   即latest
docker pull oamdev/vela-apiserver:v1.3.1
docker pull oamdev/velaux:v1.3.1
docker pull oamdev/cluster-gateway:v1.3.2

第四步：
helm repo add kubevela https://charts.kubevela.net/core
helm repo update
helm install --create-namespace -n vela-system kubevela kubevela/vela-core --version 1.2.4 --wait --debug
或者
helm package kubevela-1.3.0/charts/vela-core --destination kubevela-1.3.0/charts/
helm install --create-namespace -n vela-system kubevela kubevela-1.3.0/charts/vela-core-0.1.0.tgz

第五步：
vela addon enable velaux --version v1.3.1
vela port-forward -n vela-system addon-velaux 9082:80
选择velaux、service且需要一直保持启动中才行！


第六步-接入新集群：
docker ps 
docker inspect 98736e2e1fa4 得到IPAddress
将config-kind01-bridge中的 server: https://127.0.0.1:6443改为IPAddress
vela cluster join /mnt/f/vela/config-kind01-bridge -n kind02


第七步：
不必要
kubectl apply -f /mnt/f/vela/script/ingress-deploy.yaml
docker pull liangjw/ingress-nginx/kube-webhook-certgen:v1.1.1
docker tag liangjw/ingress-nginx/kube-webhook-certgen:v1.1.1 k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.1.1



备注：
kind delete cluster --name kind01
rm -rf $HOME/.kube

docker cp kind01-control-plane:/etc/kubernetes /etc/

vela addon disable velaux
helm delete kubevela -n vela-system
vela uninstall

2、运行make reviewable的配置

****golangci-lint****
go install 安装版本必须是v1.38.0
go install github.com/golangci/golangci-lint/cmd/golangci-lint@v1.38.0
golangci-lint --version
golangci-lint run [目录]/[文件]
golangci-lint 在/root/go/bin目录

****kustomize****
下载地址:https://github.com/kubernetes-sigs/kustomize/releases
直接二进制包安装,版本必须是v4.5.4
cp kustomize_v4.5.4_linux_amd64/kustomize  /root/go/bin
kustomize在/root/go/bin目录
kustomize version查看版本

****staticcheck****
直接二进制包安装,版本最新版本
cp staticcheck  /root/go/bin

****readme-generator****
wget https://nodejs.org/dist/v16.14.0/node-v16.14.0-linux-x64.tar.xz
tar -xvf node-v12.16.1-linux-x64.tar.xz
mv -f node-v12.16.1-linux-x64 /usr/local/
export NODEJS_HOME=/usr/local/node-v12.16.1-linux-x64
export PATH=$PATH:$NODEJS_HOME/bin:/mnt/nodejs/node_global/bin
npm install -g readme-generator-for-helm


****项目目录****
cd /mnt/d/GoLand/workspace/src/kubevela


****make reviewable****
在项目根目录执行make reviewable

****推代码到git****
git clone git@github.com:fourierr/kubevela.git
git checkout -b env_comp02
git branch
make reviewable
git add .
git commit -s -m 'Fix: trait env and storage conflict Signed-off-by: maxiangbo maxiangboo@cmbchina.com'
git push origin gitee_comp03:gitee_comp03

备注：
git log 查看commit记录
git branch 产看本地分支
git checkout -b <分支名称> 切换远程分支：
git config --global user.name 'fourierr'
git config --global user.email 'maxiangboo@qq.com'
git remote remove origin 
git remote add origin git@github.com:fourierr/kubevela.git   

****linux和window的编码转换****
在kubevela目录下
find kubevela/ -name "*.sh" | xargs dos2unix
find kubevela/ -name "*.sh" | xargs unix2dos

3、kubevela pr的流程

配置变量KUBECONFIG=/mnt/f/vela/config
配置启动参数--enable-cluster-gateway=true

在github上的master分支 fetch upstream

在ubuntu上
cd /mnt/d/GoLand/workspace/src/
git clone git@github.com:fourierr/kubevela.git
cd kubevela

git checkout -b comp_pyroscope01
git branch

设置GoLand为LF并用GoLand编辑代码

make reviewable

git add .提交所有修改的和新建的文件 或者 
git add -u提交所有被删除和修改的文件 或者 
git add -A 提交交所有被删除、被替换、被修改和新增的文件


git commit -s -m 'Feat: Add example doc for controlPlaneOnly'

git push -f origin comp_controlPlaneOnlyDoc

第二次的commit会覆盖前一次的commit




备注：
Signed-off-by: Xiangbo Ma <maxiangboo@cmbchina.com>

重新触发ci，可以通过commit --amend再force push实现重跑
git commit --amend -s -m 'Feat: Add example doc for controlPlaneOnly'
git push -f origin comp_controlPlaneOnlyDoc

查看新增和删除远程仓库
git remote -v
git remote add origin_doc git@github.com:fourierr/kubevela.io.git
git push -f origin_doc comp_crontask01:comp_crontask01
git remote rm origin

4、kubevela.io PR流程

kubevela.io 可以直接在windows上修改文件、commit、push, 且运行时需要执行
npm install 
npm run build 
npm run start

5、backport与cherry-pick流程

带有backport的tag的pr会自己合并到release-1.3分支，但是也可能失败，失败后则需要手动cherry-pick

cherry-pick的流程：
git clone git@github.com:oam-dev/kubevela.git
cd kubevela
git checkout release-1.3
git checkout -b backport-3724-to-release-1.3
拷贝需要修改的文件
git add .
git commit -s -m 'Fix: cherry-pick #3724 to delete apprev annotation. Signed-off-by: Xiangbo Ma <maxiangboo@cmbchina.com>'
git remote add downstream git@github.com:fourier/kubevela.git
git remote -v
git push -f downstream backport-3724-to-release-1.3:backport-3724-to-release-1.3

6、开发者环境搭建

需要删除vela-system下的webhook cr，即ValidatingWebhookConfiguration、MutatingWebhookConfiguration的cr,否则由于webhook admission 无法成功下发cr

kubectl scale deploy -n vela-system kubevela-vela-core --replicas=0
kubectl delete MutatingWebhookConfiguration -n vela-system  kubevela-vela-core-admission
kubectl delete ValidatingWebhookConfiguration -n vela-system kubevela-vela-core-admission

创建clusterrolebinding解决X509 Cert
kubectl create clusterrolebinding vela-core-clusterrolebinding --clusterrole=cluster-admin --user=vela-core
在goland中设置KUBECONFIG=/mnt/f/vela/config

7、git与github配置

ssh-keygen -C 'maxiangboo@qq.com' -t rsa
cd ~/.ssh/
将“cat id_rsa.pub”得到的整段结果复制，黏贴到自己的Github中的ssh-key
ssh -v git@github.com

8、本地调试kube-apiserver

复制相关文件

docker cp kind01-control-plane:/etc/kubernetes /etc/

apiserver启动参数

--advertise-address=172.18.0.2
--allow-privileged=true
--authorization-mode=Node,RBAC
--client-ca-file=/etc/kubernetes/pki/ca.crt
--enable-admission-plugins=NodeRestriction
--enable-bootstrap-token-auth=true
--etcd-cafile=/etc/kubernetes/pki/etcd/ca.crt
--etcd-certfile=/etc/kubernetes/pki/apiserver-etcd-client.crt
--etcd-keyfile=/etc/kubernetes/pki/apiserver-etcd-client.key
--etcd-servers=https://127.0.0.1:2379
--kubelet-client-certificate=/etc/kubernetes/pki/apiserver-kubelet-client.crt
--kubelet-client-key=/etc/kubernetes/pki/apiserver-kubelet-client.key
--kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname
--proxy-client-cert-file=/etc/kubernetes/pki/front-proxy-client.crt
--proxy-client-key-file=/etc/kubernetes/pki/front-proxy-client.key
--requestheader-allowed-names=front-proxy-client
--requestheader-client-ca-file=/etc/kubernetes/pki/front-proxy-ca.crt
--requestheader-extra-headers-prefix=X-Remote-Extra-
--requestheader-group-headers=X-Remote-Group
--requestheader-username-headers=X-Remote-User
--runtime-config=
--secure-port=26443
--service-account-issuer=https://kubernetes.default.svc.cluster.local
--service-account-key-file=/etc/kubernetes/pki/sa.pub
--service-account-signing-key-file=/etc/kubernetes/pki/sa.key
--service-cluster-ip-range=10.96.0.0/16
--tls-cert-file=/etc/kubernetes/pki/apiserver.crt
--tls-private-key-file=/etc/kubernetes/pki/apiserver.key

生成token

创建sa kubectl create sa apiserver-viewer 
创建rolebinding 绑定到sa kubectl create clusterrolebinding apiadmin --clusterrole cluster-admin --serviceaccount default:apiserver-viewer 
拿到secret的名字apiserver-viewer-token-scdk8 kubectl get sa apiserver-viewer -oyaml 
查看serect中的token kubectl describe secret apiserver-viewer-token-scdk8

postman访问

127.0.0.1 不行要换成wsl的地址

curl命令

curl --location --request GET 'https://127.0.0.1:6443/api/v1/namespaces/kube-system/pods/etcd-kind01-control-plane' \
--header 'Authorization: Bearer eyJhbGciOiJSUzI1NiIsImtpZCI6InlPOVdnR01STi0xcFJXX0o4eWFYRThKMVdWTjdoWVVMTm9LWmJJeU9uY0kifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJkZWZhdWx0Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZWNyZXQubmFtZSI6ImFwaXNlcnZlci12aWV3ZXItdG9rZW4tc2NkazgiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoiYXBpc2VydmVyLXZpZXdlciIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6IjYzZTk4MmUxLWFlNDctNDJiMC04MmY5LWQ4M2Q0YTg4N2QzZSIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDpkZWZhdWx0OmFwaXNlcnZlci12aWV3ZXIifQ.E55NVZUJoExkVDqlwKNVSfNZwfIYKbEnWqCr4RgWEjJK2RGDRzA_SJSpKxUrKjmFbt7vPUFjUWkMygyAUS7vRRu38IxhwsIv51rvBIQ1R5BSRSyRX7-MQEkNp_c9vpEdxz51feWn7slipPKGVwDAjMBMBu--tYM4t2FoO-7yfeLX1tQTwCo89Iv5yElEwRCBxIvAMGXi42shETN9njDnVHK-BKuYL-HBd84qtNmDbmQ3F3t67GDr5uXYHn7tRkMf0VcDIh6nVrUG1EOYZsmdvIxVOx14xWi-ynToRESpKlGm6Sq6xhGyJ9fhUnxZMyb4d1DwKdpW42d5VFCJzd18LA'