【实验0001】K8S环境下使用skywalking和Envoy 访问日志服务对service mesh进行观察
部分原创,部分缝合。
版本信息
- centos version: 7.9 minimal
- minikube version: v1.26.0
- kubernetes version: v1.24.1
- docker version: 20.10.21
- istioctl version: 1.14.1
- helm version: 3.9.2
硬件信息
- ESXI5.5.0 虚拟机 ☆开启VMX☆
- CPU:16c16T
- MEM:64GB
开始实验
- 安装操作系统centos 7.9 minimal 过程不表
- minikube安装
#安装工具包,方便操作
yum install -y net-tools lrzsz wget unzip
# 关闭防火墙
systemctl stop firewalld
systemctl disable firewalld
# 关闭selinux
sed -i 's/enforcing/disabled/' /etc/selinux/config # 永久
setenforce 0 # 临时
# 关闭swap
swapoff -a # 临时
sed -ri 's/.*swap.*/#&/' /etc/fstab # 永久
cat > /etc/sysctl.d/k8s.conf << EOF
net.bridge.bridge-nf-call-iptables=1
net.bridge.bridge-nf-call-ip6tables=1
net.ipv4.ip_forward=1
net.ipv4.tcp_tw_recycle=0
vm.swappiness=0 # 禁止使用 swap 空间,只有当系统 OOM 时才允许使用它
vm.overcommit_memory=1 # 不检查物理内存是否够用
vm.panic_on_oom=0 # 开启 OOM
fs.inotify.max_user_instances=8192
fs.inotify.max_user_watches=1048576
fs.file-max=52706963
fs.nr_open=52706963
net.ipv6.conf.all.disable_ipv6=1
net.netfilter.nf_conntrack_max=2310720
EOF
# 生效
sysctl --system
wget https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo -O /etc/yum.repos.d/docker-ce.repo
yum -y install docker-ce-20.10.21-3.el7
systemctl enable docker && systemctl start docker
cat > /etc/docker/daemon.json << EOF
{
"registry-mirrors": ["https://docker.mirrors.ustc.edu.cn/"],
"dns":["8.8.8.8","114.114.114.114"]
}
EOF
systemctl daemon-reload
systemctl restart docker
cat > /etc/yum.repos.d/kubernetes.repo << EOF
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF
#yum安装kubectl
yum install -y kubectl-1.24.1
#新建一个普通用户
useradd dev
passwd dev
usermod -a -G wheel dev
usermod -a -G docker dev
#切换到dev用户
su - dev
#后续命令全部使用dev用户执行
#如curl无法下载,请手动下载到本地再上传到服务器然后执行安装命令。
curl -LO https://storage.googleapis.com/minikube/releases/v1.26.0/minikube-linux-amd64
sudo install minikube-linux-amd64 /usr/local/bin/minikube
#上传minikube基础镜像(镜像下载神秘代码:1janEtbwY3TNH3a_c80npVg🐴pwov)
minikube_images_v1.24.1.tar
docker load -i minikube_images_v1.24.1.tar
minikube start --cpus=4 --memory 8192 --disk-size 32g --kubernetes-version=v1.24.1
#会下载两个镜像,一个stable_v0.0.32,一个preloaded-images-k8s-v18-v1.24.1-docker-overlay2-amd64
#第一次启动肯定失败,不要慌,不要ctrl+c!!!等待他自己失败。
#导入基础镜像到minikube(全入一遍以防万一)
minikube image load k8s.gcr.io/pause:3.7
minikube image load k8s.gcr.io/pause:3.6
minikube image load k8s.gcr.io/kube-scheduler:v1.24.1
minikube image load k8s.gcr.io/kube-proxy:v1.24.1
minikube image load k8s.gcr.io/kube-controller-manager:v1.24.1
minikube image load k8s.gcr.io/kube-apiserver:v1.24.1
minikube image load k8s.gcr.io/etcd:3.5.3-0
minikube image load k8s.gcr.io/coredns/coredns:v1.8.6
minikube image load gcr.io/k8s-minikube/storage-provisioner:v5
minikube stop
minikube start --cpus=4 --memory 8192 --disk-size 32g --kubernetes-version=v1.24.1
##给阿姨倒一杯卡不踢no!!!!!等待
##出现 Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default 完成
kubectl get no 出现minikube节点,minikube的安装到此结束。
export ISTIO_VERSION=1.7.1
curl -L https://istio.io/downloadIstio | sh -
sudo mv $PWD/istio-$ISTIO_VERSION/bin/istioctl /usr/local/bin/
istioctl install --set profile=demo
#启动访问日志服务
istioctl manifest install \
--set meshConfig.enableEnvoyAccessLogService=true \
--set meshConfig.defaultConfig.envoyAccessLogService.address=skywalking-oap.istio-system:11800
#给命名空间干上标签
kubectl label namespace default istio-injection=enabled
curl -sSLO https://get.helm.sh/helm-v3.0.0-linux-amd64.tar.gz
sudo tar xz -C /usr/local/bin --strip-components=1 linux-amd64/helm -f helm-v3.0.0-linux-amd64.tar.gz
# Clone SkyWalking Helm Chart
git clone https://ghproxy.com/https://github.com/apache/skywalking-kubernetes
cd skywalking-kubernetes/chart
git reset --hard dd749f25913830c47a97430618cefc4167612e75
# Update dependencies
helm dep up skywalking
# 发布 SkyWalking
helm -n istio-system install skywalking skywalking \
--set oap.storageType='h2'\
--set ui.image.tag=8.4.0 \
--set oap.image.tag=8.4.0-es7 \
--set oap.replicas=1 \
--set oap.env.SW_ENVOY_METRIC_ALS_HTTP_ANALYSIS=k8s-mesh \
--set oap.env.JAVA_OPTS='-Dmode=' \
--set oap.envoy.als.enabled=true \
--set elasticsearch.enabled=false
export POD_NAME=$(kubectl get pods -A -l "app=skywalking,release=skywalking,component=ui" -o name)
echo $POD_NAME
kubectl -n istio-system --address 0.0.0.0 port-forward $POD_NAME 8080:8080
##此时通过ip:port已经可以访问skywalking页面了,但是没有数据,在部署demo系统以后我们回来再看。
到此最恶心的一步来了!(可以直接跳过步骤,方法在最后)
curl -Lo skaffold https://storage.googleapis.com/skaffold/releases/v2.0.0/skaffold-linux-amd64 && \
sudo install skaffold /usr/local/bin/
git clone https://ghproxy.com/https://github.com/GoogleCloudPlatform/microservices-demo.git
- 修改代码中所有出现的githubURL前面增加代理https://ghproxy.com
- 修改pip源,增加go代理,入下图示例:
#下图中的修改的字符串如下:
RUN go env -w GOPROXY=https://goproxy.cn
https://ghproxy.com/
-i https://pypi.tuna.tsinghua.edu.cn/simple
![在这里插入图片描述](https://i-blog.csdnimg.cn/blog_migrate/6923b82f1447c03cb6c2cd1d6e08846b.png)
- 最后是进入microservices-demo目录下,直接执行
skaffold run
- 漫长的等待后,kubectl get po -n default 就会出现11个微服务了,其中各种报错,大家自行google或者百度吧。
kubectl --address 0.0.0.0 port-forward svc/frontend-external 8090:80
访问ip:port 即可访问demo了
![在这里插入图片描述](https://i-blog.csdnimg.cn/blog_migrate/9bd9429df43ec6d756b78be2ef735184.png)
此时再去看skywalking的仪表盘
![在这里插入图片描述](https://i-blog.csdnimg.cn/blog_migrate/b967f56029f055b477b2769c97656895.png)
直接下载目标镜像
docker pull gcr.io/google-samples/microservices-demo/emailservice:v0.4.1
docker pull gcr.io/google-samples/microservices-demo/checkoutservice:v0.4.1
docker pull gcr.io/google-samples/microservices-demo/recommendationservice:v0.4.1
docker pull gcr.io/google-samples/microservices-demo/frontend:v0.4.1
docker pull gcr.io/google-samples/microservices-demo/paymentservice:v0.4.1
docker pull gcr.io/google-samples/microservices-demo/productcatalogservice:v0.4.1
docker pull gcr.io/google-samples/microservices-demo/cartservice:v0.4.1
docker pull gcr.io/google-samples/microservices-demo/loadgenerator:v0.4.1
docker pull gcr.io/google-samples/microservices-demo/currencyservice:v0.4.1
docker pull gcr.io/google-samples/microservices-demo/shippingservice:v0.4.1
docker pull gcr.io/google-samples/microservices-demo/adservice:v0.4.1
minikube image load gcr.io/google-samples/microservices-demo/emailservice:v0.4.0
minikube image load gcr.io/google-samples/microservices-demo/checkoutservice:v0.4.0
minikube image load gcr.io/google-samples/microservices-demo/recommendationservice:v0.4.0
minikube image load gcr.io/google-samples/microservices-demo/frontend:v0.4.0
minikube image load gcr.io/google-samples/microservices-demo/paymentservice:v0.4.0
minikube image load gcr.io/google-samples/microservices-demo/productcatalogservice:v0.4.0
minikube image load gcr.io/google-samples/microservices-demo/cartservice:v0.4.0
minikube image load gcr.io/google-samples/microservices-demo/loadgenerator:v0.4.0
minikube image load gcr.io/google-samples/microservices-demo/currencyservice:v0.4.0
minikube image load gcr.io/google-samples/microservices-demo/shippingservice:v0.4.0
minikube image load gcr.io/google-samples/microservices-demo/adservice:v0.4.0
kubectl apply -f https://ghproxy.com/https://github.com/GoogleCloudPlatform/microservices-demo/blob/main/release/kubernetes-manifests.yaml
kubectl --address 0.0.0.0 port-forward svc/frontend-external 8090:80