RK3588 IREE+Vulkan ResNet50推理测试

背景

在边缘计算场景中，RK3588作为高性能ARM处理器，配合Vulkan图形API和IREE机器学习运行时，能够实现高效的模型推理。IREE（Intermediate Representation Execution Environment）是Google开源的机器学习模型编译器运行时，可将PyTorch/TensorFlow等框架的模型编译为可在多种硬件上运行的高效代码。Vulkan作为新一代跨平台图形API，其计算着色器特性非常适合并行计算任务。本文旨在对该方案进行摸底

一.性能数据【暂不考虑该框架】

• 模型:resnet50 输入:[1,3.224,224 float32] 输出:[1,1000 float32]

精度模式	CPU推理【性能太差】	Vulkan推理【结果不对】
FP32	3001.05ms MSE=0.00000	10791.2645 ms MSE:98008.86
FP16	4097.97ms MSE=0.00002	0.6807 ms MSE:3.67

二.操作步骤

2.1 搭建NFS服务,解决IREE编译时,空间不足的问题

ssh root@192.168.1.100
cd /home/rk3588
docker stop nfs-server
docker rm nfs-server
docker run -d --name nfs-server --privileged -e SHARED_DIRECTORY=/home \
           -v $PWD:/home -p 2049:2049 nfs_server

2.2 编译、安装IREE

2.2.1 挂载NFS

ssh root@192.168.1.101
mkdir -p /nfs
mount -t nfs -o nolock 192.168.1.100:/ /nfs/
mkdir /nfs/rk3588

2.2.2 安装依赖

apt install software-properties-common -y
add-apt-repository ppa:deadsnakes/ppa   #添加Python软件源
apt install python3.10
apt install python3.10-dev
wget https://bootstrap.pypa.io/get-pip.py
python3.10 get-pip.py
pip3.10 install numpy
/usr/bin/python3.10 -m pip install onnx
pip3.10 install torch
pip3.10 install torchvision
apt install google-mock -y
apt install ninja-build -y
apt install ccache -y

#安装cmake
cd /nfs/rk3588
wget https://github.com/Kitware/CMake/releases/download/v3.22.5/cmake-3.22.5-linux-aarch64.sh
bash cmake-3.22.5-linux-aarch64.sh --prefix=/usr/local --skip-license
export PATH=/usr/local/bin:$PATH

2.2.3 编译IREE

cd /nfs/rk3588
git clone --recursive https://github.com/google/shaderc.git
cd /nfs/rk3588/shaderc
cd /nfs/rk3588/shaderc/third_party
git clone --recursive https://github.com/KhronosGroup/SPIRV-Tools.git
mv SPIRV-Tools spirv-tools
cd spirv-tools/external
git clone --recursive https://github.com/KhronosGroup/SPIRV-Headers.git
mv SPIRV-Headers spirv-headers
cd /nfs/rk3588/shaderc/third_party
git clone --recursive https://github.com/KhronosGroup/glslang.git

cd /nfs/rk3588/shaderc
rm build -rf
mkdir build && cd build 
cmake -DSHADERC_SKIP_TESTS=ON ..
make -j4
make install

cd /nfs/rk3588/
git clone --recursive https://github.com/openxla/iree.git