端侧部署开源项目:https://github.com/hzpzlz/EasyDeploy
一 MNN编译动态库
环境要求
- cmake(建议使用3.10或以上版本)
- protobuf(使用3.0或以上版本)
- gcc(使用4.9或以上版本)
目前所用版本:Release 2.1.0 Eager 模式 / CUDA 后端改造 / Winograd Int8 计算实现 · alibaba/MNN · GitHub
1 Linux平台x86_64
编译选项:具体参考MNN官方说明文档->推理框架Linux / macOS编译 · 语雀
cd /path/to/MNN
./schema/generate.sh
mkdir build
cd build
cmake .. && make -j8
make install //if failed by permission denied, try "sudo make install"
编译、安装完成之后,一般会在/usr/local/include/MNN目录下生成头文件(后续需要拷贝到自己的工程中);在/usr/local/lib下生成libMNN.so。可以考虑在~/.bashrc中设置
export LD_LIBRARY_PATH=LD_LIBRARY_PATH:/usr/local/lib
运行可执行文件就不需要再export所需的环境变量了
2 Android平台arm64-v8a
编译选项:具体参考MNN官方说明文档->推理框架Android编译 · 语雀
1 在https://developer.android.com/ndk/downloads下载安装NDK,我这边使用的r19c,附个下载链接吧:https://dl.google.com/android/repository/android-ndk-r19c-linux-x86_64.zip
2 在~/.bashrc中设置环境变量
export NDKROOT=/home/hzp/NDK/android-ndk-r19c
这里的NDKROOT名字可以修改,后续编译会用到
3 编译MNN
cd /path/to/MNN
cd project/android
mkdir build_64
cd build_64
../build_64.sh
编译完成后会在build_64目录下生成Android所需的依赖libMNN.so,这里不需要
make install了,不然会把linux下的覆盖掉,只需要.so文件即可。
二 Opencv编译动态库
Opencv官方Git:GitHub - opencv/opencv: Open Source Computer Vision Library
目前所用版本:Opencv4.5.3,附下载链接 https://github.com/opencv/opencv/archive/refs/tags/4.5.3.zip
1 Linux平台x86_64
简化版本的编译,反正可以支持项目使用了0.0
cd path/to/opencv4.5.3
mkdir build
cd build
cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local ..
make -j8
sudo make install
更加复杂的编译选项可以参照OpenCV: OpenCV configuration options reference
2 Android平台arm64-v8a
该来的总是要来,终究是逃不过,搞个相对复杂些的版本。
cd path/to/opencv4.5.3
vi build_64.sh
//写入以下代码
#!/bin/bash
mkdir build_android
cd build_android
cmake -DCMAKE_BUILD_WITH_INSTALL_RPATH=ON \
-DCMAKE_TOOLCHAIN_FILE=${NDKROOT}/build/cmake/android.toolchain.cmake \
-DANDROID_NDK=${NDKROOT} \
-DANDROID_NATIVE_API_LEVEL=24 \
-DANDROID_ABI="arm64-v8a" \
-DWITH_CUDA=OFF \
-DWITH_MATLAB=OFF \
-DANDROID_STL=c++_shared \
-DBUILD_ANDROID_PROJECTS=OFF \
-DBUILD_ANDROID_EXAMPLES=OFF \
-DBUILD_DOCS=OFF \
-DBUILD_OPENCV_JAVA=OFF \
-DBUILD_PERF_TESTS=OFF \
-DBUILD_TESTS=OFF \
-DBUILD_SHARED_LIBS=ON \
-DCMAKE_INSTALL_PREFIX=/home/hzp/opencv-4.5.3/opencv_android_install/ \
../
make -j8
make install
最后一行的-DCMAKE_INSTALL_PREFIX=/home/hzp/opencv-4.5.3/opencv_android_install/ \
为安装路径,设置成你想要安装的路径即可,里面的东西后续也会用到。tips:尽量不要设置成/usr/local,会覆盖Linux版本。
三 构建自己的项目
终于来到了这个令人激动人心的环节0.0
附上Git上的项目链接,还有很多地方需要完善:GitHub - hzpzlz/EasyDeploy: To reduce deployment costs and speed up project schedules
1 构建CMakeLists.txt
这里简单讲一讲从0开始搭建CMakeLists.txt吧,具体参考Git中的CMakeLists.txt
cmake所需的最低版本和项目名称
cmake_minimum_required(VERSION 3.10.2)
project(segment)
控制项目运行的平台
if(ANDROID)
set(ARCH ${ANDROID_ABI})
message(STATUS "ANDROID")
elseif(APPLE)
message(STATUS "APPLE")
elseif(WIN32)
message(STATUS "WIN32")
elseif(UNIX)
set(ARCH "x86_64")
message(STATUS "UNIX")
else()
message(FATAL_ERROR "OTHER")
endif()
设置头文件路径
if(HPC_BACKEND STREQUAL "MNN")
set(HPC_INC ${CMAKE_SOURCE_DIR}/dependency/inc/${HPC_BACKEND}/)
elseif(HPC_BACKEND STREQUAL "NCNN")
set(HPC_INC ${CMAKE_SOURCE_DIR}/dependency/inc/${HPC_BACKEND}/)
endif()
设置依赖库路径
if(HPC_BACKEND STREQUAL "MNN")
set(HPC_LIB ${CMAKE_SOURCE_DIR}/dependency/libs/${ARCH}/${HPC_BACKEND}/libMNN.so)
# set(HPC_LIB ${CMAKE_SOURCE_DIR}/dependency/libs/${ARCH}/${HPC_BACKEND})
# LINK_DIRECTORIES(${HPC_LIB})
elseif(HPC_BACKEND STREQUAL "NCNN")
set(HPC_LIB ${CMAKE_SOURCE_DIR}/dependency/libs/${ARCH}/${HPC_BACKEND}/libNCNN.so)
endif()
设置opencv相关的头文件和依赖库
##opencv 头文件
set(OPENCV_INC ${CMAKE_SOURCE_DIR}/dependency/inc/opencv4/)
##opencv 依赖库 多个.so文件,用FILE
FILE(GLOB_RECURSE OPENCV_LIB ${CMAKE_SOURCE_DIR}/dependency/libs/${ARCH}/opencv2/*.so)
include头文件,这里有两个,分别是MNN和opencv
include_directories(${OPENCV_INC})
include_directories(${HPC_INC})
编写的demo
FILE(GLOB_RECURSE mnist_demo_cpp ${CMAKE_SOURCE_DIR}/src/mnist_demo.cpp)
FILE(GLOB_RECURSE seg_demo_cpp ${CMAKE_SOURCE_DIR}/src/seg_demo.cpp)
生成动态库
#动态库
add_library(
seglib
SHARED //SHARED:动态库 STATIC:静态库
${seg_demo_cpp}
)
#链接所需的lib
target_link_libraries(
seglib
${HPC_LIB}
${OPENCV_LIB}
)
生成可执行文件
#生成可执行文件
add_executable(
segDemo
${seg_demo_cpp}
)
#链接所需的lib
target_link_libraries(
segDemo
${HPC_LIB}
${OPENCV_LIB}
)
安装
##install
set(CMAKE_INSTALL_PREFIX ${CMAKE_SOURCE_DIR}/build/install)
set(CMAKE_INSTALL_BINDIR ${CMAKE_INSTALL_PREFIX}/bin/)
set(CMAKE_INSTALL_LIBDIR ${CMAKE_INSTALL_PREFIX}/lib/)
# set(CMAKE_INSTALL_INCLUDEDIR ${CMAKE_INSTALL_PREFIX}/inc/)
set(path_demo_output ${CMAKE_INSTALL_PREFIX}/demo/)
install(
TARGETS segDemo seglib
RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR}
LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR}
ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR}
)
# install(DIRECTORY ${path_inc_output} DESTINATION ${CMAKE_INSTALL_INCLUDEDIR})
install(FILES ${seg_demo_cpp} DESTINATION ${path_demo_output})
2 放置头文件和依赖库
项目中的头文件和依赖库放置目录
a.头文件:
/dependency/inc目录下存放所需的头文件
MNN:/dependency/inc/MNN目录存放MNN所需头文件,不过因为MNN里面的头文件includ时是#include <MNN/ErrorCode.hpp>这种形式,所以要在/dependency/inc/MNN目录下再新建一个MNN目录,然后执行
cp -r /usr/local/include/MNN/* /dependency/inc/MNN/MNN/
Opencv:也是类似的操作,/dependency/inc目录下新建opencv4,然后执行
/usr/local/include/opencv4/* /dependency/inc/opencv4/
b.依赖库:
/dependency/libs目录下存放所需的依赖
以Linux平台为例,/dependency/libs/x86_64,还是直接截图吧,清晰明了
MNN:
/usr/local/lib/libMNN.so /dependency/libs/x86_64/MNN/
Opencv:
/usr/local/lib/libopencv*.so /dependency/libs/x86_64/opencv2/
Android是类似的,只不过拷贝的是Android对应的编译结果,在此就不赘述了。
四 模型转换
使用MNN的工具将onnx转化为mnn所需的模型,编译的时候将编译选项MNN_BUILD_CONVERTER设置当成ON,在build下面就生成MNNConvert工具
执行命令
./MNNConvert -f ONNX --modelFile XXX.onnx --MNNModel XXX.mnn
将onnx转化为mnn模型
五 编写demo
一般包括前后处理和推理引擎调用,以分割为例
头文件和命名空间
#include "MNN/Interpreter.hpp"
#include "MNN/MNNDefine.h"
#include "MNN/Tensor.hpp"
#include <math.h>
#include <opencv2/opencv.hpp>
#include <iostream>
#include <stdio.h>
using namespace MNN;
读取图像和简单的预处理:
//使用opencv读取图像
cv::Mat raw_image = cv::imread(image_name.c_str());
int raw_image_height = raw_image.rows;
int raw_image_width = raw_image.cols;
//将图像resize到模型大小
cv::Mat image;
cv::resize(raw_image, image, cv::Size(MODEL_SIZE, MODEL_SIZE));
printf("input img size: %d %d \n", raw_image_height, raw_image_width);
// preprocessing
image.convertTo(image, CV_32FC3, 1.0);
//image = image / 255.0f;
std::cout<<"img channel: "<<image.channels()<<std::endl;
初始化MNN所需参数:
//初始化参数
int precision = 2;
int power = 0;
int memory = 0;
int threads = 1;
int MODEL_SIZE = 768; //模型输入大小是768*768
从.mnn加载模型
std::shared_ptr<MNN::Interpreter> net;
net.reset(MNN::Interpreter::createFromFile(model_name.c_str()));
char *version = (char *)net->getModelVersion();
printf("MNN version: %s \n", version);
Init和createSession
MNN::ScheduleConfig config;
config.numThread = threads;
config.type = static_cast<MNNForwardType>(forward);
MNN::BackendConfig backendConfig;
backendConfig.precision = (MNN::BackendConfig::PrecisionMode)precision;
backendConfig.power = (MNN::BackendConfig::PowerMode) power;
backendConfig.memory = (MNN::BackendConfig::MemoryMode) memory;
config.backendConfig = &backendConfig;
auto session = net->createSession(config);
net->releaseModel();
MNN输入
// wrapping input tensor, convert nhwc to nchw
std::vector<int> dims{1, MODEL_SIZE, MODEL_SIZE, 3};
auto nhwc_Tensor = MNN::Tensor::create<float>(dims, NULL, MNN::Tensor::TENSORFLOW);
auto nhwc_data = nhwc_Tensor->host<float>();
auto nhwc_size = nhwc_Tensor->size();
::memcpy(nhwc_data, image.data, nhwc_size);
std::string input_tensor = "input"; //转onnx时设置的inputtensor
auto inputTensor = net->getSessionInput(session, nullptr);
inputTensor->copyFromHostTensor(nhwc_Tensor);
run
net->runSession(session);
获得MNN输出,使用opencv保存图像
// get output data
std::string output_tensor_name0 = "output"; //转onnx时设置的outputtensor
MNN::Tensor *out_tensor = net->getSessionOutput(session, output_tensor_name0.c_str());
MNN::Tensor out_tensor_host(out_tensor, out_tensor->getDimensionType());
if (out_tensor->getDimensionType() == MNN::Tensor::TENSORFLOW) {
std::cout<<"using tpye TENSORFLOW"<<std::endl;
}
else if (out_tensor->getDimensionType() == MNN::Tensor::CAFFE) {
std::cout<<"using tpye CAFFE"<<std::endl;
}
out_tensor->copyToHostTensor(&out_tensor_host);
std::cout<<"host tensor size: "<<out_tensor_host.size()<<std::endl;
auto out_data = out_tensor_host.host<float>();
cv::Mat seg_res(MODEL_SIZE, MODEL_SIZE, CV_32FC1, cv::Scalar(255));
::memcpy(seg_res.data, out_data, MODEL_SIZE * MODEL_SIZE * sizeof(float));
cv::imwrite("../assets/seg.jpg", seg_res * 255.0);
一键运行:
#!/bin/bash
####目前仅使用于linux和phone的CPU后端,后续会更新DSP和GPU
ARCH='linux' # "linux" "android"
mkdir build
cd build
rm * -rf
../build.sh $ARCH
make -j8
make install
###for linux
if [ "${ARCH}" = "linux" ]; then
echo "linux ********"
./install/bin/segDemo
...
...
...
...
...
...
输入:
输出结果:
enjoy!