FastDeploy是一款全场景、易用灵活、极致高效的AI推理部署工具, 支持云边端部署。提供超过 🔥160+ Text,Vision, Speech和跨模态模型📦开箱即用的部署体验,并实现🔚端到端的推理性能优化。包括 物体检测、字符识别(OCR)、人脸、人像扣图、多目标跟踪系统、NLP、Stable Diffusion文图生成、TTS 等几十种任务场景,满足开发者多场景、多硬件、多平台的产业部署需求。官网:GitHub - PaddlePaddle/FastDeploy: ⚡️An Easy-to-use and Fast Deep Learning Model Deployment Toolkit for ☁️Cloud 📱Mobile and 📹Edge. Including Image, Video, Text and Audio 20+ main stream scenarios and 150+ SOTA models with end-to-end optimization, multi-platform and multi-framework support.
遗憾的是在FreeBSD下没有装成。
发现fastdeploy需要opencv-python,所以花了很大的精力来安装,但是也没有装上。opencv可以用Pillow代替,但是后面还是碰到没法解决的问题。
编译安装opencv-python
编译安装没完成,估计还是用pkg install opencv-python装成的。
需要安装opencv-pyhton
安装opencv-python
pip install opencv-python,但是装不上,所以选择源代码编译安装
先安装pip install scikit-build
然后下载opencv-python源代码
可以用git clone https://github.com/opencv/opencv-python
也可以在pip安装的时候拿到下载链接,然后wget下载
解压源代码:
tar -xzvf opencv-python-4.9.0.80.tar.gz
设置编译多线程:
set MAX_JOBS=8
export MAX_JOBS=8
开始编译
进入 opencv-python-4.9.0.80 目录并编译
cd opencv-python-4.9.0.80
python setup.py install
编译失败,见后面的记录。
安装FastDeploy
标准流程是cpu安装:pip install numpy opencv-python fastdeploy-python -f https://www.paddlepaddle.org.cn/whl/fastdeploy.html
我们使用命令
pip install fastdeploy-python -f https://www.paddlepaddle.org.cn/whl/fastdeploy.html
这样跳过了opencv部分,先把fastdeploy装好了。
推理
Python 推理示例
准备模型和图片
wget https://bj.bcebos.com/paddlehub/fastdeploy/ppyoloe_crn_l_300e_coco.tgz
tar xvf ppyoloe_crn_l_300e_coco.tgz
wget https://gitee.com/paddlepaddle/PaddleDetection/raw/release/2.4/demo/000000014439.jpg
python推理
# GPU/TensorRT部署参考 examples/vision/detection/paddledetection/python
import cv2
import fastdeploy.vision as vision
model = vision.detection.PPYOLOE("ppyoloe_crn_l_300e_coco/model.pdmodel",
"ppyoloe_crn_l_300e_coco/model.pdiparams",
"ppyoloe_crn_l_300e_coco/infer_cfg.yml")
im = cv2.imread("000000014439.jpg")
result = model.predict(im)
print(result)
vis_im = vision.vis_detection(im, result, score_threshold=0.5)
cv2.imwrite("vis_image.jpg", vis_im)
opencv-python这里实在装不上, 用Pillow代替,但是报错:
# GPU/TensorRT部署参考 examples/vision/detection/paddledetection/python
# import cv2
from PIL import Image
import fastdeploy.vision as vision
model = vision.detection.PPYOLOE("ppyoloe_crn_l_300e_coco/model.pdmodel",
"ppyoloe_crn_l_300e_coco/model.pdiparams",
"ppyoloe_crn_l_300e_coco/infer_cfg.yml")
# im = cv2.imread("000000014439.jpg")
im = Image.open("000000014439.jpg")
result = model.predict(im)
print(result)
vis_im = vision.vis_detection(im, result, score_threshold=0.5)
# cv2.imwrite("vis_image.jpg", vis_im)
vis_im.save("vis_image.jpg")
结论:python推理失败
C++推理
预编译环境
Release版本
平台 | 文件 | 说明 |
---|---|---|
Linux x64 | fastdeploy-linux-x64-1.0.7.tgz | g++ 8.2编译产出 |
Windows x64 | fastdeploy-win-x64-1.0.7.zip | Visual Studio 16 2019编译产出 |
Mac OSX x64 | fastdeploy-osx-x86_64-1.0.7.tgz | clang++ 10.0.0编译产出 |
没有FreeBSD的,所以我们要自己编译。
进入FastDeploy目录进行编译:
cd FastDeploy
mkdri build && cd build
cmake ..
make
可以根据自己cpu的核数x,加上-j 2*x参数 ,如4核cpu make -j 8
老规矩,编译好之后加入PATH路径,而不是放入/usr/bin目录,以免污染整个系统。
发现目录结构远比想像的要复杂,还是用make install 安装吧 。切换root账户,
cmake .. -DCMAKE_INSTALL_PREFIX=/home/xxx/work/fd
make -j 8
make install
最终使用的语句是在root账户下,在FastDeploy目录执行:
mkdir build
cd build/
cmake .. -DCMAKE_INSTALL_PREFIX=/home/skywalk/work/fd -DWITH_CAPI=ON
make -j 8
make install # 第一次运行报错,所以把下面的patch库挪到install
mkdir third_libs/install
cp -rf third_libs/patchelf/ third_libs/install/
make install
这里参数漏掉一个D ,加上之后编译不过去,也是就是DWITH_CAPI=ON编译不过去,ENABLE_PADDLE_BACKEND和ENABLE_ORT_BACKEND也都过不去。
把参数全删掉可以过去,但那样就没有用了啊!
结论:编译环境失败
准备图片
wget https://bj.bcebos.com/paddlehub/fastdeploy/ppyoloe_crn_l_300e_coco.tgz
tar xvf ppyoloe_crn_l_300e_coco.tgz
wget https://gitee.com/paddlepaddle/PaddleDetection/raw/release/2.4/demo/000000014439.jpg
推理
源码
// GPU/TensorRT部署参考 examples/vision/detection/paddledetection/cpp
#include "fastdeploy/vision.h"
int main(int argc, char* argv[]) {
namespace vision = fastdeploy::vision;
auto model = vision::detection::PPYOLOE("ppyoloe_crn_l_300e_coco/model.pdmodel",
"ppyoloe_crn_l_300e_coco/model.pdiparams",
"ppyoloe_crn_l_300e_coco/infer_cfg.yml");
auto im = cv::imread("000000014439.jpg");
vision::DetectionResult res;
model.Predict(im, &res);
auto vis_im = vision::VisDetection(im, res, 0.5);
cv::imwrite("vis_image.jpg", vis_im);
return 0;
}
把文件保存为infer_demo.c, 用gcc编译报错。
到FastDeploy/examples/runtime/cpp 目录,编译
mkdir build && cd build
cmake .. -DFASTDEPLOY_INSTALL_DIR=/home/skywalk/work/fd
make -j 8
(在没有任何参数的编译出来的环境下)编译出来一个runtime_demo文件,执行直接崩了。
结论
目前fastdeploy在FreeBSD没有调通。当然在linux下是极其好用的。
调试
pip install opencv-python报错
搞不定,下载源代码手动编译安装python setup.py install
编译时报错 No module named 'skbuild'
File "/usr/home/skywalk/work/opencv-python-headless-4.9.0.80/setup.py", line 10, in <module>
from skbuild import cmaker, setup
ModuleNotFoundError: No module named 'skbuild'
pip install scikit-build
编译安装时报错
[ 31%] Building CXX object modules/dnn/CMakeFiles/opencv_dnn.dir/misc/caffe/opencv-caffe.pb.cc.o
In file included from /usr/home/skywalk/work/opencv-python-4.9.0.80/opencv/modules/dnn/misc/caffe/opencv-caffe.pb.cc:4:
In file included from /usr/home/skywalk/work/opencv-python-4.9.0.80/opencv/modules/dnn/misc/caffe/opencv-caffe.pb.h:10:
/usr/local/include/google/protobuf/port_def.inc:210:1: error: static_assert failed due to requirement '201103L >= 201402L' "Protobuf only supports C++14 and newer."
static_assert(PROTOBUF_CPLUSPLUS_MIN(201402L), "Protobuf only supports C++14 and newer.");
^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /usr/home/skywalk/work/opencv-python-4.9.0.80/opencv/modules/dnn/misc/caffe/opencv-caffe.pb.cc:4:
/usr/home/skywalk/work/opencv-python-4.9.0.80/opencv/modules/dnn/misc/caffe/opencv-caffe.pb.h:17:2: error: This file was generated by an older version of protoc which is
#error This file was generated by an older version of protoc which is
^
/usr/home/skywalk/work/opencv-python-4.9.0.80/opencv/modules/dnn/misc/caffe/opencv-caffe.pb.h:18:2: error: incompatible with your Protocol Buffer headers. Please
#error incompatible with your Protocol Buffer headers. Please
pip install protobuf==3.20试试
不行,用opencv-python4.4.
到openv目录 手工cmke .. make -j 报错
/usr/local/include/absl/strings/internal/has_absl_stringify.h:46:8: error: no template named 'enable_if_t' in namespace 'std'; did you mean simply 'enable_if_t'?
T, std::enable_if_t<std::is_void<decltype(AbslStringify(
^~~~~
[ 2%] Built target gen_opencv_python_source
/usr/local/include/absl/meta/type_traits.h:656:1: note: 'enable_if_t' declared here
using enable_if_t = typename std::enable_if<B, T>::type;
^
[ 2%] Building CXX object 3rdparty/protobuf/CMakeFiles/libprotobuf.dir/src/google/protobuf/generated_message_table_driven_lite.cc.o
In file included from /home/skywalk/work/opencv-python-4.4.0.42/opencv/3rdparty/protobuf/src/google/protobuf/arena.cc:31:
In file included from /usr/local/include/google/protobuf/arena.h:53:
In file included from /usr/local/include/google/protobuf/arena_align.h:85:
/usr/local/include/google/protobuf/port_def.inc:210:1: error: static_assert failed due to requirement '201103L >= 201402L' "Protobuf only supports C++14 and newer."
static_assert(PROTOBUF_CPLUSPLUS_MIN(201402L), "Protobuf only supports C++14 and newer.");
尝试升级gcc:pkg upgrade gcc,但是也只升级到gcc13 还是不到14
尝试使用opencv-python3.4.17版本
报错
[ 45%] Building CXX object modules/dnn/CMakeFiles/opencv_dnn.dir/misc/caffe/opencv-caffe.pb.cc.o
In file included from /usr/home/skywalk/work/opencv-python-3.4.17.63/opencv/modules/dnn/misc/caffe/opencv-caffe.pb.cc:4:
/usr/home/skywalk/work/opencv-python-3.4.17.63/opencv/modules/dnn/misc/caffe/opencv-caffe.pb.h:17:2: error: This file was generated by an older version of protoc which is
#error This file was generated by an older version of protoc which is
^
/usr/home/skywalk/work/opencv-python-3.4.17.63/opencv/modules/dnn/misc/caffe/opencv-caffe.pb.h:18:2: error: incompatible with your Protocol Buffer headers. Please
#error incompatible with your Protocol Buffer headers. Please
^
/usr/home/skywalk/work/opencv-python-3.4.17.63/opencv/modules/dnn/misc/caffe/opencv-caffe.pb.h:19:2: error: regenerate this file with a newer version of protoc.
#error regenerate this file with a newer version of protoc.
搞不定。
不过可喜的是,
import cv2没有报错,也就是opencv可以用啊!
后来测试,发现不行
编译opencv-python-headless报错
In file included from /usr/home/skywalk/work/opencv-python-headless-4.9.0.80/opencv/modules/dnn/misc/caffe/opencv-caffe.pb.cc:4:
In file included from /usr/home/skywalk/work/opencv-python-headless-4.9.0.80/opencv/modules/dnn/misc/caffe/opencv-caffe.pb.h:10:
/usr/local/include/google/protobuf/port_def.inc:210:1: error: static_assert failed due to requirement '201103L >= 201402L' "Protobuf only supports C++14 and newer."
static_assert(PROTOBUF_CPLUSPLUS_MIN(201402L), "Protobuf only supports C++14 and newer.");
^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
在root账户下fastdeploy c编译,make install 报错
-- Installing: /root/work/fd/include/fastdeploy/utils/path.h
CMake Error at cmake_install.cmake:81 (file):
file INSTALL cannot find
"/home/skywalk/github/FastDeploy/build/third_libs/install": No such file or
directory.
尝试
cd ~/github/FastDeploy/build/third_libs
mkdir install
cp -rf patchelf/ install/
然后再make install
c推理例子编译报错:
skywalk@x250:~/github/FastDeploy/examples/runtime/cpp % gcc infer_demo.cc
In file included from /usr/local/include/fastdeploy/vision/visualize/visualize.h:17,
from /usr/local/include/fastdeploy/vision.h:78,
from infer_demo.cc:2:
/usr/local/include/fastdeploy/vision/common/result.h:16:10: fatal error: opencv2/core/core.hpp: No such file or directory
16 | #include "opencv2/core/core.hpp"
| ^~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
skywalk@x250:~/github/FastDeploy/examples/runtime/cpp %
examples/runtime/cpp目录编译生成的demo 文件runtime_demo执行报错:
./runtime_demo
[ERROR] fastdeploy/runtime/runtime.cc(105)::AutoSelectBackend The candiate backends for ModelFormat::PADDLE & Device::CPU are [ Backend::PDINFER ,Backend::PDLITE ,Backend::ORT ,Backend::OPENVINO ], but both of them have not been compiled with current FastDeploy yet.
Assertion failed: (runtime.Init(runtime_option)), function main, file /home/skywalk/github/FastDeploy/examples/runtime/cpp/infer_paddle_paddle_inference.cc, line 37.
Abort (core dumped)
fastdeploy编译报错'opencv2/imgcodecs.hpp' file not found
type.cc.o
/home/skywalk/github/FastDeploy/c_api/fastdeploy_capi/core/fd_type.cc:17:10: fatal error: 'opencv2/imgcodecs.hpp' file not found
#include <opencv2/imgcodecs.hpp>
^~~~~~~~~~~~~~~~~~~~~~~
用Pillow替代opencv推理报错
python inf.py
Traceback (most recent call last):
File "/usr/home/skywalk/py310/lib/python3.10/site-packages/fastdeploy_python-0.0.0-py3.10-freebsd-13.2-RELEASE-amd64.egg/fastdeploy/c_lib_wrap.py", line 164, in <module>
from .libs.fastdeploy_main import *
ImportError: Shared object "libdl.so.2" not found, required by "libonnxruntime.so.1.12.0"
During handling of the above exception, another exception occurred:
先搁置。