FastDeploy面向AI模型产业落地,帮助开发者简单几步即可完成AI模型在对应硬件上的部署,降低部署难度、压缩部署时间成本。支持40多个主流的AI模型在8大类常见硬件上的部署能力。
本文给大家分享如何快速搭建FastDeploy环境,并部署推理模型。
准确工作
请先使用miniConda创建一个python 3.10的环境,并将其命名为 fastdeploy
接下来的环节都基于此条件下说明。
PaddleX3安装教程-CSDN博客文章浏览阅读319次,点赞8次,收藏9次。百度飞浆AI一站式工具最新版PaddleX3的详细安装教程,根据以下步骤,可以帮助大家避坑,且一次性安装成功,助力大家将复杂的AI环境搭建标准化、流程化、简单化。原来AI开发可以如此简单。话不多说,直接上干货。
https://blog.csdn.net/li277967151/article/details/140655249注:不会使用conda搭建环境的同学,请参考这篇文章,有详细介绍。
下载FastDeploy whl安装包
FastDeply各平台官方下载链接
https://www.paddlepaddle.org.cn/whl/fastdeploy.html
请大家根据自己的实际软硬件配置情况,下载对应的whl包版本,如GPU版本、CPU版本等。
有几点说明一下:
- whl安装包文件的命名规范:fastdeploy_[cpu|gpu]_python-[版本号]-[python版本]-[python版本]-[操作系统]_[架构].whl
- cp310 或 cp39等,表示的是Python版本,即python3.10、python3.9
总之,如果你的python是3.10,请下载含cp310的whl包,同理,如果你是python3.9,则下载含cp39的whl包,依次类推。
安装FastDeploy
# 切换到fastdeploy安装包的所在目录
# 激活名称为fastdeploy的conda环境
conda activate fastdeploy
# 通过pip安装fasteploy
pip install fastdeploy_gpu_python-1.0.7-cp310-cp310-win_amd64.whl
# numpy降级(否则FD可视化时图像会有问题,但不影响推理)
pip install --upgrade numpy==1.26.4
FastDeploy安装十分方便,一条命令就搞定了。它会自动安装相关依赖,耐心等待一会即可。
输出如下图所示,未报错,即表示成功。(一般都会成功,此处没有坑)
基于FastDeploy推理
- 通用推理脚本 infer.py
import fastdeploy as fd
import cv2
import os
import numpy as np
import time
from tqdm import tqdm
def parse_arguments():
import argparse
import ast
parser = argparse.ArgumentParser()
parser.add_argument(
"--model", required=True, help="Path of PaddleClas model.")
parser.add_argument(
"--image", type=str, required=True, help="Path of test image file.")
parser.add_argument(
"--topk", type=int, default=1, help="Return topk results.")
parser.add_argument(
"--device",
type=str,
default='cpu',
help="Type of inference device, support 'cpu' or 'gpu' or 'ipu' or 'kunlunxin' or 'ascend' ."
)
parser.add_argument(
"--device_id",
type=int,
default=0,
help="Define which GPU card used to run model.")
parser.add_argument(
"--backend",
type=str,
default="default",
help="Type of inference backend, support ort/trt/paddle/openvino, default 'openvino' for cpu, 'tensorrt' for gpu"
)
return parser.parse_args()
def build_option(args):
option = fd.RuntimeOption()
if args.device.lower() == "gpu":
option.use_gpu(args.device_id)
if args.backend.lower() == "trt":
assert args.device.lower(
) == "gpu", "TensorRT backend require inference on device GPU."
option.use_trt_backend()
elif args.backend.lower() == "pptrt":
assert args.device.lower(
) == "gpu", "Paddle-TensorRT backend require inference on device GPU."
option.use_paddle_infer_backend()
option.paddle_infer_option.enable_trt = True
option.paddle_infer_option.enable_log_info = False
elif args.backend.lower() == "ort":
option.use_ort_backend()
elif args.backend.lower() == "paddle":
option.use_paddle_infer_backend()
option.paddle_infer_option.enable_log_info = False
elif args.backend.lower() == "openvino":
assert args.device.lower(
) == "cpu", "OpenVINO backend require inference on device CPU."
option.use_openvino_backend()
elif args.backend.lower() == "pplite":
assert args.device.lower(
) == "cpu", "Paddle Lite backend require inference on device CPU."
option.use_lite_backend()
return option
args = parse_arguments()
# 配置runtime,加载模型
option = build_option(args)
model_file = os.path.join(args.model, "inference.pdmodel")
params_file = os.path.join(args.model, "inference.pdiparams")
config_file = os.path.join(args.model, "inference.yml")
model = fd.vision.detection.PaddleDetectionModel(
model_file, params_file, config_file, runtime_option=option)
dump_result = dict()
im = cv2.imread(args.image)
result = model.predict(im)
print(result)
# 预测结果可视化
vis_im = fd.vision.vis_detection(im, result,score_threshold=0.5, line_size=8, font_size=2)
cv2.imwrite("visualized_result.jpg", vis_im)
print("Visualized result save in ./visualized_result.jpg")
- 将官方下载的模型或自己训练的模型,解压或复制到 infer.py 同级目录的model文件夹中,如下图所示:
- 执行脚本进行推理
python infer.py --model model --image fall.png --device gpu --backend paddle
输出结果如下:
[INFO] fastdeploy/vision/common/processors/transform.cc(45)::fastdeploy::vision::FuseNormalizeCast Normalize and Cast are fused to Normalize in preprocessing pipeline.
[INFO] fastdeploy/vision/common/processors/transform.cc(93)::fastdeploy::vision::FuseNormalizeHWC2CHW Normalize and HWC2CHW are fused to NormalizeAndPermute in preprocessing pipeline.
[INFO] fastdeploy/vision/common/processors/transform.cc(159)::fastdeploy::vision::FuseNormalizeColorConvert BGR2RGB and NormalizeAndPermute are fused to NormalizeAndPermute with swap_rb=1
[INFO] fastdeploy/runtime/runtime.cc(273)::fastdeploy::Runtime::CreatePaddleBackend Runtime initialized with Backend::PDINFER in Device::GPU.
Visualized result save in ./visualized_result.jpg
DetectionResult: [xmin, ymin, xmax, ymax, score, label_id]
563.366333,559.916443, 1558.083862, 1015.399658, 0.940546, 0
171.644043,85.243446, 527.575439, 911.452698, 0.061176, 0
331.805389,821.662109, 469.789490, 926.417908, 0.054461, 0
820.271912,561.068237, 1611.818115, 1024.686890, 0.049425, 0
860.103455,889.728699, 1275.387695, 1014.181458, 0.046738, 0
540.486694,181.668716, 1593.693604, 1024.698242, 0.036476, 0
538.231201,799.907715, 638.935425, 972.466980, 0.035621, 0
1274.074707,1181.460205, 1415.242554, 1311.077271, 0.035504, 0
至此,FastDeploy入门完成。从安装到推理都非常方便,没有一点坑。大大降低AI部署的难度,部署效率非常明显。API也十分容易理解。只要是FastDeploy支持的模型,基本上都是3句Code的事情,就可以搞定。
Amazing!Enjoy It!
附:
PPYoloE目标检测模型链接:https://pan.baidu.com/s/1u4qFk5UI6s0SeXOljQbHCQ?pwd=d9dq