libtorch部署手册

于 2024-09-09 19:10:58 发布
阅读量531 收藏 10
点赞数 5
文章标签： pytorch
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/WILLIX/article/details/142066702
版权

libtorch部署手册

x86架构

libtorch下载

在pytorch网站下载对应cuda版本的libtorch

Start Locally | PyTorch

请添加图片描述

下载后解压

unzip libtorch-shared-with-deps-2.3.1%2Bcu118.zip

示例

需要有cmake编译器大于3.16

进入libtorch目录

cd libtorch

创建example-app目录并进入

sudo mkdir example-app
cd example

创建编译文件CMakeLists.txt和example-app.cpp

// CMakeLists.txt
cmake_minimum_required(VERSION 3.16 FATAL_ERROR)
project(example-app)

# Set CUDA compiler
set(CMAKE_CUDA_COMPILER "/usr/local/cuda/bin/nvcc")  #cuda的nvcc路径，如果系统找不到的情况下需要添加
enable_language(CUDA)

find_package(Torch REQUIRED)
find_package(CUDA REQUIRED)

set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${TORCH_CXX_FLAGS}")

add_executable(example-app example-app.cpp)
target_link_libraries(example-app "${TORCH_LIBRARIES}")

set_property(TARGET example-app PROPERTY CXX_STANDARD 17)

# If using any CUDA functions or libraries, link them here
target_link_libraries(example-app ${CUDA_LIBRARIES})

# Copy Torch DLLs on Windows as needed
if (MSVC)
  file(GLOB TORCH_DLLS "${TORCH_INSTALL_PREFIX}/lib/*.dll")
  add_custom_command(TARGET example-app
                     POST_BUILD
                     COMMAND ${CMAKE_COMMAND} -E copy_if_different
                     ${TORCH_DLLS}
                     $<TARGET_FILE_DIR:example-app>)
endif (MSVC)


//example.cpp

#include <torch/torch.h>
#include <iostream>

int main() {
  torch::Tensor tensor = torch::rand({2, 3});
  std::cout << tensor << std::endl;
}

创建build目录并进入进行编译

sudo mkdir build
cd build
cmake -DCMAKE_PREFIX_PATH=/path/to/libtorch ..  //替换为实际libtorch路径

cmake --build . --config Release

运行example-app

./example-app

得到

(base) ➜  build ./example-app
 0.1078  0.4405  0.5437
 0.1887  0.9248  0.7328
[ CPUFloatType{2,3} ]

opencv3安装

需要先准备opencv3，根据下方链接进行安装

Ubuntu20安装OpenCV3（图解亲测）_ubuntu 20 安装opencv3-CSDN博客

调用模型进行图像分类推理

整体流程与示例类似

//CMakeLists.txt
cmake_minimum_required(VERSION 3.16 FATAL_ERROR)
project(test1-resnet18)

# Set CUDA compiler
set(CMAKE_CUDA_COMPILER "/usr/local/cuda/bin/nvcc")
enable_language(CUDA)

find_package(Torch REQUIRED)
find_package(CUDA REQUIRED)
find_package(OpenCV REQUIRED)  # 查找 OpenCV

add_executable(test1-resnet18 test1-resnet18.cpp)
include_directories(${OpenCV_INCLUDE_DIRS})  # 包含 OpenCV 头文件

target_link_libraries(test1-resnet18 "${TORCH_LIBRARIES}")
target_link_libraries(test1-resnet18 ${CUDA_LIBRARIES})
target_link_libraries(test1-resnet18 ${OpenCV_LIBS})  # 链接 OpenCV 库

set_property(TARGET test1-resnet18 PROPERTY CXX_STANDARD 17)

if(MSVC)
    file(GLOB TORCH_DLLS "${TORCH_INSTALL_PREFIX}/lib/*.dll")
    add_custom_command(TARGET test1-resnet18
                       POST_BUILD
                       COMMAND ${CMAKE_COMMAND} -E copy_if_different
                       ${TORCH_DLLS}
                       $<TARGET_FILE_DIR:test1-resnet18>)
endif(MSVC)


//test1-resnet18.cpp
#include <torch/torch.h>
#include <torch/script.h>  // For torch::jit::load
#include <opencv2/opencv.hpp>
#include <iostream>
#include <memory>
#include <string>
#include <opencv2/imgproc/imgproc.hpp>
#include <opencv2/highgui.hpp>

torch::Device device(torch::kCUDA);
auto image_preprocess(cv::Mat image){
    cv::cvtColor(image, image, CV_BGR2RGB);
    cv::Mat img_float;
    image.convertTo(img_float, CV_32F, 1.0/255);
    cv::resize(img_float, img_float, cv::Size(224, 224), cv::INTER_AREA);
    auto img_tensor = torch::from_blob(img_float.data, {1, 224, 224, 3}, torch::kFloat32);
    img_tensor = img_tensor.permute({0, 3, 1, 2});
    img_tensor[0][0] = img_tensor[0][0].sub_(0.485).div_(0.229);
    img_tensor[0][1] = img_tensor[0][1].sub_(0.456).div_(0.224);
    img_tensor[0][2] = img_tensor[0][2].sub_(0.406).div_(0.225);
    img_tensor = img_tensor.to(device);
    return img_tensor;
}
void infer_and_postprocess(torch::Tensor img_tensor,torch::jit::Module model){

    torch::NoGradGuard no_grad;
    auto output = model.forward({img_tensor}).toTensor();
    std::cout << output.sizes() << std::endl;
    std::cout << output.slice(/*dim=*/1,/*start=*/0,/*end=*/3) << '\n';

    // 1. Load labels
    std::string label_file = "/home/zhouhaojie/wxl/libtorch/test1-resnet18/synset_words.txt";
    std::ifstream rf(label_file.c_str());
    CHECK(rf) << "Unable to open labels file" << label_file;
    std::string line;
    std::vector<std::string> labels;
    while(std::getline(rf, line)){labels.push_back(line);}


    // 2. print predicted top-3 labels
    std::tuple<torch::Tensor, torch::Tensor> result = output.sort(-1, true);
    torch::Tensor top_scores = std::get<0>(result)[0];
    torch::Tensor top_idxs = std::get<1>(result)[0].toType(torch::kInt32);
    std::cout << top_scores[0].item<float>() << std::endl;

    for(int i=0; i<3;i++){
        int idx = top_idxs[i].item<int>();
        std::cout << "top-" << i+1 << " label: ";
        std::cout << labels[idx] << ",score: " << top_scores[0].item<float>() << std::endl;
    }
}

//摄像头输入
void camera_stream(const char* url,torch::jit::Module model){
    cv::VideoCapture cap(url); // 0 代表默认摄像头
    if (!cap.isOpened()) {
        std::cerr << "Error: Unable to open the camera" << std::endl;
        return;
    }

    cv::Mat frame;
    cv::namedWindow("Webcam Feed", cv::WINDOW_NORMAL); // 创建一个可调整大小的窗口
    cv::resizeWindow("Webcam Feed", 640, 480); // 设置窗口大小
    while (true) {
        cap >> frame; // 读取新的帧
        if (frame.empty()) {
            std::cerr << "Error: No frame captured from the camera" << std::endl;
            break;
        }

        auto img_tensor = image_preprocess(frame);
        infer_and_postprocess(img_tensor,model);
        cv::cvtColor(frame, frame, cv::COLOR_RGB2BGR);
        // 显示图像
        cv::imshow("Webcam Feed", frame);
        if (cv::waitKey(10) == 27) { // 按 'ESC' 键退出
            break;
        }
    }
    cap.release();  // 释放 VideoCapture 对象
    cv::destroyAllWindows();  // 关闭所有 OpenCV 窗口
}
void vedio_stream(const char* video_path, torch::jit::Module model) {
    cv::VideoCapture cap(video_path);  // 使用视频文件路径而非摄像头

    if (!cap.isOpened()) {
        std::cerr << "Error: Unable to open the video file" << std::endl;
        return;  // 无法打开视频文件时退出
    }

    cv::Mat frame;
    cv::namedWindow("Video Feed", cv::WINDOW_NORMAL);  // 更改窗口名称为"Video Feed"
    cv::resizeWindow("Video Feed", 640, 480);  // 设置窗口大小

    while (true) {
        cap >> frame;  // 从视频文件读取新的帧
        if (frame.empty()) {
            std::cerr << "Error: No frame captured from the video" << std::endl;
            break;  // 如果没有帧被捕获，退出循环
        }

        auto img_tensor = image_preprocess(frame);  // 假设已有的图像预处理函数
        infer_and_postprocess(img_tensor, model);  // 假设已有的模型推理和后处理函数

        cv::cvtColor(frame, frame, cv::COLOR_RGB2BGR);  // 确保显示的颜色格式正确
        cv::imshow("Video Feed", frame);  // 在窗口中显示处理后的帧

        if (cv::waitKey(10) == 27) {  // 按 'ESC' 键退出
            break;
        }
    }

    cap.release();  // 释放 VideoCapture 对象
    cv::destroyAllWindows();  // 关闭所有 OpenCV 窗口
}
// 图片输入
void image_stream(cv::Mat image,torch::jit::Module model){
    auto img_tensor = image_preprocess(image);
    infer_and_postprocess(img_tensor,model);

}

int main() {
    // 设置设备
    torch::Device device(torch::kCUDA);  // 使用 CUDA，如果不可用则使用 CPU: torch::kCPU

    // 加载预训练的模型
    auto model = torch::jit::load("/home/zhouhaojie/wxl/libtorch/test1-resnet18/traced_resnet18_model.pt");
    model.to(device);
    model.eval();
    // 摄像头输入
    // auto url = "rtsp://admin:vision123456@192.168.0.202/Streaming/Channels/1";
    // camera_stream(url,model);

    //图片输入
    // cv::Mat image = cv::imread("/home/zhouhaojie/wxl/libtorch/test1-resnet18/dog2.JPEG", cv::IMREAD_COLOR);
    // image_stream(image,model);

	//视频输入
    const char* video_path = "/home/zhouhaojie/wxl/libtorch/test1-resnet18/edge_box.mp4";
    vedio_stream(video_path,model);

    return 0;
}

调用模型进行图像识别推理（yolov8）

在官方中有写到如何使用ultralytics/examples/YOLOv8-LibTorch-CPP-Inference at main · ultralytics/ultralytics (github.com)

但存在问题：

RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!

解决方案：需要在加载模型的时候直接加载在GPU上，而不能加载好后移动到GPU

原：
        torch::jit::script::Module yolo_model;
        yolo_model = torch::jit::load(model_path);
        yolo_model.eval();
        yolo_model.to(device, torch::kFloat32);
改进后：
        torch::jit::script::Module yolo_model;
        yolo_model = torch::jit::load(model_path, device);
        yolo_model.eval();

//CMakeLists.txt
cmake_minimum_required(VERSION 3.16 FATAL_ERROR)
project(test-yolo)

# Set CUDA compiler
set(CMAKE_CUDA_COMPILER "/usr/local/cuda/bin/nvcc")
enable_language(CUDA)

find_package(Torch REQUIRED)
find_package(CUDA REQUIRED)
find_package(OpenCV REQUIRED)  # 查找 OpenCV

add_executable(test-yolo main.cpp)
include_directories(${OpenCV_INCLUDE_DIRS})  # 包含 OpenCV 头文件

target_link_libraries(test-yolo "${TORCH_LIBRARIES}")
target_link_libraries(test-yolo ${CUDA_LIBRARIES})
target_link_libraries(test-yolo ${OpenCV_LIBS})  # 链接 OpenCV 库
target_link_libraries(test-yolo pthread)
set_property(TARGET test-yolo PROPERTY CXX_STANDARD 17)

if(MSVC)
    file(GLOB TORCH_DLLS "${TORCH_INSTALL_PREFIX}/lib/*.dll")
    add_custom_command(TARGET test-yolo
                       POST_BUILD
                       COMMAND ${CMAKE_COMMAND} -E copy_if_different
                       ${TORCH_DLLS}
                       $<TARGET_FILE_DIR:test-yolo>)
endif(MSVC)


//multi_thread_yolo.cpp
#include <iostream>
#include <opencv2/core.hpp>
#include <opencv2/imgproc.hpp>
#include <opencv2/imgcodecs.hpp>
#include <opencv2/opencv.hpp>
#include <torch/torch.h>
#include <torch/script.h>
#include <string>
#include <thread>
#include <mutex>
#include <vector>

std::mutex mtx;
using torch::indexing::Slice;
using torch::indexing::None;

float generate_scale(cv::Mat& image, const std::vector<int>& target_size) {
    int origin_w = image.cols;
    int origin_h = image.rows;

    int target_h = target_size[0];
    int target_w = target_size[1];

    float ratio_h = static_cast<float>(target_h) / static_cast<float>(origin_h);
    float ratio_w = static_cast<float>(target_w) / static_cast<float>(origin_w);
    float resize_scale = std::min(ratio_h, ratio_w);
    return resize_scale;
}

float letterbox(cv::Mat &input_image, cv::Mat &output_image, const std::vector<int> &target_size) {
    if (input_image.cols == target_size[1] && input_image.rows == target_size[0]) {
        if (input_image.data == output_image.data) {
            return 1.;
        } else {
            output_image = input_image.clone();
            return 1.;
        }
    }

    float resize_scale = generate_scale(input_image, target_size);
    int new_shape_w = std::round(input_image.cols * resize_scale);
    int new_shape_h = std::round(input_image.rows * resize_scale);
    float padw = (target_size[1] - new_shape_w) / 2.;
    float padh = (target_size[0] - new_shape_h) / 2.;

    int top = std::round(padh - 0.1);
    int bottom = std::round(padh + 0.1);
    int left = std::round(padw - 0.1);
    int right = std::round(padw + 0.1);

    cv::resize(input_image, output_image,
               cv::Size(new_shape_w, new_shape_h),
               0, 0, cv::INTER_AREA);

    cv::copyMakeBorder(output_image, output_image, top, bottom, left, right,
                       cv::BORDER_CONSTANT, cv::Scalar(114, 114, 114));
    return resize_scale;
}

torch::Tensor xyxy2xywh(const torch::Tensor& x) {
    auto y = torch::empty_like(x);
    y.index_put_({"...", 0}, (x.index({"...", 0}) + x.index({"...", 2})).div(2));
    y.index_put_({"...", 1}, (x.index({"...", 1}) + x.index({"...", 3})).div(2));
    y.index_put_({"...", 2}, x.index({"...", 2}) - x.index({"...", 0}));
    y.index_put_({"...", 3}, x.index({"...", 3}) - x.index({"...", 1}));
    return y;
}

torch::Tensor xywh2xyxy(const torch::Tensor& x) {
    auto y = torch::empty_like(x);
    auto dw = x.index({"...", 2}).div(2);
    auto dh = x.index({"...", 3}).div(2);
    y.index_put_({"...", 0}, x.index({"...", 0}) - dw);
    y.index_put_({"...", 1}, x.index({"...", 1}) - dh);
    y.index_put_({"...", 2}, x.index({"...", 0}) + dw);
    y.index_put_({"...", 3}, x.index({"...", 1}) + dh);
    return y;
}

torch::Tensor nms(const torch::Tensor& bboxes, const torch::Tensor& scores, float iou_threshold) {
    if (bboxes.numel() == 0)
        return torch::empty({0}, bboxes.options().dtype(torch::kLong));
    auto x1_t = bboxes.select(1, 0).contiguous();
    auto y1_t = bboxes.select(1, 1).contiguous();
    auto x2_t = bboxes.select(1, 2).contiguous();
    auto y2_t = bboxes.select(1, 3).contiguous();

    torch::Tensor areas_t = (x2_t - x1_t) * (y2_t - y1_t);
    auto order_t = std::get<1>(
        scores.sort(/*stable=*/true, /*dim=*/0, /* descending=*/true));
    auto ndets = bboxes.size(0);
    torch::Tensor suppressed_t = torch::zeros({ndets}, bboxes.options().dtype(torch::kByte));
    torch::Tensor keep_t = torch::zeros({ndets}, bboxes.options().dtype(torch::kLong));
    auto suppressed = suppressed_t.data_ptr<uint8_t>();
    auto keep = keep_t.data_ptr<int64_t>();
    auto order = order_t.data_ptr<int64_t>();
    auto x1 = x1_t.data_ptr<float>();
    auto y1 = y1_t.data_ptr<float>();
    auto x2 = x2_t.data_ptr<float>();
    auto y2 = y2_t.data_ptr<float>();
    auto areas = areas_t.data_ptr<float>();
    int64_t num_to_keep = 0;
    for (int64_t _i = 0; _i < ndets; _i++) {
        auto i = order[_i];   //这里有问题
        if (suppressed[i] == 1)
            continue;
        keep[num_to_keep++] = i;
        auto ix1 = x1[i];
        auto iy1 = y1[i];
        auto ix2 = x2[i];
        auto iy2 = y2[i];
        auto iarea = areas[i];
        for (int64_t _j = _i + 1; _j < ndets; _j++) {
            auto j = order[_j];
            if (suppressed[j] == 1)
                continue;
            auto xx1 = std::max(ix1, x1[j]);
            auto yy1 = std::max(iy1, y1[j]);
            auto xx2 = std::min(ix2, x2[j]);
            auto yy2 = std::min(iy2, y2[j]);

            auto w = std::max(static_cast<float>(0), xx2 - xx1);
            auto h = std::max(static_cast<float>(0), yy2 - yy1);
            auto inter = w * h;
            auto ovr = inter / (iarea + areas[j] - inter);
            if (ovr > iou_threshold)
                suppressed[j] = 1;
        }
    }
    return keep_t.narrow(0, 0, num_to_keep);
}


torch::Tensor non_max_suppression(torch::Tensor& prediction, float conf_thres = 0.25, float iou_thres = 0.45, int max_det = 300) {
    auto bs = prediction.size(0);
    auto nc = prediction.size(1) - 4;
    auto nm = prediction.size(1) - nc - 4;
    auto mi = 4 + nc;
    auto xc = prediction.index({Slice(), Slice(4, mi)}).amax(1) > conf_thres;

    prediction = prediction.transpose(-1, -2);
    prediction.index_put_({"...", Slice({None, 4})}, xywh2xyxy(prediction.index({"...", Slice(None, 4)})));

    std::vector<torch::Tensor> output;
    for (int i = 0; i < bs; i++) {
        output.push_back(torch::zeros({0, 6 + nm}, prediction.device()));
    }
    
    for (int xi = 0; xi < prediction.size(0); xi++) {
        auto x = prediction[xi];
        x = x.index({xc[xi]});
        auto x_split = x.split({4, nc, nm}, 1);
        auto box = x_split[0], cls = x_split[1], mask = x_split[2];
        auto [conf, j] = cls.max(1, true);
        x = torch::cat({box, conf, j.toType(torch::kFloat), mask}, 1);
        x = x.index({conf.view(-1) > conf_thres});
        int n = x.size(0);
        if (!n) { continue; }
        // NMS
        auto c = x.index({Slice(), Slice{5, 6}}) * 7680;
        auto boxes = x.index({Slice(), Slice(None, 4)}) + c;
        auto scores = x.index({Slice(), 4});
        auto i = nms(boxes, scores, iou_thres);
        i = i.index({Slice(None, max_det)});
        output[xi] = x.index({i});
    }

    return torch::stack(output);
}

torch::Tensor clip_boxes(torch::Tensor& boxes, const std::vector<int>& shape) {
    boxes.index_put_({"...", 0}, boxes.index({"...", 0}).clamp(0, shape[1]));
    boxes.index_put_({"...", 1}, boxes.index({"...", 1}).clamp(0, shape[0]));
    boxes.index_put_({"...", 2}, boxes.index({"...", 2}).clamp(0, shape[1]));
    boxes.index_put_({"...", 3}, boxes.index({"...", 3}).clamp(0, shape[0]));
    return boxes;
}

torch::Tensor scale_boxes(const std::vector<int>& img1_shape, torch::Tensor& boxes, const std::vector<int>& img0_shape) {
    auto gain = (std::min)((float)img1_shape[0] / img0_shape[0], (float)img1_shape[1] / img0_shape[1]);
    auto pad0 = std::round((float)(img1_shape[1] - img0_shape[1] * gain) / 2. - 0.1);
    auto pad1 = std::round((float)(img1_shape[0] - img0_shape[0] * gain) / 2. - 0.1);

    boxes.index_put_({"...", 0}, boxes.index({"...", 0}) - pad0);
    boxes.index_put_({"...", 2}, boxes.index({"...", 2}) - pad0);
    boxes.index_put_({"...", 1}, boxes.index({"...", 1}) - pad1);
    boxes.index_put_({"...", 3}, boxes.index({"...", 3}) - pad1);
    boxes.index_put_({"...", Slice(None, 4)}, boxes.index({"...", Slice(None, 4)}).div(gain));
    return boxes;
}
void process_video(const std::string& camera_url, torch::jit::script::Module& yolo_model, const torch::Device& device, const std::vector<std::string>& classes) {
    cv::VideoCapture cap(camera_url); // 打开摄像头
    if (!cap.isOpened()) {
        std::cerr << "无法打开视频捕捉设备: " << camera_url << std::endl;
        return;
    }

    cv::Mat frame;
    cv::namedWindow("Webcam Feed - " + camera_url, cv::WINDOW_NORMAL); // 创建一个可调整大小的窗口
    cv::resizeWindow("Webcam Feed - " + camera_url, 640, 480); // 设置窗口大小
    while (cap.read(frame)) {
        cv::Mat input_image;
        letterbox(frame, input_image, {640, 640});

        torch::Tensor image_tensor = torch::from_blob(input_image.data, {input_image.rows, input_image.cols, 3}, torch::kByte).to(device);
        image_tensor = image_tensor.toType(torch::kFloat32).div(255);
        image_tensor = image_tensor.permute({2, 0, 1});
        image_tensor = image_tensor.unsqueeze(0);
        std::vector<torch::jit::IValue> inputs {image_tensor};

        // Inference
        torch::Tensor output = yolo_model.forward(inputs).toTensor().cpu();
        auto keep = non_max_suppression(output)[0];
        auto boxes = keep.index({Slice(), Slice(None, 4)});
        keep.index_put_({Slice(), Slice(None, 4)}, scale_boxes({input_image.rows, input_image.cols}, boxes, {frame.rows, frame.cols}));

        // Draw bounding boxes and labels
        std::lock_guard<std::mutex> lock(mtx);
        std::cout << camera_url << "检测结果：" << std::endl;
        for (int i = 0; i < keep.size(0); i++) {
            int x1 = keep[i][0].item().toFloat();
            int y1 = keep[i][1].item().toFloat();
            int x2 = keep[i][2].item().toFloat();
            int y2 = keep[i][3].item().toFloat();
            float conf = keep[i][4].item().toFloat();
            int cls = keep[i][5].item().toInt();
            std::cout << "Rect: [" << x1 << "," << y1 << "," << x2 << "," << y2 << "]  Conf: " << conf << "  Class: " << classes[cls] << std::endl;
            cv::rectangle(frame, cv::Point(x1, y1), cv::Point(x2, y2), cv::Scalar(255, 0, 0), 2);
            std::string label = classes[cls] + " " + std::to_string(conf);
            cv::putText(frame, label, cv::Point(x1, y1 - 5), cv::FONT_HERSHEY_SIMPLEX, 0.5, cv::Scalar(0, 255, 0), 1);
        }

        // Display the resulting frame
        cv::imshow("Webcam Feed - " + camera_url, frame);
        if (cv::waitKey(1) == 27) { // 按 'ESC' 键退出
            break;
        }
    }
    cap.release();
    cv::destroyAllWindows();
}
int main() {
    // Device
    torch::Device device(torch::cuda::is_available() ? torch::kCUDA :torch::kCPU);
    // Note that in this example the classes are hard-coded
    std::vector<std::string> classes {"person", "bicycle", "car", "motorcycle", "airplane", "bus", "train", "truck", "boat", "traffic light", "fire hydrant",
                                      "stop sign", "parking meter", "bench", "bird", "cat", "dog", "horse", "sheep", "cow", "elephant", "bear", "zebra",
                                      "giraffe", "backpack", "umbrella", "handbag", "tie", "suitcase", "frisbee", "skis", "snowboard", "sports ball", "kite",
                                      "baseball bat", "baseball glove", "skateboard", "surfboard", "tennis racket", "bottle", "wine glass", "cup", "fork", "knife",
                                      "spoon", "bowl", "banana", "apple", "sandwich", "orange", "broccoli", "carrot", "hot dog", "pizza", "donut", "cake", "chair",
                                      "couch", "potted plant", "bed", "dining table", "toilet", "tv", "laptop", "mouse", "remote", "keyboard", "cell phone",
                                      "microwave", "oven", "toaster", "sink", "refrigerator", "book", "clock", "vase", "scissors", "teddy bear", "hair drier", "toothbrush"};


    //图片输入
    // try {
    //     // Load the model (e.g. yolov8s.torchscript)
    //     std::string model_path = "/home/zhouhaojie/wxl/libtorch/test-yolo/ultralytics/yolov8s.torchscript";
    //     torch::jit::script::Module yolo_model;
    //     yolo_model = torch::jit::load(model_path,device);
    //     yolo_model.eval();
    //     // yolo_model.to(device, torch::kFloat32);
    //     std::cout << "模型加载成功" << std::endl;

    //     // Load image and preprocess
    //     cv::Mat image = cv::imread("/home/zhouhaojie/wxl/libtorch/test1-resnet18/dog2.JPEG");
    //     cv::Mat input_image;
    //     letterbox(image, input_image, {640, 640});

    //     torch::Tensor image_tensor = torch::from_blob(input_image.data, {input_image.rows, input_image.cols, 3}, torch::kByte).to(device);
    //     image_tensor = image_tensor.toType(torch::kFloat32).div(255);
    //     image_tensor = image_tensor.permute({2, 0, 1});
    //     image_tensor = image_tensor.unsqueeze(0);
    //     std::vector<torch::jit::IValue> inputs {image_tensor};

    //     // Inference
    //     torch::Tensor output = yolo_model.forward(inputs).toTensor().cpu();
    //     // NMS
    //     auto keep = non_max_suppression(output)[0];
    //     auto boxes = keep.index({Slice(), Slice(None, 4)});
    //     keep.index_put_({Slice(), Slice(None, 4)}, scale_boxes({input_image.rows, input_image.cols}, boxes, {image.rows, image.cols}));

    //     // Show the results
    //     for (int i = 0; i < keep.size(0); i++) {
    //         int x1 = keep[i][0].item().toFloat();
    //         int y1 = keep[i][1].item().toFloat();
    //         int x2 = keep[i][2].item().toFloat();
    //         int y2 = keep[i][3].item().toFloat();
    //         float conf = keep[i][4].item().toFloat();
    //         int cls = keep[i][5].item().toInt();
    //         std::cout << "Rect: [" << x1 << "," << y1 << "," << x2 << "," << y2 << "]  Conf: " << conf << "  Class: " << classes[cls] << std::endl;
    //         cv::rectangle(image, cv::Point(x1, y1), cv::Point(x2, y2), cv::Scalar(255, 0, 0), 2);
    //         std::string label = classes[cls] + " " + std::to_string(conf);
    //         cv::putText(image, label, cv::Point(x1, y1 - 5), cv::FONT_HERSHEY_SIMPLEX, 0.5, cv::Scalar(0, 255, 0), 1);
    //     }
    //     cv::imwrite("/home/zhouhaojie/wxl/libtorch/test-yolo/dog_detected.jpg", image);
    //     std::cout << "检测结果已保存到 bus_detected.jpg" << std::endl;
    // } catch (const c10::Error& e) {
    //     std::cout << e.msg() << std::endl;
    // }


    //摄像头输入
    try {
        // Load the model (e.g. yolov8s.torchscript)
        std::string model_path = "/home/zhouhaojie/wxl/libtorch/test-yolo/ultralytics/yolov8s.torchscript";
        torch::jit::script::Module yolo_model;
        yolo_model = torch::jit::load(model_path, device);
        yolo_model.eval();
        std::cout << "模型加载成功" << std::endl;

        // List of camera URLs
        std::vector<std::string> camera_urls = {
            "rtsp://admin:vision123456@192.168.0.201/Streaming/Channels/1",
            "rtsp://admin:vision123456@192.168.0.202/Streaming/Channels/1"
            // Add more camera URLs here
        };

        // Start threads for each camera
        std::vector<std::thread> threads;
        for (const auto& camera_url : camera_urls) {
            threads.emplace_back(process_video, camera_url, std::ref(yolo_model), std::ref(device), std::ref(classes));
        }

        // Wait for all threads to finish
        for (auto& t : threads) {
            t.join();
        }
    } catch (const c10::Error& e) {
        std::cout << e.msg() << std::endl;
    }
    return 0;
}

arm架构（jetson）

pytorch下载

在nvidia官网下载cuda对应版本的pytorch，并按照官方指令安装

CUDA Toolkit 12.4 Downloads | NVIDIA Developer

请添加图片描述

安装好后提取出其中的share、include、bin、lib即可。其他与x86架构教程相似。

WILLIX
关注
Libtorch部署模型
startllee的博客
05-30 3515
Libtorch部署显著性目标检测网络BASNet一、环境配置二、torchscript生成.pt文件三、Libtorch部署1.添加环境变量2.VS配置环境四、BASNet demo参考链接 Libtorch 是Pytorch官方提供的C++ API，使用方法可以说是很大程度上还原了Pytorch。 一、环境配置 GitHub地址：BASNet VS2017+CUDA 10.1+cuDNN 7.6.5+OpenCV 3.4.11 Pytorch 1.5.0+torchvision 0.6.0 Libto
Libtorch（c++）模型部署
君子之学必日新，日新者日进也。不日新者必日退，未有不进而不退者
07-15 1942
Libtorch（c++）模型部署
Libtorch部署与教程
专业主义
04-18 2695
vs2015编译错误解决：/jit/pickler.h(34): error C2059: 语法错误:“常量” ；error C3805: “常量”: 意外标记，应输入“}”或者“, libtorch error C2440: “初始化”: 无法从“torch::jit::script::Module”转换…的问题 libtorch环境搭建、生成库文件、测试 ......
【libtorch模型部署】VS2017中部署libtorch模型
shanhedian2013的博客
09-29 333
用libtorch部署pytorch模型，而不是用tensorrt等工具部署模型的优势在于：pytorch和libtorch同属一个生态，API语句比较接近，并且不会出现某网络层不支持的问题。理论上只要是pytorch训练的模型都能用libtorch部署，因为他们共用相同的c++代码。具体的部署流程如下：1.模型训练：首先使用PyTorch对模型进行训练，得到一个可以使用的模型。2.模型保存：训练好的模型需要保存为TorchScript模型，这是一个可以被TorchScript解释器执行的模型。
Libtorch模型部署指南
Guo_Python的博客
05-18 1721
前言：libtorch是pytorch原生支持的生态，理论上只要是pytorch训练的模型都能用libtorch部署，因为他们共用相同的c++代码。 主要参考：Pytorch官网 一. PyTorch模型转为Troch Script import torch import torchvision # An instance of your model. model = torchvision.models.resnet18() script_module = torch.jit.script(model
基于C++ libtorch部署YOLOv5（源码+图片+说明文档）.rar
04-15
1、资源内容：基于c++ libtorch部署yolov5（完整源码+说明文档+数据）.rar 2、代码特点：参数化编程、参数可方便更改、代码编程思路清晰、注释明细。 3、适用对象：计算机，电子信息工程、数学等专业的大学生课程...
基于c++ libtorch部署yolov5完整源码+数据（高分课程设计）.zip
10-18
基于c++ libtorch部署yolov5完整源码+数据（高分课程设计）.zip 已获导师指导并通过的97分的高分课程设计项目，可作为课程设计和期末大作业，下载即用无需修改，项目完整确保可以运行。 基于c++ libtorch部署yolov...
算法部署-使用TensorRT+NCNN+LibTorch部署Deeplabv3+行人分割算法-优质算法部署项目实战.zip
03-06
在本项目实战中，我们将探讨如何使用TensorRT、NCNN和LibTorch这三种高效工具来部署Deeplabv3+行人分割算法。 Deeplabv3+是一种深度学习模型，专门用于图像语义分割任务，即识别并区分图像中的不同对象。此模型在...
算法部署-使用Libtorch部署Pytorch训练出来的CRNN不定长文本序列识别模型-附项目源码-优质项目实战.zip
最新发布
05-24
在本项目中，我们将深入探讨如何使用LibTorch部署一个基于PyTorch训练的CRNN（卷积循环神经网络）模型，用于不定长文本序列识别。这个实战项目提供了详细的步骤和源代码，使得开发者能够更好地理解和应用这项技术。 ...
windows上部署libtorch
hajungong007的博客
11-02 790
ref: https://zhuanlan.zhihu.com/p/68901339 https://zhuanlan.zhihu.com/p/55292836
YOLOv5-LibTorch:通过LibTorch C ++ API部署YOLOv5进行实时对象检测
05-08
YOLOv5 LibTorch 通过LibTorch C ++ API部署YOLOv5进行实时对象检测 环境 Ubuntu 18.04 OpenCV 3.2.0 LibTorch 1.6.0 CMake 3.10.2 入门 安装OpenCV。 sudo apt-get install libopencv-dev 安装LibTorch。 wget https://download.pytorch.org/libtorch/nightly/cpu/libtorch-shared-with-deps-latest.zip unzip libtorch-shared-with-deps-latest.zip 编辑“ CMakeLists.txt”以正确配置OpenCV和LibTorch。 编译并运行。 cd build cmake .. make ./../bin/YOLOv5
Unet训练和部署代码
08-06
Unet训练和libtorch部署代码: 1、训练部分 2、libtorch部署方法
libtorch c++部署-使用GPU
热门推荐
alex1801
10-28 1万+
1、引用关系 以vs下libtorch引用CUDA为例： 包含目录： libtorch_dir/include libtorch_dir/include/torch/csrc/api/include 库目录： libtorch_dir/lib 动态库： c10.lib c10_cuda.lib torch_cpu.lib torch_cuda.lib 测试代码： #include<iostream> #include<torch/script.h> #inc
利用libtorch在C++程序上部署pytorch模型
weixin_33566282的博客
02-13 336
在torch上导出模型，新建ResNet50.py import torch import torchvision.models as models from PIL import Image import numpy as np image = Image.open("0.jpg") #图片发在了build文件夹下 image = image.resize((224, 224),Image.ANTIALIAS) image = np.asarray(image) image = image / 255 i
LibTorch部署PyTorch模型简单流程
累了就要打游戏
02-01 1739
PyTorch模型 LibTorch 一、配置LibTorch 1.1 下载 下载地址：https://pytorch.org/ 下图分别是GPU和CPU的下载地址，GPU版本需要根据CUDA的版本确定。箭头指向为Release版，下方是Debug版。 下载完成后，将…\libtorchCPU\lib添加至环境变量。 1.2 CmakeLists.txt与测试代码 （1）CmakeLists.txt cmake_minimum_required(VERSION 3.0 ...
使用libtorch部署yolox
weixin_38906715的博客
09-26 512
一、模型转换pytorch->torchscriptpython tools/export_libtorch.py -n yolox-s -c weights/best_ckpt.pth.tar二、libtorch环境配置其实也没有啥配置的，因为libtorch官方已经帮我们编译好了，直接下载下来用即可，到torch的官网中下载，然后解压即可，注意：一定要与模型转换的PyTorch版本一致，...
C++下的libtorch部署——（三）window10下的Visual Studio配置libtorch
qq_43199575的博客
08-24 5391
libtorch可以理解为C++版的pytorch，语法和pytorch基本相似。安装libtorch之前，要先安装好cuda以及对应的pytorch。
AODNet在Libtorch端的部署全过程(附详细源码)
Flag_ing的博客
10-31 1899
目录 一、部署环境 二、部署过程 1、VS2017配置Opencv的环境 2、CMake的安装 3、PyTorch —> Libtorch模型的转换 4、Libtorch端的实现 5、遇到的问题 一、部署环境 Windows10、Opencv3.4.10、Libtorch1.5.0-CPU-Release、PyTorch1.5.0-CUDA10.1、VS2017、CMake3.19 二、部署过程 1、VS2017配置Opencv的环境 参考博客。感谢这位博主的分享，亲测有效！
libtorch部署yolo
07-27
libtorch 是一个用于C++的PyTorch C++前端库，它提供了一个用于构建、训练和部署深度学习模型的高级API。而YOLO（You Only Look Once）是一种流行的目标检测算法，能够在图像中实时地检测和定位多个目标。部署 YOLO 模型需要以下步骤： 1. 安装 PyTorch 和 libtorch：首先，需要安装 PyTorch 和 libtorch。PyTorch 用于训练模型，而 libtorch 用于将训练好的模型部署到 C++ 环境中。 2. 训练 YOLO 模型：在 PyTorch 中，可以使用现成的 YOLO 模型实现，也可以自定义模型。通过加载训练数据集，定义模型结构并训练模型，可以得到一个经过训练的 YOLO 模型。 3. 导出模型：在训练完成后，可以通过 PyTorch 提供的导出功能将模型导出为 ONNX 格式或 TorchScript 格式，以便在 C++ 环境中使用。 4. 使用 libtorch 进行部署：在 C++ 程序中，使用 libtorch 库加载导出的模型，并编写代码进行图像的预处理和后处理。通过调用模型的前向传播函数，可以输入图像并获取模型的输出，即目标检测的结果。 5. 图像的预处理和后处理：在图像的预处理过程中，需要将图像转换为模型可接受的输入格式，例如将图像进行缩放、归一化和通道转换等操作。在模型的输出结果中，会包含被检测到的目标的位置和类别信息，可以通过解析输出结果并进行后处理，如非最大值抑制和边界框绘制等，来得到最终的目标检测结果。 综上所述，部署 YOLO 模型需要使用 libtorch 进行模型的加载和前向传播，同时编写相关的预处理和后处理代码，以实现目标检测任务的部署。
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值