骁龙神经处理引擎SDK参考指南（21）

最新推荐文章于 2023-12-21 15:45:29 发布

weixin_38498942

最新推荐文章于 2023-12-21 15:45:29 发布

阅读量761

点赞数

分类专栏：神经处理引擎笔记 sdk 文章标签：算法

本文链接：https://blog.csdn.net/weixin_38498942/article/details/131214562

版权

笔记同时被 3 个专栏收录

364 篇文章 44 订阅

订阅专栏

sdk

281 篇文章 30 订阅

订阅专栏

神经处理引擎

17 篇文章 8 订阅

订阅专栏

骁龙神经处理引擎SDK参考指南（21）

- 6.5 代码示例
- - 6.5.1 C++ 教程 - 构建示例

6.5 代码示例

6.5.1 C++ 教程 - 构建示例

先决条件

SNPE SDK 已按照SNPE 设置章节进行设置。
教程设置已完成。

介绍

本教程演示如何构建可在 PC 或目标设备上执行神经网络模型的 C++ 示例应用程序。

注意：虽然此示例代码不执行任何错误检查，但强烈建议用户在使用 SNPE API 时检查错误。

大多数应用程序在使用神经网络时将遵循以下模式：

获取可用运行时
负载网络
载入 UDO
设置网络生成器选项
加载网络输入
- 使用 ITensors
- 使用用户缓冲区
执行网络和过程输出
- 使用 ITensors
- 使用用户缓冲区

static zdl::DlSystem::Runtime_t runtime = checkRuntime();
std::unique_ptr<zdl::DlContainer::IDlContainer> container = loadContainerFromFile(dlc);
std::unique_ptr<zdl::SNPE::SNPE> snpe = setBuilderOptions(container, runtime, useUserSuppliedBuffers);
std::unique_ptr<zdl::DlSystem::ITensor> inputTensor = loadInputTensor(snpe, fileLine); // ITensor
loadInputUserBuffer(applicationInputBuffers, snpe, fileLine); // User Buffer
executeNetwork (snpe , inputTensor, OutputDir, inputListNum); // ITensor
executeNetwork(snpe, inputMap, outputMap, applicationOutputBuffers, OutputDir, inputListNum); // User Buffer

以下部分描述了如何实施上述每个步骤。更多详细信息，请参考位于$SNPE_ROOT/examples/NativeCpp/SampleCode/jni 的源代码文件集合。

获取可用运行时
下面的代码摘录说明了如何使用本机 API 检查特定运行时是否可用（以 GPU 运行时为例）。

zdl::DlSystem::Runtime_t checkRuntime()
{
    static zdl::DlSystem::Version_t Version = zdl::SNPE::SNPEFactory::getLibraryVersion();
    static zdl::DlSystem::Runtime_t Runtime;
    std::cout << "SNPE Version: " << Version.asString().c_str() << std::endl; //Print Version number
    if (zdl::SNPE::SNPEFactory::isRuntimeAvailable(zdl::DlSystem::Runtime_t::GPU)) {
        Runtime = zdl::DlSystem::Runtime_t::GPU;
    } else {
        Runtime = zdl::DlSystem::Runtime_t::CPU;
    }
    return Runtime;
}

负载网络
下面的代码摘录说明了如何从 SNPE 容器文件 (DLC) 加载网络。

std::unique_ptr<zdl::DlContainer::IDlContainer> loadContainerFromFile(std::string containerPath)
{
    std::unique_ptr<zdl::DlContainer::IDlContainer> container;
    container = zdl::DlContainer::IDlContainer::open(containerPath);
    return container;
}

载入 UDO

下面的代码摘录说明了如何加载 UDO 包。

bool loadUDOPackage(const std::string& UdoPackagePath)
{
    std::vector<std::string> udoPkgPathsList;
    split(udoPkgPathsList, UdoPackagePath, ',');
    for (const auto &u : udoPkgPathsList)
    {
       if (false == zdl::SNPE::SNPEFactory::addOpPackage(u))
       {
          std::cerr << "Error while loading UDO package: "<< u << std::endl;
          return false;
       }
    }
    return true;
}

SNPE 可以使用用户定义的操作 (udo) 执行网络。请参考UDO 教程来实现一个 udo。

然后在 snpe-sample 后加上“-u”选项执行。

设置网络生成器选项

以下代码演示了如何实例化 SNPE Builder 对象，该对象将用于使用给定参数执行网络。

std::unique_ptr<zdl::SNPE::SNPE> setBuilderOptions(std::unique_ptr<zdl::DlContainer::IDlContainer>& container,
                                                   zdl::DlSystem::RuntimeList runtimeList,
                                                   bool useUserSuppliedBuffers)
{
    std::unique_ptr<zdl::SNPE::SNPE> snpe;
    zdl::SNPE::SNPEBuilder snpeBuilder(container.get());
    snpe = snpeBuilder.setOutputLayers({})
       .setRuntimeProcessorOrder(runtimeList)
       .setUseUserSuppliedBuffers(useUserSuppliedBuffers)
       .build();
    return snpe;
}

加载网络输入
网络输入和输出可以是用户支持的缓冲区或 ITensors（内置 SNPE 缓冲区），但不能同时是两者。使用用户支持的缓冲区的优点是它消除了从用户缓冲区创建 ITensors 的额外副本。加载网络输入的两种方法如下所示。

使用用户缓冲区
SNPE 可以从用户支持的缓冲区创建其网络输入和输出。请注意，SNPE 期望缓冲区的值在其执行期间存在并有效。

这是一个用于从用户支持的缓冲区创建 SNPE UserBuffer 并将其存储在zdl::DlSystem::UserBufferMap中的函数。这些映射是所有输入或输出用户缓冲区的方便集合，可以传递给 SNPE 以执行网络。

免责声明：缓冲区的步幅应该已经为用户所知，不应按如下所示计算。显示的计算仅用于执行示例代码。

void createUserBuffer(zdl::DlSystem::UserBufferMap& userBufferMap,
                      std::unordered_map<std::string, std::vector<uint8_t>>& applicationBuffers,
                      std::vector<std::unique_ptr<zdl::DlSystem::IUserBuffer>>& snpeUserBackedBuffers,
                      std::unique_ptr<zdl::SNPE::SNPE>& snpe,
                      const char * name)
{
   // get attributes of buffer by name
   auto bufferAttributesOpt = snpe->getInputOutputBufferAttributes(name);
   if (!bufferAttributesOpt) throw std::runtime_error(std::string("Error obtaining attributes for input tensor ") + name);
   // calculate the size of buffer required by the input tensor
   const zdl::DlSystem::TensorShape& bufferShape = (*bufferAttributesOpt)->getDims();
   // Calculate the stride based on buffer strides, assuming tightly packed.
   // Note: Strides = Number of bytes to advance to the next element in each dimension.
   // For example, if a float tensor of dimension 2x4x3 is tightly packed in a buffer of 96 bytes, then the strides would be (48,12,4)
   // Note: Buffer stride is usually known and does not need to be calculated.
   std::vector<size_t> strides(bufferShape.rank());
   strides[strides.size() - 1] = sizeof(float);
   size_t stride = strides[strides.size() - 1];
   for (size_t i = bufferShape.rank() - 1; i > 0; i--)
   {
      stride *= bufferShape[i];
      strides[i-1] = stride;
   }
   const size_t bufferElementSize = (*bufferAttributesOpt)->getElementSize();
   size_t bufSize = calcSizeFromDims(bufferShape.getDimensions(), bufferShape.rank(), bufferElementSize);
   // set the buffer encoding type
   zdl::DlSystem::UserBufferEncodingFloat userBufferEncodingFloat;
   // create user-backed storage to load input data onto it
   applicationBuffers.emplace(name, std::vector<uint8_t>(bufSize));
   // create SNPE user buffer from the user-backed buffer
   zdl::DlSystem::IUserBufferFactory& ubFactory = zdl::SNPE::SNPEFactory::getUserBufferFactory();
   snpeUserBackedBuffers.push_back(ubFactory.createUserBuffer(applicationBuffers.at(name).data(),
                                                              bufSize,
                                                              strides,
                                                              &userBufferEncodingFloat));
   // add the user-backed buffer to the inputMap, which is later on fed to the network for execution
   userBufferMap.add(name, snpeUserBackedBuffers.back().get());
}

然后，以下函数显示了如何将输入数据从文件加载到用户缓冲区。请注意，输入值只是简单地加载到用户支持的缓冲区中，SNPE 可以在其上创建 SNPE UserBuffers，如上所示。

void loadInputUserBuffer(std::unordered_map<std::string, std::vector<uint8_t>>& applicationBuffers,
                               std::unique_ptr<zdl::SNPE::SNPE>& snpe,
                               const std::string& fileLine)
{
    // get input tensor names of the network that need to be populated
    const auto& inputNamesOpt = snpe->getInputTensorNames();
    if (!inputNamesOpt) throw std::runtime_error("Error obtaining input tensor names");
    const zdl::DlSystem::StringList& inputNames = *inputNamesOpt;
    assert(inputNames.size() > 0);
    // treat each line as a space-separated list of input files
    std::vector<std::string> filePaths;
    split(filePaths, fileLine, ' ');
    if (inputNames.size()) std::cout << "Processing DNN Input: " << std::endl;
    for (size_t i = 0; i < inputNames.size(); i++) {
        const char* name = inputNames.at(i);
        std::string filePath(filePaths[i]);
        // print out which file is being processed
        std::cout << "\t" << i + 1 << ") " << filePath << std::endl;
        // load file content onto application storage buffer,
        // on top of which, SNPE has created a user buffer
        loadByteDataFile(filePath, applicationBuffers.at(name));
    };
}

使用 ITensors

std::unique_ptr<zdl::DlSystem::ITensor> loadInputTensor (std::unique_ptr<zdl::SNPE::SNPE> & snpe , std::string& fileLine)
{
    std::unique_ptr<zdl::DlSystem::ITensor> input;
    const auto &strList_opt = snpe->getInputTensorNames();
    if (!strList_opt) throw std::runtime_error("Error obtaining Input tensor names");
    const auto &strList = *strList_opt;
    // Make sure the network requires only a single input
    assert (strList.size() == 1);
    // If the network has a single input, each line represents the input file to be loaded for that input
    std::string filePath(fileLine);
    std::cout << "Processing DNN Input: " << filePath << "\n";
    std::vector<float> inputVec = loadFloatDataFile(filePath);
    /* Create an input tensor that is correctly sized to hold the input of the network. Dimensions that have no fixed size will be represented with a value of 0. */
    const auto &inputDims_opt = snpe->getInputDimensions(strList.at(0));
    const auto &inputShape = *inputDims_opt;
    /* Calculate the total number of elements that can be stored in the tensor so that we can check that the input contains the expected number of elements.
       With the input dimensions computed create a tensor to convey the input into the network. */
    input = zdl::SNPE::SNPEFactory::getTensorFactory().createTensor(inputShape);
    /* Copy the loaded input file contents into the networks input tensor.SNPE's ITensor supports C++ STL functions like std::copy() */
    std::copy(inputVec.begin(), inputVec.end(), input->begin());
    return input;
}

执行网络和过程输出
以下代码片段使用本机 API 执行网络（在 UserBuffer 或 ITensor 模式下）并展示如何迭代新填充的输出张量。

使用用户缓冲区

void executeNetwork(std::unique_ptr<zdl::SNPE::SNPE>& snpe,
                    zdl::DlSystem::UserBufferMap& inputMap,
                    zdl::DlSystem::UserBufferMap& outputMap,
                    std::unordered_map<std::string,std::vector<uint8_t>>& applicationOutputBuffers,
                    const std::string& outputDir,
                    int num)
{
    // Execute the network and store the outputs in user buffers specified in outputMap
    snpe->execute(inputMap, outputMap);
    // Get all output buffer names from the network
    const zdl::DlSystem::StringList& outputBufferNames = outputMap.getUserBufferNames();
    // Iterate through output buffers and print each output to a raw file
    std::for_each(outputBufferNames.begin(), outputBufferNames.end(), [&](const char* name)
    {
       std::ostringstream path;
       path << outputDir << "/Result_" << num << "/" << name << ".raw";
       SaveUserBuffer(path.str(), applicationOutputBuffers.at(name));
    });
}
// The following is a partial snippet of the function
void SaveUserBuffer(const std::string& path, const std::vector<uint8_t>& buffer) {
   ...
   std::ofstream os(path, std::ofstream::binary);
   if (!os)
   {
      std::cerr << "Failed to open output file for writing: " << path << "\n";
      std::exit(EXIT_FAILURE);
   }
   if (!os.write((char*)(buffer.data()), buffer.size()))
   {
      std::cerr << "Failed to write data to: " << path << "\n";
      std::exit(EXIT_FAILURE);
   }
}

使用 ITensors

void executeNetwork(std::unique_ptr<zdl::SNPE::SNPE>& snpe,
                    std::unique_ptr<zdl::DlSystem::ITensor>& input,
                    std::string OutputDir,
                    int num)
{
    //Execute the network and store the outputs that were specified when creating the network in a TensorMap
    static zdl::DlSystem::TensorMap outputTensorMap;
    snpe->execute(input.get(), outputTensorMap);
    zdl::DlSystem::StringList tensorNames = outputTensorMap.getTensorNames();
    //Iterate through the output Tensor map, and print each output layer name
    std::for_each( tensorNames.begin(), tensorNames.end(), [&](const char* name)
    {
        std::ostringstream path;
        path << OutputDir << "/"
        << "Result_" << num << "/"
        << name << ".raw";
        auto tensorPtr = outputTensorMap.getTensor(name);
        SaveITensor(path.str(), tensorPtr);
    });
}
// The following is a partial snippet of the function
void SaveITensor(const std::string& path, const zdl::DlSystem::ITensor* tensor)
{
   ...
   std::ofstream os(path, std::ofstream::binary);
   if (!os)
   {
      std::cerr << "Failed to open output file for writing: " << path << "\n";
      std::exit(EXIT_FAILURE);
   }
   for ( auto it = tensor->cbegin(); it != tensor->cend(); ++it )
   {
      float f = *it;
      if (!os.write(reinterpret_cast<char*>(&f), sizeof(float)))
      {
         std::cerr << "Failed to write data to: " << path << "\n";
         std::exit(EXIT_FAILURE);
      }
   }
}

使用 IOBufferDataTypeMap

IOBufferDataTypeMap 用于指定网络输入/输出的预期数据类型。数据类型值包括 zdl::DlSystem::IOBufferDataType_t::FLOATING_POINT_32、zdl::DlSystem::IOBufferDataType_t::FIXED_POINT_8 和 zdl::DlSystem::IOBufferDataType_t::FIXED_POINT_16。
如果网络的输出是 FIXED_POINT_8 类型，但是用户打算访问 FLOATING_POINT_32 格式的输出。反量化操作在臂端执行。通过使用 IOBufferDataTypeMap API 将数据类型指定为 FLOATING_POINT_32，反量化操作直接添加到图形中。

以下代码片段显示了如何使用本机 API 指定缓冲区的数据类型。

void setBufferDataType(zdl::DlSystem::IOBufferDataTypeMap& bufferDataTypeMap, std::string bufferName, zdl::DlSystem::IOBufferDataType_t dataType)
{
    bufferDataTypeMap.add(bufferName.c_str(), dataType);
}
setBufferDataType(bufferDataTypeMap, "output_1", zdl::DlSystem::IOBufferDataType_t::FLOATING_POINT_32);
zdl::SNPE::SNPEBuilder snpeBuilder(container.get());
snpeBuilder.setBufferDataType(bufferDataTypeMap);

构建 C++ 应用程序
在 x86 Linux 和嵌入式 Linux 上构建和运行

首先转到 snpe-sample 基本目录。

cd $SNPE_ROOT/示例/NativeCpp/SampleCode

请注意与不同 Linux 平台关联的不同 makefile。请注意，需要根据目标平台设置 $CXX。下面是受支持目标的表格，以及它们对 $CXX 和要使用的 Makefile 的相应设置。

目标	生成文件	可能的 CXX 值	输出位置
arm-oe-linux (gcc 6.4hf)	Makefile.arm-oe-linux-gcc6.4hf	arm-oe-linux-g++	arm-oe-linux-gcc6.4hf
aarch64-oe-linux (gcc 6.4)	Makefile.aarch64-oe-linux-gcc6.4	aarch64-oe-linux-g++	aarch64-oe-linux-gcc6.4
x86_64-linux	Makefile.x86_64-linux-clang	g++	x86_64-linux-clang

export CXX=<Name of c++ cross compiler>
make -f <Makefile for the target>

注意：确保已在 $PATH 中设置编译器二进制文件的路径。

连同示例可执行文件，所有其他库都需要推送到它们各自的目标上。$LD_LIBRARY_PATH 可能还需要更新以指向支持库。您可以使用 -h 参数运行可执行文件以查看其描述。

snpe-sample -h

描述应如下所示：

DESCRIPTION:
------------
Example application demonstrating how to load and execute a neural network
using the SNPE C++ API.


REQUIRED ARGUMENTS:
-------------------
  -d  <FILE>   Path to the DL container containing the network.
  -i  <FILE>   Path to a file listing the inputs for the network.
  -o  <PATH>   Path to directory to store output results.

OPTIONAL ARGUMENTS:
-------------------
  -b  <TYPE>    Type of buffers to use [USERBUFFER, ITENSOR] (default is USERBUFFER).
  -u  <VAL,VAL> Path to UDO package with registration library for UDOs.
                Optionally, user can provide multiple packages as a comma-separated list.

运行 snpe-sample 假设先前已设置运行 AlexNet 模型或运行 Inception v3 模型的示例之一。

使用 AlexNet 模型运行snpe-sample ：

cd $SNPE_ROOT/models/alexnet/data
$SNPE_ROOT/examples/NativeCpp/SampleCode/obj/local/x86_64-linux-clang/snpe-sample -b ITENSOR -d ../dlc/bvlc_alexnet.dlc -i target_raw_list.txt -o output

结果存储在输出目录中。要处理输出，请运行以下脚本以生成分类结果。

python $SNPE_ROOT/models/alexnet/scripts/show_alexnet_classifications.py -i target_raw_list.txt -o output/ -l ilsvrc_2012_labels.txt
Classification results
cropped/trash_bin.raw     0.949348 412 ashcan, trash can, garbage can, wastebin, ash bin, ash-bin, ashbin, dustbin, trash barrel, trash bin
cropped/chairs.raw        0.365685 831 studio couch, day bed
cropped/plastic_cup.raw   0.749103 647 measuring cup
cropped/notice_sign.raw   0.722709 458 brass, memorial tablet, plaque
cropped/handicap_sign.raw 0.188248 919 street sign

在 ARM Android 上构建和运行

先决条件：您将需要 Android NDK 来构建 Android C++ 可执行文件。本教程假定您可以从 shell 调用“ndk-build”。

首先移动到 snpe-sample 的基本目录。

cd $SNPE_ROOT/examples/NativeCpp/SampleCode

要使用 clang/libc++ SNPE 二进制文件（即 arm-android-clang6.0 和 aarch64-android-clang6.0）构建 snpe-sample，请使用以下命令：

cd $SNPE_ROOT/examples/NativeCpp/SampleCode
ndk-build NDK_TOOLCHAIN_VERSION=clang APP_STL=c++_shared```


ndk-build 命令将构建 snpe-sample 的 armeabi-v7a 和 arm64-v8a 二进制文件。

- $SNPE_ROOT/examples/NativeCpp/SampleCode/obj/local/armeabi-v7a/ snpe-sample
- $SNPE_ROOT/examples/NativeCpp/SampleCode/obj/local/arm64-v8a/ snpe-sample
要运行 Android C++ 可执行文件，请将适当的 SNPE 库和可执行文件推送到 Android 目标上。

```c
export SNPE_TARGET_ARCH=arm-android-clang6.0
export SNPE_TARGET_ARCH_OBJ_DIR=armeabi-v7a
adb shell "mkdir -p /data/local/tmp/snpeexample/$SNPE_TARGET_ARCH/bin"
adb shell "mkdir -p /data/local/tmp/snpeexample/$SNPE_TARGET_ARCH/lib"
adb shell "mkdir -p /data/local/tmp/snpeexample/dsp/lib"
adb push $SNPE_ROOT/lib/$SNPE_TARGET_ARCH/ /data/local/tmp/snpeexample/$SNPE_TARGET_ARCH/lib
adb push $SNPE_ROOT/lib/dsp/ /data/local/tmp/snpeexample/dsp/lib
adb push $SNPE_ROOT/examples/NativeCpp/SampleCode/obj/local/$SNPE_TARGET_ARCH_OBJ_DIR/snpe-sample /data/local/tmp/snpeexample/$SNPE_TARGET_ARCH/bin

在目标上使用 Alexnet 模型运行snpe-sample 。这假定您已完成在 Android Target 上运行运行的设置步骤，以将所有示例数据文件和 Alexnet 模型推送到目标。

adb shell
export SNPE_TARGET_ARCH=arm-android-clang6.0
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/data/local/tmp/snpeexample/$SNPE_TARGET_ARCH/lib
export PATH=$PATH:/data/local/tmp/snpeexample/$SNPE_TARGET_ARCH/bin
cd /data/local/tmp/alexnet
snpe-sample -b ITENSOR -d bvlc_alexnet.dlc -i target_raw_list.txt -o output_sample
exit

将目标输出拉入主机端输出目录。

cd $SNPE_ROOT/models/alexnet/
adb pull /data/local/tmp/alexnet/output_sample output_sample

同样，我们可以运行解释脚本来查看分类结果。

python $SNPE_ROOT/models/alexnet/scripts/show_alexnet_classifications.py -i data/target_raw_list.txt -o output_sample/ -l data/ilsvrc_2012_labels.txt
Classification results
cropped/trash_bin.raw     0.949348 412 ashcan, trash can, garbage can, wastebin, ash bin, ash-bin, ashbin, dustbin, trash barrel, trash bin
cropped/chairs.raw        0.365685 831 studio couch, day bed
cropped/plastic_cup.raw   0.749103 647 measuring cup
cropped/notice_sign.raw   0.722709 458 brass, memorial tablet, plaque
cropped/handicap_sign.raw 0.188248 919 street sign