Jetson nano及模型部署

参考链接：

Developer Guide :: NVIDIA Deep Learning TensorRT Documentation

Developer Guide :: NVIDIA Deep Learning TensorRT Documentation

Improving INT8 Accuracy Using Quantization Aware Training and the NVIDIA TAO Toolkit | NVIDIA Technical Blog

项目：

https://github.com/beyondli/Yolo_on_Jetson

Jetson nano使用说明：

JETSON NANO - Waveshare Wiki

nano的浮点算力：0.5TFLOPS（FP16）

FP32 = float32 单精度浮点格式

64 位 base-2双精度

算力单位概述 - 知乎 (zhihu.com)

FP32、FP16、int8

FP32、FP16和INT8_fp32 int8_Stars-Chan的博客-CSDN博客

#cd outside Yolo_on_Jetson to install pytorch-quantization
python3 -m pip install --upgrade pip
git clone https://github.com/NVIDIA/TensorRT.git
cd tools/pytorch-quantization
python3 setup.py install

#added this export to ~/.bashrc
export LD_PRELOAD=/usr/lib/aarch64-linux-gnu/libgomp.so.1
source ~/.bashrc

pip3 install tqdm
pip3 install seaborn
pip3 install pycocotools

apt update
apt install python3-tk

#for Xaver AGX
python3 models/export.py --weights  ./weights/yolov5s.pt --img 640 --batch 1 --device 0

#for Orin AGX
#modify /usr/local/lib/python3.8/dist-packages/torch/nn/modules/upsampling.py line 152
def forward(self, input: Tensor) -> Tensor:
    #return F.interpolate(input, self.size, self.scale_factor, self.mode, self.align_corners,
    #                     recompute_scale_factor=self.recompute_scale_factor)
    return F.interpolate(input, self.size, self.scale_factor, self.mode, self.align_corners)

#then export model
python3 models/export.py --weights  ./weights/yolov5s.pt --img 640 --batch 1
 

python3 trt/onnx_to_trt.py --model ./weights/yolov5s.onnx --dtype fp32
mv weights/yolov5s.engine weights/yolov5s_fp32.engine

python3 trt/onnx_to_trt.py --model ./weights/yolov5s.onnx --dtype fp16
mv weights/yolov5s.engine weights/yolov5s_fp16.engine

python3 trt/onnx_to_trt.py --model weights/yolov5s.onnx --batch-size 1 --img-size 640 --calib-img-dir ../../data/COCO/calibrate_random/ --dtype int8

#evaluate fp32
python3 trt/eval_yolo_trt.py --model ./weights/yolov5s_fp32.engine 

#evaluate fp16
python3 trt/eval_yolo_trt.py --model ./weights/yolov5s_fp16.engine 

#evaluate int8
python3 trt/eval_yolo_trt.py --model ./weights/yolov5s_int8.engine 

#result images saved at images_test/ret
python3 trt/batch_test.py -m ./weights/yolov5s_fp16.engine

...
[property]
gpu-id=0
net-scale-factor=0.0039215697906911373
model-color-format=0

network-mode=0
model-engine-file=./models/yolov5s_fp32.engine

#network-mode=2
#model-engine-file=./models/yolov5s_fp16.engine

#network-mode=1
#model-engine-file=./models/yolov5s_int8.engine
#int8-calib-file=./models/yolov5s_calibration.cache

#for Xavier AGX
cd Yolo_on_Jetson/deepstream
CUDA_VER=10.2 make -C nvdsinfer_custom_impl_Yolo

#for Orin AGX
CUDA_VER=11.4 make -C nvdsinfer_custom_impl_Yolo
 

deepstream-app -c deepstream_app_config.txt

/TensorRT/tools/pytorch-quantization