基于华为atlas的烟火检测实战

1、下载官方yolov5的v6.1版本

git clone https://github.com/ultralytics/yolov5.git

git checkout v6.1

2、烟火数据集准备：

tree -d

Images/train/目录下图片

Labels/train/目录下标签

3、数据格式转化：

数据集采用labelimg标注，xml文件转化为txt文件的代码如下，

import os.path
import xml.etree.ElementTree as ET
 
# 类别
class_names = ["fire"]
# voc数据集路径
vocPath = './'
 
# xml文件路径
xmlPath = vocPath + r'\Annotations'
# xml转换后txt文件存放路径
txtPath = vocPath + r'\txts'
 
files = []
if not os.path.exists(txtPath):
    os.makedirs(txtPath)
 
for root, dirs, files in os.walk(xmlPath):
    None
 
number = len(files)
print(number)
i = 0
while i < number:
 
    name = files[i][0:-4]
    xml_name = name + ".xml"
    txt_name = name + ".txt"
    xml_file_name = os.path.join(xmlPath, xml_name)
    txt_file_name = os.path.join(txtPath, txt_name)
 
    xml_file = open(xml_file_name, encoding='gb18030',errors='ignore')
    tree = ET.parse(xml_file)
    root = tree.getroot()
 
    w = int(root.find('size').find('width').text)
    h = int(root.find('size').find('height').text)
 
    f_txt = open(txt_file_name, 'w+')
    content = ""
 
    first = True
 
    for obj in root.iter('object'):
 
        name = obj.find('name').text
        # 若只有一类 ,即 class_num = 0
        class_num = class_names.index(name)
 
        xmlbox = obj.find('bndbox')
 
        x1 = int(xmlbox.find('xmin').text)
        x2 = int(xmlbox.find('xmax').text)
        y1 = int(xmlbox.find('ymin').text)
        y2 = int(xmlbox.find('ymax').text)
 
        if first:
            content += str(class_num) + " " + \
                       str((x1 + x2) / 2 / w) + " " + str((y1 + y2) / 2 / h) + " " + \
                       str((x2 - x1) / w) + " " + str((y2 - y1) / h)
            first = False
        else:
            content += "\n" + \
                       str(class_num) + " " + \
                       str((x1 + x2) / 2 / w) + " " + str((y1 + y2) / 2 / h) + " " + \
                       str((x2 - x1) / w) + " " + str((y2 - y1) / h)
 
    print(content)
    f_txt.write(content)
    f_txt.close()
    xml_file.close()
    i += 1
    

4、配置yaml文件：

data/fire_smoke.yaml

5、启动训练：

python train.py --img 640 --epochs 100 --data ./data/fire_smoke.yaml --weights yolov5s.pt

6、Pt模型转化为onnx模型

python export.py --weights best.pt --simplify

7、模型转化为atlas模型：

mkdir -p models/yolov5_fire_smoke

新建insert_op.cfg

aipp_op {
aipp_mode : static
related_input_rank : 0
input_format : YUV420SP_U8
src_image_size_w : 640
src_image_size_h : 640
crop : false
csc_switch : true
rbuv_swap_switch : false
matrix_r0c0 : 256
matrix_r0c1 : 0
matrix_r0c2 : 359
matrix_r1c0 : 256
matrix_r1c1 : -88
matrix_r1c2 : -183
matrix_r2c0 : 256
matrix_r2c1 : 454
matrix_r2c2 : 0
input_bias_0 : 0
input_bias_1 : 128
input_bias_2 : 128
var_reci_chn_0 : 0.0039216
var_reci_chn_1 : 0.0039216
var_reci_chn_2 : 0.0039216
}

新建yolov5_add_bs1_fp16.cfg

CLASS_NUM=2
BIASES_NUM=18
BIASES=10,13,16,30,33,23,30,61,62,45,59,119,116,90,156,198,373,326
SCORE_THRESH=0.25
#SEPARATE_SCORE_THRESH=0.001,0.001,0.001,0.001,0.001,0.001,0.001,0.001,0.001,0.001
OBJECTNESS_THRESH=0.0
IOU_THRESH=0.5
YOLO_TYPE=3
ANCHOR_DIM=3
MODEL_TYPE=2
RESIZE_FLAG=0
YOLO_VERSION=5

新建fire_smoke.names

fire
smoke

将yolov5的best.onnx模型拷贝到当前目录，进行onnx转化为om，输出yolov5_add_bs1_fp16.om

输入npu-smi info

atc  --input_shape="images:1,3,640,640" --out_nodes="/model.24/Transpose:0;/model.24/Transpose_1:0;/model.24/Transpose_2:0" --output_type=FP32 --input_format=NCHW --output="./yolov5_add_bs1_fp16" --soc_version=Ascend310P3 --framework=5 --model="./best.onnx" --insert_op_conf=./insert_op.cfg

8、修改华为atlas推理的pipeline文件

修改pipeline/fire_smoke.pipeline文件

9、基于pipenine实现推理代码

实现简单的yolov5的推理函数yolov5.py，并运行

python3 yolov5.py

10、流媒体引擎ZLMediaKit搭建：

编译库

git clone https://github.com/ZLMediaKit/ZLMediaKit.git
cd ZLMediaKit/
git submodule update --init
mkdir build
cd build
cmake ..
make -j4

11、运行流媒体引擎库：

cd ZLMediaKit/release/linux/Debug
#通过-h可以了解启动参数
./MediaServer -h
#以守护进程模式启动
./MediaServer -d &

12、运行算法服务：

python3 server.py >&/dev/null&

13、运行视频处理业务：

python3 push_stream.py

在VLC中进行播放，

rtmp://10.100.1.1:19350/live/test

http://10.100.1.1:19350/live/test.live.flv

14、信创化容器制作：

实现信创化的docker file用于生成docker image，初始系统选择openeuler-20.03系统，docker file文件内容如下，

#FROM opstool/openeuler:20.09
FROM openeuler/openeuler:20.03-lts-sp3

RUN mv /usr/bin/sh /usr/bin/sh.bak && ln -s /usr/bin/bash /usr/bin/sh



#RUN cat /etc/yum.repos.d/openEuler.repo
RUN sed -i 's/http:\/\/repo.openeuler.org/https:\/\/repo.huaweicloud.com\/openeuler/g' /etc/yum.repos.d/openEuler.repo
#RUN cat /etc/yum.repos.d/openEuler.repo


#RUN yum update
RUN yum install -y cmake make
RUN yum install -y wget tar zlib-devel.aarch64
RUN yum install -y mesa-libGL.aarch64 openssl-devel 
RUN yum install -y libffi-devel
RUN yum install -y git
RUN yum install -y gcc gcc-c++.aarch64


#ffmpeg
RUN yum install -y nasm 
RUN yum install -y x264-devel.aarch64
RUN curl -O -L http://anduin.linuxfromscratch.org/BLFS/x265/x265_3.4.tar.gz && tar -xzvf x265_3.4.tar.gz && cd x265_3.4/build/linux/ && cmake -G "Unix Makefiles" -DCMAKE_INSTALL_PREFIX="$INSTALL_PATH" -DENABLE_SHARED:bool=on ../../source && make -j8 && make install

RUN wget https://pkg-config.freedesktop.org/releases/pkg-config-0.29.2.tar.gz && tar -xvf pkg-config-0.29.2.tar.gz && cd pkg-config-0.29.2 && ./configure  --with-internal-glib && make -j9 &&make install


RUN git clone https://github.com/ksvc/FFmpeg.git -b release/3.4 --depth=1 && sed -i 's/x264_bit_depth/X264_BIT_DEPTH/g' FFmpeg/libavcodec/libx264.c
RUN cd FFmpeg && ./configure --extra-libs="-lpthread -lm" --ld="g++" --enable-static --enable-pic --enable-encoder=aac --enable-encoder=libx264 --enable-gpl --enable-libx264 --enable-encoder=libx265 --enable-libx265 --enable-decoder=aac --enable-decoder=h264 --enable-decoder=hevc --enable-demuxer=aac --enable-demuxer=mov --enable-demuxer=mpegts --enable-demuxer=flv --enable-demuxer=h264 --enable-demuxer=hevc --enable-demuxer=hls --enable-muxer=h264 --enable-muxer=flv --enable-muxer=f4v  --enable-muxer=mp4 --disable-doc --enable-nonfree && make -j8 && make install


RUN wget https://www.python.org/ftp/python/3.9.12/Python-3.9.12.tgz
RUN tar -xzvf Python-3.9.12.tgz
RUN cd Python-3.9.12 &&./configure --prefix=/usr/local/python3.9.12 --enable-shared && make -j8 && make install
RUN cp /usr/local/python3.9.12/lib/libpython3.9.so.1.0 /usr/lib


#RUN export LD_LIBRARY_PATH=/usr/local/python3.9.12/lib:$LD_LIBRARY_PATH
#RUN export PATH=/usr/local/python3.9.12/bin:$PATH


RUN echo "export LD_LIBRARY_PATH=/usr/local/python3.9.12/lib:$LD_LIBRARY_PATH" >> ~/.bashrc
RUN echo "export PATH=/usr/local/python3.9.12/bin:$PATH" >> ~/.bashrc

RUN source ~/.bashrc


RUN yum install -y python3-pip
RUN echo "source /data/ai_install_packages/MindX_SDK/mxVision/set_env.sh" >> ~/.bashrc
RUN echo "source /usr/local/Ascend/ascend-toolkit/set_env.sh" >> ~/.bashrc


RUN /usr/local/python3.9.12/bin/pip3 install opencv-python opencv-python-headless Pillow -i https://pypi.tuna.tsinghua.edu.cn/simple/
RUN /usr/local/python3.9.12/bin/pip3 install attrs cloudpickle  decorator psutil scipy  synr==0.5.0 tornado absl-py -i https://pypi.tuna.tsinghua.edu.cn/simple/
RUN /usr/local/python3.9.12/bin/pip3 install absl-py Flask gunicorn tqdm requests -i https://pypi.tuna.tsinghua.edu.cn/simple/

15、docker环境部署：

#构建镜像
docker build . -t sitri/openeuler-20.03-ai:1.0.0
#保存镜像
#docker save -o myimage-20.03.tar sitri/openeuler-20.03-ai:1.0.0
#导入保存的镜像
#docker load -i myimage-20.03.tar

docker run --restart=always -itd -u root \
--network host \
--device=/dev/davinci0 \
--device=/dev/davinci_manager \
--device=/dev/devmm_svm \
--device=/dev/hisi_hdc \
-v /usr/local/dcmi:/usr/local/dcmi \
-v /var/log/npu:/var/log/npu \
-v /usr/local/Ascend/driver:/usr/local/Ascend/driver \
-v /usr/slog:/usr/slog \
-v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
-v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ \
-v /usr/local/Ascend/driver/tools/:/usr/local/Ascend/driver/tools/ \
-v /usr/local/Ascend/add-ons/:/usr/local/Ascend/add-ons/ \
-v /usr/local/Ascend/ascend-toolkit/:/usr/local/Ascend/ascend-toolkit/ \
-v /data:/data \
--name="firesmoke" \
-w /data/ai_install_packages/fire_smoke \
sitri/openeuler-20.03-ai:1.0.0 \
/bin/bash \
-c "source ~/.bashrc && gunicorn -c gunicorn_config.py server:app"

16、整体效果

基于flask实现烟火检测算法的http服务，然后实现视频解码-AI识别-结果绘制于视频上进行视频编码的业务代码。

最终效果如下，上边为业务代码、左下角为流媒体引擎代码、右下角为AI服务代码、中间为AI实时视频识别效果。

references:

文档：

昇腾社区-官网丨昇腾万里 让智能无所不及

案例：

昇腾社区-官网丨昇腾万里 让智能无所不及

github：

ascend_community_projects: MindX边缘开发套件社区代码仓库

samples: CANN Samples

容器镜像：

AscendHub