若要加载模块二进制_OpenCV图像处理-DNN模块之分类和检测应用

目录

DNN模块应用

• 读取DNN模型各层信息
  • 概述
  • 代码
• DNN实现图像分类
  • 概述
  • 代码
• SSD实现目标检测
  • 单张图片检测
  • 视频检测

DNN模块应用

Opencv中结合了很多深度学习的模块，本文主要介绍下opencv结合分类模型的一些知识点，本文主要结合googlenet进行分类案例分析和Mobilenet-SSD进行检测案例分析。

读取DNN模型各层信息

概述

✔️ OpenCV的DNN模块支持下面框架的预训练模型的前馈网络(预测图)使用：

• Caffe
• Tensorflow
• Torch
• DLDT
• Darknet

同时还支持自定义层解析、非最大抑制操作、获取各层的信息等。 OpenCV加载模型的通用API为:

cv2.dnn.readNet(model,  # 模型
                config = "", 
                framework = "" )

其中：

• model二进制训练好的网络权重文件，可能来自支持的网络框架，扩展名为如下：
  • *.caffemodel (Caffe,http://caffe.berkeleyvision.org/)
  • *.pb (TensorFlow, https://www.tensorflow.org/)
  • *.t7 | *.net (Torch, http://torch.ch/)
  • *.weights (Darknet, https://pjreddie.com/darknet/)
  • *.bin (DLDT, https://software.intel.com/openvino-toolkit)
• config针对模型二进制的描述文件，不同的框架配置文件有不同扩展名：
  • *.prototxt (Caffe, http://caffe.berkeleyvision.org/)
  • *.pbtxt (TensorFlow, https://www.tensorflow.org/)
  • *.cfg (Darknet, https://pjreddie.com/darknet/)
  • *.xml (DLDT, https://software.intel.com/openvino-toolkit)
• framework显示声明参数，说明模型使用哪个框架训练出来的。

代码

import cv2
import numpy as np

bin_model = "bvlc_googlenet.caffemodel"
protxt = "bvlc_googlenet.prototxt"

# load CNN model
net = cv2.dnn.readNet(bin_model, protxt)

# 获取各层信息
layer_names = net.getLayerNames()

for name in layer_names:
    id = net.getLayerId(name)
    layer = net.getLayer(id)
    print("layer id : %d, type : %s, name: %s"%(id, layer.type, layer.name))

print("successfully")

输出

layer id : 1, type : Convolution, name: conv1/7x7_s2
layer id : 2, type : ReLU, name: conv1/relu_7x7
layer id : 3, type : Pooling, name: pool1/3x3_s2
layer id : 4, type : LRN, name: pool1/norm1
layer id : 5, type : Convolution, name: conv2/3x3_reduce
layer id : 6, type : ReLU, name: conv2/relu_3x3_reduce
layer id : 7, type : Convolution, name: conv2/3x3
layer id : 8, type : ReLU, name: conv2/relu_3x3
layer id : 9, type : LRN, name: conv2/norm2
layer id : 10, type : Pooling, name: pool2/3x3_s2
...
successfully

DNN实现图像分类

概述

✔️ 我们使用ImageNet数据集支持1000分类的GoogleNet网络模型， 结合opencv实现图像分类标签预测。

label标签是在一个单独的文本文件中读取。

读取模型的API：

cv2.dnn.readNetFromCaffe(prototxt,
                         caffeModel = String())

其中：

• prototxt表示模型的配置文件
• caffeModel表示模型的权重二进制文件

使用模型实现预测的时候，需要读取图像作为输入，网络模型支持的输入数据是四维的输入，所以要把读取到的Mat对象转换为四维张量，OpenCV的提供的API为如下：

cv2.dnn。blobFromImage(
                        image,
                        scalefactor = 1.0,
                        size = Size(),
                        mean = Scalar(),
                        swapRB = false,
                        crop = false,
                        ddepth = CV_32F 
)

其中：

• image输入图像
• scalefactor 默认1.0
• size表示网络接受的数据大小
• mean表示训练时数据集的均值
• swapRB 是否互换Red与Blur通道
• crop剪切
• ddepth 数据类型

代码

import cv2
import numpy as np

bin_model = "bvlc_googlenet.caffemodel"
protxt = "bvlc_googlenet.prototxt"

# 加载类别
classes = None
with open("classification_classes_ILSVRC2012.txt", 'rt') as f:
    classes = f.read().rstrip('n').split('n')

# 加载模型
net = cv2.dnn.readNetFromCaffe(protxt, bin_model)

# 读取输入数据
image = cv2.imread("guinea_pig.jpg")
blob = cv2.dnn.blobFromImage(image, 1.0, (224, 224), (104, 117,123), False, crop=False)
result = np.copy(image)

# 运行模型
net.setInput(blob)
out = net.forward()

# 获取最高分的类别
out = out.flatten()
classId = np.argmax(out)
confidence = out[classId]

# 输出运行时间
t, _ = net.getPerfProfile()
label = 'cost time: %.2f ms' % (t * 1000.0 / cv2.getTickFrequency())
cv2.putText(result, label, (0, 20), cv2.FONT_HERSHEY_SIMPLEX, 0.8, (255, 255, 0), 2)

# 显示结果
label = '%s: %.4f' % (classes[classId] if classes else 'Class #%d' % classId, confidence)
cv2.putText(result, label, (0, 60), cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 0, 255), 2)

show_img = np.hstack((image, result))

输出豚鼠的预测结果

分类结果

SSD实现目标检测

✔️ OpenCV的DNN模块支持常见得对象检测模型SSD， 以及Mobile Net-SSD。

✔️ 这里我们用基于Caffe训练好的mobile-net SSD来测试目标检测。

单张图片检测

import cv2

model_bin = "../model/ssd/MobileNetSSD_deploy.caffemodel";
config_text = "../model/ssd/MobileNetSSD_deploy.prototxt";
objName = ["background",
			"aeroplane", "bicycle", "bird", "boat",
			"bottle", "bus", "car", "cat", "chair",
			"cow", "diningtable", "dog", "horse",
			"motorbike", "person", "pottedplant",
			"sheep", "sofa", "train", "tvmonitor"];

# 加载模型
net = cv2.dnn.readNetFromCaffe(config_text, model_bin)
image = cv2.imread("dog.jpg")
h = image.shape[0]
w = image.shape[1]

# 获得所有层名称与索引
layerNames = net.getLayerNames()
lastLayerId = net.getLayerId(layerNames[-1])
lastLayer = net.getLayer(lastLayerId)
print(lastLayer.type)

# 加载图片检测
blobImage = cv2.dnn.blobFromImage(image, 0.007843, (300, 300), (127.5, 127.5, 127.5), True, False);
net.setInput(blobImage)
Out = net.forward()
print(Out.shape)
for detection in Out[0,0,:,:]:
    score = float(detection[2])
    objIndex = int(detection[1])
    if score > 0.5:
        left = detection[3]*w
        top = detection[4]*h
        right = detection[5]*w
        bottom = detection[6]*h

        # 绘制
        cv2.rectangle(image, (int(left), int(top)), (int(right), int(bottom)), (255, 0, 0), thickness=2)
        cv2.putText(image, "score:%.2f, %s"%(score, objName[objIndex]),
                (int(left) - 10, int(top) - 5), cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 0, 255), 2, 8);

输出

原图

检测结果

视频检测

修改代码部分如下

# 调用前置摄像头获取视频
cap = cv2.VideoCapture(0)
while True:
    ret, frame = cap.read()
    if ret is False:
        break
    h, w = frame.shape[:2]
    blobImage = cv2.dnn.blobFromImage(frame, 0.007843, (300, 300), (127.5, 127.5, 127.5), True, False);
    net.setInput(blobImage)
    cvOut = net.forward()
    for detection in cvOut[0,0,:,:]:
        score = float(detection[2])
        objIndex = int(detection[1])
        if score > 0.5:
            left = detection[3]*w
            top = detection[4]*h
            right = detection[5]*w
            bottom = detection[6]*h

            # 绘制
            cv2.rectangle(frame, (int(left), int(top)), (int(right), int(bottom)), (255, 0, 0), thickness=2)
            cv2.putText(frame, "score:%.2f, %s"%(score, objName[objIndex]),
                    (int(left) - 10, int(top) - 5), cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 0, 255), 2, 8);
    cv2.imshow('video-ssd-demo', frame)
    c = cv2.waitKey(10)
    if c == 27:
        break

------------------------------------------可爱の分割线------------------------------------------

更多Opencv教程可以 Follow github的opencv教程，中文&English 欢迎Star❤️❤️❤️

JimmyHHua/opencv_tutorials​github.com