目录
DNN模块应用
- 读取DNN模型各层信息
- 概述
- 代码
- DNN实现图像分类
- 概述
- 代码
- SSD实现目标检测
- 单张图片检测
- 视频检测
DNN模块应用
Opencv中结合了很多深度学习的模块,本文主要介绍下opencv结合分类模型的一些知识点,本文主要结合googlenet进行分类案例分析和Mobilenet-SSD进行检测案例分析。
读取DNN模型各层信息
概述
✔️ OpenCV的DNN模块支持下面框架的预训练模型的前馈网络(预测图)使用:
- Caffe
- Tensorflow
- Torch
- DLDT
- Darknet
同时还支持自定义层解析、非最大抑制操作、获取各层的信息等。 OpenCV加载模型的通用API为:
cv2.dnn.readNet(model, # 模型
config = "",
framework = "" )
其中:
- model二进制训练好的网络权重文件,可能来自支持的网络框架,扩展名为如下:
- *.caffemodel (Caffe,http://caffe.berkeleyvision.org/)
- *.pb (TensorFlow, https://www.tensorflow.org/)
- *.t7 | *.net (Torch, http://torch.ch/)
- *.weights (Darknet, https://pjreddie.com/darknet/)
- *.bin (DLDT, https://software.intel.com/openvino-toolkit)
- config针对模型二进制的描述文件,不同的框架配置文件有不同扩展名:
- *.prototxt (Caffe, http://caffe.berkeleyvision.org/)
- *.pbtxt (TensorFlow, https://www.tensorflow.org/)
- *.cfg (Darknet, https://pjreddie.com/darknet/)
- *.xml (DLDT, https://software.intel.com/openvino-toolkit)
- framework显示声明参数,说明模型使用哪个框架训练出来的。
代码
import cv2
import numpy as np
bin_model = "bvlc_googlenet.caffemodel"
protxt = "bvlc_googlenet.prototxt"
# load CNN model
net = cv2.dnn.readNet(bin_model, protxt)
# 获取各层信息
layer_names = net.getLayerNames()
for name in layer_names:
id = net.getLayerId(name)
layer = net.getLayer(id)
print("layer id : %d, type : %s, name: %s"%(id, layer.type, layer.name))
print("successfully")
输出
layer id : 1, type : Convolution, name: conv1/7x7_s2
layer id : 2, type : ReLU, name: conv1/relu_7x7
layer id : 3, type : Pooling, name: pool1/3x3_s2
layer id : 4, type : LRN, name: pool1/norm1
layer id : 5, type : Convolution, name: conv2/3x3_reduce
layer id : 6, type : ReLU, name: conv2/relu_3x3_reduce
layer id : 7, type : Convolution, name: conv2/3x3
layer id : 8, type : ReLU, name: conv2/relu_3x3
layer id : 9, type : LRN, name: conv2/norm2
layer id : 10, type : Pooling, name: pool2/3x3_s2
...
successfully
DNN实现图像分类
概述
✔️ 我们使用ImageNet数据集支持1000分类的GoogleNet网络模型, 结合opencv实现图像分类标签预测。
label标签是在一个单独的文本文件中读取。
读取模型的API:
cv2.dnn.readNetFromCaffe(prototxt,
caffeModel = String())
其中:
- prototxt表示模型的配置文件
- caffeModel表示模型的权重二进制文件
使用模型实现预测的时候,需要读取图像作为输入,网络模型支持的输入数据是四维的输入,所以要把读取到的Mat对象转换为四维张量,OpenCV的提供的API为如下:
cv2.dnn。blobFromImage(
image,
scalefactor = 1.0,
size = Size(),
mean = Scalar(),
swapRB = false,
crop = false,
ddepth = CV_32F
)
其中:
- image输入图像
- scalefactor 默认1.0
- size表示网络接受的数据大小
- mean表示训练时数据集的均值
- swapRB 是否互换Red与Blur通道
- crop剪切
- ddepth 数据类型
代码
import cv2
import numpy as np
bin_model = "bvlc_googlenet.caffemodel"
protxt = "bvlc_googlenet.prototxt"
# 加载类别
classes = None
with open("classification_classes_ILSVRC2012.txt", 'rt') as f:
classes = f.read().rstrip('n').split('n')
# 加载模型
net = cv2.dnn.readNetFromCaffe(protxt, bin_model)
# 读取输入数据
image = cv2.imread("guinea_pig.jpg")
blob = cv2.dnn.blobFromImage(image, 1.0, (224, 224), (104, 117,123), False, crop=False)
result = np.copy(image)
# 运行模型
net.setInput(blob)
out = net.forward()
# 获取最高分的类别
out = out.flatten()
classId = np.argmax(out)
confidence = out[classId]
# 输出运行时间
t, _ = net.getPerfProfile()
label = 'cost time: %.2f ms' % (t * 1000.0 / cv2.getTickFrequency())
cv2.putText(result, label, (0, 20), cv2.FONT_HERSHEY_SIMPLEX, 0.8, (255, 255, 0), 2)
# 显示结果
label = '%s: %.4f' % (classes[classId] if classes else 'Class #%d' % classId, confidence)
cv2.putText(result, label, (0, 60), cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 0, 255), 2)
show_img = np.hstack((image, result))
输出豚鼠的预测结果
SSD实现目标检测
✔️ OpenCV的DNN模块支持常见得对象检测模型SSD, 以及Mobile Net-SSD。
✔️ 这里我们用基于Caffe训练好的mobile-net SSD来测试目标检测。
单张图片检测
import cv2
model_bin = "../model/ssd/MobileNetSSD_deploy.caffemodel";
config_text = "../model/ssd/MobileNetSSD_deploy.prototxt";
objName = ["background",
"aeroplane", "bicycle", "bird", "boat",
"bottle", "bus", "car", "cat", "chair",
"cow", "diningtable", "dog", "horse",
"motorbike", "person", "pottedplant",
"sheep", "sofa", "train", "tvmonitor"];
# 加载模型
net = cv2.dnn.readNetFromCaffe(config_text, model_bin)
image = cv2.imread("dog.jpg")
h = image.shape[0]
w = image.shape[1]
# 获得所有层名称与索引
layerNames = net.getLayerNames()
lastLayerId = net.getLayerId(layerNames[-1])
lastLayer = net.getLayer(lastLayerId)
print(lastLayer.type)
# 加载图片检测
blobImage = cv2.dnn.blobFromImage(image, 0.007843, (300, 300), (127.5, 127.5, 127.5), True, False);
net.setInput(blobImage)
Out = net.forward()
print(Out.shape)
for detection in Out[0,0,:,:]:
score = float(detection[2])
objIndex = int(detection[1])
if score > 0.5:
left = detection[3]*w
top = detection[4]*h
right = detection[5]*w
bottom = detection[6]*h
# 绘制
cv2.rectangle(image, (int(left), int(top)), (int(right), int(bottom)), (255, 0, 0), thickness=2)
cv2.putText(image, "score:%.2f, %s"%(score, objName[objIndex]),
(int(left) - 10, int(top) - 5), cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 0, 255), 2, 8);
输出
视频检测
修改代码部分如下
# 调用前置摄像头获取视频
cap = cv2.VideoCapture(0)
while True:
ret, frame = cap.read()
if ret is False:
break
h, w = frame.shape[:2]
blobImage = cv2.dnn.blobFromImage(frame, 0.007843, (300, 300), (127.5, 127.5, 127.5), True, False);
net.setInput(blobImage)
cvOut = net.forward()
for detection in cvOut[0,0,:,:]:
score = float(detection[2])
objIndex = int(detection[1])
if score > 0.5:
left = detection[3]*w
top = detection[4]*h
right = detection[5]*w
bottom = detection[6]*h
# 绘制
cv2.rectangle(frame, (int(left), int(top)), (int(right), int(bottom)), (255, 0, 0), thickness=2)
cv2.putText(frame, "score:%.2f, %s"%(score, objName[objIndex]),
(int(left) - 10, int(top) - 5), cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 0, 255), 2, 8);
cv2.imshow('video-ssd-demo', frame)
c = cv2.waitKey(10)
if c == 27:
break
------------------------------------------可爱の分割线------------------------------------------
更多Opencv教程可以 Follow github的opencv教程,中文&English 欢迎Star❤️❤️❤️
JimmyHHua/opencv_tutorialsgithub.com