Python应用案例——基于Keras, OpenCV和MobileNet口罩佩戴识别

代码骑士

已于 2024-09-14 10:36:47 修改

阅读量1.2k

点赞数 7

分类专栏： # Python应用项目案例算法及源码1 文章标签： python keras 开发语言

于 2024-02-21 09:39:55 首次发布

本文链接：https://blog.csdn.net/qq_51701007/article/details/136203080

版权

Python应用项目案例算法及源码1 专栏收录该内容

6 篇文章 1 订阅

订阅专栏

5、加载MobileNetV2模型，并在其基础上添加一个全连接层（head）进行分类

6、训练神经网络的头部

7、评估一个神经网络模型在测试集上的性能，并将模型保存到磁盘

8、绘制训练过程中的损失和准确率

9、封装检测并预测人脸口罩函数

10、使用OpenCV的DNN模块进行人脸检测和口罩检测

1、导入所需库

#https://www.kaggle.com/prithwirajmitra/covid-face-mask-detection-dataset
# import the necessary packages
# pip install imutils
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.applications import MobileNetV2
from tensorflow.keras.layers import AveragePooling2D
from tensorflow.keras.layers import Dropout
from tensorflow.keras.layers import Flatten
from tensorflow.keras.layers import Dense
from tensorflow.keras.layers import Input
from tensorflow.keras.models import Model
from tensorflow.keras.optimizers import Adam
from tensorflow.keras.applications.mobilenet_v2 import preprocess_input
from tensorflow.keras.preprocessing.image import img_to_array
from tensorflow.keras.preprocessing.image import load_img
from tensorflow.keras.utils import to_categorical #Converts a class vector (integers) to binary class matrix
from sklearn.preprocessing import LabelBinarizer #accepts Categorical data as input and returns an Numpy array
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report
from imutils import paths #image processing functions such as translation, rotation, resizing, skeletonization, and displaying Matplotlib images easier with OpenCV and both Python 2.7 and Python 3
import matplotlib.pyplot as plt
import numpy as np
import os

这段代码导入了以下模块和函数：

ImageDataGenerator：用于数据增强和预处理的类；
MobileNetV2：基于MobileNetV2架构的预训练模型；
AveragePooling2D、Dropout、Flatten、Dense、Input：Keras中的一些常见层；
Model：用于构建神经网络模型的类；
Adam：优化器之一；
preprocess_input、img_to_array、load_img：用于图像预处理的函数；
to_categorical、LabelBinarizer：用于将类别向量转换为二进制类别矩阵的函数；
train_test_split：用于将数据集划分为训练集和测试集的函数；
classification_report：用于评估分类模型性能的函数；
paths：用于处理图像路径的函数；
matplotlib.pyplot：用于绘制图像的库；
numpy：用于进行数值计算的库；
os：用于处理文件和目录的库。

2、加载人脸口罩检测数据集

# initialize the initial learning rate, number of epochs to train for,
# and batch size
INIT_LR = 1e-4
EPOCHS = 20
BS = 32

#r means the string will be treated as raw string
#E:\Download\archive (1)\New Masks Dataset\dataset
#DIRECTORY = r"D:\python\Project0-Python-MachineLearning\Lesson82-Face-Mask-Detection\dataset"
DIRECTORY = "../New_Masks_Dataset/dataset"
CATEGORIES = ["with_mask", "without_mask"]

# grab the list of images in dataset directory, then initialize
# the list of data (i.e., images) and class images
print("[INFO] loading images...")

data = []
labels = []

for category in CATEGORIES:
    path = os.path.join(DIRECTORY, category)
    for img in os.listdir(path):
    	img_path = os.path.join(path, img)
    	image = load_img(img_path, target_size=(224, 224))
    	image = img_to_array(image)
    	image = preprocess_input(image)

    	data.append(image)
    	labels.append(category)

首先定义了一些常量，如学习率、训练轮数和批量大小。然后设置了数据集的目录路径和类别标签。接下来，遍历每个类别，读取该类别下的所有图像，并将它们添加到数据列表中。同时，将对应的类别标签添加到标签列表中。最后，打印出加载图像的信息。

3、对标签进行独热编码

使用LabelBinarizer()和to_categorical()函数将标签转换为独热编码形式

# perform one-hot encoding on the labels
lb = LabelBinarizer()
labels = lb.fit_transform(labels)
labels = to_categorical(labels)

data = np.array(data, dtype="float32")
labels = np.array(labels)

(trainX, testX, trainY, testY) = train_test_split(data, labels,
	test_size=0.20, stratify=labels, random_state=42)

4、构建一个用于数据增强的训练图像生成器

设置旋转范围、缩放范围、宽度偏移范围、高度偏移范围、剪切范围和水平翻转等参数

#stratify parameter will preserve the proportion of target as in original dataset, in the train and test datasets as well.

# construct the training image generator for data augmentation
aug = ImageDataGenerator(
	rotation_range=20,
	zoom_range=0.15,
	width_shift_range=0.2,
	height_shift_range=0.2,
	shear_range=0.15,
	horizontal_flip=True,
	fill_mode="nearest") #https://fairyonice.github.io/Learn-about-ImageDataGenerator.html

5、加载MobileNetV2模型，并在其基础上添加一个全连接层（head）进行分类

# load the MobileNetV2 network, ensuring the head FC layer sets are left off
baseModel = MobileNetV2(weights=None, include_top=False, input_tensor=Input(shape=(224, 224, 3)))

#baseModel = MobileNetV2(weights="imagenet", include_top=False, input_tensor=Input(shape=(224, 224, 3)))
#执行上句必须开启全局代理
#如果仍不行则下载模型，手动加载
#Exception: URL fetch failure on https://storage.googleapis.com/tensorflow/keras-applications/mobilenet_v2/mobilenet_v2_weights_tf_dim_ordering_tf_kernels_1.0_224_no_top.h5: None -- [Errno 11004] getaddrinfo failed
#E:\Download\mobilenet_v2_weights_tf_dim_ordering_tf_kernels_1.0_224_no_top.h5

#model_h5 = r'E:\Download\mobilenet_v2_weights_tf_dim_ordering_tf_kernels_1.0_224_no_top.h5'
baseModel = MobileNetV2(weights='mobilenet_v2_weights_tf_dim_ordering_tf_kernels_1.0_224_no_top.h5', include_top=False, input_tensor=Input(shape=(224, 224, 3)))

# construct the head of the model that will be placed on top of the
# the base model
headModel = baseModel.output
headModel = AveragePooling2D(pool_size=(7, 7))(headModel)
headModel = Flatten(name="flatten")(headModel)
headModel = Dense(128, activation="relu")(headModel)
headModel = Dropout(0.5)(headModel)
headModel = Dense(2, activation="softmax")(headModel)

# place the head FC model on top of the base model (this will become the actual model we will train)
model = Model(inputs=baseModel.input, outputs=headModel)

# loop over all layers in the base model and freeze them so they will *not* be updated during the first training process
for layer in baseModel.layers:
	layer.trainable = False

# compile our model
print("[INFO] compiling model...")
opt = Adam(lr=INIT_LR, decay=INIT_LR / EPOCHS)
model.compile(loss="binary_crossentropy", optimizer=opt, metrics=["accuracy"])

首先，它加载了一个不包含顶层全连接层的MobileNetV2模型。然后，在基础模型的基础上构建了一个新的模型，该模型的输入与基础模型相同，输出为经过一系列操作后的全连接层。最后，将基础模型的所有层冻结，以便在训练过程中不会更新它们，然后编译新模型。

6、训练神经网络的头部

# train the head of the network
print("[INFO] training head...")
H = model.fit(
	aug.flow(trainX, trainY, batch_size=BS),
	steps_per_epoch=len(trainX) // BS,
	validation_data=(testX, testY),
	validation_steps=len(testX) // BS,
	epochs=EPOCHS)

首先，它使用数据增强（aug）对训练数据（trainX和trainY）进行预处理，然后使用模型（model）进行训练。训练过程中，每个批次的大小为BS，每个周期（epoch）的训练步数为训练数据长度除以批次大小，验证数据为测试数据（testX和testY），验证步数为测试数据长度除以批次大小。训练周期数为EPOCHS。

7、评估一个神经网络模型在测试集上的性能，并将模型保存到磁盘

使用model.predict()方法对测试集进行预测，得到预测结果predIdxs。
使用np.argmax()函数找到每个图像对应的最大预测概率的标签索引。
使用classification_report()函数生成一个格式化的分类报告，展示模型在测试集上的准确率、召回率等指标。
将训练好的模型保存到磁盘，文件名为"mask_detector.model"，格式为"h5"。

# make predictions on the testing set
print("[INFO] evaluating network...")
predIdxs = model.predict(testX, batch_size=BS)

# for each image in the testing set we need to find the index of the
# label with corresponding largest predicted probability
predIdxs = np.argmax(predIdxs, axis=1)

# show a nicely formatted classification report
print(classification_report(testY.argmax(axis=1), predIdxs, target_names=lb.classes_))

# serialize the model to disk
print("[INFO] saving mask detector model...")
model.save("mask_detector.model", save_format="h5")

[INFO] evaluating network...
              precision    recall  f1-score   support

   with_mask       0.99      1.00      1.00       101
without_mask       1.00      0.99      1.00       101

    accuracy                           1.00       202
   macro avg       1.00      1.00      1.00       202
weighted avg       1.00      1.00      1.00       202

[INFO] saving mask detector model...

8、绘制训练过程中的损失和准确率

首先，设置了图表的样式为"ggplot"，然后创建了一个图形。接着，分别绘制了训练损失（train_loss）、验证损失（val_loss）、训练准确率（train_acc）和验证准确率（val_acc）随迭代次数（Epoch）的变化曲线。最后，设置了图表的标题、坐标轴标签和图例位置，并显示了图表。

# plot the training loss and accuracy
N = EPOCHS
plt.style.use("ggplot")
plt.figure()
plt.plot(np.arange(0, N), H.history["loss"], label="train_loss")
plt.plot(np.arange(0, N), H.history["val_loss"], label="val_loss")
plt.plot(np.arange(0, N), H.history["accuracy"], label="train_acc")
plt.plot(np.arange(0, N), H.history["val_accuracy"], label="val_acc")
plt.title("Training Loss and Accuracy")
plt.xlabel("Epoch #")
plt.ylabel("Loss/Accuracy")
plt.legend(loc="lower left")
plt.show()
#plt.savefig("plot.png")

9、封装检测并预测人脸口罩函数

# import the necessary packages
from tensorflow.keras.applications.mobilenet_v2 import preprocess_input
from tensorflow.keras.preprocessing.image import img_to_array
from tensorflow.keras.models import load_model
from imutils.video import VideoStream
import numpy as np
import imutils
import time
import cv2
import os

def detect_and_predict_mask(frame, faceNet, maskNet):
	# grab the dimensions of the frame and then construct a blob
	# from it
	(h, w) = frame.shape[:2]
	blob = cv2.dnn.blobFromImage(frame, 1.0, (224, 224),
		(104.0, 177.0, 123.0))

	# pass the blob through the network and obtain the face detections
	faceNet.setInput(blob)
	detections = faceNet.forward()
	print(detections.shape)

	# initialize our list of faces, their corresponding locations,
	# and the list of predictions from our face mask network
	faces = []
	locs = []
	preds = []

	# loop over the detections
	for i in range(0, detections.shape[2]):
		# extract the confidence (i.e., probability) associated with
		# the detection
		confidence = detections[0, 0, i, 2]

		# filter out weak detections by ensuring the confidence is
		# greater than the minimum confidence
		if confidence > 0.5:
			# compute the (x, y)-coordinates of the bounding box for
			# the object
			box = detections[0, 0, i, 3:7] * np.array([w, h, w, h])
			(startX, startY, endX, endY) = box.astype("int")

			# ensure the bounding boxes fall within the dimensions of
			# the frame
			(startX, startY) = (max(0, startX), max(0, startY))
			(endX, endY) = (min(w - 1, endX), min(h - 1, endY))

			# extract the face ROI, convert it from BGR to RGB channel
			# ordering, resize it to 224x224, and preprocess it
			face = frame[startY:endY, startX:endX]
			face = cv2.cvtColor(face, cv2.COLOR_BGR2RGB)
			face = cv2.resize(face, (224, 224))
			face = img_to_array(face)
			face = preprocess_input(face)

			# add the face and bounding boxes to their respective
			# lists
			faces.append(face)
			locs.append((startX, startY, endX, endY))

	# only make a predictions if at least one face was detected
	if len(faces) > 0:
		# for faster inference we'll make batch predictions on *all*
		# faces at the same time rather than one-by-one predictions
		# in the above `for` loop
		faces = np.array(faces, dtype="float32")
		preds = maskNet.predict(faces, batch_size=32)

	# return a 2-tuple of the face locations and their corresponding
	# locations
	return (locs, preds)

这段代码定义了一个名为detect_and_predict_mask的函数，用于检测并预测人脸口罩。函数接收三个参数：frame（摄像头捕获的一帧图像），faceNet（用于人脸检测的网络模型）和maskNet（用于口罩检测的网络模型）。

函数首先获取输入图像的尺寸，并将其转换为一个blob。然后，通过调用faceNet.forward()方法，使用faceNet对图像进行人脸检测。接下来，遍历检测到的人脸，计算每个检测框的坐标，并将人脸ROI从BGR格式转换为RGB格式，调整大小为224x224，并进行预处理。将处理后的人脸添加到faces列表中，同时记录其位置信息。

如果检测到了至少一张人脸，就使用maskNet对所有人脸进行口罩检测。将所有检测到的人脸堆叠成一个numpy数组，并一次性进行批量预测。最后，返回一个包含人脸位置和对应口罩检测结果的元组。

10、使用OpenCV的DNN模块进行人脸检测和口罩检测

#OpenCV’s DNN Module
#In order to use DNN face detector in OpenCV, you first need to download the Caffe files from the OpenCV repository, 
#the deploy.prototxt file defines the network architecture and res10_300x300_ssd_iter_140000.caffemodel has the weights 
#of the layers.
prototxtPath = r"face_detector\deploy.prototxt"
weightsPath = r"face_detector\res10_300x300_ssd_iter_140000.caffemodel"
faceNet = cv2.dnn.readNet(prototxtPath, weightsPath)

# load the face mask detector model from disk
maskNet = load_model("mask_detector.model")

# initialize the video stream
print("[INFO] starting video stream...")
vs = VideoStream(src=1).start()

# loop over the frames from the video stream
while True:
	# grab the frame from the threaded video stream and resize it
	# to have a maximum width of 400 pixels
	frame = vs.read()
	frame = imutils.resize(frame, width=400)

	# detect faces in the frame and determine if they are wearing a
	# face mask or not
	(locs, preds) = detect_and_predict_mask(frame, faceNet, maskNet)

	# loop over the detected face locations and their corresponding
	# locations
	for (box, pred) in zip(locs, preds):
		# unpack the bounding box and predictions
		(startX, startY, endX, endY) = box
		(mask, withoutMask) = pred

		# determine the class label and color we'll use to draw
		# the bounding box and text
		label = "Mask" if mask > withoutMask else "No Mask"
		color = (0, 255, 0) if label == "Mask" else (0, 0, 255)

		# include the probability in the label
		label = "{}: {:.2f}%".format(label, max(mask, withoutMask) * 100)

		# display the label and bounding box rectangle on the output
		# frame
		cv2.putText(frame, label, (startX, startY - 10),
			cv2.FONT_HERSHEY_SIMPLEX, 0.45, color, 2)
		cv2.rectangle(frame, (startX, startY), (endX, endY), color, 2)

	# show the output frame
	cv2.imshow("Frame", frame)
	key = cv2.waitKey(1) & 0xFF

	# if the `q` key was pressed, break from the loop
	if key == ord("q"):
		break

# do a bit of cleanup
cv2.destroyAllWindows()
vs.stop()

首先，从OpenCV仓库下载Caffe文件（deploy.prototxt和res10_300x300_ssd_iter_140000.caffemodel），然后加载口罩检测器模型（mask_detector.model）。接着，初始化视频流并开始循环处理每一帧图像。

在循环中，首先从线程视频流中读取帧并将其调整为最大宽度为400像素。然后，使用faceNet对帧中的面部进行检测，并判断是否佩戴口罩。接下来，遍历检测到的面部位置及其对应的预测结果。对于每个面部位置，解包边界框和预测结果，确定要绘制的类标签和颜色。将概率包含在标签中，并在输出帧上显示标签和边界框矩形。

最后，如果按下“q”键，跳出循环。在循环结束后，进行一些清理工作，关闭所有窗口并停止视频流。