基于CNN制作的疲劳驾驶检测系统

最新推荐文章于 2024-09-16 14:49:33 发布

启诚工作室

最新推荐文章于 2024-09-16 14:49:33 发布

阅读量994

点赞数 15

文章标签：人工智能 python cnn 神经网络深度学习

本文链接：https://blog.csdn.net/2301_80759678/article/details/141200399

版权

摘要：疲劳驾驶是一个全球性的交通安全问题，它指的是驾驶员在长时间驾驶或身体、精神状态不佳的情况下操作车辆，从而导致注意力下降、反应迟钝、判断力减弱等现象。这种状态下驾驶极易引发交通事故，对公共安全构成严重威胁。基于此背景下，本工作室基于CNN制作了疲劳驾驶检测系统。

1 项目成果展示

2 主要算法

2.1 脸部检测算法

系统使用Dlib库的68点面部特征检测模型，通过摄像头捕捉人脸，提取关键点坐标、人脸框及角度信息，以评估驾驶员的疲劳状态。具体步骤包括：

初始化人脸检测器
设置面部标志预测
获取脸部位置和特征检测器
定位双眼关键点
开启摄像头捕获视频流

Dlib库中的68个特征点模型如描述所示：

2.2 眼睛检测算法

基于Eye Aspect Ratio (EAR)算法的眨眼检测原理如下：当人眼睁开时，EAR会在一个特定范围内波动；而当眼睛闭合时，EAR会迅速下降，理论上接近于0。如果EAR值低于预设的阈值，则认为眼睛处于闭合状态；当EAR值从一个较高的数值迅速下降到低于这个阈值，并随后迅速回升到阈值之上时，则被判定为一次眨眼。为了准确检测眨眼次数，需要确定一个连续帧数作为一次眨眼的标准，由于眨眼动作通常很快，一般只需要1到3帧就可以完成一次眨眼。

眼部特征图如下：

2.3 打哈欠检测算法

基于Mouth Aspect Ratio (MAR)算法的哈欠检测方法如下：利用Dlib库提取嘴部的6个关键特征点（51、59、53、57的纵坐标和49、55的横坐标），通过这些点的坐标计算嘴部张开的程度。当嘴部张开（如打哈欠）时，点51、59、53、57的纵坐标差值增大，导致MAR值显著增加；相反，当嘴部闭合时，MAR值会迅速减小。

嘴部主要取六个参考点，如下图：

3 项目介绍

3.1 运行环境:

Python3.9 以上。同时还要安装以下子库

1.opencv-python 2.dlib 3.scipy 4.numpy 5.playsound 6.joblib 7.imutils
8.scikit-learn 9.tensorflow 10.imgaug 11.pygame 12.opencv-python-headless
13.keras

3.2 主函数:

主文件夹中包含main函数和UI函数，main函数和UI函数的展现如下：

main函数:

import pygame
import cv2
import dlib
import numpy as np
import time  # 添加这一行
from keras.models import load_model
from UI import DrowsinessDetectionUI  # 导入 UI 类

# 路径设置
detector_path = 'C:/Users/moemo/Desktop/fatiguedriving-detection/src/models/shape_predictor_68_face_landmarks.dat'
model_path = 'C:/Users/moemo/Desktop/fatiguedriving-detection/src/models/drowsiness_detection_model.h5'
sound_path1 = 'C:/Users/moemo/Desktop/fatiguedriving-detection/sounds/alert_sound1.wav'
sound_path2 = 'C:/Users/moemo/Desktop/fatiguedriving-detection/sounds/alert_sound2.wav'
sound_path3 = 'C:/Users/moemo/Desktop/fatiguedriving-detection/sounds/alert_sound3.wav'

# 初始化 Pygame 用于播放警报音效
pygame.init()
pygame.mixer.init()

# 加载警报音效
alert_sound1 = pygame.mixer.Sound(sound_path1)  # 闭眼检测的警报
alert_sound2 = pygame.mixer.Sound(sound_path2)  # 打哈欠的警报
alert_sound3 = pygame.mixer.Sound(sound_path3)  # 长时间识别不到人脸的警报

# 初始化人脸检测器和预测器
detector = dlib.get_frontal_face_detector()
predictor = dlib.shape_predictor(detector_path)
model = load_model(model_path)

def calculate_ear(shape):
    left_eye = np.array([(shape.part(i).x, shape.part(i).y) for i in range(36, 42)])
    right_eye = np.array([(shape.part(i).x, shape.part(i).y) for i in range(42, 48)])
    def ear(eye):
        A = np.linalg.norm(eye[1] - eye[5])
        B = np.linalg.norm(eye[2] - eye[4])
        C = np.linalg.norm(eye[0] - eye[3])
        return (A + B) / (2.0 * C)
    ear_left = ear(left_eye)
    ear_right = ear(right_eye)
    return (ear_left + ear_right) / 2.0

def calculate_mar(shape):
    mouth = np.array([(shape.part(i).x, shape.part(i).y) for i in range(48, 68)])
    A = np.linalg.norm(mouth[3] - mouth[9])
    B = np.linalg.norm(mouth[0] - mouth[6])
    return A / B

# 启动视频捕获
cap = cv2.VideoCapture(0)

print("Starting video capture...")

# 初始化状态变量
last_face_time = time.time()
last_drowsiness_time = time.time()
last_yawn_time = time.time()
face_lost_interval = 3
drowsiness_detection_interval = 2
yawn_detection_interval = 1.5
drowsiness_detected = False
yawn_detected = False

no_face_detected_counter = 0
no_face_detected_threshold = 5

yawn_mar_history = []
yawn_mar_history_size = 5
yawn_mar_threshold = 0.3

yawn_count = -1
drowsiness_count = 0
initialbug = -1

drowsiness_triggered = False

# 创建 UI 实例
ui = DrowsinessDetectionUI()

try:
    while True:
        ret, frame = cap.read()
        if not ret:
            print("Error: Could not read frame.")
            break

        gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
        gray = cv2.equalizeHist(gray)
        clahe = cv2.createCLAHE(clipLimit=1.5, tileGridSize=(8, 8))
        gray = clahe.apply(gray)
        alpha = 1.2
        beta = 0
        gray = cv2.convertScaleAbs(gray, alpha=alpha, beta=beta)

        rects = detector(gray, 0)
        current_time = time.time()

        if ui.detection_active:  # 仅当检测处于活动状态时才更新计数和警报
            if len(rects) == 0:
                no_face_detected_counter += 1
                if no_face_detected_counter > no_face_detected_threshold:
                    if current_time - last_face_time > face_lost_interval:
                        alert_sound3.play()  # 播放长时间识别不到人脸的警报
                        last_face_time = current_time
                    no_face_detected_counter = 0
            else:
                no_face_detected_counter = 0
                last_face_time = current_time

                for rect in rects:
                    (x, y, w, h) = (rect.left(), rect.top(), rect.width(), rect.height())
                    cv2.rectangle(frame, (x, y), (x + w, y + h), (0, 255, 0), 2)
                    shape = predictor(gray, rect)
                    ear = calculate_ear(shape)
                    mar = calculate_mar(shape)

                    if ear < 0.25 and mar > 0.3:
                        if not drowsiness_detected:
                            drowsiness_detected = True
                            last_drowsiness_time = current_time
                        elif current_time - last_drowsiness_time > drowsiness_detection_interval:
                            if not drowsiness_triggered:
                                alert_sound1.play()  # 播放闭眼检测的警报
                                drowsiness_count += 1
                                drowsiness_triggered = True
                    else:
                        drowsiness_detected = False
                        drowsiness_triggered = False

                    yawn_mar_history.append(mar)
                    if len(yawn_mar_history) > yawn_mar_history_size:
                        yawn_mar_history.pop(0)
                    avg_yawn_mar = np.mean(yawn_mar_history)
                    if avg_yawn_mar > yawn_mar_threshold:
                        if not yawn_detected:
                            yawn_detected = True
                            last_yawn_time = current_time
                            if initialbug != -1:
                                alert_sound2.play()  # 播放打哈欠的警报
                            yawn_count += 1
                            initialbug += 1
                    else:
                        yawn_detected = False

                    for i in range(68):
                        (x, y) = (shape.part(i).x, shape.part(i).y)
                        cv2.circle(frame, (x, y), 2, (255, 0, 0), -1)

        ui.update_display(frame, yawn_count, drowsiness_count)

        if not ui.handle_events():
            break

except KeyboardInterrupt:
    print("Program interrupted by user.")
except Exception as e:
    print(f"Unexpected error: {e}")
finally:
    cap.release()
    ui.quit()

UI函数：

import pygame
import cv2
import numpy as np


class DrowsinessDetectionUI:
    def __init__(self, width=800, height=600, video_width=640, video_height=480):
        pygame.init()
        self.screen_width = width
        self.screen_height = height
        self.video_width = video_width
        self.video_height = video_height
        self.screen = pygame.display.set_mode((self.screen_width, self.screen_height))
        pygame.display.set_caption("Drowsiness Detection")
        self.font = pygame.font.SysFont(None, 36)
        self.large_font = pygame.font.SysFont(None, 72)  # 大字体用于显示暂停信息

        # 加载背景图片
        self.background = pygame.image.load('C:/Users/moemo/Desktop/fatiguedriving-detection/bg/background.jpg')
        self.background = pygame.transform.scale(self.background, (self.screen_width, self.screen_height))

        # 按钮定义（向下移动了100像素）
        self.button_start = pygame.Rect(680, 130, 100, 40)  # y坐标从80改为130
        self.button_pause = pygame.Rect(680, 180, 100, 40)  # y坐标从130改为180
        self.button_quit = pygame.Rect(680, 230, 100, 40)  # y坐标从180改为230

        # 按钮边框颜色
        self.border_color = (192, 192, 192)  # 银白色
        self.border_thickness = 5

        # 初始化状态
        self.detection_active = True  # 默认检测是开启的

    def draw_buttons(self):
        for button in [self.button_start, self.button_pause, self.button_quit]:
            # 绘制按钮边框
            pygame.draw.rect(self.screen, self.border_color,
                             button.inflate(self.border_thickness * 2, self.border_thickness * 2),
                             self.border_thickness)
            # 绘制按钮背景
            pygame.draw.rect(self.screen, (0, 0, 0), button)

            if button == self.button_start:
                text = "Start"
            elif button == self.button_pause:
                text = "Pause"
            else:
                text = "Quit"

            button_text = self.font.render(text, True, (255, 255, 255))
            self.screen.blit(button_text, (button.x + 5, button.y + 5))

    def update_display(self, frame, yawn_count, drowsiness_count):
        self.screen.blit(self.background, (0, 0))  # 绘制背景

        # 显示打哈欠和疲劳闭眼的次数
        yawn_text = self.font.render(f"Yawns: {yawn_count}", True, (255, 0, 0))
        drowsiness_text = self.font.render(f"Drowsiness: {drowsiness_count}", True, (255, 0, 0))
        self.screen.blit(yawn_text, (10, 30))
        self.screen.blit(drowsiness_text, (10, 70))

        if self.detection_active:
            # 将 OpenCV 图像转换为 Pygame 表面
            frame_rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
            frame_surface = pygame.surfarray.make_surface(np.transpose(frame_rgb, (1, 0, 2)))

            # 计算实时窗口的位置（左下角并留边界）
            x_offset = 20  # 左边距
            y_offset = self.screen_height - self.video_height - 15  # 底边距

            # 绘制银白色边框
            border_color = (192, 192, 192)  # 银白色
            border_thickness = 5
            pygame.draw.rect(self.screen, border_color,
                             (x_offset - border_thickness, y_offset - border_thickness,
                              self.video_width + 2 * border_thickness, self.video_height + 2 * border_thickness),
                             border_thickness)

            self.screen.blit(frame_surface, (x_offset, y_offset))
        else:
            # 显示“Detection Paused”信息
            paused_text = self.large_font.render("Detection Paused", True, (255, 0, 0))
            text_rect = paused_text.get_rect(center=(self.screen_width // 2, self.screen_height // 2))
            self.screen.blit(paused_text, text_rect)

        # 绘制按钮
        self.draw_buttons()

        pygame.display.flip()

    def handle_events(self):
        for event in pygame.event.get():
            if event.type == pygame.QUIT:
                return False
            if event.type == pygame.MOUSEBUTTONDOWN:
                if self.button_start.collidepoint(event.pos):
                    print("Start Detection button clicked")
                    self.detection_active = True
                elif self.button_pause.collidepoint(event.pos):
                    print("Pause Detection button clicked")
                    self.detection_active = False
                elif self.button_quit.collidepoint(event.pos):
                    return False
        return True

    def quit(self):
        pygame.quit()

2.3 主要文件说明:

1.data_preprocessing 人脸检测和面部关键点识别，将结果保存到 JSON 文件中。

2.prepare_data 从图像数据集中提取面部特征，并将这些特征数据保存为 JSON 文件。

3.verify_data 验证 JSON 数据文件中的类别分布，确保每个类别的样本计数是正确的。

4.train_model 数据预处理、模型构建、训练和保存。

5.update_features 更新 JSON 文件中的特征数据。

6.validate_model 加载模型、加载数据以及对数据进行预测。

7.realtime_drowsiness_detection_with_alert 对人脸进行检测，分析面部特征，最后发出警报。