使用TensorFlow.js从网络摄像头进行实时AI情感检测

最新推荐文章于 2024-07-02 20:42:38 发布

寒冰屋

最新推荐文章于 2024-07-02 20:42:38 发布

阅读量1k

点赞数

分类专栏：人工智能文章标签： TensorFlow.js 情感检测 AI

原文链接：https://www.codeproject.com/Articles/5276821/Real-Time-AI-Emotion-Detection-from-a-Webcam-With

版权

人工智能专栏收录该内容

564 篇文章 57 订阅

订阅专栏

下载源-10.6 MB

使用支持HTML5的现代网络浏览器，我们可以轻松访问许多API，例如网络摄像头。当我们需要访问实时数据时，它特别有用。在本文中，我们将使用网络摄像头访问实时数据，并使用转移学习来检测脾气暴躁。

如果您一直在看本系列，那么本文中使用的代码应该看起来很熟悉。我们将使用相同的迁移学习技术，将预先训练的MobileNet模型与我们的自定义实时训练数据结合起来。

配置

创建一个HTML文档并通过导入所需的TensorFlow.js、MobileNet和KNN分类器模型开始。

<script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs/dist/tf.min.js"> </script>
<script src="https://cdn.jsdelivr.net/npm/@tensorflow-models/mobilenet"></script>
<script src="https://cdn.jsdelivr.net/npm/@tensorflow-models/knn-classifier"></script>

我们将使用视频标签来使用来自实时摄像机的图像，而不是使用图像或画布标签。

<video autoplay muted id="webcam" width="224" height="224"></video>

我们还需要“脾气暴躁”和“中性”按钮，才能将实况视频的图像添加到我们的训练数据中：

<button id="grumpy">Grumpy</button>
<button id="neutral">Neutral</button>

让我们添加另一个标签，以在页面而不是控制台上输出我们的预测。

<div id="prediction"></div>

我们最终的HTML文件如下所示：

<html lang="en">
    <head>
        <meta charset="UTF-8">
        <script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs/dist/tf.min.js"> </script>
        <script src="https://cdn.jsdelivr.net/npm/@tensorflow-models/mobilenet"></script>
        <script src="https://cdn.jsdelivr.net/npm/@tensorflow-models/knn-classifier"></script>
    </head>
    <body>
        <div>
            <h1>Real-Time Grumpiness detection using Tensorflow.js</h1>
        
            <div style="width:100%">
            <video autoplay muted id="webcam" width="224" height="224" style=" margin: auto;"></video>
            </div>
 
            <h3>How are you feeling?</h3>
            <button id="grumpy">Grumpy</button>
            <button id="neutral">Neutral</button>
 
            <div id="prediction"></div>
 
            <script src="grumpinessClassifier.js"></script>
        </div>
    </body>
</html>

获取实时视频源

现在是时候转到JavaScript文件了，我们将通过设置一些重要的变量来开始：

let knn;
let model;
 
const classes = ['Grumpy', 'Neutral'];
const video = document.getElementById('webcam');

我们还需要加载模型并创建KNN分类器的实例：

let knn;
let model;
 
const classes = ['Grumpy', 'Neutral'];
const video = document.getElementById('webcam');

我们现在可以设置网络摄像头以获取视频图片：

const setupWebcam = async () => {
    return new Promise((resolve, reject) => {
        const _navigator = navigator;
        navigator.getUserMedia = navigator.getUserMedia ||
        _navigator.webkitGetUserMedia || _navigator.mozGetUserMedia ||
        _navigator.msGetUserMedia;
        if (navigator.getUserMedia) {
            navigator.getUserMedia({video: true},
                stream => {
                    video.srcObject = stream;
                    video.addEventListener('loadeddata', () => resolve(), false);
                },
            error => reject());
        }
    });
}

使用KNN分类器

现在我们从网络摄像头获取数据，我们将继续使用knn的addExample方法添加示例训练数据。

const addExample = label => {
    // getting the intermediate activation of MobileNet 'conv_preds' and passing that to the KNN classifier.
    const feature = model.infer(video, 'conv_preds');

    // Pass the intermediate activation to the classifier
    knn.addExample(feature, label);
};

// add an example to the specified class on button click
document.getElementById('grumpy').addEventListener('click', () => addExample(0));
document.getElementById('neutral').addEventListener('click', () => addExample(1));

将代码放在一起

这是我们的grumpinessClassifier.js文件的最终外观：

let knn;
let model;
 
knn = knnClassifier.create();
const classes = ['Grumpy', 'Neutral'];
const video = document.getElementById('webcam');
 
async function loadKnnClassifier() {
 
    console.log('Model is Loading..');
    model = await mobilenet.load();
    console.log('Model loaded successfully!');
 
    await setupWebcam();
 
    // Reading the image from the webcam and associate the image with a specific class 
    const addExample = label => {
        // getting the intermediate activation of MobileNet 'conv_preds' and passing that to the KNN classifier.
        const feature = model.infer(video, 'conv_preds');
 
        // Pass the intermediate activation to the classifier
        knn.addExample(feature, label);
    };
 
    // add an example to the specified class on button click
    document.getElementById('grumpy').addEventListener('click', () => addExample(0));
    document.getElementById('neutral').addEventListener('click', () => addExample(1));
 
    while(true) {
        if (knn.getNumClasses() > 0) {
            // Getting activation from mobilenet for the webcam video
            const feature = model.infer(video, 'conv_preds');
            // getting the top prediction from the classifier module
            const prediction = await knn.predictClass(feature);
 
            // printing the prediction with confidence score on the screen
            document.getElementById('prediction').innerText = `
                Predicted emotion: ${classes[prediction.classIndex]}\n
                Probability of prediction: ${prediction.confidences[prediction.classIndex].toFixed(2)}
                `;
        }
 
        // wait for the next animation frame 
        await tf.nextFrame();
    }
}
 
const setupWebcam = async () => {
    return new Promise((resolve, reject) => {
        const _navigator = navigator;
        navigator.getUserMedia = navigator.getUserMedia ||
        _navigator.webkitGetUserMedia || _navigator.mozGetUserMedia ||
        _navigator.msGetUserMedia;
        if (navigator.getUserMedia) {
            navigator.getUserMedia({video: true},
                stream => {
                    video.srcObject = stream;
                    video.addEventListener('loadeddata', () => resolve(), false);
                },
            error => reject());
        }
    });
}
 
loadKnnClassifier();