手语识别_使用深度学习进行手语识别

手语识别

TL;DR It is presented a dual-cam first-vision translation system using convolutional neural networks. A prototype was developed to recognize 24 gestures. The vision system is composed of a head-mounted camera and a chest-mounted camera and the machine learning model is composed of two convolutional neural networks, one for each camera.

TL; DR 提出了一种使用卷积神经网络的双摄像头第一视觉翻译系统。 开发了一个原型来识别24个手势。 视觉系统由头戴式摄像头和胸部安装式摄像头组成，机器学习模型由两个卷积神经网络组成，每个卷积神经网络一个。

介绍 (Introduction)

Sign language recognition is a problem that has been addressed in research for years. However, we are still far from finding a complete solution available in our society.

手语识别是多年来研究中已经解决的问题。 但是，我们离我们的社会还没有找到一个完整的解决方案。

Among the works developed to address this problem, the majority of them have been based on basically two approaches: contact-based systems, such as sensor gloves; or vision-based systems, using only cameras. The latter is way cheaper and the boom of deep learning makes it more appealing.

在为解决这个问题而开发的作品中，大部分都基于两种方法：基于接触的系统，例如传感器手套； 或基于视觉的系统，仅使用摄像头。 后者更便宜，而深度学习的兴起使其更具吸引力。

This post presents a prototype of a dual-cam first-person vision translation system for sign language using convolutional neural networks. The post is divided into three main parts: the system design, the dataset, and the deep learning model training and evaluation.

这篇文章介绍了使用卷积神经网络的双摄像头第一人称视觉翻译系统的原型。 该职位分为三个主要部分：系统设计，数据集以及深度学习模型的训练和评估。

视觉系统 (VISION SYSTEM)

Vision is a key factor in sign language, and every sign language is intended to be understood by one person located in front of the other, from this perspective, a gesture can be completely observable. Viewing a gesture from another perspective makes it difficult or almost impossible to be understood since every finger position and movement will not be observable.

视觉是手语中的一个关键因素，每种手语都应由位于另一人面前的一个人理解，从这个角度来看，手势可以完全观察到。 由于无法观察到每个手指的位置和移动，因此从另一个角度查看手势很难或几乎无法理解。

Trying to understand sign language from a first-vision perspective has the same limitations, some gestures will end up looking the same way. But, this ambiguity can be solved by locating more cameras in different positions. In this way, what a camera can’t see, can be perfectly observable by another camera.

试图从第一视觉的角度理解手语具有相同的局限性，某些手势最终将以相同的方式出现。 但是，可以通过在不同位置放置更多摄像机来解决这种歧义。 这样，一台摄像机看不到的东西可以被另一台摄像机完全观察到。

The vision system is composed of two cameras: a head-mounted camera and a chest-mounted camera. With these two cameras we obtain two different views of a sign, a top-view, and a bottom-view, that works together to identify signs.

视觉系统由两个摄像头组成：头戴式摄像头和胸部安装式摄像头。 使用这两个摄像头，我们可以获得标牌的两种不同视图，即顶视图和底视图，它们可以一起识别标牌。