使用OpenCV和Tensorflow在排球中跟踪球

介绍 (Introduction)

After the first experience of applying AI in sport, I was inspired to continue. Home exercises are looked like an insignificant goal and I targeted team plays.

在将AI应用于体育方面的第一次经验之后，我受到启发继续前进。 家庭练习看起来像是一个微不足道的目标，我的目标是团队比赛。

AI in sports is a pretty new thing. There are a few interesting works:

体育中的AI是一件很新的事物。 有一些有趣的作品：

• Basketball

  篮球

• Tennis

  网球

• Volleyball

  排球

I am a big fan of playing volleyball, so let’s talk about the last reference. This a site of one Austrian institute who analyzes games of a local amateur league. There are some documents to read and even more important — open video dataset.

我非常喜欢打排球，所以让我们谈谈最后的参考。 这是一个奥地利研究所的站点，负责分析当地业余联赛的比赛。 有一些文档需要阅读，更重要的是-打开视频数据集。

Volleyball is a complex game with many different aspects. So I started with a small but very important piece — the ball.

排球是一个复杂的比赛，有很多不同的方面。 所以我从一个很小但很重要的部分开始-球。

Ball tracking is a pretty famous task. Google gives a lot of links but many of them are just a demo. Obviously, recognize and track a big ball of distinguished color in front of a camera cannot be compared with real game ball detection, where the ball is tiny, moving fast and blended into the background.

跟踪球是一项非常著名的任务。 Google提供了很多链接，但其中许多只是一个演示。 显然，识别并追踪摄像机前颜色显着的大球无法与真实的游戏球检测相提并论，在真实的游戏中，球很小，移动速度很快并融合到背景中。

And finally, we want to get something like that:

最后，我们想要得到这样的东西：

Before start let’s notice some detail about the video dataset:

在开始之前，让我们注意有关视频数据集的一些细节：

• the camera is static and located behind the court
  相机是静态的，位于球场后面
• skill level allows to see the ball freely (Professionals hit so hard that it is almost impossible to see the ball without TV replay)
  技能水平允许自由观看球(专业人士打得如此之重，以至于没有电视重播几乎看不到球)
• The ball color, blue and yellow, does not contrast much with the floor, unfortunately. That makes all color-based approaches meaningless
  不幸的是，球的颜色(蓝色和黄色)与地板的对比度不高。 这使得所有基于颜色的方法都变得毫无意义

解 (Solution)

So far most obvious approach — with colors — does not work, I used a fact the ball is moving.Then let’s find moving objects and pick the ball, sounds easy.

到目前为止，最明显的方法(使用颜色)是行不通的，我使用了一个事实，即球在移动。然后让我们找到移动的物体并捡起球，听起来很简单。

OpenCV contains tools to detect moving object with background removal:

OpenCV包含用于通过背景去除来检测运动对象的工具：

mask = backSub.apply(frame)     
mask = cv.dilate(mask, None)     
mask = cv.GaussianBlur(mask, (15, 15),0)     
ret,mask = cv.threshold(mask,0,255,cv.THRESH_BINARY | cv.THRESH_OTSU)

And such picture:

这样的图片：

Transformed into:

转化成：

In this example, the ball is on top and human brain and eye can easily detect it. How did we decide? Some rules could be deduced from the picture:

在此示例中，球在顶部，人的大脑和眼睛可以轻松地检测到它。 我们如何决定？ 从图片中可以得出一些规则：

• the ball is a blob
  球是一团
• it is the highest blob on the picture
  它是图片上的最高斑点

The second rule does not work well. In this picture, for example, the highest blob is the referee’s shoulder.

第二条规则效果不佳。 例如，在这张照片中，最高的斑点是裁判的肩膀。

But highest-blob approach gives an initial data for further steps.

但是最高blob方法为后续步骤提供了初始数据。

We can collect these blobs and train a classifier to distinguish the ball.

我们可以收集这些斑点并训练分类器来区分球。

This dataset looks like that:

该数据集如下所示：

In terms of AI — it is a binary classification of color images, very similar to Cats-vs-Dogs challenge.

在AI方面，它是彩色图像的二进制分类，与Cats-vs-Dogs挑战非常相似。

There are many ways to implement, but the most popular approach is with VGG neural network.

有很多实现方法，但是最流行的方法是使用VGG神经网络 。

A problem — ball pictures are very small and multiple convolution layers do not fit there. So I had to cut VGG to a very simple architecture:

一个问题-球形图片非常小，并且多个卷积层不适合此处。 因此，我不得不将VGG简化为一个非常简单的体系结构：

model = Sequential([Convolution2D(32,(3,3), activation='relu',       input_shape=input_shape),         MaxPooling2D(),            Convolution2D(64,(3,3), activation='relu'),         
 MaxPooling2D(),         
Flatten(),         
Dense(64, activation='relu'),         
Dropout(0.1),         
Dense(2, activation='softmax')       
])
     
model.compile(loss="categorical_crossentropy",   optimizer=SGD(lr=0.01), metrics=["accuracy"])

The model is simple and produces mediocre results: about 20% of false-positives and about 30% of false-negatives.That is better than nothing but, of course, not enough.

该模型很简单，并且产生了中等的结果：大约20％的假阳性和大约30％的假阴性，这总比没有好，但当然还不够。

The model applied to the game generates many false balls:

应用于游戏的模型会产生许多错误的球：

There are actually two kinds of false balls:

实际上有两种假球：

• they appear in random places in random time
  它们出现在随机时间的随机位置
• the model consistently makes a mistake, recognizing something else as a ball
  该模型始终犯错，将其他东西识别为球

弹道 (Trajectories)

As the next step, there is an idea, the ball does not move randomly, but follows parabolic or linear trajectories.

下一步，有一个想法，球不会随机移动，而是遵循抛物线或线性轨迹。

Validation of blob movements on this geometry will cut off random and consistent mistakes.

验证此几何体上的斑点运动将消除随机和一致的错误。

There is an example of recorded trajectories during one ball play:

有一个在一个球比赛中记录的轨迹的示例：

Where directed paths are in blue, static — in green, and random are grey.

定向路径为蓝色，静态(绿色)和随机路径为灰色。

Only blue trajectories are interesting. They consist of at least 3 points and have a direction. The direction is very important because the next point could be forecasted in case if it is missed in the real stream and no new paths detected.

只有蓝色的轨迹很有趣。 它们至少包含3个点，并有一个方向。 方向非常重要，因为如果在实际流中错过了下一个点且未检测到新路径，则可以预测下一个点。

This logic applied to the clip generates a pretty realistic tracking:

应用于剪辑的此逻辑生成了一个非常逼真的跟踪：

链接 (Links)

Github repo

Github回购

翻译自: https://towardsdatascience.com/ball-tracking-in-volleyball-with-opencv-and-tensorflow-3d6e857bd2e7