无人驾驶汽车入门_无人驾驶汽车的世界

无人驾驶汽车入门

by David Brailovsky

戴维·布雷洛夫斯基(David Brailovsky)

无人驾驶汽车的世界 (The world through the eyes of a self-driving car)

可视化神经网络用来识别交通灯的图像的哪一部分 (Visualizing which part of an image a neural network uses to recognize traffic lights)

In my last post I described how I trained a ConvNet (Convolutional Neural Network) to recognize traffic lights in dash-cam images. The best performing single network achieved an impressive accuracy of >94%.

在上一篇文章中，我描述了如何训练ConvNet(卷积神经网络)以识别行车记录仪图像中的交通信号灯。 性能最佳的单个网络达到了令人印象深刻的准确度，即> 94％。

While ConvNets are very good at learning to classify images, they are also somewhat of a black box. It’s hard to tell what they’re doing once they’re trained. Since I never explicitly “told” the network to focus on traffic lights, it’s possible that it’s using some other visual cues in the images to predict the correct class. Maybe it’s looking for static cars to predict a red light? ?

尽管ConvNets非常擅长学习对图像进行分类，但它们在某种程度上还是一个黑匣子。 培训后很难说出他们在做什么。 由于我从没有明确地“说服”网络关注交通信号灯，因此它可能会在图像中使用其他视觉提示来预测正确的类别。 也许它正在寻找静态汽车来预测红灯？ ？

In this post I describe a very simple and useful method for visualizing what part of an image the network uses for its prediction. The approach involves occluding parts of the image and seeing how that changes the network’s prediction. This approach has been described in “Visualizing and Understanding Convolutional Networks”.

在这篇文章中，我描述了一种非常简单实用的方法，用于可视化网络将其用于预测的图像的哪一部分。 该方法涉及遮挡图像的各个部分，并查看其如何改变网络的预测。 在“ 可视化和理解卷积网络 ”中已经描述了这种方法。

Self-driving cars today use much more sophisticated methods for detecting objects in a scene, as well as many more sensors as inputs. The ConvNet we examine throughout the post should be seen as a simplified version of what self-driving cars actually use. Nonetheless, the visualization method described in this post can be useful and adapted for different kinds of neural network applications.

如今，自动驾驶汽车使用更加复杂的方法来检测场景中的物体，并使用更多的传感器作为输入。 我们在整个文章中讨论的ConvNet应该被视为自动驾驶汽车实际使用的简化版本。 尽管如此，本文中描述的可视化方法仍然有用，并适用于不同种类的神经网络应用。

You can download a notebook file with the code I used from here.

您可以从此处下载带有我使用的代码的笔记本文件。

例子1 (Example 1)

I started with the following image which has a red traffic light:

我从下面的图像开始，该图像具有红色的交通信号灯：

The network predicts this image has a red traffic light with 99.99% probability. Next I generated many versions of this image with a grey square patch in different positions. More specifically, a 64 x 64 sliding square with a step size of 16 pixels.

网络预测该图像具有99.99％概率的红色交通信号灯。 接下来，我生成了此图像的许多版本，并在不同位置带有灰色正方形补丁。 更具体地说，是一个64 x 64的滑动正方形，步长为16像素。

I ran each image through the network and recorded the probability it predicted for the class “red”. Below you can see a plot of a heat-map of those recorded probabilities.

我通过网络运行每个图像，并记录了它对“红色”类的预测概率。 在下面，您可以看到这些记录的概率的热图图。

The color represents the probability of the class “red” when there was a square patch covering that position. Darker color means lower probability. There’s a smoothing effect because I averaged the probabilities each pixel got for all the patches that covered it.

颜色表示当有一个方形补丁覆盖该位置时，类别为“红色”的可能性。 深色表示较低的可能性。 之所以会有平滑效果，是因为我平均了覆盖它的所有色块每个像素获得的概率。

Then I plotted the heat-map on top of the original image:

然后，我将热图绘制在原始图像的顶部：

Very cool! ? The lowest probability is exactly when covering the traffic light. I then repeated this process with a smaller patch size of 16x16:

很酷！ ？ 最低的概率恰好是在遮盖交通信号灯时。 然后，我使用较小的补丁尺寸16x16重复此过程：

Exactly on the traffic light! ?

恰好在交通信号灯上！ ？

例子2 (Example 2)

I kept examining more images and came across this interesting example:

我不断检查更多图像，并遇到了这个有趣的示例：

The ConvNet predicted the class “green” with 99.99% probability for this image. I generated another heat-map by sliding a patch of size 32x32 and a step size of 16 pixels:

ConvNet预测此图像的类别为“绿色”，概率为99.99％。 我通过滑动尺寸为32x32的补丁和16像素的步长生成了另一个热图：

Hmm… something’s not right ?. The lowest probability for “green” that any patched image got was 99.909% which is still very high. The image with the lowest probability was:

嗯...不对劲吗？ 修补后的任何图像出现“绿色”的可能性最低，为99.909％，这仍然很高。 概率最低的图像是：

That actually looks fine, it covers the traffic light perfectly. So why was the network still predicting “green” with a high probability? Could be because of the second green traffic light in the image. I repeated the sliding patch process on the patched image above and plotted the heat-map:

实际上看起来不错，它可以完美覆盖交通信号灯。 那么，为什么网络仍然以很高的概率预测“绿色”呢？ 可能是因为图像中的第二个绿色交通信号灯。 我在上面的修补图像上重复了滑动修补过程，并绘制了热图：

Much better! ? After hiding the second traffic light, the probability for “green” dropped close to zero, 0.25% to be exact.

好多了！ ？ 隐藏第二个交通信号灯后，“绿色”的概率下降到接近于零，精确到0.25％。

看着错误 (Looking at mistakes)

Next I wanted to see if I could learn anything interesting by using this technique to understand some the network’s misclassifications. Many of the mistakes were caused by having two traffic lights in the scene, one green and one red. It was pretty obvious that the other traffic light is the part of the image that caused the mistake in those cases.

接下来，我想看看是否可以通过使用这种技术了解网络的一些错误分类来学习任何有趣的东西。 许多错误是由场景中的两个交通信号灯，一个绿色和一个红色引起的。 很明显，在这些情况下，另一个交通信号灯是导致错误的图像部分。

Another type of mistake was when the network predicted there’s no traffic light in the scene when there actually was. Unfortunately this technique was not very useful for understanding the reason the network got it wrong since there was no specific part of the image it focused on.

另一类错误是当网络预测实际存在时，场景中没有交通信号灯。 不幸的是，该技术对于理解网络出错的原因不是很有用，因为它没有关注图像的特定部分。

The last kind of mistake I looked at was when the network predicted a traffic light when there actually wasn’t one. See the example below:

我所看到的最后一种错误是当网络实际上预测不到一个交通信号灯时。 请参阅以下示例：

And with the heat-map plotted on top:

并在顶部绘制热图：

Looks like the network confused the parking sign light for a traffic light. Interesting to see that it was just the right parking sign and not the left one.

好像网络将停车标志灯换成了交通信号灯一样。 有趣的是，它只是右停车标志，而不是左停车标志。

结论 (Conclusion)

This method is very simple yet effective to gain insights into what a ConvNet is focusing on in an image. Unfortunately it doesn’t tell us why it’s focusing on that part.

此方法非常简单，但有效地了解了ConvNet在图像中关注的内容。 不幸的是，它没有告诉我们为什么它专注于这一部分。

I also experimented a little with generating a saliency map as described in “Deep Inside Convolutional Networks”, but didn’t get any visually pleasing results.

我还做了一些尝试，如“ 深度内卷积网络 ”中所述，生成了显着性图，但是没有得到任何视觉上令人愉悦的结果。

If you know of any other interesting ways to understand what ConvNets are doing, please write a comment below ?

如果您知道了解ConvNets正在做什么的其他有趣方式，请在下面写评论？

If you enjoyed reading this post, please tap ♥ below!

如果您喜欢阅读这篇文章，请点击下面的♥ ！

翻译自: https://www.freecodecamp.org/news/what-is-my-convnet-looking-at-7b0533e4d20e/

无人驾驶汽车入门