神经网络算法是黑盒_在黑盒内窥视如何欺骗神经网络

神经网络算法是黑盒

Neural networks get a bad reputation for being black boxes. And while it certainly takes creativity to understand their decision making, they are really not as opaque as people would have you believe.

神经网络被誉为黑匣子，因此声誉不佳。 尽管理解他们的决策当然需要创造力，但实际上他们并没有人们想像的那么模糊。

In this tutorial, I’ll show you how to use backpropagation to change the input as to classify it as whatever you would like.

在本教程中，我将向您展示如何使用反向传播更改输入以将其分类为所需的内容。

Follow along using this colab.

继续使用此colab 。

(This work was co-written with Alfredo Canziani ahead of an upcoming video)

(此作品是在即将上映的视频之前与Alfredo Canziani共同撰写的)

人类像黑匣子 (Humans as black boxes)

Let’s consider the case of humans. If I show you the following input:

让我们考虑人类的情况。 如果我向您显示以下输入：

there’s a good chance you have no idea whether this is a 5 or a 6. In fact, I believe that I could even make a case for convincing you that this might also be an 8.

还有你不知道这是否是一个5或6。其实一个很好的机会，我相信，我甚至可以充分的理由说服你，这也可能是8。

Now, if you asked a human what they would have to do to make something more into a 5 you might visually do something like this:

现在，如果您问一个人，他们将要做些什么才能将更多的东西变成5，那么您可能会在视觉上执行以下操作：

And if I wanted you to make this more into an 8, you might do something like this:

如果我希望您将其设置为8，则可以执行以下操作：

Now, the answer to this question is not easy to explain in a few if statements or by looking at a few coefficients (yes, I’m looking at you regression). Unfortunately, with certain types of inputs (images, sound, video, etc…) explainability certainly becomes much harder but not impossible.

现在，用几个if语句或查看几个系数不容易解释这个问题的答案(是的，我正在看您的回归)。 不幸的是，对于某些类型的输入(图像，声音，视频等)，可解释性当然会变得更加困难， 但并非不可能 。

询问神经网络 (Asking the neural network)

How would a neural network answer the same questions I posed above? Well, to answer that, we can use gradient ascent to do exactly that.

神经网络将如何回答我上面提出的相同问题？ 好吧，要回答这个问题，我们可以使用梯度上升来做到这一点。

Here’s how the neural network thinks we would need to modify the input to make it more into a 5.

这是神经网络认为我们需要修改输入以使其更多地变为5。

There are two interesting results from this. First, the black areas are where the network things we need to remove pixel density from. Second, the yellow areas are where it thinks we need to add more pixel density.

有两个有趣的结果。 首