机器视觉科学计算可视化_模因视觉：对模因进行分类的科学

机器视觉科学计算可视化

As a person of culture and science, I decided to build a model to identify memes. This problem is far simpler than the Image-Net competition and so a simpler solution is appropriate. I will demonstrate this by comparing the “Meme Vision” framework to ResNet-50 (the winner of Image-Net 2015).

作为文化和科学界的人，我决定建立一个模型来识别模因。 这个问题比Image-Net竞争要简单得多，因此更简单的解决方案是合适的。 我将通过比较“ Meme Vision”框架与ResNet-50 (2015年Image-Net的获胜者)来证明这一点。

方法：Meme Vision框架 (Method: Meme Vision framework)

In a previous article I explained the radial histogram method;

在上一篇文章中，我解释了径向直方图方法。

(TL;DR — it measures the distribution of color in each segment of the image)

(TL； DR-它测量图像各部分中的颜色分布)

Below we see how this can reduce images to very low dimensional representations.

下面我们看到如何将图像缩小为非常低的尺寸表示。

Basic radial color histogram example with 3 bins per color channel and 4 segments (giving 4*3³=108 features)

基本的径向颜色直方图示例，每个颜色通道具有3个bin，并具有4个细分(给出4 *3³= 108个特征)

The final Meme Vision model uses a few extra steps:

最终的Meme Vision模型使用了一些额外的步骤：

• Convert from RGB to HSV - color degradation is less of a problem to computers when viewed in the HSV palette.

  从RGB转换为HSV-在HSV调色板中查看时，颜色降低对计算机来说不是问题。

• Log transformation of pixel counts to help focus on the little differences.
  像素计数的对数转换可帮助您关注微小差异。
• Use 8 bins per channel (instead of 3) to distinguish similar color shades, which results 2048 features (instead of 108).
  每个通道使用8个bin(而不是3个)来区分相似的色度，从而产生2048个特征(而不是108个)。

• Feed these features into a linear support vector machine.

  将这些特征输入线性支持向量机 。

How to make and test your own MemeVision model

如何制作和测试自己的MemeVision模型

Radial color histogram features processed using this package. (ResNet-50 transfer used for the comparison below was implemented using Tensorflow).

使用此程序包处理的径向颜色直方图特征。 (以下比较中使用的ResNet-50传输是使用Tensorflow实现的 )。

结果：Meme Vision与ResNet-50传输 (Result: Meme Vision vs ResNet-50 transfer)

The performance of Meme Vision is compared to a neural net based upon ResNet-50 (training only the final layer for 80 epochs). The dataset used consists of 5716 memes classified under 115 labels — we use only 20% for training and the rest as the test set.

将Meme Vision的性能与基于ResNet-50的神经网络进行了比较(仅训练最后一个层达80个纪元)。 所使用的数据集包括分类为115个标签的5716个模因 -我们仅使用20％的训练量，其余的用作测试集。

Unsurprisingly, training and predicting is orders of magnitude quicker under the Meme Vision framework. The difference in accuracy is not important, the neural would certainly outperform Meme Vision if left to train for longer.

毫不奇怪，在Meme Vision框架下，训练和预测要快几个数量级。 准确性的差异并不重要，如果长时间训练，神经系统肯定会胜过Meme Vision。

The dataset:

数据集：

结论：有什么意义？ (Conclusion: What was the point?)

Firstly, if you scrape 400,000 images from Reddit over one year (as I did), then classify each meme using the Meme Vision model (as I did), then cross reference with the metadata related to each image (as I did), then you will be able to compute the relative popularity of each meme over time (see below). My next article will be a closer look at the results of this meme popularity analysis, along with a surprising discovery of great cultural significance.

首先，如果您在一年中从Reddit抓取了40万张图像(如我所做的)，然后使用Meme Vision模型对每个模因进行了分类(如我所做的那样)，然后将与每个图像相关的元数据进行了交叉引用(如我所做的)，然后您将能够计算出每个模因在一段时间内的相对受欢迎程度(请参见下文)。 我的下一篇文章将仔细研究这种模因流行度分析的结果，以及具有重大文化意义的令人惊讶的发现。

Secondly, and the main point of this article, is that what I did would have been harder if I had set out to use a more complex model. Admittedly there will difficult edge cases where a neural net is necessary to identify a meme (e.g. memes converted to black & white). But the point of Meme Vision is speed — it can be used to quickly pick all the low hanging fruit, leaving difficult cases for a more robust & expensive model.

其次，本文的重点是，如果我打算使用一个更复杂的模型，那么我所做的工作将会更加困难。 诚然，在困难的情况下，需要神经网络来识别模因(例如，转换为黑白的模因)。 但是Meme Vision的重点是速度-它可用于快速摘取所有悬而未决的水果，从而为更坚固，更昂贵的型号保留了困难的情况。

So next time you have a problem and your first thought is to consider implementing a capsule network as devised by Geoffrey Hinton (the ‘Godfather of deep learning’)… perhaps stop and think “is my problem really as complex as those being tackled by Geoffrey Hinton?”.

因此，下次您遇到问题时，您的第一个想法就是考虑实施由Geoffrey Hinton(“深度学习的教父”)设计的胶囊网络 ...也许停下来想一想，“我的问题真的和Geoffrey所解决的问题一样复杂吗？欣顿？”。

This is unrelated to the work I do at Machine Medicine Technologies.

这与我在 Machine Medicine Technologies 所做的工作无关 。

翻译自: https://towardsdatascience.com/meme-vision-framework-e90a9a7a4187

机器视觉科学计算可视化