一步一步教你建立自己的账户_自我监督的学习挑战和下一步

一步一步教你建立自己的账户

Self-supervised learning is a type of learning (representations of image, text, audio, and so on) where the data (not labelled by humans) itself provides some sort of supervision. One of the main ideas is to force the network to learn useful features by doing predictions on some pre-text tasks.

自我监督学习是一种学习(图像，文本，音频等的表示形式)，其中数据(未由人类标记)本身提供某种监督。 主要思想之一是通过对一些前置任务进行预测来迫使网络学习有用的功能。

Recently I had a chance to attend a virtual workshop on self-supervised learning (https://sslwin.org/). Ideas and methods on SSL go back to decades ago but it is an active area of research now since many researchers are realizing the limits of supervised learning on “fixed datasets”. Many prominent researchers in the SSL area gave talks in the workshop and I thought it would be good to share a summary/view on this sought-after topic.

最近，我有机会参加了一个关于自我监督学习的虚拟研讨会( https://sslwin.org/ )。 关于SSL的思想和方法可以追溯到几十年前，但是由于许多研究人员已经意识到“固定数据集”上的监督学习的局限性，所以它一直是研究的活跃领域。 SSL领域的许多著名研究人员在研讨会上进行了演讲，我认为最好就此热门话题分享摘要/观点。

演示地址

Self-Supervision as a Path to a Post-Dataset Era — Alexei Alyosha Efros

自我监督是进入数据集后时代的道路— Alexei Alyosha Efros

Prof. Alyosha gave a nice overview of the “Pre-dataset Era” in Computer Vision followed by “The Dataset Era (2000-present)” to emphasize how Datasets played a crucial role in speeding up the progress in the field. How it became easy to benchmark multiple methods and know if a new method is in-fact improving the state of the art (SOTA)?

Alyosha教授对“计算机视觉”中的“数据集前时代”进行了很好的概述，随后对“数据集时代(2000年至今)”进行了很好的概述，以强调数据集如何在加快该领域的进展中起关键作用。 对多种方法进行基准测试并知道一种新方法实际上是否正在改进现有技术(SOTA)，这变得多么容易？

He talks about how datasets for training became common and issues such as “Dataset bias” started creeping in.

他谈到了用于训练的数据集如何变得普遍，以及诸如“数据集偏差”之类的问题开始蔓延。

We are raising a generation of algorithms who can only cram for the test (set). — Alyosha Efros

我们正在培养一代只能为测试(集合)填充的算法。 — Alyosha Efros

He then talks about going beyond fixed datasets and proposes to use continual, online learning. He calls it Test-Time Training. I especially liked his points around “smoothness” being an important constraint for this sort of learning. He also mentions that video is a great frontier for self-supervised learning due to its streaming property. He presents an interesting work on self-supervised learning for visual correspondence using a graph constructed from a `palindrome’ of frames.

然后，他谈论了超越固定数据集的问题，并建议使用连续在线学习。 他称其为“测试时间培训” 。 我特别喜欢他关于“平稳”的观点，这种观点是这种学习的重要限制。 他还提到，由于视频具有流媒体特性，因此它是自我监督学习的重要前沿。 他提出了一项有趣的工作，即使用从框架“回文”构建的图形进行视觉监督的自我监督学习。

演示地址

Self-Supervision & Modularity: Cornerstones for Generalization in Embodied Agents — Deepak Pathak

自我监督和模块性：在具体代理中进行泛化的基石-Deepak Pathak

The next talk by Deepak continues to touch upon similar points. He focuses first on the question “Why do we need self-supervision?”. He then poses three questions on SSL’s (1) Goal (2) Setup and (3) Efficiency. He attributes the slow progress in the field of “robotics” or “embodied agents” to continually, changing test data.

迪帕克(Deepak)的下一次演讲继续涉及类似的问题。 他首先关注“为什么我们需要自我监督？”这个问题。 然后，他对SSL的(1)目标(2)设置和(3)效率提出了三个问题。 他将“机器人”或“实体代理”领域的缓慢进展归因于不断变化的测试数据。

Self-supervised intrinsic motivation to explore the environment could be the key in learning skills needed to perform various tasks

自我监督的探索环境的内在动机可能是执行各种任务所需的学习技能的关键

By predicting the consequences of the action and re-iterating on scenarios where the prediction was bad, the agent is able to continually learn on its own. They use the disagreement of multiple models as the notion of curiosity.

通过预测操作的后果并在预测不佳的情况下重复执行此操作，代理可以自行继续学习。 他们使用多种模型的分歧作为好奇心的概念。

He further discusses recent work on incorporating multi-step “planning” in explorations (ICML 2020). They showed that the agent is able to perform as good as with an oracle which knows the rewards in the environment. In a very interesting work, they showed that the robot learned in this manner could imitate human using one sample. Finally, he discusses the idea of bringing modularity to hardware where the controller for each limb/motor could be trained and shared across all the motors/limbs.

他进一步讨论了将多步骤“计划”纳入勘探的最新工作( ICML 2020 )。 他们表明，该代理能够像知道环境奖励的神谕一样出色地工作。 在一项非常有趣的工作中，他们表明以这种方式学习的机器人可以使用一个样本来模仿人类。 最后，他讨论了将模块化引入硬件的想法，其中可以训练每个肢体/电动机的控制器并在所有电动机/肢体之间共享。

演示地址

Multi-view Invariance and Grouping for Self-Supervised Learning — Ishan Misra

自主学习的多视图不变性和分组— Ishan Misra

Ishan talks about two key properties that are important for learning representations: (1) Multi-view invariance (2) Grouping. Throughout his talk, he discusses different approaches against these properties.

Ishan讨论了对于学习表示形式很重要的两个关键属性：(1)多视图不变性(2)分组。 在整个演讲中，他讨论了针对这些属性的不同方法。

He argues “pretext” task (pre-2019) based learning representations aren’t actually learning semantically meaningful representations. His own work PIRL uses contrastive learning (CL) to learn features that are invariant to a pre-text task. It has “multi-view invariance” to some extent but it is weak in grouping.

他认为基于“前置”任务(2019年前)的学习表示实际上并不是在学习语义上有意义的表示。 他自己的作品PIRL使用对比学习(CL)来学习与前置任务无关的功能。 它在某种程度上具有“多视图不变性”，但是在分组方面较弱。

In Contrastive Learning for SSL the positives lack the notion of grouping since its the same sample.

在SSL的对比学习中，肯定因素缺乏分组的概念，因为其样本相同。

In his work AVID-CMA, authors used the combined features of video and audio to group similar segments to introduce “grouping” in the CL loss. He then presents “SwAV” which is an online algorithm that uses a swapping prediction mechanism to predict the cluster assignment (codes) of a view from the representation of another view.

在他的作品AVID-CMA中，作者使用视频和音频的组合功能将相似的片段分组，以在CL损失中引入“分组”。 然后，他提出“ SwAV ”，这是一种在线算法，它使用交换预测机制从另一个视图的表示中预测一个视图的群集分配(代码)。

There were other interesting talks in the workshop and I plan to summarize them in the next part of this post.

研讨会上还有其他有趣的演讲，我计划在本文的下一部分中进行总结。

翻译自: https://medium.com/swlh/self-supervised-learning-challenges-and-next-steps-d324d265a31c

一步一步教你建立自己的账户