我想像蔡徐坤一样打球
I recently started a new newsletter focus on AI education. TheSequence is a no-BS( meaning no hype, no news etc) AI-focused newsletter that takes 5 minutes to read. The goal is to keep you up to date with machine learning projects, research papers and concepts. Please give it a try by subscribing below:
我最近开始了一份有关AI教育的新时事通讯。 TheSequence是无BS(意味着没有炒作，没有新闻等)，它是专注于AI的新闻通讯，需要5分钟的阅读时间。 目标是让您了解机器学习项目，研究论文和概念的最新动态。 请通过以下订阅尝试一下：
Creating agents that resemble the cognitive abilities of the human brain has been one of the most elusive goals of the artificial intelligence(AI) space. Recently, I’ve been spending time on a couple of scenarios that relate to imagination in deep learning systems which reminded me of a very influential paper Alphabet’s subsidiary DeepMind published last year in this subject.
创建类似于人脑认知能力的代理一直是人工智能(AI)空间最难以捉摸的目标之一。 最近，我一直在花一些时间讨论深度学习系统中与想象有关的场景，这使我想起了去年Alphabet的子公司DeepMind在该主题上发表的非常有影响力的论文。
Imagination is one of those magical features of the human mind that differentiate us from other species. From the neuroscience standpoint, imagination is the ability of the brain to form images or sensations without any immediate sensorial input. Imagination is a key element of our learning process as it enable us to apply knowledge to specific problems and better plan for future outcomes. As we execute tasks in our daily lives, we are constantly “imagining” potential outcomes in order to optimize our actions. Not surprisingly, imagination is often perceived as a foundational enables of planning from a cognitive standpoint.
想象力是人类思维中将我们与其他物种区分开的那些神奇特征之一。 从神经科学的角度来看，想象力是大脑在没有任何直接感觉输入的情况下形成图像或感觉的能力。 想象力是我们学习过程的关键要素，因为它使我们能够将知识应用于特定问题并更好地计划未来的结果。 当我们执行日常生活中的任务时，我们不断地“想象”潜在的结果，以优化我们的行动。 毫不奇怪，从认知的角度来看，想象力通常被认为是进行计划的基础。
Incorporating imagination into artificial intelligence(AI) agents have long been an elusive goal of researchers in the space. Imagine( very appropriately 😉 ) AI programs that are not only able to lean new tasks to plan and reason about the future. Recently, we have seen some impressive results in the area of adding imagination to AI agents in systems such as AlphaGo. It has been precisely the DeepMind team who has been helping to formulate to initial theory of imagination-augmented AI agents. Last year, they published a new revision of a famous research paper that outlined one of the first neural network architectures to achieve this goal.
长期以来，将想象力整合到人工智能(AI)代理中一直是该领域研究人员难以实现的目标。 想象(非常恰当地😉)人工智能程序，它们不仅能够依靠新任务来计划和推理未来。 最近，我们在为AlphaGo等系统的AI代理增加想象力方面取得了令人瞩目的成果。 正是DeepMind团队一直在帮助构想具有想象力增强的AI代理的初始理论。 去年，他们发表了一篇著名研究论文的新修订版，其中概述了实现这一目标的首批神经网络体系结构之一。
How can we define imagination in the context of AI agents? In the case of DeepMind, they define imagination-augmented agents to systems that include the following characteristics:
我们如何在AI代理的背景下定义想象力？ 在DeepMind的情况下，他们为包含以下特征的系统定义了增强想象力的代理：
Deep reinforcement learning(RL) is often seen as the hallmark of imagination-augmented AI agents as it attempts to correlate observations with actions. However, deep RL systems typically require large amounts of training that results on knowledge tailored to a very specific set of tasks in an environment. The DeepMind paper proposes an alternative to traditional models by using models that use environment simulations to learn to “interpret” imperfect predictions. The idea is to have parallel models that use simulations to extract useful knowledge that can be used in the core model. Just like we often judge the level of imagination of an individual [that guy has no imagination], we can see the imagination models as an augmented capability of deep learning programs.
深度强化学习(RL)通常被视为增强想象力的AI代理的标志，因为它试图将观察结果与动作相关联。 但是，深度RL系统通常需要进行大量的培训，这些培训需要针对针对环境中非常特定的一组任务量身定制的知识。 DeepMind论文提出了一种替代传统模型的方法，即使用通过环境仿真来学习“解释”不完美预测的模型。 这个想法是要有并行模型，这些并行模型使用模拟来提取可用于核心模型的有用知识。 就像我们经常判断一个人的想象力水平(那个家伙没有想象力)一样，我们可以将想象力模型视为深度学习程序的增强功能。
I2A架构 (The I2A Architecture)
To enable “imagination” in deep learning agents, the DeepMind team relied on a clever neural network architecture known as I2A. The key element of the I2A architecture is a component called Imagination Core, that uses an environmental model to, given information about the current environment, make predictions about its future state. Given a past state and current action, the environment model predicts the next state and any number of signals from the environment. The I2A architecture, rolls out the environment model over multiple time steps into the future, by initializing the imagined trajectory with the present time real observation, and subsequently feeding simulated observations into the model. The actions produced in each rollout help to define the agent policy that is then used by the imagination core module.
为了在深度学习代理中实现“想象力”，DeepMind团队依赖于称为I2A的聪明的神经网络架构。 I2A体系结构的关键元素是一个称为Imagination Core的组件，该组件使用环境模型，在获得有关当前环境的信息的情况下，对其未来状态进行预测。 给定过去的状态和当前的动作，环境模型可以预测下一个状态以及来自环境的任何数量的信号。 I2A体系结构通过使用当前的实时观测值初始化想象的轨迹，然后将模拟的观测值输入到模型中，在未来的多个时间步骤中推出环境模型。 每个部署中产生的操作有助于定义代理策略，然后由想象力核心模块使用该策略。
One of the key elements of the I2A architecture are the rollout encoders that are responsible for “interpreting” the information produced by the imagination core, extracting any information useful for the agent’s decision, or even ignoring it when necessary.
I2A体系结构的关键要素之一是部署编码器，这些编码器负责“解释”由想象力核心生成的信息，提取对代理的决策有用的任何信息，甚至在必要时将其忽略。
玩推箱子 (Playing Sokoban)
To see the I2A model in action, the DeepMind team created an implementation that tried to play the famous Sokoban game. Sokoban is a classic planning problem, where the agent has to push a number of boxes onto given target locations. Because boxes can only be pushed (as opposed to pulled), many moves are irreversible, and mistakes can render the puzzle unsolvable. A human player is thus forced to plan moves ahead of time. The imagination augmented models showed impressive abilities to learn from imperfect environments like Sokoban as shown in the following video:
为了查看I2A模型的运行情况，DeepMind团队创建了一个尝试玩著名的推箱子游戏的实现。 推箱子是一个经典的计划问题，代理商必须将多个箱子推入给定的目标位置。 因为只能推箱子(而不是拉箱子)，所以许多动作都是不可逆的，而且错误会使拼图变得无法解决。 因此，人类玩家被迫提前计划动作。 想象力增强模型展示了令人印象深刻的能力，可从不完美的环境(如推箱子)中学习，如以下视频所示：
The DeepMind team benchmarked the I2A model against more traditional deep RL techniques and the results were remarkable. I2A achieved a shocking 85% performance which was vastly superior to other strategies.
DeepMind团队对照更传统的深度RL技术对I2A模型进行了基准测试，结果令人瞩目。 I2A达到了令人震惊的85％的性能，大大优于其他策略。
One of the most impressive takeaways from the Sokoban experiments was the ability of imagination-augmented agents to imagine trajectories in potentially imperfect environment models and ignore inaccurate information. This is particularly relevant given the growing number of scenarios that require AI agents to operate with imperfect information and limited data.
推箱子实验最令人印象深刻的收获之一是增强想象力的特工能够在可能不完美的环境模型中想象轨迹并忽略不准确信息的能力。 鉴于越来越多的场景需要AI代理使用不完善的信息和有限的数据进行操作，因此这特别相关。
Imagination is one of those key capabilities that can open the door to a new generation of AI agents. Techniques like I2A are still in the very nascent state but can become a key building block of reinforcement learning architectures in which agents are not only able to learn the present but to “imagine” the future.
想象力是可以为新一代AI代理打开大门的关键功能之一。 诸如I2A之类的技术仍处于起步阶段，但可以成为强化学习体系结构的关键组成部分，在这种体系结构中，代理人不仅能够学习现在，而且能够“想象”未来。
我想像蔡徐坤一样打球
扫一扫
1万+
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
