机器学习 声音 分角色
Bart. Elmo. Bert. Kermit. Marge. What do they have in common?
巴特 埃尔莫 伯特 柯米特 玛格 他们有什么共同点？
They’re all beloved fictional characters from TV shows many of us watched when we were young. But that’s not all — they’re also all AI models.
它们都是我们许多人在年轻时观看的电视节目中最受欢迎的虚构人物。 但这还不是全部-它们都是AI模型。
In 2018, researchers at the Allen Institute published the language model ELMo. The lead author, Matt Peters, said the team brainstormed many acronyms for their model, and ELMo instantly stuck as a “whimsical but memorable” choice.
2018年，艾伦研究所(Allen Institute)的研究人员发布了语言模型ELMo 。 主要作者马特·彼得斯(Matt Peters)说，该团队对其模型进行了许多头脑风暴，而ELMo立即成为“异想天开但令人难忘”的选择。
What started out as an inside joke has become a full-blown trend.
最初由内心的笑话变成了一种成熟的趋势。
Google AI followed with BERT, an incredibly powerful and now widely used Transformer-based language model. Then, many more: ERNIE, KERMIT, BART, GROVER, etc. OpenAI almost named GPT-2 “Snuffleupagus, or Snuffy for short.” Just last month, Facebook AI published MARGE.
Google AI紧随其后的是BERT ，这是一种功能强大且现已广泛使用的基于Transformer的语言模型。 然后，还有更多： ERNIE ， KERMIT ， BART ， GROVER等。OpenAI 几乎将GPT-2命名为“ Snuffleupagus或简称Snuffy”。 就在上个月，Facebook AI发布了MARGE 。
This article gives an overview on all of these models and, of course, their character inspirations below:
本文概述了所有这些模型，以及下面它们的特征灵感：
ELMo(2018)：语言模型的嵌入 (ELMo (2018): Embeddings from Language Models)
The one that started this trend, ELMo is a deep contextualized word representation approach that is able to capture more characteristics about words (syntax, semantics, and more).
引发这一趋势的一种方法是ELMo ， 它是一种深层的上下文化词表示方法，能够捕获有关词的更多特征(语法，语义等)。
A big challenge in representing words as vectors (“word embeddings”) was that a word would be represented by the same vector no matter what context it was used in. However, “current” has different meanings in “what’s your current job?” versus “that is a strong river current” — we can’t just use one fixed representation for both “current”s!
将单词表示为矢量(“单词嵌入”)的一大挑战是，无论使用何种上下文，一个单词都将由相同的矢量表示。但是，“当前”在“您目前的工作是什么？”中具有不同的含义。 而不是“那是强劲的河流潮流 ” —我们不能仅对两个“ 潮流”使用一个固定的表示！
Thus, contextualized word embeddings were created to capture a word’s context in its representation as well. Rather than only reading a word at a time, ELMo reads the entire sentence for context before assigning every word its embedding, which is done using a bi-directional LSTM.
因此，创建情境字的嵌入到捕捉词的上下文中的表现也是如此。 ELMo不仅一次读取一个单词，还为上下文分配了整个句子，然后才为每个单词分配嵌入，这是使用双向LSTM完成的。
ELMo was a great step forward in natural language processing (NLP) research using language modeling. For an illustrated explanation of ELMo, I would highly recommend this resource.
ELMo在使用语言建模的自然语言处理(NLP)研究方面迈出了一大步。 对于ELMo的图解说明，我强烈推荐该资源 。
Outside machine learning and NLP, Elmo is an adorable furry red Muppet on the children’s show Sesame Street. Elmo likes surprises, pizza, and bubble baths and won the longest giggle award on the Golden Grover Awards. It first appeared on screen in 1980.
除了机器学习和NLP外，Elmo是儿童表演芝麻街上的可爱毛茸茸的红色布偶。 Elmo喜欢惊喜，披萨和泡泡浴，并获得了Golden Grover奖中最长的傻笑奖 。 它于1980年首次出现在屏幕上。
BERT(2019)：变压器的双向编码器表示 (BERT (2019): Bidirectional Encoder Representations from Transformers)
Google further transformed pre-training in NLP by introducing BERT, a new Transformer-based language model that was the first to allow deeply bidirectional and unsupervised representation.
Google通过引入 BERT (一种基于Transformer的新语言模型)进一步转变了NLP中的预培训，这是第一个允许深度 双向和无监督表示的模型。
Deeply bidirectional means in capturing contextual embeddings, BERT represents each word using both its previous and next context. (In comparison, ELMo is shallowly bidirectional.) Simply conditioning each word on its previous and next words can be problematic, so BERT randomly masks some of the words and conditions each word bidirectionally to predict the masked words.
BERT是捕获上下文嵌入的一种深度双向方法，它使用上一个和下一个上下文来表示每个单词。 (相比之下，ELMo是浅双向的。)仅对每个单词的前一个单词和下一个单词进行条件调整可能会出现问题，因此BERT会随机屏蔽某些单词，并双向地调整每个单词以预测被屏蔽的单词。
In its initial release, BERT already achieved impressive results in question-and-answering and natural language understanding tasks. BERT and other Transformer-based architectures have been a cornerstone of NLP research in the past year.
在最初的发行版中，BERT在问答和自然语言理解任务中已经取得了令人瞩目的成果。 过去一年，BERT和其他基于Transformer的体系结构一直是NLP研究的基石。
Outside machine learning and NLP, Bert is a friendly yellow character on Sesame Street. In his free time, he likes to read Boring Stories, eat oatmeal, and study pigeons.
除了机器学习和NLP外，Bert在芝麻街上是一个友好的黄色人物。 在业余时间，他喜欢阅读《 无聊的故事》 ，吃燕麦片和研究鸽子。
ERNIE(2019)：双重麻烦 (ERNIE (2019): Double Trouble)
You can’t have Bert without his best buddy Ernie — good thing researchers developed ERNIE (Sun et al.), ERNIE (Zhang et al.), and even ERNIE 2.0!
没有他最好的伙伴Ernie，您将无法拥有Bert，好东西研究人员开发了ERNIE (Sun等人)， ERNIE (Zhang等人)，甚至ERNIE 2.0 ！
The first ERNIE (Enhanced Representation through Knowledge Integration) presented a language model that extended BERT’s word masking strategy to entity-level and phrase-level masking. In doing so, this ERNIE can implicitly learn the prior knowledge of phrases, entities, and relationships between them during the training process.
第一个ERNIE (通过知识集成增强表示)提出了一种语言模型，该模型将BERT的单词屏蔽策略扩展到实体级和短语级屏蔽 。 这样，该ERNIE可以在训练过程中隐式学习短语，实体以及它们之间的关系的先验知识。
Less than two months later, a second ERNIE (Enhanced Language Representation with Informative Entities) was published. This ERNIE presented a language model that incorporated knowledge graphs to optimize for having as much information as possible. Knowledge graphs are a powerful way to represent data points and relations that link them together.
不到两个月后，第二本ERNIE (信息实体的增强语言表示)发布了。 该ERNIE提出了一种语言模型，该模型合并了知识图以进行优化以获取尽可能多的信息。 知识图是表示数据点和将它们链接在一起的关系的有效方法。
Outside machine learning and NLP, Ernie is a troublemaker whose life mission is to annoy Bert on Sesame Street. He is extremely fond of his rubber duckie and once famously said “I can’t hear you, I have a banana in my ear!”
除了机器学习和NLP之外，Ernie是一个麻烦制造者，其人生使命是惹恼芝麻街上的 Bert 。 他非常喜欢他的橡皮鸭，并且曾经有句著名的话：“我听不到您的声音，我的耳朵里有香蕉！”
KERMIT(2019)：插入变换制作的Kontextuell编码器表示形式 (KERMIT (2019): Kontextuell Encoder Representations Made by Insertion Transformations)
KERMIT is an insertion-based generative architecture that models the joint distribution, decompositions (i.e. its marginals), and conditionals together. The researchers found that KERMIT worked well for a few select tasks, including machine translation.
KERMIT是基于插入的生成体系结构，可对关节分布，分解(即其边际)和条件一起建模。 研究人员发现，KERMIT在某些选定的任务(包括机器翻译)中表现良好。
In case you were wondering like I was, “kontextuell” is Swedish for “contextual”. We know the researchers were quite intentional on this naming because later in the paper, they write “Then, like its friends ELMo (Peters et al., 2018), BERT (Devlin et al., 2019), and ERNIE (Sun et al., 2019), we can also use KERMIT…”
如果您像我一样想知道，“ kontextuell”是瑞典语的“上下文”。 我们知道研究人员在命名时非常有意，因为他们在论文的后面写道：“然后， 像它的朋友 ELMo(Peters等，2018)，BERT(Devlin等，2019)和ERNIE(Sun等， 。，2019)，我们也可以使用KERMIT…”
Outside machine learning and NLP, Kermit is an iconic singing frog that has graced many Muppet productions (Sam and Friends, Sesame Street, The Muppet Show, etc.) and memes (But That’s None of My Business, Evil Kermit, etc.). Introduced in 1955, Kermit is the oldest TV character on this list.
在机器学习和NLP之外，Kermit是一个标志性的唱歌青蛙，它使Muppet的许多作品( Sam和Friends，芝麻街，The Muppet Show等)和模因( 但不关我的事 ， Evil Kermit等)得到了赞誉 。 Kermit于1955年推出，是该列表中最古老的电视角色。
BART(2019)：双向和自回归变压器 (BART (2019): Bidirectional and Auto-Regressive Transformers)
Facebook AI built on BERT, GPT, and previous NLP pre-training work to create BART, a new pretrained model for text generation and comprehension that combines both bidirectional and auto-regressive Transformers.
基于BERT，GPT和以前的NLP预训练工作的Facebook AI创建了BART ，这是一种用于文本生成和理解的新的预训练模型，该模型结合了双向和自回归变压器。
BERT used masking in its bidirectional encoder, which meant that masked/missing words were predicted independently. As a result, BERT can’t be used for text generation.
BERT在其双向编码器中使用了屏蔽，这意味着屏蔽/缺失字是独立预测的。 结果，BERT不能用于文本生成。
In contrast, GPT is auto-regressive, which means that it predicts a future word from a set of words given a context in a forward direction. As a result, GPT can’t learn bidirectional interactions.
相反，GPT是自回归的，这意味着GPT从给定上下文中向前的一组单词中预测未来的单词。 结果，GPT无法学习双向交互。
BART combines these fundamental ideas together: 1) a document with random spans of text replaced with masks is encoded bidirectionally, then 2) the likelihood of the unmodified document is predicted with an autoregressive decoder.
BART将这些基本思想结合在一起：1)对用文本掩码替换的随机文本跨度的文档进行双向编码，然后2)用自回归解码器预测未修改文档的可能性。
Outside machine learning and NLP, Bart is one of TV’s most well-known young rebels. You can find Bart and his endless prank calls on The Simpsons.
除了机器学习和NLP外，Bart是电视界最知名的年轻叛军之一。 您可以在《辛普森一家》中找到Bart和他无休止的恶作剧电话。
格罗夫(2019) (GROVER (2019))
The NLP field has progressed very quickly in recent years to achieve highly believable summarization and translation. However, these technologies can also be used for less-than-positive purposes, such as AI-generated fake news and propaganda.
近年来，NLP领域取得了飞速发展，以实现高度可信的摘要和翻译。 但是，这些技术也可以用于不太积极的目的，例如AI生成的假新闻和宣传。
To combat this, researchers created GROVER, a publicly available generator that can write realistic and controlled fake news. GROVER’s purpose is so others could practice against it to develop better techniques for classifying AI-generated fake news from real, human-written news. At the time, the best discriminator could only distinguish AI-generated fake news from real news with 73% accuracy.
为了解决这个问题，研究人员创建了GROVER ， GROVER是一种公开可用的生成器，可以编写真实且受控的假新闻 。 GROVER的目的是让其他人可以实践它，以开发出更好的技术，以将AI生成的假新闻从真实的人工编写的新闻中分类。 当时，最佳判别器只能以73％的准确度区分AI生成的假新闻和真实新闻。
(In a baffling naming decision, there’s no acronym here — it’s called GROVER just because.)
(在一个令人困惑的命名决定中，这里没有首字母缩写，仅因为它而被称为GROVER。)
Outside machine learning and NLP, Grover is a furry, blue Sesame Street character who loves to help (or tries to help) others. Hopefully GROVER can help the AI world in addressing disinformation as much as Grover helps his friends.
在机器学习和NLP之外，格罗弗是一位毛茸茸的蓝色芝麻街角色，他喜欢帮助(或尝试帮助)他人。 希望GROVER可以像Grover帮助他的朋友一样，帮助AI世界解决虚假信息。
MARGE(2020)：检索和生成的多语言自动编码器 (MARGE (2020): Multilingual Autoencoder that Retrieves and Generates)
Published very recently by Facebook AI, MARGE is a new “a pre-trained sequence-to-sequence model learned with an unsupervised multilingual multi-document paraphrasing objective.”
MARGE由Facebook AI最近发布，是一种新的“预先训练的序列到序列模型，它是在无监督的多语言多文档措辞目标下学习的。”
In simpler words, it’s a pre-trained language model that generates text by 1) retrieving related text in other languages, 2) reconstructing the original text by finding patterns within them.
用简单的话来说，它是一个经过训练的语言模型，它通过以下步骤生成文本：1) 检索其他语言的相关文本，2)通过在其中查找模式来重建原始文本。
The researchers found that MARGE can successfully perform paraphrasing, translation, multi-document summarization, and information retrieval tasks all without any fine tuning. MARGE scored up to 35.8 on BLEU (Bilingual Evaluation Understudy), a metric for language translations, which is considered quite high for a model without fine tuning.
研究人员发现，MARGE可以成功执行释义，翻译，多文档摘要和信息检索任务，而无需进行任何微调。 MARGE在BLEU (双语评估研究)(语言翻译指标)上的得分高达35.8，对于没有微调的模型来说，这是相当高的。
Outside machine learning and NLP, Marge is a fictional character on The Simpsons. Mother of three kids (including Bart), Marge is also well-known for her tall, blue (bleu? 👀) hair.
在机器学习和NLP之外，Marge是《辛普森一家》中的虚构人物。 玛格(Marge)是三个孩子(包括巴特)的母亲，她以蓝色的高发着称。
(With both BART and MARGE, perhaps Facebook AI likes the Simpsons a lot?)
(对于BART和MARGE，也许Facebook AI非常喜欢Simpsons吗？)
最后的想法 (Final Thoughts)
It’s pretty cool that researchers pay homage to previous works in this funny, harmless way. Perhaps future state-of-the-art machine learning models will be named HERMiONE or ZUKo — I eagerly await the day for AI researchers to expand to other fiction realms.
研究人员以一种有趣，无害的方式向以前的工作致敬，这真是太酷了。 也许未来最先进的机器学习模型将被命名为HERMiONE或ZUKo-我热切地等待着AI研究人员将其扩展到其他小说领域的一天。
If there are any other AI models named after fictional characters, let me know!
如果还有其他以虚构人物命名的AI模型，请告诉我 ！
Thank you for reading! Subscribe to read more about research, resources, and issues related to AI.
感谢您的阅读！ 订阅以阅读有关AI的研究，资源和问题的更多信息。
Catherine Yeo is a CS undergraduate at Harvard interested in AI/ML/NLP, fairness and ethics, and everything related. Feel free to suggest ideas or say hi to her on Twitter.
Catherine Yeo是哈佛大学的CS本科生，对AI / ML / NLP，公平与道德以及所有相关方面感兴趣。 随时在 Twitter上 提出想法或向她打招呼 。
翻译自: https://towardsdatascience.com/machine-learnings-obsession-with-kids-tv-show-characters-728edfb43b3c
机器学习 声音 分角色
4332
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
