openai-gpt_您可以使用OpenAI GPT-3语言模型做什么？

openai-gpt

探索GPT-3：语言生成的新突破 (Exploring GPT-3: A New Breakthrough in Language Generation)

Substantial enthusiasm surrounds OpenAI’s GPT-3 language model, recently made accessible to beta users of the “OpenAI API”.

热情高涨的是OpenAI的GPT-3语言模型，该模型最近可供 “ OpenAI API”的 beta用户访问 。

什么是GPT-3？ (What is GPT-3?)

It seems like only last year that we were arguing about whether the slow-release rollout of the 1.5 billion parameter Generative Pretrained Transformer-2 (GPT-2) was reasonable. If the debate seems recent, that’s because it is (writing from 2020): The notorious GPT-2 model was announced by OpenAI in February 2019, but it wasn’t fully released until nearly 9 months later (although it was replicated before that). The release schedule was admittedly somewhat experimental, meant more to foster discussion of responsible open publishing, rather than a last-ditch effort to avert an AI apocalypse. That didn’t stop critics from questioning the hype-boosting publicity advantages of an ominous release cycle.

似乎仅在去年，我们就在争论15亿参数的Generative Pretrained Transformer-2(GPT-2) 的缓释部署是否合理。 如果辩论似乎是最近的，那是因为它是(写于2020年)：臭名昭著的GPT-2模型由OpenAI于2019年2月宣布，但直到将近9个月后才完全发布(尽管在此之前已被复制 ) 。 不可否认的是，发布时间表只是实验性的，更多的是要促进对负责任的公开发布的讨论，而不是为了避免AI启示而付出的最后努力。 但这并没有阻止 批评者质疑不祥释放周期的宣传效果。

All that is a bit moot by now because not only has OpenAI trained a much larger language model in GPT-3, but you can sign up to access it through their new API. Comparing GPT-3 to GPT-2 is like comparing apples to, well, raisins, because the model is about that much larger. While GPT-2 weighed in at a measly 1.542 billion parameters (with smaller release versions at 117, 345, and 762 million), the full-sized GPT-3 has 175 billion parameters. GPT-3 was also matched with a larger dataset for pre-training: 570GB of text compared to 40GB for GPT-2.

到目前为止，所有这些都还没有定论，因为OpenAI不仅在GPT-3中训练了更大的语言模型，而且您可以注册以通过其新API访问它。 将GPT-3与GPT-2进行比较就像将苹果与葡萄干进行比较一样，因为该模型要大得多。 GPT-2的参数仅为15.42亿(较小版本的117、345和7.62亿)，而完整版GPT-3的参数为1750亿。 GPT-3还与更大的预训练数据集匹配：570GB的文本，而GPT-2为40GB。

Approximate size comparison of GPT-2, represented by a human skeleton, and GPT-3 approximated by the bones of a Tyrannosaurus rex. Illustration by William Matthew in the public domain, published in 1905. GPT-3 has more than 100x more parameters than GPT-2.

GPT-2(由人类骨骼代表)和GPT-3(由霸王龙的骨骼近似)的大小比较。 William Matthew在公共领域的插图， 于1905 年 出版 。GPT-3的参数比GPT-2多100倍。

GPT-3 is the largest natural language processing (NLP) transformer released to date, eclipsing the previous record, Microsoft Research’s Turing-NLG at 17B parameters, by about 10 times. Unsurprisingly there has been plenty of excitement surrounding the model, and, given the plethora of GPT-3 demonstrations on Twitter and elsewhere, OpenAI has apparently been pretty accommodating in providing beta access to the new API. This has resulted in an explosion of demos: some good, some bad, all interesting. Some of these demos are now being touted as soon-to-be-released products, and in some cases may actually be useful. One thing’s for certain, NLP has come a long way from the days when naming guinea pigs or writing nonsensical sci-fi scripts were killer apps.

GPT-3是迄今为止发布的最大的自然语言处理(NLP)转换器，比以前的记录(Microsoft Research的Turing-NLG的17B参数)高出约10倍。 毫无疑问，该模型周围有很多令人兴奋的地方，并且考虑到Twitter和其他地方的GPT-3演示过多，OpenAI显然很乐于提供对新API的beta访问。 这导致了演示的爆炸式增长：一些好事，一些坏事，都有趣。 这些演示中的一些现在被吹捧为即将发布的产品，在某些情况下可能实际上是有用的。 可以肯定的是，自从命名豚鼠或编写荒谬的科幻脚本成为杀手级应用程序的那一天起，NLP已经走了很长一段路。

GPT-3的创意写作 (Creative Writing with the Help of GPT-3)

Unsurprisingly, several nearly passable blog posts have been written with the help of GPT-3, as experimenters get access to the API and try things out. Almost certainly the most thorough and visible investigation into GPT-3 for creative writing comes from Gwern Branwen at gwern.net. Having followed the NLP progress at OpenAI over the years, Gwern describes GPT-1 as “adorable,” GPT-2 as “impressive,” and GPT-3 as “scary” in their varying capabilities to mimic human language and style in text. Gwern has spent a substantial amount of time exploring the capabilities of GPT-3 and its predecessors, and the resulting musings on the current generation of GPT model and what might be holding it back are worth a read.

毫不奇怪，随着GPT-3的出现，实验人员可以访问API并进行尝试，从而撰写了几篇几乎可以通过的博客 文章 。 几乎可以肯定，最彻底，最明显的调查GPT-3的创作来源于Gwern Branwen在gwern.net 。 多年来，在遵循OpenAI上NLP的进展之后，Gwern将GPT-1形容为“可爱”，将GPT-2形容为“令人印象深刻”，将GPT-3形容为“恐怖”，因为它们具有模仿人类语言和文本样式的多种功能。 Gwern花了大量时间探索GPT-3及其前身的功能，因此对当前一代GPT模型产生的沉思以及可能使它陷入困境的值得一读 。

The OpenAI API does not currently facilitate a way of directly fine-tuning or training the GPT-3 model for specific tasks. Gwern argues, however, that the ability of GPT-3 to mimic writing styles and generate different types of output merely from a dialogue-like interaction with the experimenter amounts to a kind of emergent meta-learning. This wasn’t present in GPT-2, and Gwern posits the transformer attention mechanism as the means to facilitate this capability.

OpenAI API当前不支持直接微调或训练GPT-3模型以完成特定任务的方法。 但是，Gwern认为，GPT-3仅通过与实验者的类似于对话的交互来模仿写作风格并产生不同类型的输出的能力就构成了一种新兴的元学习。 G