Benedict Evans：Ways to think about AGI思考 AGI 的方法:

​Benedict Evans本文发布于2024 年 5 月 4 日

How do we think about a fundamentally unknown and unknowable risk, when the experts agree only that they have no idea?
当专家们一致认为他们一无所知时，我们如何看待根本上未知和不可知的风险？

The manuscript for ‘A Logic Named Joe’
《乔的逻辑》手稿

In 1946, my grandfather, writing as ‘Murray Leinster’, published a science fiction story called ‘A Logic Named Joe’. Everyone has a computer (a ‘logic’) connected to a global network that does everything from banking to newspapers and video calls. One day, one of these logics, ‘Joe’, starts giving helpful answers to any request, anywhere on the network: invent an undetectable poison, say, or suggest the best way to rob a bank. Panic ensues - ‘Check your censorship circuits!’ - until they work out what to unplug. (My other grandfather, meanwhile, was using computers to spy on the Germans, and then the Russians.)
1946 年，我的祖父以“Murray Leinster”的名义出版了一部科幻小说，名为《名为乔的逻辑》。每个人都有一台连接到全球网络的计算机（一个“逻辑”），可以执行从银行业务到报纸和视频通话的所有操作。有一天，这些逻辑之一“乔”开始对网络上任何地方的任何请求提供有用的答案：例如，发明一种无法检测到的毒药，或者提出抢劫银行的最佳方法。恐慌随之而来——“检查你的审查电路！”——直到他们弄清楚要拔掉什么。 （与此同时，我的另一位祖父正在使用计算机监视德国人，然后是俄罗斯人。）

For as long as we’ve thought about computers, we’ve wondered if they could make the jump from mere machines, shuffling punch-cards and databases, to some kind of ‘artificial intelligence’, and wondered what that would mean, and indeed, what we’re trying to say with the word ‘intelligence’. There’s an old joke that ‘AI’ is whatever doesn’t work yet, because once it works, people say ‘that’s not AI - it’s just software’. Calculators do super-human maths, and databases have super-human memory, but they can’t do anything else, and they don’t understand what they’re doing, any more than a dishwasher understands dishes, or a drill understands holes. A drill is just a machine, and databases are ‘super-human’ but they’re just software. Somehow, people have something different, and so, on some scale, do dogs, chimpanzees and octopuses and many other creatures. AI researchers have come to talk about this as ‘general intelligence’ and hence making it would be ‘artificial general intelligence’ - AGI.
自从我们思考计算机以来，我们就想知道它们是否能够从纯粹的机器、打孔卡和数据库跳跃到某种“人工智能”，并想知道这意味着什么，事实上， ，我们想用“智能”这个词来表达什么。有一个老笑话说，“人工智能”是指还没有发挥作用的东西，因为一旦它发挥作用，人们就会说“这不是人工智能——这只是软件”。计算器具有超人的数学能力，数据库具有超人的记忆力，但它们不能做任何其他事情，而且它们不明白自己在做什么，就像洗碗机了解盘子或钻头了解孔一样。钻机只是一台机器，数据库是“超人”，但它们只是软件。不知何故，人们有一些不同的东西，在某种程度上，狗、黑猩猩、章鱼和许多其他生物也是如此。人工智能研究人员开始将其称为“通用智能”，因此将其称为“通用人工智能”——AGI。

If we really could create something in software that was meaningfully equivalent to human intelligence, it should be obvious that this would be a very big deal. Can we make software that can reason, plan, and understand? At the very least, that would be a huge change in what we could automate, and as my grandfather and a thousand other science fiction writers have pointed out, it might mean a lot more.
如果我们真的能够在软件中创造出与人类智能同等的东西，那么显然这将是一件非常大的事情。我们能开发出能够推理、计划和理解的软件吗？至少，这将是我们可以实现自动化的巨大变化，正如我的祖父和其他一千位科幻小说作家所指出的那样，这可能意味着更多。

Every few decades since 1946, there’s been a wave of excitement that sometime like this might be close, each time followed by disappointment and an ‘AI Winter’, as the technology approach of the day slowed down and we realised that we needed an unknown number of unknown further breakthroughs. In 1970 the AI pioneer Marvin Minsky claimed that in “from three to eight years we will have a machine with the g