OpenAI GPT-3 API error: “This model‘s maximum context length is 4097 tokens“

题意：OpenAI GPT-3 API 错误：“此模型的最大上下文长度为 4097 个 token”

问题背景：

I am making a request to the completions endpoint. My prompt is 1360 tokens, as verified by the Playground and the Tokenizer. I won't show the prompt as it's a little too long for this question.

我正在向补全（completions）端点发送一个请求。我的提示（prompt）包含1360个标记（tokens），这一点已通过（Playground）和标记器（Tokenizer）验证。我不会展示这个提示，因为它对于这个问题来说有点过长。

Here is my request to openai in Nodejs using the openai npm package.

以下是我在 Node.js 中使用 openai npm 包向 OpenAI 发送请求的代码。

const response = await openai.createCompletion({
  model: 'text-davinci-003',
  prompt,
  max_tokens: 4000,
  temperature: 0.2
})

When testing in the playground my total tokens after response are 1374.

在 Playground 中测试时，我的响应后的总 token 数是 1374。

When submitting my prompt via the completions API I am getting the following error:

当我通过 completions API 提交我的提示时，我收到以下错误：

error: {
  message: "This model's maximum context length is 4097 tokens, however you requested 5360 tokens (1360 in your prompt; 4000 for the completion). Please reduce your prompt; or completion length.",
  type: 'invalid_request_error',
  param: null,
  code: null
}

If you have been able to solve this one, I'd love to hear how you did it.

如果你已经解决了这个问题，我很想知道你是怎么做到的。

问题解决：

The max_tokens parameter is shared between the prompt and the completion. Tokens from the prompt and the completion all together should not exceed the token limit of a particular OpenAI model.

`max_tokens` 参数在提示和完成之间是共享的。提示和完成的 token 总数不应超过特定 OpenAI 模型的 token 限制。

As stated in the official OpenAI article:

正如 OpenAI 官方文章中所述：

Depending on the model used, requests can use up to 4097 tokens shared between prompt and completion. If your prompt is 4000 tokens, your completion can be 97 tokens at most.

根据所使用的模型，请求可以使用最多 4097 个 token，在提示和完成之间共享。如果你的提示是 4000 个 token，那么你的完成最多可以有 97 个 token。

The limit is currently a technical limitation, but there are often creative ways to solve problems within the limit, e.g. condensing your prompt, breaking the text into smaller pieces, etc.

这个限制目前是技术上的限制，但通常可以通过创造性的方法在限制范围内解决问题，例如：精简提示、将文本拆分成更小的部分等。

Note: For counting tokens before(!) sending an API request, see this answer.

注意：有关在发送 API 请求之前计算 token 的信息，请参阅 [这个回答](https://stackoverflow.com/a/71453182)。

GPT-4 and GPT-4 Turbo models:

LATEST MODEL	DESCRIPTION	MAX TOKENS	TRAINING DATA
gpt-4-1106-preview	GPT-4 Turbo The latest GPT-4 model with improved instruction following, JSON mode, reproducible outputs, parallel function calling, and more. Returns a maximum of 4,096 output tokens. This preview model is not yet suited for production traffic. Learn more.	128,000 tokens	Up to Apr 2023
gpt-4-vision-preview	GPT-4 Turbo with vision Ability to understand images, in addition to all other GPT-4 Turbo capabilties. Returns a maximum of 4,096 output tokens. This is a preview model version and not suited yet for production traffic. Learn more.	128,000 tokens	Up to Apr 2023
gpt-4	Currently points to gpt-4-0613. See continuous model upgrades.	8,192 tokens	Up to Sep 2021
gpt-4-0613	Snapshot of gpt-4 from June 13th 2023 with improved function calling support.	8,192 tokens	Up to Sep 2021
gpt-4-32k	Currently points to gpt-4-32k-0613. See continuous model upgrades.	32,768 tokens	Up to Sep 2021
gpt-4-32k-0613	Snapshot of gpt-4-32k from June 13th 2023 with improved function calling support.	32,768 tokens	Up to Sep 2021
gpt-4-0314 (Legacy)	Snapshot of gpt-4 from March 14th 2023 with function calling support. This model version will be deprecated on June 13th 2024.	8,192 tokens	Up to Sep 2021
gpt-4-32k-0314 (Legacy)	Snapshot of gpt-4-32k from March 14th 2023 with function calling support. This model version will be deprecated on June 13th 2024.	32,768 tokens	Up to Sep 2021

LATEST MODEL	DESCRIPTION	MAX TOKENS	TRAINING DATA
gpt-3.5-turbo-1106	Updated GPT 3.5 Turbo The latest GPT-3.5 Turbo model with improved instruction following, JSON mode, reproducible outputs, parallel function calling, and more. Returns a maximum of 4,096 output tokens. Learn more.	16,385 tokens	Up to Sep 2021
gpt-3.5-turbo	Currently points to gpt-3.5-turbo-0613. Will point to gpt-3.5-turbo-1106 starting Dec 11, 2023. See continuous model upgrades.	4,096 tokens	Up to Sep 2021
gpt-3.5-turbo-16k	Currently points to gpt-3.5-turbo-0613. Will point to gpt-3.5-turbo-1106 starting Dec 11, 2023. See continuous model upgrades.	16,385 tokens	Up to Sep 2021
gpt-3.5-turbo-instruct	Similar capabilities as text-davinci-003 but compatible with legacy Completions endpoint and not Chat Completions.	4,096 tokens	Up to Sep 2021
gpt-3.5-turbo-0613 (Legacy)	Snapshot of gpt-3.5-turbo from June 13th 2023. Will be deprecated on June 13, 2024.	4,096 tokens	Up to Sep 2021
gpt-3.5-turbo-16k-0613 (Legacy)	Snapshot of gpt-3.5-16k-turbo from June 13th 2023. Will be deprecated on June 13, 2024.	16,385 tokens	Up to Sep 2021
gpt-3.5-turbo-0301 (Legacy)	Snapshot of gpt-3.5-turbo from March 1st 2023. Will be deprecated on June 13th 2024.	4,096 tokens	Up to Sep 2021

GPT-3 models (Legacy):

LATEST MODEL	DESCRIPTION	MAX TOKENS	TRAINING DATA
text-curie-001	Very capable, faster and lower cost than Davinci.	2,049 tokens	Up to Oct 2019
text-babbage-001	Capable of straightforward tasks, very fast, and lower cost.	2,049 tokens	Up to Oct 2019
text-ada-001	Capable of very simple tasks, usually the fastest model in the GPT-3 series, and lowest cost.	2,049 tokens	Up to Oct 2019
davinci	Most capable GPT-3 model. Can do any task the other models can do, often with higher quality.	2,049 tokens	Up to Oct 2019
curie	Very capable, but faster and lower cost than Davinci.	2,049 tokens	Up to Oct 2019
babbage	Capable of straightforward tasks, very fast, and lower cost.	2,049 tokens	Up to Oct 2019
ada	Capable of very simple tasks, usually the fastest model in the GPT-3 series, and lowest cost.	2,049 tokens	Up to Oct 2019

GPT base models:

LATEST MODEL	DESCRIPTION	MAX TOKENS	TRAINING DATA
babbage-002	Replacement for the GPT-3 ada and babbage base models.	16,384 tokens	Up to Sep 2021
davinci-002	Replacement for the GPT-3 curie and davinci base models.	16,384 tokens	Up to Sep 2021