SamplingParams参数

sampling_params = SamplingParams(temperature=0.8, top_p=0.95)

Sampling parameters for text generation.
我们遵循OpenAI文本完成API (https://platform.openai.com/docs/api-reference/completions/create)的采样参数。
此外，我们支持beam search，这是OpenAI不支持的。
n:为给定提示符返回的输出序列的数目。
best_of:从提示符生成的输出序列的数目。从这些best_of序列中，返回前n个序列。
best_of必须大于或等于n。use_beam_search为True时的波束宽度。默认情况下，best_of
设为n。从提示符生成的输出序列的数目。从这些best_of序列中，返回前n个序列。best_of必须大于或等于n。当use_beam_search为True时，这将被视为beam宽度。默认情况下，best_of被设置为n。
presence_penalty:根据新tokens到目前为止是否出现在生成的文本中来惩罚它们的浮点数。值> 0鼓励模型使用新的tokens，而值< 0鼓励模型重复tokens。
frequency_penalty:根据新标记在生成文本中的出现频率对其进行惩罚的浮点数。值> 0鼓励模型使用新的令牌，而值< 0鼓励模型重复令牌。
repetition_penalty:根据新标记到目前为止是否出现在提示符和生成的文本中来惩罚它们的浮点数。值> 1鼓励模型使用新的令牌，而值< 1鼓励模型重复令牌。
温度:控制采样随机性的浮点数。较低的值使模型更具确定性，而较高的值使模型更具随机性。0表示贪婪抽样。
top_p:控制要考虑的顶级令牌的累积概率的浮点数。必须在(0,1)内。设置为1以考虑所有令牌。
top_k:控制要考虑的顶级令牌数量的整数。设置为-1以考虑所有令牌。
min_p:表示要考虑的令牌的最小概率的浮点数，相对于最可能的令牌的概率。必须在[0,1]范围内。设置为0禁用此功能。
seed:用于生成的随机种子。
use_beam_search:是否使用波束搜索而不是采样。
length_penalty:根据序列的长度惩罚序列的浮点数。用于波束搜索。
early_stop:控制束流搜索的停止条件。它接受以下值:True，一旦有best_of完整的候选项，生成就停止;False，当应用启发式并停止生成时，它不太可能找到更好的候选者;“never”，其中束搜索过程只在没有更好的候选者时停止(规范束搜索算法)。
stop:生成时停止生成的字符串列表。返回的输出将不包含停止字符串。
stop_token_ids:在生成令牌时停止生成的令牌列表。返回的输出将包含停止标记，除非停止标记是特殊标记。
include_stop_str_in_output:是否在输出文本中包含停止字符串。默认为False。
ignore_eos:是否忽略EOS token，生成EOS token后继续生成token。
max_tokens:每个输出序列生成的最大令牌数量。
min_tokens:在生成EOS或stop_token_ids之前，每个输出序列生成的最小令牌数量
logprobs:每个输出令牌返回的日志概率数。注意，实现遵循OpenAI API:返回结果包括logprobs最可能令牌的日志概率，以及所选令牌。API将始终返回采样令牌的日志概率，因此响应中可能有多达logprobs+1个元素。
prompt_logprobs:每个提示令牌返回的日志概率数。
detokenize:是否对输出进行去标记化。默认为True。
skip_special_tokens:是否跳过输出中的特殊标记。
spaces_between_special_tokens:是否在输出中的特殊标记之间添加空格。默认为True。
logits_processors:基于先前生成的令牌修改logit的函数列表，并可选地将提示令牌作为第一个参数。
truncate_prompt_tokens:如果设置为整数k，将只使用提示符中的最后k个标记(即左截断)。默认为None(即不截断)。

repetition_penalty: Float that penalizes new tokens based on whether they appear in the prompt and the generated text so far. Values > 1 encourage the model to use new tokens, while values < 1 encourage the model to repeat tokens.
temperature: Float that controls the randomness of the sampling. Lower values make the model more deterministic, while higher values make the model more random. Zero means greedy sampling.
top_p: Float that controls the cumulative probability of the top tokens to consider. Must be in (0, 1]. Set to 1 to consider all tokens.
top_k: Integer that controls the number of top tokens to consider. Set to -1 to consider all tokens.
min_p: Float that represents the minimum probability for a token to be considered, relative to the probability of the most likely token.Must be in [0, 1]. Set to 0 to disable this.
seed: Random seed to use for the generation.
use_beam_search: Whether to use beam search instead of sampling.
length_penalty: Float that penalizes sequences based on their length. Used in beam search.
early_stopping: Controls the stopping condition for beam search. It accepts the following values: True, where the generation stops as soon as there are best_of complete candidates; False, where an heuristic is applied and the generation stops when is it very unlikely to find better candidates; “never”, where the beam search procedure only stops when there cannot be better candidates (canonical beam search algorithm).
stop: List of strings that stop the generation when they are generated.The returned output will not contain the stop strings.
stop_token_ids: List of tokens that stop the generation when they are generated. The returned output will contain the stop tokens unless the stop tokens are special tokens.
include_stop_str_in_output: Whether to include the stop strings in output text. Defaults to False.
ignore_eos: Whether to ignore the EOS token and continue generating tokens after the EOS token is generated.
max_tokens: Maximum number of tokens to generate per output sequence.
min_tokens: Minimum number of tokens to generate per output sequence before EOS or stop_token_ids can be generated
logprobs: Number of log probabilities to return per output token.Note that the implementation follows the OpenAI API: The return result includes the log probabilities on the logprobs most likely tokens, as well the chosen tokens. The API will always return the log probability of the sampled token, so there may be up to logprobs+1 elements in the response.
prompt_logprobs: Number of log probabilities to return per prompt token.
detokenize: Whether to detokenize the output. Defaults to True.
skip_special_tokens: Whether to skip special tokens in the output.
spaces_between_special_tokens: Whether to add spaces between special tokens in the output. Defaults to True.
logits_processors: List of functions that modify logits based on previously generated tokens, and optionally prompt tokens as a first argument.
truncate_prompt_tokens: If set to an integer k, will use only the last k tokens from the prompt (i.e., left truncation). Defaults to None (i.e., no truncation).