输出多少个token，模型调用多少次，结果的随机性原理

一直以为模型接受用户输入，模型直接输出答案，而事实并非如此；事实：输出多少个token，模型调用多少次

案例一

def generate(model, vocab, start_text, length, seq_length, temperature, device):
    model.eval()
    with torch.no_grad():
        input_text = start_text
        for _ in range(length):
            inputs = torch.tensor([vocab.word2id[w] for w in input_text[-seq_length:].split()],
                                  dtype=torch.long).unsqueeze(0).to(device)
            outputs = model(inputs)
            predictions = F.softmax(outputs[-1] / temperature, dim=0).cpu().numpy()
            sampled_idx = np.random.choice(len(vocab.vocab), p=predictions)
            sampled_char = vocab.id2word[sampled_idx]
            input_text = input_text + ' ' + sampled_char
        print(input_text)
        return input_text

案例二、

import torch
from transformers import GPT2LMHeadModel, GPT2Tokenizer
 
 
model = GPT2LMHeadModel.from_pretrained("gpt2", torchscript=True).eval()
 
# tokenizer
tokenizer = GPT2Tokenizer.from_pretrained("gpt2")
in_text = "Lionel Messi is a"
in_tokens = torch.tensor(tokenizer.encode(in_text))
 
# inference
token_eos = torch.tensor([198]) # line break symbol
out_token = None
i = 0
with torch.no_grad():
    while out_token != token_eos:
        logits, _ = model(in_tokens)
        out_token = torch.argmax(logits[-1, :], dim=0, keepdim=True)
        in_tokens = torch.cat((in_tokens, out_token), 0)
        text = tokenizer.decode(in_tokens)
        print(f'step {i} input: {text}', flush=True)
        i += 1
 
out_text = tokenizer.decode(in_tokens)
print(f' Input: {in_text}')
print(f'Output: {out_text}')

输出：
step 0 input: Lionel Messi is a player
step 1 input: Lionel Messi is a player who
step 2 input: Lionel Messi is a player who has
step 3 input: Lionel Messi is a player who has been
step 4 input: Lionel Messi is a player who has been a
step 5 input: Lionel Messi is a player who has been a key
step 6 input: Lionel Messi is a player who has been a key part
step 7 input: Lionel Messi is a player who has been a key part of
step 8 input: Lionel Messi is a player who has been a key part of the
step 9 input: Lionel Messi is a player who has been a key part of the team
step 10 input: Lionel Messi is a player who has been a key part of the team’s
step 11 input: Lionel Messi is a player who has been a key part of the team’s success
step 12 input: Lionel Messi is a player who has been a key part of the team’s success.
step 13 input: Lionel Messi is a player who has been a key part of the team’s success.

Input: Lionel Messi is a
Output: Lionel Messi is a player who has been a key part of the team’s success.