使用生成模型来进行批次生成的简易代码（T5模型）

记录一下如何简单的利用生成模型T5来进行一系列句子的生成：

import torch
from transformers import T5Tokenizer, T5ForConditionalGeneration

#设定gpu
device=torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
#建立tokenizer
tokenizer = T5Tokenizer.from_pretrained("t5-base")
#建立模型
model = T5ForConditionalGeneration.from_pretrained("t5-base")
model.to(device)

#输入前缀
task_prefix = "translate English to German: "
# use different length sentences to test batching
#输入句子列表
sentences = ["The house is wonderful.", "I like to work in NYC."]

#将输入tokenize并放到GPU上
inputs = tokenizer([task_prefix + sentence for sentence in sentences], return_tensors="pt", padding=True)
inputs.to(device)

#模型生成
output_sequences = model.generate(
    input_ids=inputs["input_ids"],
    attention_mask=inputs["attention_mask"],
    do_sample=False,  # disable sampling to test if batching affects output
)

print(tokenizer.batch_decode(output_sequences, skip_special_tokens=True))

输出结果为：

非常方便，主要参考：https://huggingface.co/docs/transformers/model_doc/t5