huggingface transformers库中LlamaForCausalLM

最新推荐文章于 2024-07-19 16:36:18 发布

Jerry_liu20080504

最新推荐文章于 2024-07-19 16:36:18 发布

阅读量5.2k

点赞数 1

文章标签： python transformers

本文链接：https://blog.csdn.net/Jerry_liu20080504/article/details/132585937

版权

新手入门笔记。

LlamaForCausalLM 的使用示例，这应该是一段推理代码。

from transformers import AutoTokenizer, LlamaForCausalLM

model = LlamaForCausalLM.from_pretrained(PATH_TO_CONVERTED_WEIGHTS)
tokenizer = AutoTokenizer.from_pretrained(PATH_TO_CONVERTED_TOKENIZER)

prompt = "Hey, are you conscious? Can you talk to me?"
inputs = tokenizer(prompt, return_tensors="pt")

# Generate
generate_ids = model.generate(inputs.input_ids, max_length=30)
tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0]

参考：

Llama2
https://huggingface.co/docs/transformers/v4.32.1/en/model_doc/llama2#transformers.LlamaForCausalLM