langchain中使用huggingface中的模型作为LLM可以离线运行

hehui0921

已于 2024-02-29 18:58:45 修改

阅读量925

点赞数 2

分类专栏： huggingface LangChain 文章标签： langchain java 前端

于 2024-02-27 17:49:34 首次发布

本文链接：https://blog.csdn.net/oHeHui1/article/details/136328583

版权

LangChain 同时被 2 个专栏收录

59 篇文章 4 订阅

订阅专栏

huggingface

33 篇文章 0 订阅

订阅专栏

文章介绍了如何使用HuggingFace的Pipeline接口，结合Transformer库，特别是GPT2和THUDM/chatglm3-6b模型进行文本生成，展示了如何创建chain-of-thoughtprompt并演示了在GPU上高效运行的示例。

摘要由CSDN通过智能技术生成

from langchain_community.llms.huggingface_pipeline import HuggingFacePipeline

hf = HuggingFacePipeline.from_model_id(
    model_id="gpt2",
    task="text-generation",
    pipeline_kwargs={"max_new_tokens": 50},
)

from langchain_community.llms.huggingface_pipeline import HuggingFacePipeline
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline


from langchain.prompts import PromptTemplate

template = """Question: {question}

Answer: Let's think step by step."""
prompt = PromptTemplate.from_template(template)

chain = prompt | hf

question = "What is the  result  of  1+ 1?"

print(chain.invoke({"question": question}))

# Use a pipeline as a high-level helper
!pip install langchain_community langchain
from transformers import pipeline

from langchain_community.llms import HuggingFacePipeline
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
from langchain.prompts import PromptTemplate

pipe = pipeline("text-generation", model="HuggingFaceH4/zephyr-7b-beta")

hf = HuggingFacePipeline(pipeline=pipe)



template = """Question: {question}

Answer: Let's think step by step."""
prompt = PromptTemplate.from_template(template)

chain = prompt | hf

question = "What is the  result  of  1+ 1?"

print(chain.invoke({"question": question}))

以下使用清华的大模型GPU运行：

from transformers import pipeline
from langchain_community.llms import HuggingFacePipeline
import time
hf = HuggingFacePipeline.from_model_id(
    model_id="THUDM/chatglm3-6b",
    task="text-generation",
    device=0,
    model_kwargs={"trust_remote_code":True},
    pipeline_kwargs={"max_new_tokens": 500},
)

for i in range(10):
    a = time.time()
    print(hf.invoke("西游记中描写了哪些人物"))
    print(time.time()-a)

也可以作为链式来调用.

from transformers import pipeline
from langchain_community.llms import HuggingFacePipeline
import time
from langchain.prompts import PromptTemplate
hf = HuggingFacePipeline.from_model_id(
    model_id="THUDM/chatglm3-6b",
    task="text-generation",
    device=0,
    model_kwargs={"trust_remote_code":True},
    pipeline_kwargs={"max_new_tokens": 500},
)
template = """{question}"""
prompt = PromptTemplate.from_template(template)
chain = prompt | hf
question = "西游记中描写了哪些人物?"
for i in range(10):
    a = time.time()
    print(chain.invoke({"question": question}))
    print(time.time()-a)