开源大模型模型加载方式清单

新说一二

已于 2024-08-10 20:37:10 修改

阅读量44

点赞数

分类专栏： AI行业信息大表格文章标签：语言模型人工智能

于 2024-08-09 14:32:57 首次发布

本文链接：https://blog.csdn.net/beifeng20200101/article/details/141061079

版权

AI行业信息大表格专栏收录该内容

4 篇文章 0 订阅

订阅专栏

                    
                        
                    
                    序号加载方式库类型社区说明备注
1from_pretrainedtransformersSDK APIhuggingface.co通过AutoModelForCausalLM、AutoModel等加载
如：model = 
AutoModelForCausalLM.from_pretrained(
 model_dir,
 device_map=device,
 trust_remote_code=True,
 output_attentions=True
)
2pipelinetransformersSDK APIhuggingface.co如：model_path = snapshot_download(‘uer/roberta-base-finetuned-dianping-chinese’)
pipe = pipeline(“text-classification”, model=model_path)
result =pipe(“这个是示例！”) 
3from_pretrainedmodelscopeSDK APImodelscope.cn通过AutoModelForCausalLM、AutoModel等加载
如：model =AutoModelForCausalLM.from_pretrained(
 “Qwen/Qwen1.5-1.8B-Chat-GPTQ-Int4”,
 device_map=“auto”
)
4pipelinemodelscopeSDK APImodelscope.cn如：from modelscope.pipelines import pipeline
word_segmentation = pipeline(‘word-segmentation’)
5OllamalangchainHTTP APIhttps://ollama.com/ 
 https://www.langchain.com/from langchain.chains import LLMChain
from langchain.llms import Ollama
ollama_llm = Ollama(model=“qwen2”)
需要安装ollama，并启动相关模型
6OllamallamaindexHTTP APIhttps://ollama.com/ 
 https://www.llamaindex.ai/from llama_index.llms.ollama import Ollama
llm = Ollama(model=“qwen2”)
response = llm.complete(“Who is Laurie Voss?”)
print(response)
需要安装ollama，并启动相关模型
7

                

序号	加载方式	库	类型	社区	说明	备注
1	from_pretrained	transformers	SDK API	huggingface.co	通过AutoModelForCausalLM、AutoModel等加载如：model = AutoModelForCausalLM.from_pretrained( model_dir, device_map=device, trust_remote_code=True, output_attentions=True )
2	pipeline	transformers	SDK API	huggingface.co	如：model_path = snapshot_download(‘uer/roberta-base-finetuned-dianping-chinese’) pipe = pipeline(“text-classification”, model=model_path) result =pipe(“这个是示例！”)
3	from_pretrained	modelscope	SDK API	modelscope.cn	通过AutoModelForCausalLM、AutoModel等加载如：model =AutoModelForCausalLM.from_pretrained( “Qwen/Qwen1.5-1.8B-Chat-GPTQ-Int4”, device_map=“auto” )
4	pipeline	modelscope	SDK API	modelscope.cn	如：from modelscope.pipelines import pipeline word_segmentation = pipeline(‘word-segmentation’)
5	Ollama	langchain	HTTP API	https://ollama.com/ https://www.langchain.com/	from langchain.chains import LLMChain from langchain.llms import Ollama ollama_llm = Ollama(model=“qwen2”)	需要安装ollama，并启动相关模型
6	Ollama	llamaindex	HTTP API	https://ollama.com/ https://www.llamaindex.ai/	from llama_index.llms.ollama import Ollama llm = Ollama(model=“qwen2”) response = llm.complete(“Who is Laurie Voss?”) print(response)	需要安装ollama，并启动相关模型
7