ollama部署模型

1.下载ollamaDownload Ollama on Windows

Download Ollamahttps://ollama.com/download

2.在全局环境变量中配置模型地址

```

OLLAMA_HOST   0.0.0.0:11434

OLLAMA_MODELS D:\models_ollama

```

3.下载并运行模型

```

ollama run qwen:1.8b

```

4.访问本地模型

使用openai API

pip install openai

# Example: reuse your existing OpenAI setup

from openai import OpenAI

# Point to the local server

client = OpenAI(base_url="http://localhost:11434/v1", api_key="none")

completion = client.chat.completions.create(

  model="qwen:1.8b",

  messages=[

    {"role": "user", "content": "讲一个50字以内的笑话"}

  ],

  temperature=0.7,

  top_p=0.95,

)

print(completion.choices[0].message)

使用langchain

from langchain_community.llms.openai import OpenAI

from langchain_community.chat_models.openai import ChatOpenAI

from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler

from langchain_community.llms import Tongyi

 

# 创建一个OpenAI集成对象，用于与OpenAI模型交互

# 注意：API只支持流式输出,并且要指定模型

# llm = OpenAI(base_url="http://localhost:11434/v1", api_key="none",model="qwen:1.8b")

llm = ChatOpenAI(base_url="http://localhost:11434/v1", api_key="none",streaming=True,model="qwen:1.8b",

    callbacks=[StreamingStdOutCallbackHandler()]  )

# llm = Tongyi(base_url="http://localhost:11434/v1", api_key="none",model="qwen:1.8b")

llm.invoke("讲一个50字以内的笑话")

---------------------------------------------------------------------------------------------------

ollama常用命令：

```

ollama  serve       Start ollama

ollama  create      Create a model from a Modelfile

ollama  show        Show information for a model

ollama  run         Run a model

ollama  pull        Pull a model from a registry

ollama  push        Push a model to a registry

ollama  list        List models

ollama  ps          List running models

ollama  cp          Copy a model

ollama  rm          Remove a model

ollama  help        Help about any command