大模型从入门到应用——LangChain：模型（Models）-[大型语言模型（LLMs）：LLM的异步API、自定义LLM包装器、虚假LLM和人类输入LLM（Human Input LLM）]

分类目录：《大模型从入门到应用》总目录

LangChain系列文章：

• 基础知识
• 快速入门
  • 安装与环境配置
  • 链（Chains）、代理（Agent:）和记忆（Memory）
  • 快速开发聊天模型
• 模型（Models）
  • 基础知识
  • 大型语言模型（LLMs）
    • 基础知识
    • LLM的异步API、自定义LLM包装器、虚假LLM和人类输入LLM（Human Input LLM）
    • 缓存LLM的调用结果
    • 加载与保存LLM类、流式传输LLM与Chat Model响应和跟踪tokens使用情况
  • 聊天模型（Chat Models）
    • 基础知识
    • 使用少量示例和响应流式传输
  • 文本嵌入模型
    • Aleph Alpha、Amazon Bedrock、Azure OpenAI、Cohere等
    • Embaas、Fake Embeddings、Google Vertex AI PaLM等
• 提示（Prompts）
  • 基础知识
  • 提示模板
    • 基础知识
    • 连接到特征存储
    • 创建自定义提示模板和含有Few-Shot示例的提示模板
    • 部分填充的提示模板和提示合成
    • 序列化提示信息
  • 示例选择器（Example Selectors）
  • 输出解析器（Output Parsers）
• 记忆（Memory）
  • 基础知识
  • 记忆的类型
    • 会话缓存记忆、会话缓存窗口记忆和实体记忆
    • 对话知识图谱记忆、对话摘要记忆和会话摘要缓冲记忆
    • 对话令牌缓冲存储器和基于向量存储的记忆
  • 将记忆添加到LangChain组件中
  • 自定义对话记忆与自定义记忆类
  • 聊天消息记录
  • 记忆的存储与应用
• 索引（Indexes）
  • 基础知识
  • 文档加载器（Document Loaders）
  • 文本分割器（Text Splitters）
  • 向量存储器（Vectorstores）
  • 检索器（Retrievers）
• 链（Chains）
  • 基础知识
  • 通用功能
    • 自定义Chain和Chain的异步API
    • LLMChain和RouterChain
    • SequentialChain和TransformationChain
    • 链的保存（序列化）与加载（反序列化）
  • 链与索引
    • 文档分析和基于文档的聊天
    • 问答的基础知识
    • 图问答（Graph QA）和带来源的问答（Q&A with Sources）
    • 检索式问答
    • 文本摘要（Summarization）、HyDE和向量数据库的文本生成
• 代理（Agents）
  • 基础知识
  • 代理类型
  • 自定义代理（Custom Agent）
  • 自定义MRKL代理
  • 带有ChatModel的LLM聊天自定义代理和自定义多操作代理（Custom MultiAction Agent）
  • 工具
    • 基础知识
    • 自定义工具（Custom Tools）
    • 多输入工具和工具输入模式
    • 人工确认工具验证和Tools作为OpenAI函数
  • 工具包（Toolkit）
  • 代理执行器（Agent Executor）
    • 结合使用Agent和VectorStore
    • 使用Agents的异步API和创建ChatGPT克隆
    • 处理解析错误、访问中间步骤和限制最大迭代次数
    • 为代理程序设置超时时间和限制最大迭代次数和为代理程序和其工具添加共享内存
  • 计划与执行
• 回调函数（Callbacks）

----### LLM的异步API
LangChain通过利用asyncio库为LLM提供异步支持。异步支持对于同时调用多个LLM特别有用，因为这些调用是网络绑定的。当前，支持的LLMs包括OpenAI、PromptLayerOpenAI、ChatOpenAI和Anthropic，后续还会提供其他LLMs的异步支持。我们可以使用agenerate方法异步调用OpenAI LLM：

import time
import asyncio

from langchain.llms import OpenAI

def generate_serially():
    llm = OpenAI(temperature=0.9)
    for _ in range(10):
        resp = llm.generate(["Hello, how are you?"])
        print(resp.generations[0][0].text)

async def async_generate(llm):
    resp = await llm.agenerate(["Hello, how are you?"])
    print(resp.generations[0][0].text)

async def generate_concurrently():
    llm = OpenAI(temperature=0.9)
    tasks = [async_generate(llm) for _ in range(10)]
    await asyncio.gather(*tasks)

s = time.perf_counter()
# 如果在Jupyter之外运行，请使用asyncio.run(generate_concurrently())
await generate_concurrently() 
elapsed = time.perf_counter() - s
print('\033[1m' + f"Concurrent executed in {elapsed:0.2f} seconds." + '\033[0m')

s = time.perf_counter()
generate_serially()
elapsed = time.perf_counter() - s
print('\033[1m' + f"Serial executed in {elapsed:0.2f} seconds." + '\033[0m')

输出：

I'm doing well, thank you. How about you?
    
I'm doing well, thank you. How about you?
    
I'm doing well, how about you?
    
I'm doing well, thank you. How about you?
    
I'm doing well, thank you. How about you?
    
I'm doing well, thank you. How about yourself?
    
I'm doing well, thank you! How about you?
    
I'm doing well, thank you. How about you?
    
I'm doing well, thank you! How about you?
    
I'm doing well, thank you. How about you?
[1mConcurrent executed in 1.39 seconds.

I'm doing well, thank you. How about you?
    
I'm doing well, thank you. How about you?
    
I'm doing well, thank you. How about you?
    
I'm doing well, thank you. How about you?
    
I'm doing well, thank you. How about yourself?
    
I'm doing well, thanks for asking. How about you?
    
I'm doing well, thanks! How about you?
    
I'm doing well, thank you. How about you?
    
I'm doing well, thank you. How about yourself?
    
I'm doing well, thanks for asking. How about you?
[1mSerial executed in 5.77 seconds.

编写自定义LLM包装器

本部分介绍如何创建自定义的LLM包装器，这部分的内容使我们可以使用自己的LLM或与LangChain支持的包装器不同的包装器。一个自定义LLM只需要实现一件必须的事情：

• 一个_call方法，它接受一个字符串、一些可选的停止词，并返回一个字符串。

它还可以实现一个可选的事情:

• 一个_identifying_params属性，用于帮助打印这个类。应该返回一个字典。

让我们实现一个非常简单的自定义LLM，它只返回输入的前$N$个字符：

from typing import Any, List, Mapping, Optional

from langchain.callbacks.manager import CallbackManagerForLLMRun
from langchain.llms.base import LLM
class CustomLLM(LLM):
    n: int
  
    @property
    def _llm_type(self) -> str:
        return "custom"
    
    def _call(
        self,
        prompt: str,
        stop: Optional[List[str]] = None,
        run_manager: Optional[CallbackManagerForLLMRun] = None,
    ) -> str:
        if stop is not None:
            raise ValueError("stop kwargs are not permitted.")
        return prompt[:self.n]
    
    @property
    def _identifying_params(self) -> Mapping[str, Any]:
        """Get the identifying parameters."""
        return {"n": self.n}

我们现在可以像使用其他LLM一样使用它了：

llm = CustomLLM(n=10)
llm("This is a foobar thing")

输出：

'This is a '

我们还可以打印LLM并查看它的自定义打印：

print(llm)

输出：

CustomLLM
Params: {'n': 10}

虚假LLM

LangChain提供了一个用于测试的虚假LLM类，这使我们可以模拟对LLM的调用，并模拟LLM以特定方式响应时会发生什么。我们从在代理中使用FakeLLM开始：

from langchain.llms.fake import FakeListLLM
from langchain.agents import load_tools
from langchain.agents import initialize_agent
from langchain.agents import AgentType

tools = load_tools(["python_repl"])

responses=[
    "Action: Python REPL\nAction Input: print(2 + 2)",
    "Final Answer: 4"
]
llm = FakeListLLM(responses=responses)

agent = initialize_agent(tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True)
 

agent.run("whats 2 + 2")

日志输出：

> Entering new AgentExecutor chain...
Action: Python REPL
Action Input: print(2 + 2)
Observation: 4

Thought:Final Answer: 4

> Finished chain.

输出：

'4'

人类输入LLM（Human Input LLM）

与fake LLM类似，LangChain提供了一个伪LLM课程，可用于测试、调试或教育目的。这使我们可以模拟对LLM的调用，并模拟人类接收提示时的响应方式，我们从在代理中使用Human Input LLM开始：

from langchain.llms.human import HumanInputLLM
from langchain.agents import load_tools
from langchain.agents import initialize_agent
from langchain.agents import AgentType

由于我们后面将使用WikipediaQueryRun工具，如果我们还没有安装wikipedia包，我们需要安装它：

pip install wikipedia

接着我们就可以开始使用wikipedia：

tools = load_tools(["wikipedia"])
llm = HumanInputLLM(prompt_func=lambda prompt: print(f"\n===PROMPT====\n{prompt}\n=====END OF PROMPT======"))
agent = initialize_agent(tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True)
agent.run("What is 'Bocchi the Rock!'?")

日志输出：

> Entering new AgentExecutor chain...

===PROMPT====
Answer the following questions as best you can. You have access to the following tools:

Wikipedia: A wrapper around Wikipedia. Useful for when you need to answer general questions about people, places, companies, historical events, or other subjects. Input should be a search query.

Use the following format:

Question: the input question you must answer
Thought: you should always think about what to do
Action: the action to take, should be one of [Wikipedia]
Action Input: the input to the action
Observation: the result of the action
... (this Thought/Action/Action Input/Observation can repeat N times)
Thought: I now know the final answer
Final Answer: the final answer to the original input question

Begin!

Question: What is 'Bocchi the Rock!'?
Thought:
=====END OF PROMPT======
I need to use a tool.
Action: Wikipedia
Action Input: Bocchi the Rock!, Japanese four-panel manga and anime series.
Observation: Page: Bocchi the Rock!
Summary: Bocchi the Rock! (ぼっち・ざ・ろっく!, Bocchi Za Rokku!) is a Japanese four-panel manga series written and illustrated by Aki Hamaji. It has been serialized in Houbunsha's seinen manga magazine Manga Time Kirara Max since December 2017. Its chapters have been collected in five tankōbon volumes as of November 2022.
An anime television series adaptation produced by CloverWorks aired from October to December 2022. The series has been praised for its writing, comedy, characters, and depiction of social anxiety, with the anime's visual creativity receiving acclaim.

Page: Manga Time Kirara
Summary: Manga Time Kirara (まんがタイムきらら, Manga Taimu Kirara) is a Japanese seinen manga magazine published by Houbunsha which mainly serializes four-panel manga. The magazine is sold on the ninth of each month and was first published as a special edition of Manga Time, another Houbunsha magazine, on May 17, 2002. Characters from this magazine have appeared in a crossover role-playing game called Kirara Fantasia.

Page: Manga Time Kirara Max
Summary: Manga Time Kirara Max (まんがタイムきららMAX) is a Japanese four-panel seinen manga magazine published by Houbunsha. It is the third magazine of the "Kirara" series, after "Manga Time Kirara" and "Manga Time Kirara Carat". The first issue was released on September 29, 2004. Currently the magazine is released on the 19th of each month.
Thought:
===PROMPT====
Answer the following questions as best you can. You have access to the following tools:

Wikipedia: A wrapper around Wikipedia. Useful for when you need to answer general questions about people, places, companies, historical events, or other subjects. Input should be a search query.

Use the following format:

Question: the input question you must answer
Thought: you should always think about what to do
Action: the action to take, should be one of [Wikipedia]
Action Input: the input to the action
Observation: the result of the action
... (this Thought/Action/Action Input/Observation can repeat N times)
Thought: I now know the final answer
Final Answer: the final answer to the original input question

Begin!

Question: What is 'Bocchi the Rock!'?
Thought:I need to use a tool.
Action: Wikipedia
Action Input: Bocchi the Rock!, Japanese four-panel manga and anime series.
Observation: Page: Bocchi the Rock!
Summary: Bocchi the Rock! (ぼっち・ざ・ろっく!, Bocchi Za Rokku!) is a Japanese four-panel manga series written and illustrated by Aki Hamaji. It has been serialized in Houbunsha's seinen manga magazine Manga Time Kirara Max since December 2017. Its chapters have been collected in five tankōbon volumes as of November 2022.
An anime television series adaptation produced by CloverWorks aired from October to December 2022. The series has been praised for its writing, comedy, characters, and depiction of social anxiety, with the anime's visual creativity receiving acclaim.

Page: Manga Time Kirara
Summary: Manga Time Kirara (まんがタイムきらら, Manga Taimu Kirara) is a Japanese seinen manga magazine published by Houbunsha which mainly serializes four-panel manga. The magazine is sold on the ninth of each month and was first published as a special edition of Manga Time, another Houbunsha magazine, on May 17, 2002. Characters from this magazine have appeared in a crossover role-playing game called Kirara Fantasia.

Page: Manga Time Kirara Max
Summary: Manga Time Kirara Max (まんがタイムきららMAX) is a Japanese four-panel seinen manga magazine published by Houbunsha. It is the third magazine of the "Kirara" series, after "Manga Time Kirara" and "Manga Time Kirara Carat". The first issue was released on September 29, 2004. Currently the magazine is released on the 19th of each month.
Thought:
=====END OF PROMPT======
These are not relevant articles.
Action: Wikipedia
Action Input: Bocchi the Rock!, Japanese four-panel manga series written and illustrated by Aki Hamaji.
Observation: Page: Bocchi the Rock!
Summary: Bocchi the Rock! (ぼっち・ざ・ろっく!, Bocchi Za Rokku!) is a Japanese four-panel manga series written and illustrated by Aki Hamaji. It has been serialized in Houbunsha's seinen manga magazine Manga Time Kirara Max since December 2017. Its chapters have been collected in five tankōbon volumes as of November 2022.
An anime television series adaptation produced by CloverWorks aired from October to December 2022. The series has been praised for its writing, comedy, characters, and depiction of social anxiety, with the anime's visual creativity receiving acclaim.
Thought:
===PROMPT====
Answer the following questions as best you can. You have access to the following tools:

Wikipedia: A wrapper around Wikipedia. Useful for when you need to answer general questions about people, places, companies, historical events, or other subjects. Input should be a search query.

Use the following format:

Question: the input question you must answer
Thought: you should always think about what to do
Action: the action to take, should be one of [Wikipedia]
Action Input: the input to the action
Observation: the result of the action
... (this Thought/Action/Action Input/Observation can repeat N times)
Thought: I now know the final answer
Final Answer: the final answer to the original input question

Begin!

Question: What is 'Bocchi the Rock!'?
Thought:I need to use a tool.
Action: Wikipedia
Action Input: Bocchi the Rock!, Japanese four-panel manga and anime series.
Observation: Page: Bocchi the Rock!
Summary: Bocchi the Rock! (ぼっち・ざ・ろっく!, Bocchi Za Rokku!) is a Japanese four-panel manga series written and illustrated by Aki Hamaji. It has been serialized in Houbunsha's seinen manga magazine Manga Time Kirara Max since December 2017. Its chapters have been collected in five tankōbon volumes as of November 2022.
An anime television series adaptation produced by CloverWorks aired from October to December 2022. The series has been praised for its writing, comedy, characters, and depiction of social anxiety, with the anime's visual creativity receiving acclaim.

Page: Manga Time Kirara
Summary: Manga Time Kirara (まんがタイムきらら, Manga Taimu Kirara) is a Japanese seinen manga magazine published by Houbunsha which mainly serializes four-panel manga. The magazine is sold on the ninth of each month and was first published as a special edition of Manga Time, another Houbunsha magazine, on May 17, 2002. Characters from this magazine have appeared in a crossover role-playing game called Kirara Fantasia.

Page: Manga Time Kirara Max
Summary: Manga Time Kirara Max (まんがタイムきららMAX) is a Japanese four-panel seinen manga magazine published by Houbunsha. It is the third magazine of the "Kirara" series, after "Manga Time Kirara" and "Manga Time Kirara Carat". The first issue was released on September 29, 2004. Currently the magazine is released on the 19th of each month.
Thought:These are not relevant articles.
Action: Wikipedia
Action Input: Bocchi the Rock!, Japanese four-panel manga series written and illustrated by Aki Hamaji.
Observation: Page: Bocchi the Rock!
Summary: Bocchi the Rock! (ぼっち・ざ・ろっく!, Bocchi Za Rokku!) is a Japanese four-panel manga series written and illustrated by Aki Hamaji. It has been serialized in Houbunsha's seinen manga magazine Manga Time Kirara Max since December 2017. Its chapters have been collected in five tankōbon volumes as of November 2022.
An anime television series adaptation produced by CloverWorks aired from October to December 2022. The series has been praised for its writing, comedy, characters, and depiction of social anxiety, with the anime's visual creativity receiving acclaim.
Thought:
=====END OF PROMPT======
It worked.
Final Answer: Bocchi the Rock! is a four-panel manga series and anime television series. The series has been praised for its writing, comedy, characters, and depiction of social anxiety, with the anime's visual creativity receiving acclaim.

> Finished chain.

输出：

"Bocchi the Rock! is a four-panel manga series and anime television series. The series has been praised for its writing, comedy, characters, and depiction of social anxiety, with the anime's visual creativity receiving acclaim."

参考文献：
[1] LangChain 🦜️🔗 中文网，跟着LangChain一起学LLM/GPT开发：https://www.langchain.com.cn/
[2] LangChain中文网 - LangChain 是一个用于开发由语言模型驱动的应用程序的框架：http://www.cnlangchain.com/