课程视频链接:书生·浦语大模型全链路开源体系_哔哩哔哩_bilibili
大模型的趋势是从专用模型(不同方向细分模型)转变为通用大模型(一个模型可以应对多种任务或者多种模态)。书生·浦语大模型(InternLM2)是当前优秀的开源通用大语言模型(LLM),其也提供了7B和20B的InternLM、InternLM-Chat、InternLM-Base、InternLM-Math、InternLM-XComposer供我们调用。
我个人的兴趣点在于大模型的应用,在课程中了解到从模型到应用的一般流程如下(图片来源课程视频):
图中,我特别关注了构建智能体模块。大模型聚焦于推理,其在环境的交互能力上功能欠缺,这里的交互指的是:例如,你询问LLM今天的天气并希望LLM根据今天的气温提供出行建议,当前的LLM并不能给出实时的建议。
为了解决实时性交互的问题(即LLM通过推理结果执行Action),智能体(Agent)的概念便被提出了。Agent的核心在于Action的执行,视频中提到了Lagent轻量级智能体框架,这里也简单阅读了一下源码。
在分析Lagent源码之前,可以先看datawhalechina开源的一个仓库: tiny-universe ,其中的TinyAgent项目构建了一个迷你Agent,学习这个项目有助于理解Lagent。TinyAgent遵循React模式,通过:Thought/Action/Action Input/Observation 不断循环迭代的方式交替生成推理轨迹和执行特定任务,使得LLM能够更好的完成实时交互任务。下面给出了项目中Agent.py的源码,可以看到:
- Agent的初始化阶段主要负责加载 ReAct Prompt、加载 LLM Model、加载 tools。
- Agent的text_completion阶段涉及到两次 LLM Model 的调用。第一次是调用 LLM 获取Action需要用到的 plugin_name 以及 plugin_args,而后Agent调用Plugin的call方法得到obsercation并加入到对话中。第二次调用是 LLM 通过前面的得到的对话进一步得到 Tought 和 Final Answer。
TOOL_DESC = """{name_for_model}: Call this tool to interact with the {name_for_human} API. What is the {name_for_human} API useful for? {description_for_model} Parameters: {parameters} Format the arguments as a JSON object."""
REACT_PROMPT = """Answer the following questions as best you can. You have access to the following tools:
{tool_descs}
Use the following format:
Question: the input question you must answer
Thought: you should always think about what to do
Action: the action to take, should be one of [{tool_names}]
Action Input: the input to the action
Observation: the result of the action
... (this Thought/Action/Action Input/Observation can be repeated zero or more times)
Thought: I now know the final answer
Final Answer: the final answer to the original input question
Begin!
"""
class Agent:
def __init__(self, path: str = '') -> None:
self.path = path
self.tool = Tools()
self.system_prompt = self.build_system_input()
self.model = InternLM2Chat(path)
def build_system_input(self):
tool_descs, tool_names = [], []
for tool in self.tool.toolConfig:
tool_descs.append(TOOL_DESC.format(**tool))
tool_names.append(tool['name_for_model'])
tool_descs = '\n\n'.join(tool_descs)
tool_names = ','.join(tool_names)
sys_prompt = REACT_PROMPT.format(tool_descs=tool_descs, tool_names=tool_names)
return sys_prompt
def parse_latest_plugin_call(self, text):
plugin_name, plugin_args = '', ''
i = text.rfind('\nAction:')
j = text.rfind('\nAction Input:')
k = text.rfind('\nObservation:')
if 0 <= i < j: # If the text has `Action` and `Action input`,
if k < j: # but does not contain `Observation`,
text = text.rstrip() + '\nObservation:' # Add it back.
k = text.rfind('\nObservation:')
plugin_name = text[i + len('\nAction:') : j].strip()
plugin_args = text[j + len('\nAction Input:') : k].strip()
text = text[:k]
return plugin_name, plugin_args, text
def call_plugin(self, plugin_name, plugin_args):
plugin_args = json5.loads(plugin_args)
if plugin_name == 'google_search':
return '\nObservation:' + self.tool.google_search(**plugin_args)
def text_completion(self, text, history=[]):
text = "\nQuestion:" + text
response, his = self.model.chat(text, history, self.system_prompt)
print(response)
plugin_name, plugin_args, response = self.parse_latest_plugin_call(response)
if plugin_name:
response += self.call_plugin(plugin_name, plugin_args)
response, his = self.model.chat(response, history, self.system_prompt)
return response, his
从上述流程可以看出,TinyAgent似乎只能进行一次工具调用,这也让我对Lagent是如何实现Thought/Action/Action Input/Observation的循环迭代产生好奇。直接打开Lagent的开源仓库,从源码目录结构可以看出代码核心是:actions、agents和llms,这里也简单对相关代码进行分析。
1. BaseModel类(llms目录下)
该框架兼容OpenAI API、Transformer、LMDeploy。因此这里先简单分析一下BaseModel类,GPTAPI、HFTransformer、LMDeployPipeline等类都继承了BaseModel,并重写或者补充了一些方法。
class BaseModel:
is_api: bool = False
def __init__(self,
path: str,
tokenizer_only: bool = False,
template_parser: 'LMTemplateParser' = LMTemplateParser,
meta_template: Optional[List[Dict]] = None,
*,
max_new_tokens: int = 512,
top_p: float = 0.8,
top_k: float = 40,
temperature: float = 0.8,
repetition_penalty: float = 1.0,
stop_words: Union[List[str], str] = None):
...
def generate(self, inputs: Union[str, List[str]], **gen_params) -> str:
...
def stream_generate(self, inputs: str, **gen_params) -> List[str]:
...
def chat(self, inputs: Union[List[dict], List[List[dict]]], **gen_params):
...
def generate_from_template(self, inputs: Union[List[dict],
List[List[dict]]],
**gen_params):
...
def stream_chat(self, inputs: List[dict], **gen_params):
...
def tokenize(self, prompts: Union[str, List[str], List[dict],
List[List[dict]]]):
...
def update_gen_params(self, **kwargs):
...
2. BaseAction类(actions目录下)
BaseAction定义了所有actions的基类并给出了自定义action的方法。
class BaseAction(metaclass=AutoRegister(TOOL_REGISTRY, ToolMeta)):
"""Base class for all actions.
Args:
description (:class:`Optional[dict]`): The description of the action.
Defaults to ``None``.
parser (:class:`Type[BaseParser]`): The parser class to process the
action's inputs and outputs. Defaults to :class:`JsonParser`.
enable (:class:`bool`): Whether the action is enabled. Defaults to
``True``.
Examples:
* simple tool
.. code-block:: python
class Bold(BaseAction):
'''Make text bold'''
def run(self, text: str):
'''
Args:
text (str): input text
Returns:
str: bold text
'''
return '**' + text + '**'
action = Bold()
* toolkit with multiple APIs
.. code-block:: python
class Calculator(BaseAction):
'''Calculator'''
@tool_api
def add(self, a, b):
'''Add operation
Args:
a (int): augend
b (int): addend
Returns:
int: sum
'''
return a + b
@tool_api
def sub(self, a, b):
'''Subtraction operation
Args:
a (int): minuend
b (int): subtrahend
Returns:
int: difference
'''
return a - b
action = Calculator()
"""
...
3. react.py(agents目录下)
ReAct继承自BaseAgent,其流程其实和TinyAgent差不多。为了实现可以不断调用工具并在适当调用次数下得到Final Answer,这里的做法是:
- 做法一:设置一个 max_turn,一旦到达最大次数就输出Final Answer
- 做法二:在Prompt中提到LLM一旦认为可以结束tools调用了就会输出{thought}和{finish}。LLM通过自主判断是否输出"final answer",一旦出现“final answer”,ReActProtocol中的Parse函数便会返回设定 action_return.type == self._action_executor.finish_action.name,于是函数调用停止。
If you already know the answer, or you do not need to use tools,
please using the following format to reply:
```
{thought}the thought process to get the final answer
{finish}final answer
```
CALL_PROTOCOL_EN = """You are a assistant who can utilize external tools.
{tool_description}
To use a tool, please use the following format:
```
{thought}Think what you need to solve, do you need to use tools?
{action}the tool name, should be one of [{action_names}]
{action_input}the input to the action
```
The response after utilizing tools should using the following format:
```
{response}the results after call the tool.
```
If you already know the answer, or you do not need to use tools,
please using the following format to reply:
```
{thought}the thought process to get the final answer
{finish}final answer
```
Begin!"""
class ReAct(BaseAgent):
"""An implementation of ReAct (https://arxiv.org/abs/2210.03629)
Args:
llm (BaseModel or BaseAPIModel): a LLM service which can chat
and act as backend.
action_executor (ActionExecutor): an action executor to manage
all actions and their response.
protocol (ReActProtocol): a wrapper to generate prompt and
parse the response from LLM / actions.
max_turn (int): the maximum number of trails for LLM to generate
plans that can be successfully parsed by ReAct protocol.
Defaults to 4.
"""
def __init__(self,
llm: Union[BaseModel, BaseAPIModel],
action_executor: ActionExecutor,
protocol: ReActProtocol = ReActProtocol(),
max_turn: int = 4) -> None:
self.max_turn = max_turn
super().__init__(
llm=llm, action_executor=action_executor, protocol=protocol)
def chat(self, message: Union[str, dict, List[dict]],
**kwargs) -> AgentReturn:
if isinstance(message, str):
inner_history = [dict(role='user', content=message)]
elif isinstance(message, dict):
inner_history = [message]
elif isinstance(message, list):
inner_history = message[:]
else:
raise TypeError(f'unsupported type: {type(message)}')
offset = len(inner_history)
agent_return = AgentReturn()
default_response = 'Sorry that I cannot answer your question.'
for turn in range(self.max_turn):
prompt = self._protocol.format(
chat_history=[],
inner_step=inner_history,
action_executor=self._action_executor,
force_stop=(turn == self.max_turn - 1))
response = self._llm.chat(prompt, **kwargs)
inner_history.append(dict(role='assistant', content=response))
thought, action, action_input = self._protocol.parse(
response, self._action_executor)
action_return: ActionReturn = self._action_executor(
action, action_input)
action_return.thought = thought
agent_return.actions.append(action_return)
if action_return.type == self._action_executor.finish_action.name:
agent_return.response = action_return.format_result()
break
inner_history.append(self._protocol.format_response(action_return))
else:
agent_return.response = default_response
agent_return.inner_steps = inner_history[offset:]
return agent_return
总结:第一节课我主要关注了从模型到应用的流程,重点了解了智能体的构建。从tinyAgent开源项目入手,结合Lagent源码简单知悉了ReAct模式下模型调用工具的流程。