Qwen的FunctionCall模版整理

原创已于 2025-01-09 14:43:50 修改

· 1.1k 阅读

1 ·

版权

文章标签：

#prompt #nlp #自然语言处理 #人工智能

于 2024-12-12 21:15:54 首次发布

Qwen的chattemplate一看太复杂了，似乎最开始有tools的说明，assisatant会输出tool call并加入对话，以下给出Qwen官方的一个demo和自己的注释说明

基本Function call的实现划分为4步骤

定义函数，并使用Json格式存储函数说明
定义对话Prompt模板，将tools和message送给LLM处理
解析LLM的<tool_call></tool_call>并执行函数得到返回结果<tool_response></tool_response>
函数返回结果送给LLM继续处理query

{%- if tools %} #是否存在工具类，在对话最开始提供
    {{- '<|im_start|>system\n' }}
    {%- if messages[0]['role'] == 'system' %}
        {{- messages[0]['content'] }}
    {%- else %}
        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
    {%- endif %}
    {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
    {%- for tool in tools %}
        {{- "\n" }}
        {{- tool | tojson }}
    {%- endfor %}
    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
{%- else %} #正常对话第一步走这个流程
    {%- if messages[0]['role'] == 'system' %}
        {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
    {%- else %}
        {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
    {%- endif %}
{%- endif %}
{%- for message in messages %} #遍历多轮对话信息
    {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
    {%- elif message.role == "assistant" %}
        {{- '<|im_start|>' + message.role }}
        {%- if message.content %}
            {{- '\n' + message.content }}
        {%- endif %}
        {%- for tool_call in message.tool_calls %}
            {%- if tool_call.function is defined %}
                {%- set tool_call = tool_call.function %}
            {%- endif %}
            {{- '\n<tool_call>\n{"name": "' }}
            {{- tool_call.name }}
            {{- '", "arguments": ' }}
            {{- tool_call.arguments | tojson }}
            {{- '}\n</tool_call>' }}
        {%- endfor %}
        {{- '<|im_end|>\n' }}
    {%- elif message.role == "tool" %}
        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
            {{- '<|im_start|>user' }}
        {%- endif %}
        {{- '\n<tool_response>\n' }}
        {{- message.content }}
        {{- '\n</tool_response>' }}
        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
            {{- '<|im_end|>\n' }}
        {%- endif %}
    {%- endif %}
{%- endfor %}
{%- if add_generation_prompt %}  #一般需要add_generation_prompt=True，直到模型输出eos_token
    {{- '<|im_start|>assistant\n' }}
{%- endif %}

接下来是官方的一个demo，写了温度获取的函数，LLM调用这个函数，传入适当的参数并执行函数获得返回结果，LLM基于返回的结果再思考下一步的行动，代码如下：

import json

def get_current_temperature(location: str, unit: str = "celsius"):
    """Get current temperature at a location.

    Args:
        location: The location to get the temperature for, in the format "City, State, Country".
        unit: The unit to return the temperature in. Defaults to "celsius". (choices: ["celsius", "fahrenheit"])

    Returns:
        the temperature, the location, and the unit in a dict
    """
    return {
        "temperature": 26.1,
        "location": location,
        "unit": unit,
    }


def get_temperature_date(location: str, date: str, unit: str = "celsius"):
    """Get temperature at a location and date.

    Args:
        location: The location to get the temperature for, in the format "City, State, Country".
        date: The date to get the temperature for, in the format "Year-Month-Day".
        unit: The unit to return the temperature in. Defaults to "celsius". (choices: ["celsius", "fahrenheit"])

    Returns:
        the temperature, the location, the date and the unit in a dict
    """
    return {
        "temperature": 25.9,
        "location": location,
        "date": date,
        "unit": unit,
    }


def get_function_by_name(name):
    if name == "get_current_temperature":
        return get_current_temperature
    if name == "get_temperature_date":
        return get_temperature_date

TOOLS = [
    {
        "type": "function",
        "function": {
            "name": "get_current_temperature",
            "description": "Get current temperature at a location.",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": 'The location to get the temperature for, in the format "City, State, Country".',
                    },
                    "unit": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"],
                        "description": 'The unit to return the temperature in. Defaults to "celsius".',
                    },
                },
                "required": ["location"],
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "get_temperature_date",
            "description": "Get temperature at a location and date.",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": 'The location to get the temperature for, in the format "City, State, Country".',
                    },
                    "date": {
                        "type": "string",
                        "description": 'The date to get the temperature for, in the format "Year-Month-Day".',
                    },
                    "unit": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"],
                        "description": 'The unit to return the temperature in. Defaults to "celsius".',
                    },
                },
                "required": ["location", "date"],
            },
        },
    },
]
MESSAGES = [
    {"role": "system", "content": "You are Qwen, created by Alibaba Cloud. You are a helpful assistant.\n\nCurrent Date: 2024-09-30"},
    {"role": "user",  "content": "What's the temperature in San Francisco now? How about tomorrow?"},
]

#</tool_call>是151657的special token
from transformers import AutoModelForCausalLM, AutoTokenizer,Qwen2Tokenizer,Qwen2ForCausalLM
model_path = "/home/wangsong/qwen"

model = Qwen2ForCausalLM.from_pretrained(
    model_path,
    torch_dtype="auto",
    device_map="cpu"
)

tokenizer = Qwen2Tokenizer.from_pretrained(model_path)

text = tokenizer.apply_chat_template(MESSAGES, tools=TOOLS, add_generation_prompt=True, tokenize=False)

经过Qwen的chat_template处理后，得到的待输入文本如下

""" 
<|im_start|>system
You are Qwen, created by Alibaba Cloud. You are a helpful assistant.

Current Date: 2024-09-30

# Tools

You may call one or more functions to assist with the user query.

You are provided with function signatures within <tools></tools> XML tags:
<tools>
{"type": "function", "function": {"name": "get_current_temperature", "description": "Get current temperature at a location.", "parameters": {"type": "object", "properties": {"location": {"type": "string", "description": "The location to get the temperature for, in the format \"City, State, Country\"."}, "unit": {"type": "string", "enum": ["celsius", "fahrenheit"], "description": "The unit to return the temperature in. Defaults to \"celsius\"."}}, "required": ["location"]}}}
{"type": "function", "function": {"name": "get_temperature_date", "description": "Get temperature at a location and date.", "parameters": {"type": "object", "properties": {"location": {"type": "string", "description": "The location to get the temperature for, in the format \"City, State, Country\"."}, "date": {"type": "string", "description": "The date to get the temperature for, in the format \"Year-Month-Day\"."}, "unit": {"type": "string", "enum": ["celsius", "fahrenheit"], "description": "The unit to return the temperature in. Defaults to \"celsius\"."}}, "required": ["location", "date"]}}}
</tools>

For each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:
<tool_call>
{"name": <function-name>, "arguments": <args-json-object>}
</tool_call><|im_end|>
<|im_start|>user
What's the temperature in San Francisco now? How about tomorrow?<|im_end|>
<|im_start|>assistant

"""


inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512)
output_text = tokenizer.batch_decode(outputs)[0][len(text):]

LLM理解query和tools,生成<tool_call>相关的json object作为response

""" 
<tool_call>
{"name": "get_temperature_date", "arguments": {"location": "San Francisco, USA", "date": "2024-10-05"}}
</tool_call>
<tool_call>
{"name": "get_temperature_date", "arguments": {"location": "San Francisco, USA", "date": "2024-10-06"}}
</tool_call><|im_end|>"""

解析工具调用output并执行函数，添加到message后续里让模型继续处理

import re
def try_parse_tool_calls(content: str):
    """Try parse the tool calls.
    解析生成的工具调用为一条消息，并将其添加到消息列表中，以便模型了解所使用的工具。
    """
    tool_calls = []
    offset = 0
    for i, m in enumerate(re.finditer(r"<tool_call>\n(.+)?\n</tool_call>", content)):
        if i == 0:
            offset = m.start()
        try:
            func = json.loads(m.group(1))
            tool_calls.append({"type": "function", "function": func})
            if isinstance(func["arguments"], str):
                func["arguments"] = json.loads(func["arguments"])
        except json.JSONDecodeError as e:
            print(f"Failed to parse tool calls: the content is {m.group(1)} and {e}")
            pass
    if tool_calls:
        if offset > 0 and content[:offset].strip():
            c = content[:offset]
        else:
            c = ""
        return {"role": "assistant", "content": c, "tool_calls": tool_calls}
    return {"role": "assistant", "content": re.sub(r"<\|im_end\|>$", "", content)}
def execute_tool_calls(MESSAGES):
    """
    #获取工具的结果并将其添加到消息列表中，以便模型了解工具调用的结果。
    """
    if tool_calls := MESSAGES[-1].get("tool_calls", None):
        for tool_call in tool_calls:
            if fn_call := tool_call.get("function"):
                fn_name: str = fn_call["name"]
                fn_args: dict = fn_call["arguments"]
                #调用函数，获得执行结果
                fn_res: str = json.dumps(get_function_by_name(fn_name)(**fn_args))

                MESSAGES.append({
                    "role": "tool",
                    "name": fn_name,
                    "content": fn_res,
                })
#对LLM生成的function_call解析并执行，然后添加到对话历史让LLM继续处理
MESSAGES.append(try_parse_tool_calls(output_text))
execute_tool_calls(MESSAGES)
"""
[{'role': 'system', 'content': 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.\n\nCurrent Date: 2024-09-30'}, 
{'role': 'user', 'content': "What's the temperature in San Francisco now? How about tomorrow?"}, 
{'role': 'assistant', 'content': '', 'tool_calls': [{'type': 'function', 'function': {'name': 'get_temperature_date', 'arguments': {'location': 'San Francisco, USA', 'date': '2024-10-05'}}}, {'type': 'function', 'function': {'name': 'get_temperature_date', 'arguments': {'location': 'San Francisco, USA', 'date': '2024 Venue'}}}]},
 {'role': 'tool', 'name': 'get_temperature_date', 'content': '{"temperature": 25.9, "location": "San Francisco, USA", "date": "2024-10-05", "unit": "celsius"}'},
  {'role': 'tool', 'name': 'get_temperature_date', 'content': '{"temperature": 25.9, "location": "San Francisco, USA", "date": "2024 Venue", "unit": "celsius"}'}]
"""
text = tokenizer.apply_chat_template(MESSAGES, tools=TOOLS, add_generation_prompt=True, tokenize=False)

将函数执行结果构造Message输入LLM

""""
<|im_start|>system
You are Qwen, created by Alibaba Cloud. You are a helpful assistant.

Current Date: 2024-09-30

# Tools

You may call one or more functions to assist with the user query.

You are provided with function signatures within <tools></tools> XML tags:
<tools>
{"type": "function", "function": {"name": "get_current_temperature", "description": "Get current temperature at a location.", "parameters": {"type": "object", "properties": {"location": {"type": "string", "description": "The location to get the temperature for, in the format \"City, State, Country\"."}, "unit": {"type": "string", "enum": ["celsius", "fahrenheit"], "description": "The unit to return the temperature in. Defaults to \"celsius\"."}}, "required": ["location"]}}}
{"type": "function", "function": {"name": "get_temperature_date", "description": "Get temperature at a location and date.", "parameters": {"type": "object", "properties": {"location": {"type": "string", "description": "The location to get the temperature for, in the format \"City, State, Country\"."}, "date": {"type": "string", "description": "The date to get the temperature for, in the format \"Year-Month-Day\"."}, "unit": {"type": "string", "enum": ["celsius", "fahrenheit"], "description": "The unit to return the temperature in. Defaults to \"celsius\"."}}, "required": ["location", "date"]}}}
</tools>

For each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:
<tool_call>
{"name": <function-name>, "arguments": <args-json-object>}
</tool_call><|im_end|>
<|im_start|>user
What's the temperature in San Francisco now? How about tomorrow?<|im_end|>
<|im_start|>assistant
<tool_call>
{"name": "get_temperature_date", "arguments": {"location": "San Francisco, USA", "date": "2024-10-05"}}
</tool_call>
<tool_call>
{"name": "get_temperature_date", "arguments": {"location": "San Francisco, USA", "date": "2024 Venue"}}
</tool_call><|im_end|>
<|im_start|>user
<tool_response>
{"temperature": 25.9, "location": "San Francisco, USA", "date": "2024-10-05", "unit": "celsius"}
</tool_response>
<tool_response>
{"temperature": 25.9, "location": "San Francisco, USA", "date": "2024 Venue", "unit": "celsius"}
</tool_response><|im_end|>
<|im_start|>assistant

"""



inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512)
output_text = tokenizer.batch_decode(outputs)[0][len(text):]
print(output_text)
"""
The current temperature in San Francisco is 26.1°C. Tomorrow, it will be approximately 26.1°F.<|im_end|>
"""
#后续进一步处理
# MESSAGES.append(try_parse_tool_calls(output_text))
# text = tokenizer.apply_chat_template(MESSAGES, tools=TOOLS, add_generation_prompt=True, tokenize=False)
# inputs = tokenizer(text, return_tensors="pt").to(model.device)
# outputs = model.generate(**inputs, max_new_tokens=512)
# output_text = tokenizer.batch_decode(outputs)[0][len(text):]

关注tool相关的special tokens，将基于这些tokens解析并执行函数，并将结果以json格式拼接为Message
缺点：每次生成tool_call和获得的tool_response都会添加到上下文中，让对话的长度增长的非常快，对LLM的长文本能力、结构化数据分析与生成能力、响应速度和资源要求都有很高的限制，如何优化长对话场景下的能力，是AGent当前面临的主要问题之一