LLM的函数调用，让LLM能够使用api接口

最新推荐文章于 2025-02-22 20:44:48 发布

HNU混子

最新推荐文章于 2025-02-22 20:44:48 发布

阅读量1.3k

点赞数 9

文章标签： python ai 语言模型

本文链接：https://blog.csdn.net/xhhgffd/article/details/144591304

版权

Function Calling

Function Calling可以把大模型与各种软件工具进行集成。理论上来说，LLM是做文字接龙的，它可以理解文字，也可以生成文字，那么这样一个LLM是如何进行接口调用，函数调用的呢？其实是通过协议来完成的。LLM集成开发工程师通过prompt或者微调等方式与LLM之间规定了某种协议，它指导大模型在触发函数调用时返回某种特定规格的字符串，在这个字符串里面包含了函数的参数信息以及函数名，然后通过哈希的方式映射到函数指针从而进行函数调用。

目前来说实现函数调用有两种方式，第一种是openai格式的函数调用，但是这需要你能获得一个模型的api_url，这个url有两种方式获得，第一种是你调用的是一个在线的模型，你需要从模型的官网来获得api_url和api_key。例如：

https://www.dmxapi.com/v1

也有第二种方式能获得，你可以通过模型管理工具或者推理框架来下载huggingface或者github上的开源大模型参数，然后用ollama或者vllm等软件来启动本地大模型的服务，启动之后软件会为你提供一个localhost url，比如：

https://localhost:9888/v1

然后就可以利用这个url和python代码实现与在线模型的调用没什么区别的chat了，然后可以通过openai格式将tools作为参数传入：

tools = [
  {
      "type": "function",
      "function": {
          "name": "get_delivery_date",
          "description": "Get the delivery date for a customer's order. Call this whenever you need to know the delivery date, for example when a customer asks 'Where is my package'",
          "parameters": {
              "type": "object",
              "properties": {
                  "order_id": {
                      "type": "string",
                      "description": "The customer's order ID.",
                  },
              },
              "required": ["order_id"],
              "additionalProperties": False,
          },
      }
  }
]

messages = [
  {
      "role": "system",
      "content": "You are a helpful customer support assistant. Use the supplied tools to assist the user."
  },
  {
      "role": "user",
      "content": "Hi, can you tell me the delivery date for my order?"
  }
]

response = openai.chat.completions.create(
  model="gpt-4o",
  messages=messages,
  tools=tools,
)

这种方式进行函数调用很简单，可以参考openai的官网：

https://platform.openai.com/docs/guides/function-calling

这里不再赘述。

本地载入模型进行function calling

第二种方式就是将本地的模型载入，通过from_pretrained()方法来得到模型，并进行推理：

AutoModelForCausalLM.from_pretrained(model_path)

在这种情况下，因为没有url，我们没有办法通过openai的格式把tools传进去，所以我们需要通过prompt工程与LLM约定一个function格式，然后我们在LLM的返回文本中捕获这个function格式并自己通过json解析的方式把函数名和参数解析出来，再自己调用。

【首先假设你已经在你的电脑上安装了jupyter和conda环境，并配置了大模型推理需要用到的环境】

然后看一下代码：

from transformers import AutoModelForCausalLM, Trainer, TrainingArguments,AutoTokenizer
import json
import torch
import re

model_path = '/media/hnu/LLM/qwen2.5-7B/'
tokenizer = AutoTokenizer.from_pretrained(model_path)
model = AutoModelForCausalLM.from_pretrained(model_path)
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
model = model.to(device)

首先我们导入了必要的包和tokenizer模型和推理模型。

def generate_tool_request(prompt,_device,max_length=100):
    inputs = tokenizer(prompt, return_tensors="pt").to(_device)
    outputs = model.generate(inputs['input_ids'], max_length=max_length,repetition_penalty=1.2,  
    eos_token_id=tokenizer.eos_token_id, 
    pad_token_id=tokenizer.pad_token_id)
    generated_ids = outputs[0][inputs['input_ids'].shape[1]:]  
    return tokenizer.decode(generated_ids, skip_special_tokens=True)

然后我们把生成回答的过程包装成一个函数，注意存储模型的设备和input_token的设备必须一样才能进行推理否则会报错。

def get_todays_weather(location:str) -> str:
	return f'the weather of {location} is sunny'

def add_number(a:int,b:int) -> int:
	return a+b

tools = {
	"get_todays_weather":get_todays_weather,
	"add_number":add_number
}

tools_json = [
	{
		'name':'get_todays_weather',
		'inputs':[('location','str')],
		'result':['str'],
		'description':'given a location and then return the weather of given location on today'
	},
	{
		'name':'add_number',
		'inputs':[('a','int'),('b','int')],
		'result':['int'],
		'description':'given two integer a and b,return the sum of a and b : a+b'
	}
]
def build_tool_prompts(prompt:str)->str:
	rst = 'You are an assitant of user and you should obey the rules of system and give help to user.\n'
	rst += 'system:You have the following tools for you to call:' 
	tmp_prompt = ''
	for tool in tools_json:
		tmp_prompt+='{name: '
		tmp_prompt+=tool['name']+', input args: '
		for each_arg in tool['inputs']:
			tmp_prompt+=f'{each_arg[0]}:{each_arg[1]} '
		tmp_prompt+=',tool\'s result type: '
		for  each_type in tool['result']:
			tmp_prompt+=f'{each_type} '
		tmp_prompt+=',and the description of this tool is: '+tool['description']+'}'
	rst += tmp_prompt
	rst += f'\nsystem:you can decide if you are need to call tools,if you are, just return a json object with label <function> and </function> of tool_calling such as following: '
	rst += '<function>{\'tool\':\'tool_name\',\'args\':{\'args_name\':\'args\',\'args_name\':\'args\'}}</function>.\n'
	rst += '''!!! you should: Don't generate any text outside of your role.\n'''
	rst += '''For example:{ 
	User:
	what's the weather like today in ShangHai?
	Assistant:
	<function>{"tool":"get_todays_weather","args":{"location":"ShangHai"}}</function>.
	}'''
	# rst += '''2.strictly obey the rules and don't generate any other things that not related to the need of user.'''
	rst += f'\nUser: \n{prompt}'
	rst += '\nAssistant:'
	print(rst)
	return rst

def get_func_json(res):
	matches = re.findall(r'<function>\s*(.*?)\s*</function>',res)
	return matches

在这里我们定义了两个给大模型用的工具函数，然后定义了一个tools映射用于把大模型返回的函数名映射到函数执行对象上。tools_json存储了工具库中的所有工具的信息，然后我们定义了一个函数来构造prompt，这里的构造prompt过程需要一点点调，因为确实大模型的输出非常多样，一般你也解释不了也控制不了。

然后我们用gey_func_json函数通过正则表达式捕获<function></function>，从而得到function的json信息。

def call_tool(Json_list):
	rst = []
	for Json_txt in Json_list:
		try:
			request = json.loads(Json_txt)
			tool = request.get('tool')
			arguments = request.get("args", {})
			if tool not in tools:
				return False, f"Invalid tool: {tool}. Available tools: {list(tools.keys())}"
			else:
				print('function calling ... ========================= ')
				rst.append(tools[tool](**arguments))
		except json.JSONDecodeError:
			return "Invalid tool request format."
		except Exception as e:
			return f"Error during tool execution: {e}"
	return rst

随后我们定义了一个函数来调用这个函数，它接受一个包含了多个函数json的list，因为大模型可能不止调用一次函数，然后解析其中的函数名和参数列表再传给函数。

最后我们测试一下：

user_prompts = 'what is the sum of 20089 and 96820'
instruction = build_tool_prompts(user_prompts)
response = generate_tool_request(instruction,device)

print(response)
func_json = get_func_json(response)
if len(func_json)!=0:
	rst = call_tool(func_json)
	print(rst)

首先看一下我们构建的prompt:

You are an assitant of user and you should obey the rules of system and give help to user.
system:You have the following tools for you to call:{name: get_todays_weather, input args: location:str ,tool's result type: str ,and the description of this tool is: given a location and then return the weather of given location on today}{name: add_number, input args: a:int b:int ,tool's result type: int ,and the description of this tool is: given two integer a and b,return the sum of a and b : a+b}
system:you can decide if you are need to call tools,if you are, just return a json object with label <function> and </function> of tool_calling such as following: <function>{'tool':'tool_name','args':{'args_name':'args','args_name':'args'}}</function>.
!!! you should: Don't generate any text outside of your role.
For example:{ 
	User:
	what's the weather like today in ShangHai?
	Assistant:
	<function>{"tool":"get_todays_weather","args":{"location":"ShangHai"}}</function>.
	}
User: 
what is the sum of 20089 and 96820
Assistant: