ollama llama3.1 8b openbuddy中文模型

凌紫宸

于 2024-08-06 02:46:23 发布

阅读量19

点赞数

openbuddy 对于 llama3.1 8b 模型进行了少量的中文训练实现了不错的中文理解处理，以下是使用社区提供的gguf 格式，制作一个ollama 新模型

模型制作

下载模型
下载gguf 文件，推荐使用加速工具
Modelfile
参考了llama3.1 的

FROM /home/models/openbuddy-llama3.1-8b-v22.1-131k-q4_k_m.gguf
 
TEMPLATE """{{ if .Messages }}
{{- if or .System .Tools }}<|start_header_id|>system<|end_header_id|>
{{- if .System }}
 
{{ .System }}
{{- end }}
{{- if .Tools }}
 
You are a helpful assistant with tool calling capabilities. When you receive a tool call response, use the output to format an answer to the orginal use question.
{{- end }}<|eot_id|>
{{- end }}
{{- range $i, $_ := .Messages }}
{{- $last := eq (len (slice $.Messages $i)) 1 }}
{{- if eq .Role "user" }}<|start_header_id|>user<|end_header_id|>
{{- if and $.Tools $last }}
 
Given the following functions, please respond with a JSON for a function call with its proper arguments that best answers the given prompt.
 
Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}. Do not use variables.
 
{{ $.Tools }}
{{- end }}
 
{{ .Content }}<|eot_id|>{{ if $last }}<|start_header_id|>assistant<|end_header_id|>
 
{{ end }}
{{- else if eq .Role "assistant" }}<|start_header_id|>assistant<|end_header_id|>
{{- if .ToolCalls }}
 
{{- range .ToolCalls }}{"name": "{{ .Function.Name }}", "parameters": {{ .Function.Arguments }}}{{ end }}
{{- else }}
 
{{ .Content }}{{ if not $last }}<|eot_id|>{{ end }}
{{- end }}
{{- else if eq .Role "tool" }}<|start_header_id|>ipython<|end_header_id|>
 
{{ .Content }}<|eot_id|>{{ if $last }}<|start_header_id|>assistant<|end_header_id|>
 
{{ end }}
{{- end }}
{{- end }}
{{- else }}
{{- if .System }}<|start_header_id|>system<|end_header_id|>
 
{{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|>
 
{{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|>
 
{{ end }}{{ .Response }}{{ if .Response }}<|eot_id|>{{ end }}"""
PARAMETER stop "<|start_header_id|>"
PARAMETER stop "<|end_header_id|>"
PARAMETER stop "<|eot_id|>"

构建

使用

测试的函数调用

appv3.py

import openai
import json
 
 
def add(a, b):
    return a + b
 
 
def sub(a, b):
    return a - b
 
 
def send_email(userid, mailContent):
    return f"send email to {userid} with content: {mailContent}"
 
 
 
def add_role(userid, roleid):
    return f"add role {roleid} to user {userid}"
 
openai.api_key = "demo"
openai.base_url = "http://localhost:11434/v1/"
 
funcs = {
    "add": add,
    "sub": sub,
    "send_email":send_email,
    "add_role":add_role
}
 
tools = [{
    "type": "function",
            "function": {
                "name": "add",
                "description": "Add two numbers together",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "a": {"type": "number", "description": "First number"},
                        "b": {"type": "number", "description": "Second number"}
                    },
                    "required": ["a", "b"]
                }
            }
},
    {
    "type": "function",
            "function": {
                "name": "sub",
                "description": "计算两个数的差",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "a": {"type": "number", "description": "First number"},
                        "b": {"type": "number", "description": "Second number"}
                    },
                    "required": ["a", "b"]
                }
            }
},
 {
    "type": "function",
            "function": {
                "name": "add_role",
                "description": "给用户添加权限",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "userid": {"type": "string", "description": "用户id"},
                        "roleid": {"type": "string", "description": "角色id"}
                    },
                    "required": ["userid", "roleid"]
                }
            }
},
 {
    "type": "function",
            "function": {
                "name": "send_email",
                "description": "给用户发送邮件",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "userid": {"type": "string", "description": "用户id"},
                        "mailContent": {"type": "string", "description": "邮件内容"}
                    },
                    "required": ["userid", "mailContent"]
                }
            }
}
]
response = openai.chat.completions.create(
    stream=False,
    model="rongfengliang/openbuddy-llama3.1",
    messages=[
        {"role": "system", "content": "你是一个人工智能助手，你可以帮助用户处理数据，发送邮件等，但是你必须使用中文"},
        {"role": "user", "content": "给用户001发送数据处理已经完成的邮件通知同时添加角色0002"},
    ],
    tools=tools,
    tool_choice="auto",
)
message = response.choices[0].message
print(message)
if message.tool_calls:
    result = response.choices[0].message.tool_calls[0].function
    print(result)
    func = funcs.get(result.name)
    params = json.loads(result.arguments)
    print(func(**params))
else:
    print(message.content)

1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.
36.
37.
38.
39.
40.
41.
42.
43.
44.
45.
46.
47.
48.
49.
50.
51.
52.
53.
54.
55.
56.
57.
58.
59.
60.
61.
62.
63.
64.
65.
66.
67.
68.
69.
70.
71.
72.
73.
74.
75.
76.
77.
78.
79.
80.
81.
82.
83.
84.
85.
86.
87.
88.
89.
90.
91.
92.
93.
94.
95.
96.
97.
98.
99.
100.
101.
102.
103.
104.
105.
106.
107.
108.
109.
110.
111.

效果

说明

目前基于openbuddy支持中文的进行函数调用比原生的是好一些，原生的有时有出现莫名其妙的中文信息

参考资料

https://openbuddy.ai/
https://github.com/OpenBuddy/OpenBuddy
https://huggingface.co/OpenBuddy/openbuddy-llama3.1-8b-v22.1-131k
https://huggingface.co/sunnyyy/openbuddy-llama3.1-8b-v22.1-131k-Q4_K_M-GGUF

原创作者: rongfengliang 转载于: https://blog.51cto.com/rongfengliang/11663749

凌紫宸

关注

0
点赞
踩
0

收藏

觉得还不错? 一键收藏
0
评论
ollama llama3.1 8b openbuddy中文模型

openbuddy 对于 llama3.1 8b 模型进行了少量的中文训练实现了不错的中文理解处理，以下是使用社区提供的gguf 格式，制作一个ollama 新模型模型制作下载模型下载gguf 文件，推荐使用加速工具Modelfile 参考了llama3.1 的登录后复制 FROM /home/models/ope...
复制链接

扫一扫