使用LlamaIndex和Gradient微调GPT-3.5-Turbo以获得更好的结果

在这篇文章中，我们将全面介绍如何使用LlamaIndex和Gradient，微调GPT-3.5-turbo以获得更好的模型输出效果。我们将通过生成训练数据和评估数据集，使用OpenAIFinetuneEngine进行微调，并最终进行评估，详细探讨在微调后的性能提升效果。

数据设置

首先，我们需要下载用于生成训练数据的PDF文件。

!curl https://www.ipcc.ch/report/ar6/wg2/downloads/report/IPCC_AR6_WGII_Chapter03.pdf --output IPCC_AR6_WGII_Chapter03.pdf

生成训练集和评估集

我们会从下载的PDF文件中生成40个不同部分的问题进行训练数据集和评估数据集的创建。

训练集生成

from llama_index.core import SimpleDirectoryReader
from llama_index.llms.openai import OpenAI
from llama_index.core.evaluation import DatasetGenerator
import random

documents = SimpleDirectoryReader(
    input_files=["IPCC_AR6_WGII_Chapter03.pdf"]
).load_data()

random.seed(42)
random.shuffle(documents)

gpt_35_llm = OpenAI(model="gpt-3.5-turbo", temperature=0.3)

question_gen_query = (
    "You are a Teacher/ Professor. Your task is to setup "
    "a quiz/examination. Using the provided context, formulate "
    "a single question that captures an important fact from the "
    "context. Restrict the question to the context information provided."
)

dataset_generator = DatasetGenerator.from_documents(
    documents[:50],
    question_gen_query=question_gen_query,
    llm=gpt_35_llm,
)

questions = dataset_generator.generate_questions_from_nodes(num=40)
print("Generated ", len(questions), " questions")

with open("train_questions.txt", "w") as f:
    for question in questions:
        f.write(question + "\n")

评估集生成

dataset_generator = DatasetGenerator.from_documents(
    documents[50:],  # since we generated ~1 question for 40 documents, we can skip the first 40
    question_gen_query=question_gen_query,
    llm=gpt_35_llm,
)

questions = dataset_generator.generate_questions_from_nodes(num=40)
print("Generated ", len(questions), " questions")

with open("eval_questions.txt", "w") as f:
    for question in questions:
        f.write(question + "\n")

初步评估

from llama_index.core import VectorStoreIndex
from llama_index.core import Settings
from datasets import Dataset
from ragas import evaluate
from ragas.metrics import answer_relevancy, faithfulness

Settings.context_window = 2048

index = VectorStoreIndex.from_documents(
    documents,
)
query_engine = index.as_query_engine(similarity_top_k=2, llm=gpt_35_llm)

questions = []
with open("eval_questions.txt", "r") as f:
    for line in f:
        questions.append(line.strip())

contexts = []
answers = []

for question in questions:
    response = query_engine.query(question)
    contexts.append([x.node.get_content() for x in response.source_nodes])
    answers.append(str(response))

ds = Dataset.from_dict(
    {
        "question": questions,
        "answer": answers,
        "contexts": contexts,
    }
)

result = evaluate(ds, [answer_relevancy, faithfulness])
print(result)

使用GPT-4生成训练数据

from llama_index.llms.openai import OpenAI
from llama_index.finetuning.callbacks import OpenAIFineTuningHandler
from llama_index.core.callbacks import CallbackManager

finetuning_handler = OpenAIFineTuningHandler()
callback_manager = CallbackManager([finetuning_handler])

llm = OpenAI(model="gpt-3.5-turbo", temperature=0.3)
llm.callback_manager = callback_manager

questions = []
with open("train_questions.txt", "r") as f:
    for line in f:
        questions.append(line.strip())

index = VectorStoreIndex.from_documents(documents)

query_engine = index.as_query_engine(similarity_top_k=2, llm=llm)

for question in questions:
    response = query_engine.query(question)

创建OpenAIFinetuneEngine

from llama_index.finetuning import OpenAIFinetuneEngine

finetuning_handler.save_finetuning_events("finetuning_events.jsonl")

finetune_engine = OpenAIFinetuneEngine(
    "gpt-3.5-turbo",
    "finetuning_events.jsonl",
)
finetune_engine.finetune()

评估微调后的模型

from llama_index.llms.openai import OpenAI
from datasets import Dataset
from ragas import evaluate
from ragas.metrics import answer_relevancy, faithfulness

ft_llm = finetune_engine.get_finetuned_model(temperature=0.3)

Settings.llm = ft_llm
Settings.context_window = 2048

index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine(similarity_top_k=2, llm=ft_llm)

questions = []
with open("eval_questions.txt", "r") as f:
    for line in f:
        questions.append(line.strip())

contexts = []
answers = []

for question in questions:
    response = query_engine.query(question)
    contexts.append([x.node.get_content() for x in response.source_nodes])
    answers.append(str(response))

ds = Dataset.from_dict(
    {
        "question": questions,
        "answer": answers,
        "contexts": contexts,
    }
)

result = evaluate(ds, [answer_relevancy, faithfulness])
print(result)

结果和结论

通过微调后的模型，我们得到了更高的评估分数：

• 答复相关性： 0.9725 -> 0.9607
• 可信性： 0.7325 -> 0.7917

虽然答复相关性稍微下降，但可信性显著提高，这说明模型在回答问题时更加切题和准确。

参考资料

• LlamaIndex 文档
• OpenAI API Documentation

可能遇到的错误

1. 网络连接错误: 下载PDF时可能会遇到网络连接问题，建议检查网络连接并重试。
2. API授权错误: 确保环境变量中正确设置了API Key。
3. 内存不足: 在生成大量数据或进行训练时，可能会遇到内存不足的情况，建议使用更高配置的机器或分批处理数据。

如果你觉得这篇文章对你有帮助，请点赞，关注我的博客，谢谢!