用于知识图谱集成的 LLM 提示工程技术

最新推荐文章于 2024-09-26 17:23:00 发布

AI引路星

最新推荐文章于 2024-09-26 17:23:00 发布

阅读量581

点赞数 12

分类专栏：成长学习程序人生人工智能文章标签：知识图谱人工智能机器学习深度学习算法代码源代码管理

本文链接：https://blog.csdn.net/Java_rich/article/details/142563006

版权

程序人生同时被 3 个专栏收录

45 篇文章 2 订阅

订阅专栏

人工智能

40 篇文章 7 订阅

订阅专栏

成长学习

37 篇文章 8 订阅

订阅专栏

自从 ChatGPT 大肆宣传以来，大量大型语言模型（LLM）变体以各种方式被广泛实施，并彻底改变了我们与 AI 的交互方式。通过简单的提示工程技术，用户可以根据自己的喜好调整模型输出，无论他们在编程语言方面的专业水平如何.从本质上讲，LLM 似乎有可能将自然语言转变为一种新的编程语言。为了更好地理解提示工程的能力，我们将演练一个用例，将用户问题的 LLM 模型响应转换为知识图谱，例如：

用户问题:

“如何在 2024 年自学生成式人工智能”

输出：

希望这份入门指南可以成为你们中的一些人尝试更优雅、更全面的解决方案的起点。

什么是提示工程？

大型语言模型（LLM）通常通过自我监督深度学习技术对数万亿个单词进行训练。从头开始培训 LLM 需要大量的时间和资源。幸运的是，有几种技术可以根据自定义要求调整模型。以下是三种常见的技术，从最高到最低的计算成本列出：

训练前：这是指语言模型（LLM）开发的早期阶段，这通常是最耗时的过程。在此阶段，模型通过自我监督学习从大型语料库中学习。
微调：它描述了添加利基和专业数据集以修改 LLM 以使其更符合特定目标的过程。它与提示工程不同，因为微调可以更新LLM权重和参数。
Prompt Engineering：更改模型输出的最具成本效益的方法。它的范围从简单的指令调优机制（本文主要讨论）到更高级的机制，如 RAG（检索增强生成）、CoT（思维链）或 ART（自动推理和工具我们e）等。

不同的模型对某些提示指令的反应非常不同。以下提示技术与 OpenAI 模型配合良好，但可能不适用于其他模型。

提示工程技术

我们将研究我们指导模型方式的微小变化如何显著改变输出。尽管提示可能以截然不同的格式呈现，但它们通常遵循 4 个组成部分的相同结构：指令、上下文、输入数据、输出指示器。

我们可以将所有组件合并到一个指令消息中，供 LLM 解释。或者，LLM 集成框架 LangChain 提供了将每个组件分离为单独单元的工具，从而为模型提供更清晰的信息。我们将使用 LangChain 构建采用 OpenAI 模型的 LLM 链，然后在各种指令线索上进行实验以改变模型响应。

1. 基线提示模板

LangChain 提供 PromptTemplate 来创建一个简单的提示指令，该指令需要两个基本参数：template 和 input_variables。它的工作原理类似于 python 中的字符串格式，其中变量用 { } 括起来。例如，在下面，我们定义模板以提供一致的指令以馈送到模型中。由于模板是作为整个字符串传递的，因此在使用 PromptTemplate 创建模板时，我们显式要求模型将 {question} 视为用户的输入变量。

template = """Question: {question}"""
prompt = PromptTemplate(
	template=template, 
	input_variables=["question"])

为了应用提示模板，我们将其输入到如下定义的 LLM 链中。

import os
from langchain.chains import LLMChain
from langchain.llms import OpenAI
from langchain.prompts import PromptTemplate

# save the OpenAI token as environment variable
os.environ["OPENAI_API_KEY"] = "<your_openai_token>"

# create a model that allows output of maximum 1000 tokens
openai_model = OpenAI(max_tokens=1000)
chain = LLMChain(
		prompt=prompt, 
		llm=openai_model)

让 LLM 链遍历 user 问题。当然，您也可以探索您感兴趣的其他问题。

question = "How to self study Generative Artificial Intelligence in 2024"
chain_response = chain.run(question)

模型响应

1. Understand the basics: Before diving into any complex topic, it is important to have a strong foundation. Begin your study by understanding the basic concepts of artificial intelligence, such as machine learning, deep learning, and neural networks. You can find various online resources, such as YouTube tutorials, online courses, and blogs, to help you with this.

2. Learn programming languages: A good understanding of programming languages is essential for working with generative artificial intelligence. Some popular languages used in AI are Python, Java, and C++. Choose one or two languages to focus on and practice writing code to build your skills.

3. Study mathematical concepts: Generative AI involves a lot of mathematical concepts, such as linear algebra, calculus, and probability. These concepts are used to design and train AI models. It is important to have a basic understanding of these concepts, so consider taking a course or using online resources to learn them.
...

2. 一步一步思考

在提示中添加简单的提示 “think step by step” 可以指示模型进行更顺序、更合乎逻辑的响应，更适合生成知识图谱。

template2 = """
Question: {question}
Let's think step by step.
""

然后，我们使用上面定义的相同链来封装 template2，模型将生成以下响应。

prompt2 = PromptTemplate(template=template2, input_variables=["question"])
chain2 = LLMChain(prompt=prompt2, llm=openai_model)
chain2_response = chain2.run(question)

模型响应

Step 1: Set your goals
Start by setting clear and achievable goals for yourself. This will give you a sense of direction and motivation throughout your self-study journey. For example, your goal could be to understand the fundamentals of Generative Artificial Intelligence and be able to apply it in real-world scenarios by the end of 2024.

Step 2: Identify your learning resources
There are plenty of resources available for self-studying Generative Artificial Intelligence. Some of the best resources include online courses, books, research papers, and tutorials. Take some time to research and identify the resources that best suit your learning style and goals.

Step 3: Create a study schedule
Once you have your learning resources, create a study schedule that works for you. It is important to be consistent and dedicated in your studies to effectively learn and retain the material. Set aside a specific time each day or week to focus on studying Generative Artificial Intelligence.
...

3. 给 Model 一个角色

指定角色或角色是提供模型生成答案所依据的上下文和视角的另一种有用方法。

template3 = """
Question: {question}

You are a Course Instructor who designs a progressive syllabus based on the Question. \\
Break it down into modules which contains a list the topics, concepts, or themes.
"""

模型响应

Module 1: Introduction to Generative Artificial Intelligence
- History and evolution of Generative Artificial Intelligence
- Key concepts and principles
- Applications and use cases
- Ethical considerations

Module 2: Machine Learning Fundamentals
- Basics of machine learning
- Types of machine learning algorithms
- Supervised, unsupervised, and reinforcement learning
- Deep learning and neural networks

Module 3: Generative Adversarial Networks (GANs)
- Understanding GANs and their components
- Training and optimization techniques
- Applications in image and text generation
- Challenges and limitations

...

您可能希望根据用户正在寻找的答案类型来修改角色或角色。

4. 生成格式化输出

此外，我们可以指示模型生成具有特定格式（e.g. JSON、XML、HTML）的输出。该模型足够智能，可以通过提示“以 JSON 格式提供答案”来重新组织其 JSON 中的响应。此技术对于以编程方式解析具有可靠性和一致性的输出特别有用。然后，这些输出可以馈送到数据分析管道中的后续步骤中。

template4 = """Question: {question}
Answer: Let's think step by step.
Provide Answer in JSON format.
"""

模型响应

{
		"Module 1": "Introduction to Generative Artificial Intelligence",
		"Topics": [
				"Definition and Basic Concepts of AI",
				"Overview of Generative AI",
				"History and Evolution of Generative AI",
				"Applications of Generative AI"
		],
		"Module 2": "Machine Learning Fundamentals",
		"Topics": [
				"Supervised, Unsupervised, and Reinforcement Learning",
				"Neural Networks and Deep Learning",
				"Data Preprocessing and Feature Engineering",
				"Model Evaluation and Validation"
		],

"Module 3": "Generative Models",
"Topics": [
"Probabilistic Models",
"Generative Adversarial Networks (GANs)",
"Variational Autoencoders (VAEs)",
"Auto-Regressive Models",
"Generative Flow Models"
],
...

5. 提供具体说明

让我们探索如何向提示添加更多详细信息，使输出更好地与知识图谱的预期格式保持一致，如下所示。

{
	"input": ... ,
	"output": ... ,
	"module": ...
}

为了获得所需的格式，我们将创建一个专门用于格式化的附加提示。第 3 节的模型响应将通过 “文本” input_variables输入到此提示中，然后生成具有这些特定要求的响应。

JSON 对象列表
每个对象都有三个键：“input”、“output”和“module”
保持措辞一致和简洁
“module” 是数字

template5 = """
Text: {text}
Response:
Format Text above as a list of JSON objects with keys "input", "output" and "module".
"input" and "output" represent one and only one key concept respectively and "output" has dependency on "input".
Keep consistent and concise wordings (two to three phrases) for "input" and "output".
Do not include Module in the "input" or "output".
"module" is a numeric value indicates the Module in the syllabus.
"""

我们修改 LLM 链以在 model_response上运行，并将 {text} 元素解释为输入变量。

format_prompt = PromptTemplate(template=template5, input_variables=["text"])
format_chain = LLMChain(prompt=format_prompt, llm=openai_model)
formatted_response = format_chain.run(model_response)
print(formatted_response)

模型响应

...
	{
    "input": "Machine Learning",
    "output": "Basic concepts and principles of machine learning",
    "module": 2
  },
  {
    "input": "Machine Learning",
    "output": "Types of machine learning (supervised, unsupervised, reinforcement)",
    "module": 2
  },
  {
    "input": "Machine Learning",
    "output": "Algorithms commonly used in Generative AI (e.g. neural networks, deep learning)",
    "module": 2
  }
...

6. One-Shot/Few-Shot 提示

你可能会注意到，该模型并没有准确地解释这些要求，例如，即使被明确告知“将括号内的概念和''e.g''之后的概念提取为单独的对象”，它仍然给我们“输出”：“生成式人工智能中常用的算法（例如神经网络、深度学习）”。

面对这样的抽象指令，即使是人类也难以理解确切的含义。我们通过示例学习效果最好，同样，提供示例输出（即 one-shot / few-shot 示例）是一种强大的技术，可以让模型快速适应并立即提高对所需格式方向的响应。

template6 = """

Text: {text}

Response:
Format Text above as a list of JSON objects with keys "input", "output" and "module".
"input" and "output" represent one and only one key concept respectively and "output" has dependency on "input".
Keep consistent and concise wordings (two to three phrases) for "input" and "output".
Do not include Module in the "input" or "output".
"module" is a numeric value indicates the Module in the syllabus.

Extract all concepts within the bracket and after ```e.g.``` as seperate "output" objects.
An example:
```
Text: "Module 6: \
- A (e.g. B, C)"

Response:
{{
    "input": "A",
    "output": "B",
    "module": 6
}},
{{
    "input": "A",
    "output": "C",
    "module": 6
}}

```
"""

模型响应

...
{
    "input": "Basic concepts and principles of machine learning",
    "output": "Machine Learning",
    "module": 2
},
{
    "input": "Types of machine learning",
    "output": "Supervised Learning",
    "module": 2
},
{
    "input": "Types of machine learning",
    "output": "Unsupervised Learning",
    "module": 2
},
{
    "input": "Types of machine learning",
    "output": "Reinforcement Learning",
    "module": 2
},
{
    "input": "Algorithms commonly used in Generative AI",
    "output": "Neural Networks",
    "module": 2
}
...

生成知识图谱

我们将使用 Network Graph 从模型输出生成知识图谱。Python 库 pyvis 让我们能够轻松创建交互式节点和边。这是通过使用 add_edge 函数并馈送 “input” 和 “output” 节点以在它们之间创建有向边来实现的。节点大小由 “module” 的值进一步自定义。

有时，由于 token 的限制，响应可能会不完整，因此，我们会将响应截断到最后一个完整的对象。然后我们使用 json.loads 将 JSON 字符串加载为字典格式以进行进一步处理。

import json
json_response = formatted_response[:formatted_response.rfind("}")+1] + "]"
json_list = json.loads(json_response)

以下代码片段处理创建知识图谱的其余步骤。

from pyvis.network import Network
from IPython.core.display import HTML

net = Network(height="750px", width="100%", bgcolor="#222222",
		font_color="white", notebook=True,
		cdn_resources='in_line', directed=True)

for i in json_list:
	input = i['input']
	output = i['output']
	effort = i['effort']
	# add nodes
	net.add_node(input, title=input)
	net.add_node(output, title=output, value=effort)
	# add edges
	net.add_edge(input, output)

# save and open the graph
net.save_graph("pyvis.html")
HTML(filename="pyvis.html")

因此，我们将看到这样的知识图谱。