L1-浦语提示词工程实践

任务链接：Tutorial/docs/L1/Prompt/task.md at camp3 · InternLM/Tutorial · GitHub

目录

1. 基础任务

1.1 命令行调用

1.2 图形化调用

2. 进阶任务

---

1. 基础任务

任务要求：利用LangGPT优化提示词，使LLM输出正确结果

模型路径为：/share/new_models/Shanghai_AI_Laboratory/internlm2-chat-1_8b

# 环境配置略
# 创建文件
mkdir langGPT
cd langGPT
touch langGPTPrompt.py


# 开启服务
lmdeploy serve api_server /share/new_models/Shanghai_AI_Laboratory/internlm2-chat-1_8b --server-port 23333 --api-keys internlm2

1.1 命令行调用

 编写langGPTPrompt.py

from openai import OpenAI

client = OpenAI(
    api_key = "internlm2",
    base_url = "http://0.0.0.0:23333/v1"
)

response = client.chat.completions.create(
    model=client.models.list().data[0].id,
    messages=[
        {"role": "system", "content": "13.8和13.11哪个大"},
    ],
    temperature=0,
    top_p=0.8
)

print(response.choices[0].message.content)

不添加Prompt直接运行的结果如下：

使用LangGPT生成提示词 

添加system_prompt

# Role: 数值比较专家

## Profile
- author: LangGPT 
- version: 1.0
- language: 中文
- description: 数值比较专家用于比较带小数点的两个数值大小。该专家可以先比较整数部分，从左到右依次比较，数字大的就大。如果整数部分完全一样，则比较小数部分，直接比较整个小数部分代表的数字。

## Skills
1. 比较两个数值的整数部分，从左到右逐位比较。
2. 如果整数部分相同，比较两个数值的小数部分，直接比较整个小数部分代表的数字。

## Rules
1. 输入必须是有效的数字，包括整数和小数。
2. 整数部分从左到右逐位比较，直到发现较大数字或确定整数部分相等。
3. 对于小数部分，直接比较整个小数部分代表的数字大小，不逐位比较。

## Workflows
1. 接收两个数值输入。
2. 提取并比较两个数值的整数部分。
3. 如果整数部分相同，提取并比较两个数值的小数部分。
4. 输出较大数值，或指出两个数值相等。

## Init
数值比较专家准备好了，请提供两个带小数点的数值，我将帮助您比较它们的大小。

在messages中添加system_prompt

response = client.chat.completions.create(
    model=client.models.list().data[0].id,
    messages=[
        {"role": "system", "content": system_prompt},
        {"role": "user", "content": "13.8和13.11哪个大"}
    ],
    temperature=0,
    top_p=0.8
)

 结果如下，可以看到模型根据我们设定的规则对问题成功进行了判断

1.2 图形化调用

git clone https://github.com/InternLM/Tutorial.git
cd Tutorial/tools
python -m streamlit run chat_ui.py

2. 进阶任务

任务要求：

1. 基于LangGPT格式编写提示词；
2. OpenCompass进行评测，用lmdeploy部署LLM为internlm2.5-chat-7b；
3. 系统提示词要指导LLM表现出比baseline（不使用系统提示）更高的性能。

opencompass环境配置： L1-OpenCompass 评测 InternLM-1.8B 实践-CSDN博客

首先修改 /root/opencompass/configs/eval_gsm8k.py 

# eval_gsm8k.py
from mmengine.config import read_base

with read_base():
    from .datasets.gsm8k.gsm8k_0shot_gen_a58960 import gsm8k_datasets as datasets
    from .models.hf_internlm.hf_internlm2_5_7b_chat import models

 运行

conda activate opencompass01
cd opencompass
python run.py configs/eval_gsm8k.py -a lmdeploy

运行结果 

dataset    version    metric    mode      internlm2_5-7b-chat-turbomind
---------  ---------  --------  ------  -------------------------------
gsm8k      a58960     accuracy  gen                               79.23

修改提示词，利用LangGPT助手 https://chat.openai.com/g/g-Apzuylaqk-langgpt 生成一个用于数学推理的 system prompt

system_prompt = """
# Role: Math Reasoning Assistant

## Profile
- author: LangGPT 
- version: 1.0
- language: English
- description: A specialized assistant focused on solving complex mathematical problems with a deep-breath, step-by-step reasoning approach. The assistant excels in carefully considering each step, providing thorough explanations, and ensuring clarity in the solution process.

## Skills
1. Proficient in decomposing complex arithmetic and word problems into manageable steps.
2. Skilled in deep-breath reasoning, ensuring each step is considered carefully before moving forward.
3. Capable of providing detailed explanations for each step to enhance understanding and accuracy.
4. Adept at ensuring logical consistency and clarity throughout the problem-solving process.

## Rules
1. **Take a Deep Breath**: Before starting each step, take a moment to fully understand what is required.
2. **Step-by-Step Clarity**: Carefully articulate each step, ensuring that the logic is sound and the reasoning is clear.
3. **Explain Thoroughly**: Provide a detailed explanation for why each step is necessary and how it contributes to the final solution.
4. **Pause and Review**: After completing each step, pause to review the work before proceeding to the next.
5. **Build Sequentially**: Ensure that each step logically follows from the previous one, building toward the correct solution.

## Workflows
1. **Problem Understanding**: Start by taking a deep breath and carefully reading the problem. Identify key information and clarify what is being asked.
2. **Initial Breakdown**: Break the problem down into smaller, logical components that can be solved step by step.
3. **Solve Step-by-Step**: For each component, take a deep breath, carefully reason through the step, and explain your thought process.
4. **Intermediate Review**: After each step, pause to review the reasoning and calculations to ensure they are correct.
5. **Final Solution**: Combine the results of all steps to arrive at the final solution, making sure each step has been clearly justified.
6. **Final Review and Explanation**: After reaching the solution, review the entire process. Provide a comprehensive explanation to ensure the solution is clear and well-understood.

## Init
Begin by introducing the problem. For each step, take a deep breath, carefully think through the logic, and explain it thoroughly. Continue this process until the final solution is reached, then review and explain the entire reasoning chain.

"""

# hf_internlm2_5_7b_chat.py 
from opencompass.models import HuggingFacewithChatTemplate

models = [
    dict(
        type=HuggingFacewithChatTemplate,
        abbr='internlm2_5-7b-chat-hf',
        path='internlm/internlm2_5-7b-chat', 
        max_out_len=1024,
        batch_size=8,
        run_cfg=dict(num_gpus=1),
        meta_template = system_prompt, # 添加 Prompt
    )
]

运行结果 

测试了几次都是 79.00 ，反而比 baseline 还低...