OpenPrompt使用过程记录

OpenPrompt介绍

OpenPrompt is a research-friendly framework that is equipped with efficiency, modularity, and extendibility, and its combin-ability allows the freedom to combine different PLMs, task formats, and prompting modules in a unified paradigm. Users could expediently deploy prompt-learning frameworks and eval-uate the generalization of them on different NLP tasks without constraints. （来源: https://arxiv.org/pdf/2111.01998.pdf）代码地址

OpenPrompt安装

官方命令
方式一：使用pip安装

pip install openprompt

方式二：使用Git安装（推荐）

git clone https://github.com/thunlp/OpenPrompt.git
cd OpenPrompt
pip install -r requirements.txt
python setup.py install

代码实战

demo实现

官方文档对于如何使用OpenPrompt框架给出了示例。
新建一个sample.py文件，将以下代码放入文件中。（此处参考了该篇博文）

# step1:
import torch
from openprompt.data_utils import InputExample
classes = [ # There are two classes in Sentiment Analysis, one for negative and one for positive
    "negative",
    "positive"
]
dataset = [ # For simplicity, there's only two examples
    # text_a is the input text of the data, some other datasets may have multiple input sentences in one example.
    InputExample(
        guid = 0,
        text_a = "Albert Einstein was one of the greatest intellects of his time.",
    ),
    InputExample(
        guid = 1,
        text_a = "The film was badly made.",
    ),
]
# step 2
from openprompt.plms import load_plm
plm, tokenizer, model_config, WrapperClass = load_plm("bert", "bert-base-cased")
#step3
from openprompt.prompts import ManualTemplate
promptTemplate = ManualTemplate(
    text = '{"placeholder":"text_a"} It was {"mask"}',
    tokenizer = tokenizer,
)
#step4
from openprompt.prompts import ManualVerbalizer
promptVerbalizer = ManualVerbalizer(
    classes = classes,
    label_words = {
        "negative": ["bad"],
        "positive": ["good", "wonderful", "great"],
    },
    tokenizer = tokenizer,
)
#step 5
from openprompt import PromptForClassification
promptModel = PromptForClassification(
    template = promptTemplate,
    plm = plm,
    verbalizer = promptVerbalizer,
)
#step 6
from openprompt import PromptDataLoader

data_loader = PromptDataLoader(
    dataset=dataset,
    tokenizer=tokenizer,
    template=promptTemplate,
    tokenizer_wrapper_class=WrapperClass,
)
# step 7
# making zero-shot inference using pretrained MLM with prompt
promptModel.eval()
with torch.no_grad():
    for batch in data_loader:
        logits = promptModel(batch)
        preds = torch.argmax(logits, dim=-1)
        print(classes[preds])
# predictions would be 1, 0 for classes 'positive', 'negative'

运行结果

代码分析

Step 1: 定义一个NLP任务

根据研究目标确定一个NLP任务（数据集是什么以及想要从数据中获取哪些信息），这一步的实质是确定任务的类别和inputExample。本实验以情感分析为例，判断情感正向（positive）和情感负向（negative）

# step1：确定NLP任务
import torch
from openprompt.data_utils import InputExample
# 1. 确定类别：在情感分析任务当中，共有两个类别，分别是negative和positive
classes = [
    "negative",
    "positive"
]
# 2.确定数据集：为了简单起见，这里只有两个例子。text_a是数据的输入文本，一些其他数据集可能在一个例子中有多个输入句子
dataset = [
    InputExample(
        guid = 0,
        text_a = "Albert Einstein was one of the greatest intellects of his time.",
    ),
    InputExample(
        guid = 1,
        text_a = "The film was badly made.",
    ),
    InputExample(
        guid = 2,
        text_a = "That is wonderful.",
    ),
]

Step 2: 确定预训练语言模型（PLM）

根据任务需要选择一个合适的预训练语言模型，不同的模型会产生不同的效果。OpenPrompt与huggingface上的模型兼容。

# step2：定义预训练语言模型(PLMs)作为backbone。
from openprompt.plms import load_plm
plm, tokenizer, model_config, WrapperClass = load_plm("bert", "bert-base-cased") # 此处使用bert模型

Step 3: 定义模板（Template）

Template是原始输入的修改器，也是prompt-learning中最重要的模块之一。本例采用的格式是 [x],It was [Z]，x对应代码中的text_a，应填入输入语句。Z对于mask，是语言模型的预测结果。

# step3:定义一个模板
from openprompt.prompts import ManualTemplate
promptTemplate = ManualTemplate(
    text = '{"placeholder":"text_a"} It was {"mask"}',
    tokenizer = tokenizer,
)

Step 4: 答案映射（Verbalizer）

Verbalizer是prompt-learning中另一个重要的（但不是必须的）部分，它将原始标签（已经将它们定义为类）映射到一组标签词上。本例中，我们把负面类别映射到单词bad，把正面的类别投射到单词good, wonderful, great。

# 答案映射
from openprompt.prompts import ManualVerbalizer
promptVerbalizer = ManualVerbalizer(
    classes = classes,
    label_words = {
        "negative": ["bad"],
        "positive": ["good", "wonderful", "great"],
    },
    tokenizer = tokenizer,
)

Step 5: 构造PromptModel

将PLM，Template和Verbalizer组合成一个PromptModel。注意，虽然这个例子简单地将这三个模块结合在一起，但实际上可以在它们之间定义一些复杂的互动关系。

 Step 5: 构造PromptModel
from openprompt import PromptForClassification
promptModel = PromptForClassification(
    template = promptTemplate,
    plm = plm,
    verbalizer = promptVerbalizer,
)

Step 6: 构造PromptDataLoader

PromptDataLoader是一个Prompt版本的pytorch Dataloader，包含了一个Tokenizer，一个Template 和一个 TokenizerWrapper.

# Step 6: :构造PromptDataLoader
from openprompt import PromptDataLoader
data_loader = PromptDataLoader(
    dataset = dataset,
    tokenizer = tokenizer,
    template = promptTemplate,
    tokenizer_wrapper_class=WrapperClass,
)

Step 7: 训练和预测

将Model处理为PromptModel，DataLoader处理为PromptDataLoader就可以向基于Pytorch的其他机器学习任务一样完成训练和测试。

import torch

# 使用带有prompt的预训练 MLM 进行 zero-shot 预测
promptModel.eval()
with torch.no_grad():
    for batch in data_loader:
        logits = promptModel(batch)
        preds = torch.argmax(logits, dim = -1)
        print(classes[preds])
# predictions would be 1, 0 for classes 'positive', 'negative'

在上述过程中，没有用任何训练数据对Bert进行调整，就实现了zero-shot的情感分类。

其他实验

step1：下载数据集

在OpenPrompt项目中的datasets文件夹执行以下命令：

bash download_text_classification.sh

step2：获取源码

git clone git@github.com:thunlp/OpenPrompt.git

step3：运行脚本

执行 few-shot 任务

bash scripts/run_fewshot.sh ```

报错信息

OSError: Can't load the configuration of '../plm_cache/roberta-large'. 
If you were trying to load it from 'https://huggingface.co/models', 
make sure you don't have a local directory with the same name. 
Otherwise, make sure '../plm_cache/roberta-large' is the correct 
path to a directory containing a config.json file

出现这个错误的原因是没有正确配置roberta-large模型。
方法一：可以在huggingface上手动下载roberta-large的配置文件；

MODEL_NAME_OR_PATH="../plm_cache/roberta-large"

方法二：可以把run_fewshot.sh文件中的MODEL_NAME_OR_PATH更改为：MODEL_NAME_OR_PATH="roberta-large"实现自动下载。(如果失败了，可以多试几次)

# 官方配置
MODEL_NAME_OR_PATH="../plm_cache/roberta-large"
# 更改为自动下载
MODEL_NAME_OR_PATH="roberta-large"

另外，run_fewshot.sh文件中还需要修改以下内容
1.OPENPROMPTPATH目标地址应是OpenPrompt的项目地址。
2.CUDA_VISIBLE_DEVICES配置应根据实验环境进行相应的调整。

报错信息

缺少ckpts文件夹，需要在KnowledgeablePromptTuning文件夹下新建。

实验过程

实验结果

执行 zero-shot 任务

首先要更改配置信息 run_zeroshot.sh

实验过程

实验结果

参考资料

[1] https://github.com/thunlp/OpenPrompt
[2] https://github.com/thunlp/KnowledgeablePromptTuning
[3] https://blog.csdn.net/qq_39328436/article/details/121744344
[4] https://blog.csdn.net/m0_46684880/article/details/129998689