【快速上手ERNIE 3.0】对话意图识别实战

最新推荐文章于 2024-09-24 11:31:10 发布
最新推荐文章于 2024-09-24 11:31:10 发布
阅读量3.3k 收藏 18
点赞数
分类专栏： 人工智能 文章标签： 深度学习 机器学习 人工智能
原文链接：https://aistudio.baidu.com/aistudio/projectdetail/2017202?contributionType=1
版权

基于预训练模型的槽位填充和意图识别

意图识别原来如此

意图识别是指分析用户的核心需求，输出与查询输入最相关的信息，例如在搜索中要找电影、查快递、市政办公等需求，这些需求在底层的检索策略会有很大的不同，错误的识别几乎可以确定找不到能满足用户需求的内容，导致产生非常差的用户体验；在对话过程中要准确理解对方所想表达的意思，这是具有很大挑战性的任务。
例如用户输入查询“仙剑奇侠传”时，我们知道“仙剑奇侠传”既有游戏又有电视剧还有新闻、图片等等，如果我们通过用户意图识别发现该用户是想看“仙剑奇侠传”电视剧的，那我们直接把电视剧作为结果返回给用户，就会节省用户的搜索点击次数，缩短搜索时间，大大提升使用体验。而在对话中如果对方说“我的苹果从不出现卡顿”，那么我们就能通过意图识别判断出此刻的苹果是一个电子设备，而非水果，这样对话就能顺利进行下去。

总之，意图识别的准确性能在很大程度上影响着搜索的准确性和对话系统的智能性。

本示例将展示如何使用ERNIE预训练模型完成任务型对话中的槽位填充和意图识别任务，这两个任务是一个pipeline型任务对话系统的基石。

本示例使用的数据集为CrossWOC中文对话数据集。该数据集包含多个领域，包括景点，餐馆，酒店，交通等。

原始数据和数据处理脚本请参见 CrossWOZ。

环境要求

  • PaddlePaddle

    本项目依赖于 PaddlePaddle 2.3 及以上版本，请参考 安装指南 进行安装

  • PaddleNLP

    pip install --upgrade paddlenlp -i https://pypi.org/simple

  • Python

    Python的版本要求 3.7+

AI Studio平台默认安装了Paddle和PaddleNLP，并定期更新版本。 如需手动更新Paddle，可参考飞桨安装说明，安装相应环境下最新版飞桨框架。

使用如下命令确保安装最新版PaddleNLP：

!python -m pip install paddlepaddle-gpu==2.3.0.post101 -f https://www.paddlepaddle.org.cn/whl/linux/mkl/avx/stable.html
!pip install --upgrade paddlenlp

Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Looking in links: https://www.paddlepaddle.org.cn/whl/linux/mkl/avx/stable.html
Requirement already satisfied: paddlepaddle-gpu==2.3.0.post101 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (2.3.0.post101)
Requirement already satisfied: numpy>=1.13 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlepaddle-gpu==2.3.0.post101) (1.19.5)
Requirement already satisfied: Pillow in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlepaddle-gpu==2.3.0.post101) (8.2.0)
Requirement already satisfied: protobuf>=3.1.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlepaddle-gpu==2.3.0.post101) (3.14.0)
Requirement already satisfied: six in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlepaddle-gpu==2.3.0.post101) (1.16.0)
Requirement already satisfied: requests>=2.20.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlepaddle-gpu==2.3.0.post101) (2.24.0)
Requirement already satisfied: astor in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlepaddle-gpu==2.3.0.post101) (0.8.1)
Requirement already satisfied: decorator in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlepaddle-gpu==2.3.0.post101) (4.4.2)
Requirement already satisfied: opt-einsum==3.3.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlepaddle-gpu==2.3.0.post101) (3.3.0)
Requirement already satisfied: paddle-bfloat==0.1.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlepaddle-gpu==2.3.0.post101) (0.1.2)
Requirement already satisfied: chardet<4,>=3.0.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests>=2.20.0->paddlepaddle-gpu==2.3.0.post101) (3.0.4)
Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests>=2.20.0->paddlepaddle-gpu==2.3.0.post101) (2019.9.11)
Requirement already satisfied: idna<3,>=2.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests>=2.20.0->paddlepaddle-gpu==2.3.0.post101) (2.8)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests>=2.20.0->paddlepaddle-gpu==2.3.0.post101) (1.25.11)
[33mWARNING: You are using pip version 22.0.4; however, version 22.1.2 is available.
You should consider upgrading via the '/opt/conda/envs/python35-paddle120-env/bin/python -m pip install --upgrade pip' command.[0m[33m
[0mLooking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Requirement already satisfied: paddlenlp in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (2.3.4)
Requirement already satisfied: colorama in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp) (0.4.4)
Requirement already satisfied: colorlog in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp) (4.1.0)
Requirement already satisfied: tqdm in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp) (4.64.0)
Requirement already satisfied: sentencepiece in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp) (0.1.96)
Requirement already satisfied: datasets>=2.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp) (2.3.2)
Requirement already satisfied: paddlefsl in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp) (1.1.0)
Requirement already satisfied: paddle2onnx in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp) (0.9.8)
Requirement already satisfied: seqeval in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp) (1.2.2)
Requirement already satisfied: multiprocess<=0.70.12.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp) (0.70.11.1)
Requirement already satisfied: protobuf<=3.20.0,>=3.1.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp) (3.14.0)
Requirement already satisfied: jieba in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp) (0.42.1)
Requirement already satisfied: dill<0.3.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp) (0.3.3)
Requirement already satisfied: pyarrow>=6.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from datasets>=2.0.0->paddlenlp) (8.0.0)
Requirement already satisfied: huggingface-hub<1.0.0,>=0.1.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from datasets>=2.0.0->paddlenlp) (0.8.1)
Requirement already satisfied: pandas in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from datasets>=2.0.0->paddlenlp) (1.1.5)
Requirement already satisfied: requests>=2.19.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from datasets>=2.0.0->paddlenlp) (2.24.0)
Requirement already satisfied: responses<0.19 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from datasets>=2.0.0->paddlenlp) (0.18.0)
Requirement already satisfied: importlib-metadata in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from datasets>=2.0.0->paddlenlp) (4.2.0)
Requirement already satisfied: xxhash in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from datasets>=2.0.0->paddlenlp) (3.0.0)
Requirement already satisfied: packaging in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from datasets>=2.0.0->paddlenlp) (21.3)
Requirement already satisfied: numpy>=1.17 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from datasets>=2.0.0->paddlenlp) (1.19.5)
Requirement already satisfied: aiohttp in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from datasets>=2.0.0->paddlenlp) (3.8.1)
Requirement already satisfied: fsspec[http]>=2021.05.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from datasets>=2.0.0->paddlenlp) (2022.5.0)
Requirement already satisfied: six>=1.9 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from protobuf<=3.20.0,>=3.1.0->paddlenlp) (1.16.0)
Requirement already satisfied: scikit-learn>=0.21.3 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from seqeval->paddlenlp) (0.24.2)
Requirement already satisfied: typing-extensions>=3.7.4.3 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from huggingface-hub<1.0.0,>=0.1.0->datasets>=2.0.0->paddlenlp) (4.2.0)
Requirement already satisfied: filelock in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from huggingface-hub<1.0.0,>=0.1.0->datasets>=2.0.0->paddlenlp) (3.0.12)
Requirement already satisfied: pyyaml>=5.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from huggingface-hub<1.0.0,>=0.1.0->datasets>=2.0.0->paddlenlp) (5.1.2)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from packaging->datasets>=2.0.0->paddlenlp) (3.0.8)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests>=2.19.0->datasets>=2.0.0->paddlenlp) (1.25.11)
Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests>=2.19.0->datasets>=2.0.0->paddlenlp) (2019.9.11)
Requirement already satisfied: chardet<4,>=3.0.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests>=2.19.0->datasets>=2.0.0->paddlenlp) (3.0.4)
Requirement already satisfied: idna<3,>=2.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests>=2.19.0->datasets>=2.0.0->paddlenlp) (2.8)
Requirement already satisfied: threadpoolctl>=2.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-learn>=0.21.3->seqeval->paddlenlp) (2.1.0)
Requirement already satisfied: scipy>=0.19.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-learn>=0.21.3->seqeval->paddlenlp) (1.6.3)
Requirement already satisfied: joblib>=0.11 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-learn>=0.21.3->seqeval->paddlenlp) (0.14.1)
Requirement already satisfied: multidict<7.0,>=4.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from aiohttp->datasets>=2.0.0->paddlenlp) (6.0.2)
Requirement already satisfied: frozenlist>=1.1.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from aiohttp->datasets>=2.0.0->paddlenlp) (1.3.0)
Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from aiohttp->datasets>=2.0.0->paddlenlp) (4.0.2)
Requirement already satisfied: attrs>=17.3.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from aiohttp->datasets>=2.0.0->paddlenlp) (21.4.0)
Requirement already satisfied: asynctest==0.13.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from aiohttp->datasets>=2.0.0->paddlenlp) (0.13.0)
Requirement already satisfied: charset-normalizer<3.0,>=2.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from aiohttp->datasets>=2.0.0->paddlenlp) (2.1.0)
Requirement already satisfied: aiosignal>=1.1.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from aiohttp->datasets>=2.0.0->paddlenlp) (1.2.0)
Requirement already satisfied: yarl<2.0,>=1.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from aiohttp->datasets>=2.0.0->paddlenlp) (1.7.2)
Requirement already satisfied: zipp>=0.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from importlib-metadata->datasets>=2.0.0->paddlenlp) (3.8.0)
Requirement already satisfied: python-dateutil>=2.7.3 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pandas->datasets>=2.0.0->paddlenlp) (2.8.2)
Requirement already satisfied: pytz>=2017.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pandas->datasets>=2.0.0->paddlenlp) (2019.3)
[33mWARNING: You are using pip version 22.0.4; however, version 22.1.2 is available.
You should consider upgrading via the '/opt/conda/envs/python35-paddle120-env/bin/python -m pip install --upgrade pip' command.[0m[33m
[0m

示例流程

与大多数NLP任务相同，本次示例的展示流程分为以下四步：

首先我们从数据准备开始。

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-LOcCxGdS-1656682353368)(https://ai-studio-static-online.cdn.bcebos.com/dd30e17318fb48fabb5701fd8a97be8176a1e372dd134cc0826e58cb5401933d)]

数据准备

数据准备流程如下：

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-BxwWWnCI-1656682353369)(https://ai-studio-static-online.cdn.bcebos.com/89ba02da6f914297ae2fc438d1c9f773556f226652134fb684ac0186bfa5bb7d)]

1. 使用load_dataset()自定义数据集

使用官方脚本预处理过的数据集已经上传至项目根目录，观察数据集格式后我们可以根据数据格式写出数据文件读取函数，传入load_dataset()。即可创建数据集。

from paddlenlp.datasets import load_dataset
import json

# 读取标签文件并创建label_map
def get_label_map(filename):
    with open(filename, "r", encoding="utf8") as f:
        label_list = json.load(f)
    
    id2label = dict([(idx, label) for idx, label in enumerate(label_list)])
    label2id = dict([(label, idx) for idx, label in enumerate(label_list)])
    return id2label, label2id

id2slot, slot2id = get_label_map('slot_labels.json')
id2intent, intent2id = get_label_map('intent_labels.json')

intent_weight = [1] * len(intent2id)

# 根据本地文件格式定义数据读取生成器
def read(filename):
    with open(filename, "r", encoding="utf8") as f:
        data = json.load(f)
    for entry in data:
        yield {
            'words': entry[0],
            'slots': entry[1],
            'intents': entry[2],
            'history': entry[4],
        }

# 将生成器传入load_dataset
train_ds = load_dataset(read, filename='train.json', lazy=False)
dev_ds = load_dataset(read, filename='test.json', lazy=False)

for idx in range(2):
    print(train_ds[idx]['words'])
    print(train_ds[idx]['slots'])
    print(train_ds[idx]['intents'])
    print(train_ds[idx]['history'])
    print()

['你', '好', '，', '麻', '烦', '帮', '我', '推', '荐', '一', '个', '门', '票', '免', '费', '的', '景', '点', '。']
['O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'B+Inform+景点+门票', 'I+Inform+景点+门票', 'O', 'O', 'O', 'O']
['General+greet+none+none', 'Request+景点+名称+']
[]

['你', '好', '，', '您', '可', '以', '选', '择', '故', '宫', '，', '八', '达', '岭', '长', '城', '，', '颐', '和', '园', '或', '者', '红', '砖', '美', '术', '馆', '。']
['O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'B+Inform+景点+名称', 'I+Inform+景点+名称', 'I+Inform+景点+名称', 'I+Inform+景点+名称', 'I+Inform+景点+名称', 'O']
['General+greet+none+none']
['你好，麻烦帮我推荐一个门票免费的景点。']

关于更多自定义数据集相关内容，请参考如何自定义数据集

2. 加载 paddlenlp.transformers.AutoTokenizer用于数据处理

文本数据在输入 ERNIE 预训练模型之前，需要经过数据处理转化为Feature。这一过程通常包括分词，token to id，add special token等步骤。

PaddleNLP对于各种预训练模型已经内置了相应的tokenizer，指定想要使用的模型名字即可加载对应的tokenizer。

可以通过调用tokenizer中的方法简单的完成上述数据处理。

import paddlenlp

# 设置模型名称
MODEL_NAME = 'ernie-3.0-medium-zh'
tokenizer = paddlenlp.transformers.AutoTokenizer.from_pretrained(MODEL_NAME)

[2022-06-29 15:11:04,512] [    INFO] - We are using <class 'paddlenlp.transformers.ernie.tokenizer.ErnieTokenizer'> to load 'ernie-3.0-medium-zh'.
[2022-06-29 15:11:04,515] [    INFO] - Already cached /home/aistudio/.paddlenlp/models/ernie-3.0-medium-zh/ernie_3.0_medium_zh_vocab.txt
[2022-06-29 15:11:04,575] [    INFO] - tokenizer config file saved in /home/aistudio/.paddlenlp/models/ernie-3.0-medium-zh/tokenizer_config.json
[2022-06-29 15:11:04,659] [    INFO] - Special tokens file saved in /home/aistudio/.paddlenlp/models/ernie-3.0-medium-zh/special_tokens_map.json

3. 调用map()方法批量处理数据

由于我们传入了lazy=False，所以我们使用load_dataset()自定义的数据集是MapDataset对象。

MapDataset是paddle.io.Dataset的功能增强版本。其内置的map()方法适合用来进行批量数据集处理。

map()方法传入的是一个用于数据处理的function。正好可以与tokenizer相配合。

以下是本示例中的用法：

from functools import partial
import numpy as np

def convert_example(example, tokenizer, use_history=False, no_entity_id=0, max_seq_len=512, mode='train'):
    
    # 调用tokenizer的数据处理方法把文本转为id
    tokenized_input = tokenizer(
        example['words'],
        is_split_into_words=True,
        max_seq_len=max_seq_len)
    
    # 对槽位标签做截断，保证和input_id等长
    slot_labels = example['slots']
    if len(tokenized_input['input_ids']) - 2 < len(slot_labels):
        slot_labels = slot_labels[:len(tokenized_input['input_ids']) - 2]
    
    # 根据label_map将槽位标签转为id
    tokenized_input['slot_labels'] = [no_entity_id] + [slot2id[label] for label in slot_labels] + [no_entity_id]
    
    # 由于任务中的意图识别是多标签分类，需要把意图标签转为类似one-hot的格式
    intent_labels = np.zeros(len(intent2id), dtype='int64')
    for l in example['intents']:
        intent_labels[intent2id[l]] = 1
        if mode == 'train':
            # 统计训练集中每个意图的正样本数
            intent_weight[intent2id[l]] += 1
    tokenized_input['intent_labels'] = intent_labels
    
    # 将历史对话用[SEP]拼起来并转成id
    if use_history:
        tokenized_history = tokenizer(
            tokenizer.cls_token+tokenizer.sep_token.join(example['history']),
            max_seq_len=max_seq_len)
        tokenized_input['history_ids'] = tokenized_history['input_ids']
    else:
        tokenized_input['history_ids'] = []
    return tokenized_input

use_history = False
max_seq_length = 512

train_trans_func = partial(
        convert_example,
        tokenizer=tokenizer,
        mode='train',
        use_history=use_history,
        max_seq_len=max_seq_length)

dev_trans_func = partial(
        convert_example,
        tokenizer=tokenizer,
        mode='dev',
        use_history=use_history,
        max_seq_len=max_seq_length)

train_ds.map(train_trans_func, lazy=False)   
dev_ds.map(dev_trans_func, lazy=False)

# 根据意图的正样本数和总样本数为不同样本赋予不同的权重
for intent, intent_id in intent2id.items():
    neg_pos = (len(train_ds) - intent_weight[intent_id]) / intent_weight[intent_id]
    intent_weight[intent_id] = np.log10(neg_pos)


for idx in range(2):
    print(train_ds[idx]['input_ids'])
    print(train_ds[idx]['token_type_ids'])
    print(train_ds[idx]['slot_labels'])
    print(train_ds[idx]['intent_labels'])
    print()

[1, 226, 170, 4, 1298, 1934, 836, 75, 426, 1645, 7, 27, 232, 1039, 783, 453, 5, 561, 180, 12043, 2]
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 0, 0, 0, 0, 0]
[1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 0 0 0 0 0 0 0 0]

[1, 226, 170, 4, 892, 48, 22, 352, 790, 470, 915, 4, 643, 302, 1560, 84, 257, 4, 3540, 14, 509, 172, 123, 536, 1520, 188, 133, 774, 12043, 2]
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 4, 4, 4, 4, 0, 0]
[1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 0 0 0 0 0 0 0 0]

从以上结果可以看出，数据集中的example已经被转换成了模型可以接收的feature，包括input_ids、token_type_ids、slot_labels和intent_labels。
其中：

  • input_ids: 表示输入文本的token ID。
  • token_type_ids: 表示token所属的句子（Transformer类预训练模型支持单句以及句对输入）。
  • slot_labels: 槽位标签，其长度与输入文本相同。
  • intent_labels: 意图标签，是一个长度等于总便签数的列表，标签对应位置为1，其余为0.

更多有关数据处理的内容，请参考数据处理。

4. Batchify和数据读入

使用paddle.io.BatchSampler和paddlenlp.data中提供的方法把数据组成batch。

然后使用paddle.io.DataLoader接口多线程异步加载数据。

batchify_fn详解：

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-E1hIZxwD-1656682353370)(https://ai-studio-static-online.cdn.bcebos.com/30e43d4659384375a2a2c1b890ca5a995c4324d7168e49cebf1d2a1e99161f7d)]

import paddle
from paddlenlp.data import Stack, Dict, Pad
from paddlenlp.data import DataCollatorWithPadding

batch_size = 20

# 初始化BatchSampler
train_batch_sampler = paddle.io.BatchSampler(
    train_ds, batch_size=batch_size, shuffle=True)

dev_batch_sampler = paddle.io.BatchSampler(
    dev_ds, batch_size=batch_size, shuffle=False)

# 定义batchify_fn
batchify_fn = lambda samples, fn=Dict({
    "input_ids": Pad(axis=0, pad_val=tokenizer.pad_token_id), 
    "token_type_ids": Pad(axis=0, pad_val=tokenizer.pad_token_type_id),
    "slot_labels": Pad(axis=0, pad_val=0, dtype="int64"),
    "intent_labels": Stack(dtype="float32"),
    "history_ids": Pad(axis=0, pad_val=tokenizer.pad_token_id)
}): fn(samples)

# batchify_fn = DataCollatorWithPadding(tokenizer)

# 初始化DataLoader
train_data_loader = paddle.io.DataLoader(
    dataset=train_ds,
    batch_sampler=train_batch_sampler,
    collate_fn=batchify_fn,
    return_list=True)

dev_data_loader = paddle.io.DataLoader(
    dataset=dev_ds,
    batch_sampler=dev_batch_sampler,
    collate_fn=batchify_fn,
    return_list=True)

更多PaddleNLP内置的batchify相关API，请参考collate。

到这里数据集准备就全部完成了，下一步我们需要组网并设计loss function。

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-fqYo5Qxc-1656682353371)(https://ai-studio-static-online.cdn.bcebos.com/fdcb44a00ede4ce08ae2652931556fb58cc903f686bf491792489353d2800e7d)]

模型结构

使用PaddleNLP一键加载预训练模型

以下项目以ERNIE为例，介绍如何将预训练模型多任务学习同时完成意图识别和槽位填充任务。

本例中的意图识别和槽位填充本质上是一个句子分类任务和一个序列标注任务。将两者的loss结合即可实现多任务学习。

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-UIXGPaFy-1656682353372)(https://ai-studio-static-online.cdn.bcebos.com/d9ff881921d74602acb6eb27c8523cb50285f07a7beb4a3cbfa1edbd9b3f9c5c)]

from src.models import JointErnie

model = JointErnie.from_pretrained(MODEL_NAME, 
                                   intent_dim=len(intent2id), 
                                   slot_dim=len(slot2id), 
                                   dropout=0.1, 
                                   use_history=use_history)

[2022-06-29 15:11:15,483] [    INFO] - Already cached /home/aistudio/.paddlenlp/models/ernie-3.0-medium-zh/ernie_3.0_medium_zh.pdparams
W0629 15:11:15.487289  4452 gpu_context.cc:278] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 11.2, Runtime API Version: 10.1
W0629 15:11:15.491648  4452 gpu_context.cc:306] device: 0, cuDNN Version: 7.6.

设计loss function

JointErnie模型会取出ErnieModel的sequence_output接入一个输出维度为槽位类别数的线性层得到slot_logits，并将pooled_output接入一个输出维度为意图类别数的线性层得到intent_logit.

所以本示例中的loss由slot_loss和intent_loss两部分组成，我们需要自己定义loss function。

槽位填充相当于在每个token的位置进行一次多分类任务，意图识别相当于对整句话做一个多标签分类任务。所以设计的loss function如下：

class NLULoss(paddle.nn.Layer):
    def __init__(self, pos_weight):
        super(NLULoss, self).__init__()

        self.intent_loss_fn = paddle.nn.BCEWithLogitsLoss(pos_weight=paddle.to_tensor(pos_weight))
        self.slot_loss_fct = paddle.nn.CrossEntropyLoss()

    def forward(self, logits, slot_labels, intent_labels):
        slot_logits, intent_logits = logits

        slot_loss = self.slot_loss_fct(slot_logits, slot_labels)
        intent_loss = self.intent_loss_fn(intent_logits, intent_labels)

        return slot_loss + intent_loss

选择网络结构后，我们需要设置Fine-Tune优化策略。

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-2GvbMDjd-1656682353373)(https://ai-studio-static-online.cdn.bcebos.com/7eca6595f338409498149cb586c077ba4933739810cf436080a2292be7e0a92d)]

设置Fine-Tune优化策略

适用于ERNIE/BERT这类Transformer模型的学习率为warmup的动态学习率。



图3：动态学习率示意图 
# 训练过程中的最大学习率
learning_rate = 3e-5 

# 训练轮次
epochs = 1

# 学习率预热比例
warmup_proportion = 0.0

# 权重衰减系数，类似模型正则项策略，避免模型过拟合
weight_decay = 0.0

max_grad_norm = 1.0

num_training_steps = len(train_data_loader) * epochs

# 学习率衰减策略
lr_scheduler = paddlenlp.transformers.LinearDecayWithWarmup(learning_rate, num_training_steps,
                                    warmup_proportion)

decay_params = [
    p.name for n, p in model.named_parameters()
    if not any(nd in n for nd in ["bias", "norm"])
]

# 定义优化器
optimizer = paddle.optimizer.AdamW(
    learning_rate=lr_scheduler,
    parameters=model.parameters(),
    weight_decay=weight_decay,
    apply_decay_param_fun=lambda x: x in decay_params,
    grad_clip=paddle.nn.ClipGradByGlobalNorm(max_grad_norm))

现在万事俱备，我们可以开始训练模型。

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-rnxN76qi-1656682353375)(https://ai-studio-static-online.cdn.bcebos.com/6975542d488f4f75b385fe75d574a3aaa8e208f5e99f4acd8a8e8aea3b85c058)]

模型训练与评估

模型训练的过程通常有以下步骤：

  1. 从dataloader中取出一个batch data
  2. 将batch data喂给model，做前向计算
  3. 将前向计算结果传给损失函数，计算loss。
  4. loss反向回传，更新梯度。重复以上步骤。

每训练一个epoch后，程序对调用evaluation()方法分别计算两个任务的F1 score。

from src.utils import evaluation

criterion = NLULoss(intent_weight)
global_step = 0

for epoch in range(1, epochs + 1):
    for step, batch in enumerate(train_data_loader, start=1):
        global_step += 1
        input_ids, token_type_ids, slot_labels, intent_labels, history_ids = batch
        logits = model(input_ids, token_type_ids, history_ids=history_ids)
        loss = criterion(logits, slot_labels, intent_labels)

        if global_step % 1000 == 0 :
            print("global step %d, epoch: %d, batch: %d, loss: %.5f" % (global_step, epoch, step, loss))

        loss.backward()
        optimizer.step()
        lr_scheduler.step()
        optimizer.clear_grad()
    
    print('\nEval begin...')
    evaluation(model, dev_data_loader)

global step 1000, epoch: 1, batch: 1000, loss: 0.17512
global step 2000, epoch: 1, batch: 2000, loss: 0.09479
global step 3000, epoch: 1, batch: 3000, loss: 0.02178
global step 4000, epoch: 1, batch: 4000, loss: 0.07578

Eval begin...
Total samples: 8476
Slot F1: 0.9397
Intent F1: 0.6043

model.eval()
# convert to static graph with specific input description
model = paddle.jit.to_static(
    model,
    input_spec=[
        paddle.static.InputSpec(
            shape=[None, None], dtype="int64"),  # input_ids
        paddle.static.InputSpec(
            shape=[None, None], dtype="int64")  # segment_ids
    ])
# save converted static graph model
paddle.jit.save(model, "infer_model")

config = paddle.inference.Config("infer_model" + ".pdmodel",
                                         "infer_model" + ".pdiparams")
config.disable_gpu()
config.switch_use_feed_fetch_ops(False)

predictor = paddle.inference.create_predictor(config)

input_handles = [
            predictor.get_input_handle(name)
            for name in predictor.get_input_names()
        ]
output_handles = [
            predictor.get_output_handle(name)
            for name in predictor.get_output_names()
        ]

infer_batchify_fn = lambda samples, fn=Dict(
        {
            "input_ids": Pad(axis=0, pad_val=tokenizer.pad_token_id),
            "token_type_ids": Pad(axis=0, pad_val=tokenizer.pad_token_type_id)
        }): fn(samples)

infer_data_loader = paddle.io.DataLoader(
            dataset=dev_ds,
            batch_sampler=dev_batch_sampler,
            collate_fn=infer_batchify_fn,
            num_workers=0,
            return_list=True)

for data in infer_data_loader:
    for input_field, input_handle in zip(data, input_handles):
        input_handle.copy_from_cpu(input_field.numpy() if isinstance(
            input_field, paddle.Tensor) else input_field)
    predictor.run()
    output = [
        output_handle.copy_to_cpu() for output_handle in output_handles
    ]
    print(output)

[1m[35m--- Running analysis [ir_graph_build_pass][0m
[1m[35m--- Running analysis [ir_graph_clean_pass][0m
[1m[35m--- Running analysis [ir_analysis_pass][0m
[32m--- Running IR pass [simplify_with_basic_ops_pass][0m
[32m--- Running IR pass [layer_norm_fuse_pass][0m
[37m---    Fused 0 subgraphs into layer_norm op.[0m
[32m--- Running IR pass [attention_lstm_fuse_pass][0m
[32m--- Running IR pass [seqconv_eltadd_relu_fuse_pass][0m
[32m--- Running IR pass [seqpool_cvm_concat_fuse_pass][0m
[32m--- Running IR pass [mul_lstm_fuse_pass][0m
[32m--- Running IR pass [fc_gru_fuse_pass][0m
[37m---    fused 0 pairs of fc gru patterns[0m
[32m--- Running IR pass [mul_gru_fuse_pass][0m
[32m--- Running IR pass [seq_concat_fc_fuse_pass][0m
[32m--- Running IR pass [gpu_cpu_squeeze2_matmul_fuse_pass][0m
[32m--- Running IR pass [gpu_cpu_reshape2_matmul_fuse_pass][0m
[32m--- Running IR pass [gpu_cpu_flatten2_matmul_fuse_pass][0m
[32m--- Running IR pass [matmul_v2_scale_fuse_pass][0m
[32m--- Running IR pass [gpu_cpu_map_matmul_v2_to_mul_pass][0m
I0629 15:14:50.948932  4452 fuse_pass_base.cc:57] ---  detected 41 subgraphs
[32m--- Running IR pass [gpu_cpu_map_matmul_v2_to_matmul_pass][0m
I0629 15:14:50.950546  4452 fuse_pass_base.cc:57] ---  detected 12 subgraphs
[32m--- Running IR pass [matmul_scale_fuse_pass][0m
[32m--- Running IR pass [gpu_cpu_map_matmul_to_mul_pass][0m
[32m--- Running IR pass [fc_fuse_pass][0m
I0629 15:14:51.109480  4452 fuse_pass_base.cc:57] ---  detected 41 subgraphs
[32m--- Running IR pass [repeated_fc_relu_fuse_pass][0m
[32m--- Running IR pass [squared_mat_sub_fuse_pass][0m
[32m--- Running IR pass [conv_bn_fuse_pass][0m
[32m--- Running IR pass [conv_eltwiseadd_bn_fuse_pass][0m
[32m--- Running IR pass [conv_transpose_bn_fuse_pass][0m
[32m--- Running IR pass [conv_transpose_eltwiseadd_bn_fuse_pass][0m
[32m--- Running IR pass [is_test_pass][0m
[32m--- Running IR pass [runtime_context_cache_pass][0m
[1m[35m--- Running analysis [ir_params_sync_among_devices_pass][0m
[1m[35m--- Running analysis [adjust_cudnn_workspace_size_pass][0m
[1m[35m--- Running analysis [inference_op_replace_pass][0m
[1m[35m--- Running analysis [ir_graph_to_program_pass][0m
I0629 15:14:51.213642  4452 analysis_predictor.cc:1007] ======= optimize end =======
I0629 15:14:51.222798  4452 naive_executor.cc:102] ---  skip [feed], feed -> token_type_ids
I0629 15:14:51.222833  4452 naive_executor.cc:102] ---  skip [feed], feed -> words_ids
I0629 15:14:51.225849  4452 naive_executor.cc:102] ---  skip [linear_80.tmp_1], fetch -> fetch
I0629 15:14:51.225895  4452 naive_executor.cc:102] ---  skip [linear_81.tmp_1], fetch -> fetch


[array([[[ 6.7814775, -3.1593466, -4.1632333, ..., -3.7419374,
         -4.0669   , -4.0492043],
        [11.554589 , -4.407591 , -4.827274 , ..., -4.422597 ,
         -4.71447  , -5.1776814],
        [11.688357 , -4.48243  , -4.6533675, ..., -4.3366413,
         -4.8391576, -5.326421 ],
        ...,
        [13.176665 , -4.1729155, -4.728089 , ..., -4.333272 ,
         -4.9682946, -5.3904204],
        [13.158845 , -4.175857 , -4.732219 , ..., -4.3447914,
         -4.961181 , -5.3807683],
        [13.149474 , -4.165904 , -4.7227573, ..., -4.349148 ,
         -4.969923 , -5.3987556]],

       [[11.390071 , -4.130967 , -3.9426858, ..., -3.426007 ,
         -5.054478 , -5.846622 ],
        [11.302519 , -3.725232 , -4.90538  , ..., -4.3901167,
         -4.0434284, -4.9326787],
        [11.920896 , -4.1674213, -4.367908 , ..., -4.548027 ,
         -4.3104568, -5.063277 ],
        ...,
        [13.779258 , -4.145918 , -4.428103 , ..., -4.0718894,
         -4.791253 , -5.410724 ],
        [13.770816 , -4.140418 , -4.4266276, ..., -4.071643 ,
         -4.7830377, -5.3985143],
        [13.779096 , -4.1328306, -4.4179735, ..., -4.0831933,
         -4.791615 , -5.4085517]],

       [[ 7.456374 , -2.5692027, -4.24109  , ..., -3.810097 ,
         -3.3252873, -4.0920987],
        [ 8.574683 , -2.2495499, -4.775242 , ..., -4.3209004,
         -3.6838706, -4.957653 ],
        [ 8.927279 , -3.9099152, -3.0694826, ..., -3.6171744,
         -4.7853646, -5.089115 ],
        ...,
        [13.01001  , -3.5385768, -4.4491134, ..., -4.2443357,
         -4.598784 , -5.3697066],
        [12.998647 , -3.5454564, -4.4575524, ..., -4.237321 ,
         -4.6142797, -5.3851476],
        [12.970467 , -3.5336912, -4.4596734, ..., -4.2461004,
         -4.6133094, -5.3774924]],

       ...,

       [[10.560348 , -3.765053 , -3.6276329, ..., -1.9620574,
         -4.5778565, -5.420154 ],
        [10.790459 , -2.9837272, -4.1767244, ..., -3.850499 ,
         -3.7639039, -5.470011 ],
        [11.995274 , -3.8022048, -2.9080052, ..., -3.2904434,
         -4.5493007, -5.2190175],
        ...,
        [13.353181 , -3.8353279, -4.085818 , ..., -3.3101988,
         -4.67991  , -5.3841143],
        [13.35966  , -3.8365638, -4.078427 , ..., -3.313433 ,
         -4.686747 , -5.3946424],
        [13.341045 , -3.8436782, -4.0968795, ..., -3.314594 ,
         -4.680817 , -5.384107 ]],

       [[ 8.224532 , -3.576959 , -4.44025  , ..., -3.8914473,
         -3.3392422, -3.2988927],
        [12.265505 , -3.804784 , -4.2402215, ..., -4.5345654,
         -4.5437865, -5.1761727],
        [12.476783 , -3.9929006, -3.3433301, ..., -4.0739303,
         -4.920037 , -5.2187667],
        ...,
        [13.492063 , -4.144082 , -4.431912 , ..., -4.124772 ,
         -4.6841397, -5.151851 ],
        [13.48388  , -4.1280503, -4.42345  , ..., -4.140751 ,
         -4.657219 , -5.1178355],
        [13.465259 , -4.097384 , -4.4179034, ..., -4.151209 ,
         -4.643125 , -5.101619 ]],

       [[10.527641 , -3.2411811, -3.5474231, ..., -3.203087 ,
         -4.1240087, -5.4090724],
        [12.255399 , -3.2481868, -3.9800825, ..., -4.3268104,
         -4.399975 , -5.4881363],
        [12.489131 , -3.5077653, -2.9631546, ..., -3.8105638,
         -4.5532346, -5.2542653],
        ...,
        [13.400847 , -3.7305331, -4.1903267, ..., -4.0007954,
         -4.6963935, -5.4153113],
        [13.391182 , -3.726018 , -4.188986 , ..., -4.0083184,
         -4.6931624, -5.4093204],
        [13.394914 , -3.7251866, -4.1796756, ..., -3.9906137,
         -4.7131515, -5.4258523]]], dtype=float32), array([[  1.3510668,  -2.093196 ,  -6.227682 , ..., -10.037692 ,
        -10.521099 ,  -9.284265 ],
       [ -6.9154387,  -6.724922 ,  -8.537305 , ..., -15.176067 ,
        -14.752896 , -14.109042 ],
       [ -4.570983 ,  -6.6087685,  -3.3213701, ..., -10.563331 ,
         -9.909536 ,  -8.56572  ],
       ...,
       [ -8.539383 ,  -7.15563  ,  -7.552422 , ..., -15.469097 ,
        -14.777759 , -14.084039 ],
       [ -4.0770936,  -6.7344694,  -3.8842864, ..., -10.432816 ,
        -10.365968 ,  -9.091926 ],
       [ -7.797839 ,  -6.611453 ,  -6.9742765, ..., -15.78601  ,
        -15.158116 , -14.320247 ]], dtype=float32)]
        -10.962064 , -10.418475 ]], dtype=float32)]

引入对话历史

在该数据集的原始论文中，提到了一种改进模型的手段。即将对话历史的特征传入模型。可以增强模型的表现。

这里给大家举个例子：

  • 当前句：“好，玩累了周边有酒店可以休息吗？”

  • 意图：“Request+景点+周边酒店”

  • 历史：[“你好，帮我找一个20到50元之间的景点，谢谢。”,“八达岭长城怎么样？门票35元。”]

在本示例中，可以通过修改数据处理函数的的use_history参数来控制是否使用这一策略。

关于该策略的详细信息和原始baseline模型,请参考CrossWOZ。

更多任务

关于更多任务型对话的示例和数据集，可以参考PaddleNLP中的DGU

以上内容实现基于PaddleNLP，开源不易，希望大家多多支持~
记得给PaddleNLP点个小小的Star⭐

GitHub地址：https://github.com/PaddlePaddle/PaddleNLP
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-DWytmI2G-1656682353376)(https://ai-studio-static-online.cdn.bcebos.com/c54e958d7e50498381f2e89c84e2368723ff1db2f7c148518b1efa8890f93f0b)]

更多使用方法可参考PaddleNLP教程

加入交流群，一起学习吧

现在就加入PaddleNLP的QQ技术交流群，一起交流NLP技术吧！

作者仅为AiStudio搬运，原项目链接：https://aistudio.baidu.com/aistudio/projectdetail/2017202?contributionType=1

抽取](https://aistudio.baidu.com/aistudio/projectdetail/1317771)

加入交流群，一起学习吧

现在就加入PaddleNLP的QQ技术交流群，一起交流NLP技术吧！

作者仅为AiStudio搬运，原项目链接：https://aistudio.baidu.com/aistudio/projectdetail/2017202?contributionType=1

ERNIE 3.0: 大规模知识增强的预训练语言理解和生成框架
土豆的博客
03-07 761
预训练模型在自然语言处理（NLP）任务中取得了最先进的成果。最近的工作，如T5和GPT-3，表明预训练语言模型的规模扩大可以提高其泛化能力。尤其是GPT-3模型，拥有1750亿参数，展示了其在零样本/少量样本学习中的强大任务无关学习能力。然而，这些大规模模型是在纯文本上训练的，缺乏诸如语言知识和世界知识等显式表示。此外，大多数大规模模型是通过自回归方式训练的，这限制了其在下游语言理解任务中的表现。
【从零开始实现意图识别】中文对话意图识别详解
热门推荐
牧子川的博客
11-23 1万+
text表示待检测的文本，intent代表文本的类别标签，slots是文本中包括的所有槽位以及对应的槽值，以字典形式给出。"text": "搜索西红柿的做法。","slots": {"ingredient": "西红柿"}本项目中在原始数据集中新增了部分数据，用来平衡数据。
人工智能-项目实践-意图识别-基于PaddleNLP的对话意图识别.zip
12-14
人工智能-项目实践-意图识别-基于PaddleNLP的对话意图识别 1.比赛介绍 意图识别是指分析用户的核心需求，输出与查询输入最相关的信息，例如在搜索中要找电影、查快递、市政办公等需求，这些需求在底层的检索策略会有很大的不同，错误的识别几乎可以确定找不到能满足用户需求的内容，导致产生非常差的用户体验；在对话过程中要准确理解对方所想表达的意思，这是具有很大挑战性的任务。 意图识别的准确性能在很大程度上影响着搜索的准确性和对话系统的智能性，在本赛题中我们需要选手对中文对话进行意图识别。 2.数据集介绍 训练数据：大约1.2万条中文对话 测试数据：3000条无标注对话 3.提交样式 评分使用准确率进行评分，准确率值越大越好。 实操方案不允许使用外部数据集，允许使用公开的外部预训练模型。 实操方案需要在指定平台进行评分，提交csv格式。
基于ERNIR3.0文本分类：(KUAKE-QIC)意图识别多分类(单标签)
m0_63642362的博客
07-31 2655
基于ERNIR3.0文本分类以中医疗搜索检索词意图分类(KUAKE-QIC)为例【多分类(单标签)】
基于百度自研模型ERNIE进行对话意图识别任务
m0_63642362的博客
12-05 1396
基于ERNIE模型，在CrossWOZ数据集上进行意图识别任务。可以应用于客服回复，机器人问答等场景。
LLM意图识别器实践
python1234_的博客
06-29 1395
利用 Ollama 和 LangChain 强化条件判断语句的智能提示分类❝本文译自Supercharging If-Statements With Prompt Classification Using Ollama and LangChain一文，以Lumos工具为例，讲解了博主在工程实践中，如何基于LangChain框架和本地LLM优雅实现了通用的意图识别工具。我以前写过不少关于 Lumos 的内容，所以这次我就简短介绍一下。
意图识别
qq_37228811的博客
02-14 707
本质是分类任务，多用在搜索引擎和智能问答中。 解决方法 1、基于词典以及模板的规则方法 核心是领域词典的构建程度，词典足够好，覆盖范围广，匹配更准确。每一个领域都有自己的词典，如电影、书籍、歌曲；一个query分配给上述三个领域其中之一，最高的匹配度和重合度就是结果。 2、基于机器学习模型来对用户的意图进行判别 标注语料应用ML或者DL训练模型进行测试。 应用fasttext进行文本分...
ERNIE 3.0中文预训练模型进行MSRA序列标注
06-01
ERNIE 3.0中文预训练模型进行MSRA序列标注 文章链接：...
百度ERNIE系列预训练语言模型浅析(3)-ERNIE3.0
weixin_40566713的博客
05-29 2964
关键词：百亿参数大模型 \Transformer-XL\Knowledge graph预训练中加入知识图谱三元组，模型基本单元从2.0的transformer换成transformer-XL百度文心可以体验模型效果：https://wenxin.baidu.com/wenxin/ernie。
百度Ernie 3.0 学习笔记——预备知识
最新发布
Bonnie831的博客
09-24 1136
Ernie 预备知识。
slot_filling_and_intent_detection_of_SLU:广告位填充，意图检测，联合训练，ATIS和SNIPS数据集，Facebook的多语言数据集，MIT语料库，电子商务购物助手（ECSA）数据集，CoNLL2003 NER，ELMo，BERT，XLNet
05-23
口语填空和意图检测任务 插槽填充和意图检测的基本模型： 论文“具有焦点机制的编码器-解码器用于基于序列标签的口语理解”的“焦点”部分的实现 。 基于的BLSTM-CRF的实现 插槽填充和意图检测任务联合培训的实施 。 基本型号+ / / 数据集教程： （英语/西班牙语/泰语） （无意图） （无意图） （无意图） 部分 描述 所需的包 如何在ATIS数据集上报告意图检测的性能 教程A：带有预训练的单词嵌入 教程A：使用预训练的单词嵌入进行插槽填充和意图检测 教程B：使用ELMo 教程B：使用ElMo进行插槽填充和意图检测 教程C：使用BERT 教程C：插槽填充和BERT意图检测 教程D：使用XLNET 教程D：使用XLNET进行插槽填充和意图检测 结果 某些数据集上不同方法的结果 推论模式 推论模式 参考 如何引用？ 设置 python 3.6.x py
张帆：搜索引擎的查询意图识别
05-29
在搜索引擎中的意图识别，主要是为了解决通用搜索的通用展现形式和用户意图的冲突，对于用户的一些意图，比如购物、地图等需求在通用网页搜索的展示情况下，满足的并不好。但不可能要求用户记住几十种需求和相应的垂直网站，所以可以在通用网站中判断用户的查询意图，并帮助用户解析查询词，然后到多家垂直网站中搜索，再整合结果并重新排序后嵌入到通用搜索中。本演讲将剖析意图搜索中涉及的技术和难点，以及未来的趋势。
鸿蒙HarmonyOS开发实战—AI功能开发（助手类意图识别）
shudaoshanQAQ的博客
01-24 1113
随着人机交互越来越普遍，设备需要理解用户下达的各种指令，方便用户的操作。助手类意图识别能够利用机器学习技术，对用户发送给设备的文本消息进行语义分析和意图识别，进而衍生出各种智能的应用场景，使设备更智慧、更智能。
预训练ernie模型微调语句分类实战
qq_39276262的博客
08-16 865
transformer+预训练ernie模型微调语句分类实战 文件名称为 dummydata 这里的label是这样的： 2.使用Datasets库加载该数据集 1.读取自定义json数据集文件 这是我json文件的位置：F:\bert意图识别\data\dummydata.jsonl 这里设置了自动将数据集按照0.1的比例将数据集分为训练集和测试集然后dataset是一个DatasetDict类里面有训练集，测试集，验证集（其中验证集内容和训练集相同） 该函数返回三个可迭代的torc
【实践】基于ERNIE实现9项GLUE任务
PaddlePaddle
10-22 388
基于ERNIE实现9项GLUE任务
NLP实战：快递单信息抽取-基于ERNIE1.0 预训练模型
m0_62237233的博客
05-03 2652
推荐使用MapDataset()自定义数据集。每条数据包含一句文本和这个文本中每个汉字以及数字对应的label标签。之后，还需要对输入句子进行数据处理，如切词，映射词表id等。ERNIE是百度开创性提出的基于知识增强的持续学习语义理解框架，该框架将大数据预训练与多源丰富知识相结合，通过持续学习技术，不断吸收海量文本数据中词汇、结构、语义等方面的知识，实现模型效果不断进化。
【文本分类】意图识别
weixin_42551154的博客
03-12 6780
意图识别
Web搜索中的用户意图的自动识别
killtayoto的专栏
10-29 4088
是否能和如何能让查询意图的国陈自动化。首先需要介绍的是一个人工的调查，然后提出问题：导航类和信息类的识别，实验结果正确率达到了90%。
ernie 3.0使用
05-23
ERNIE 3.0 是百度公司推出的一种自然语言处理模型，它是一种基于深度学习的预训练语言模型。ERNIE 3.0 在自然语言理解和自然语言生成方面都有非常优秀的表现。它使用了大量的预训练数据，包括互联网上的文本、图像和知识库等，能够进行多任务学习，实现了对于自然语言的深度理解。这使得ERNIE 3.0 在文本分类、情感分析、命名实体识别、关系抽取、问答系统等任务中都能够得到较好的效果。 如果您想使用ERNIE 3.0，可以通过访问百度AI平台（https://ai.baidu.com/）来获得API接口或者SDK等资源。在使用ERNIE 3.0时，需要先加载模型并进行预处理，然后将输入数据输入到模型中进行推理，最后获得模型的输出结果。
评论 1
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值