Bert+BiLSTM+attention+TextCNN来实现文本分类

Makup

已于 2022-12-15 22:34:06 修改

阅读量2.1k

点赞数 1

分类专栏： Pytorch-深度学习文章标签： python 深度学习

于 2022-12-15 22:33:36 首次发布

本文链接：https://blog.csdn.net/m0_62342639/article/details/128335654

版权

本文介绍了如何结合Bert、BiLSTM、注意力机制和TextCNN来实现文本分类任务。代码中展示了模型的定义、在AG_NEWS数据集上的训练与评估，并利用预训练的GloVe词向量增强表示。同时，文章还提供了整合预训练Bert模型和BiLSTM的部分代码。

摘要由CSDN通过智能技术生成

import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
from torchtext.datasets import text_classification
from torchtext.data import Field, BucketIterator

class TextCNN(nn.Module):
    def __init__(self, vocab_size, embedding_dim, output_dim, kernel_dim, kernel_sizes):
        super().__init__()

        # Embedding layer
        self.embedding = nn.Embedding(vocab_size, embedding_dim)

        # convolutional layers
        self.convs = nn.ModuleList([
            nn.Conv2d(1, kernel_dim, (k, embedding_dim))
            for k in kernel_sizes
        ])

        # dropout
        self.dropout = nn.Dropout(0.5)

        # final, fully-connected layer
        self.fc = nn.Linear(len(kernel_sizes) * kernel_dim, output_dim)

    def forward(self, text):
        # text = [batch_size, sent_length]

        embedded = self.embedding(text)
        # embedded = [batch_size, sent_length, emb_dim]

        embedded = embedded.unsqueeze(1)
        # embedded = [batch_size, 1, sent_length, emb_dim]

        conved = [F.relu(conv(embedded)).squeeze(3) for conv in self.convs]
        # conved_n = [batch_size, kernel_dim, sent_length - kernel_size[n] + 1]

        pooled = [F.max_pool1d(conv, conv.shape[2]).squeeze(2) for conv in conved]
        # pooled_n = [batch_size, kernel_dim]

        cat = self.dropout(torch.cat(pooled, dim=1))
        # cat = [batch_size, kernel_dim * len(kernel_sizes)]

        return self.fc(cat)
        # return [batch_size, output_dim]

def main():
    # Set up the fields for text and labels
    TEXT = Field(lower=True, tokenize='spacy')
    LABEL = Field(sequential=False, is_target=True)

    # Use the AG_NEWS dataset for classification
    train