使用huggingface预训练模型进行分类任务

模型地址：chinese-roberta-wwm-ext

使用本地的下载好的chinese-roberta-wwm-ext模型，如果使用tensorflow，pip库中的tensorflow和tensorflow-gpu版本需要一致，不然会报下面这个错。

None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won’t be available and only tokenizers, configuration and file/data utilities can be used.

没有找到 PyTorch 和 TensorFlow >= 2.0

调用模型又报下面这个错。

ImportError:
AutoModelForMaskedLM requires the PyTorch library but it was not found in your environment. Checkout the instructions on the
installation page: https://pytorch.org/get-started/locally/ and follow the ones that match your environment.

找不到解决方法，只好转战pytorch。最终：

• python：3.8
• torch：1.12.0+cu113
• torchvision：0.13.0+cu113
• cuda：11.3

对于 pytorch_model.bin 的理解

pytorch_model.bin是一个已经训练好的预训练模型，包含了训练好的模型权重参数，可以继续训练，或者进行推理、评估。

使用Bert模型训练文本分类任务：这个方法使用AutoModelForMaskedLM加载模型，调用这部分代码的时候，loss值一直不变，没有找到原因。

基于 chinese-roberta-wwm-ext 微调

训练+验证

from transformers import AutoTokenizer, AutoModelForSequenceClassification
from transformers import logging
import torch
import torch.nn as nn
from torch.utils.data import DataLoader, RandomSampler, TensorDataset
import numpy as np
from tqdm import tqdm
import os
import csv


logging.set_verbosity_error()
os.environ["TOKENIZERS_PARALLELISM"] = "false"
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
csv.field_size_limit(500 * 1024 * 1024)


model_dir = "chinese-roberta-wwm-ext"
train_file = "chinese-roberta-wwm-ext/data/train.csv"
val_file = "chinese-roberta-wwm-ext/data/val.csv"
max_length = 128
num_classes = 2
batch_size = 32
epoch = 2


class GenDateSet():
    def __init__(self, tokenizer, train_file, val_file, max_length=128, batch_size=32):
        self.train_file = train_file
        self.val_file = val_file
        self.max_length = max_length
        self.batch_size = batch_size
        self.tokenizer = tokenizer

    def gen_data(self, file):
        if not os.path.exists(file):
            raise Exception("数据集不存在")
        input_ids = []
        input_types = []
        input_masks = []
        labels = []
        with open(file, encoding='utf8') as f:
            data = csv.reader(f, delimiter=',')

            for index, item in enumerate</