【huggingface bertmodel类使用理解】

关于在BertModel.from_pretrained 之后如何取bert的输出的理解，参考了huggingface的文档：

先给一个手写图：

需要注意的是：pooler_output 和 last_hidden_state [0] 不相等。前者是后者经过一个MLP层得来，这个MLP层是在预训练时经过NSP任务的微调。

所以如果使用单文本分类任务，使用 last_hidden_state [0] 再自己加一个 MLP 层也许更为合适。

引用huggingface文档：

pooler_output (torch.FloatTensor of shape (batch_size, hidden_size)) — Last layer hidden-state of the first token of the sequence (classification token) after further processing through the layers used for the auxiliary pretraining task. E.g. for BERT-family of models, this returns the classification token after processing through a linear layer and a tanh activation function. The linear layer weights are trained from the next sentence prediction (classification) objective during pretraining

参考SimCSE代码的写法：