bert论文中提到:“GPT uses a sentence separator ([SEP]) and classifier token ([CLS]) which are only introduced at fine-tuning time; BERT learns [SEP], [CLS] and sentence A/B embeddings during pre-training.”
说明[CLS]不是bert原创,在GPT中就有。在GPT中用于句子的分类,判断[CLS]开头的句子是不是下一句。