huggingface使用预训练模型入门

huggingface使用预训练模型入门

huggingface的官方文档的地址：https://huggingface.co/docs/transformers/quicktour。在官方文档中提供了两种加载预训练模型的方式：一是pipline，二是为pytorch/tensorflow/flax框架加载的预训练模型，本文档只对pytorch进行介绍，其他架构除了名字使用方法一样。

pipline

官方的pipline是针对任务的，具体有哪些任务如下所示，不做过多介绍，比较简单。

使用方式就是：classfier = pipline(task=task_name, model=model_name)， classfier(inputs)，huggingface提供的

pytorch的预训练模型加载

都是在AutoClass模块下进行加载。分成三种：Generic model classes (无头纯模型)，Generic pretraining classes(模型进行预训练)， NLP/CV/AUDIO/MultiModle。

AutoModel加载无头模型

huggingface文档中加载AutoModel有两种方式：from_config/from_pretrain

from_config

from transformers import AutoConfig, AutoModel

# Download configuration from huggingface.co and cache.
config = AutoConfig.from_pretrained("bert-base-cased")
model = AutoModel.from_config(config)

from_pretrained(model_name or model_path)

from transformers import AutoConfig, AutoModel

# Download model and configuration from huggingface.co and cache.
model = AutoModel.from_pretrained("bert-base-cased")

# Update configuration during loading,也就是说可以传入自己定义的一些关键字，会更新到model_config中
model = AutoModel.from_pretrained("bert-base-cased", output_attentions=True)
model.config.output_attentions

# Loading from a TF checkpoint file instead of a PyTorch model (slower)
config = AutoConfig.from_pretrained("./tf_model/bert_tf_model_config.json")
model = AutoModel.from_pretrained(
    "./tf_model/bert_tf_checkpoint.ckpt.index", from_tf=True, config=config
)

AutoModelForPretraining 加载用于预训练的模型

两种方式：from_config/from_pretrain

from_config

from transformers import AutoConfig, AutoModelForPreTraining

# Download configuration from huggingface.co and cache.
config = AutoConfig.from_pretrained("bert-base-cased")
model = AutoModelForPreTraining.from_config(config)

from_pretrained

from transformers import AutoConfig, AutoModelForPreTraining

# Download model and configuration from huggingface.co and cache.
model = AutoModelForPreTraining.from_pretrained("bert-base-cased")

# Update configuration during loading
model = AutoModelForPreTraining.from_pretrained("bert-base-cased", output_attentions=True)
model.config.output_attentions

# Loading from a TF checkpoint file instead of a PyTorch model (slower) , 可以加载TensorFlow模型
config = AutoConfig.from_pretrained("./tf_model/bert_tf_model_config.json")
model = AutoModelForPreTraining.from_pretrained(
    "./tf_model/bert_tf_checkpoint.ckpt.index", from_tf=True, config=config
)

AutoModel加载带头模型

huggingface提供了很管带有任务头的模型，这里只举个典型例子，加载方式和上述一样，都是from_config/from_pretrain

from_config

from transformers import AutoConfig, AutoModelForMaskedLM

# Download configuration from huggingface.co and cache.
config = AutoConfig.from_pretrained("bert-base-cased")
model = AutoModelForMaskedLM.from_config(config)

from_pretrained

from transformers import AutoConfig, AutoModelForMaskedLM

# Download model and configuration from huggingface.co and cache.
model = AutoModelForMaskedLM.from_pretrained("bert-base-cased")

# Update configuration during loading
model = AutoModelForMaskedLM.from_pretrained("bert-base-cased", output_attentions=True)
model.config.output_attentions

# Loading from a TF checkpoint file instead of a PyTorch model (slower)
config = AutoConfig.from_pretrained("./tf_model/bert_tf_model_config.json")
model = AutoModelForMaskedLM.from_pretrained(
    "./tf_model/bert_tf_checkpoint.ckpt.index", from_tf=True, config=config
)