transformer pipeline 的使用

 Pipeline支持的任务类型

可以用一下代码进行查询

 from transformers.pipelines import SUPPORTED_TASKS
for k,v in SUPPORTED_TASKS.items():
    print(k,v["type"])

audio-classification audio
automatic-speech-recognition multimodal
text-to-audio text
feature-extraction multimodal
text-classification text
token-classification text
question-answering text
table-question-answering text
visual-question-answering multimodal
document-question-answering multimodal
fill-mask text
summarization text
translation text
text2text-generation text
text-generation text
zero-shot-classification text
zero-shot-image-classification multimodal
zero-shot-audio-classification multimodal
image-classification image
image-feature-extraction image
image-segmentation multimodal
image-to-text multimodal
object-detection multimodal
zero-shot-object-detection multimodal
depth-estimation image
video-classification video
mask-generation multimodal
image-to-image image

pipeline能做的任务：详见transformers网页

Pipeline的创建与使用方式

如何使用pipeline详见：详见pipeline使用方法

方式一
直接的指定相应的任务以及相关的模型

# pipeline中的重要参数
# model:指定需要使用到的模型
# device:设置是否在gpu显卡设置上，默认在cpu上
pipe = pipeline("text-classification",model="uer/roberta-base-finetuned-dianping-chinese",device=0)

示例1：

question_answer = pipeline(task='question-answering',model='luhua/chinese_pretrain_mrc_roberta_wwm_ext_large')

result = question_answer(question="What is the name of the repository?",
    context="The name of the repository is huggingface/transformers",
)
print(result)

 示例2：

classifier = pipeline(task='ner')
preds = classifier('Hugging Face is a Frence company based in New York city!')
preds

方式二
可以通过先加载模型以及指定分词器
然后再通过pipeline进行加载完成相应的任务

from transformers import AutoModelForSequenceClassification,AutoTokenizer
model = AutoModelForSequenceClassification.from_pretrained("uer/roberta-base-finetuned-dianping-chinese")
tokenizer = AutoTokenizer.from_pretrained("uer/roberta-base-finetuned-dianping-chinese")
pipes = pipeline("text-classification",model=model,tokenizer=tokenizer)

相关练习https://www.kaggle.com/code/haofucool/transformers-pipeline