spaCy 并行分词

最新推荐文章于 2024-06-24 14:55:20 发布

weixin_30737363

最新推荐文章于 2024-06-24 14:55:20 发布

阅读量305

点赞数

文章标签： javascript 人工智能 ViewUI

原文链接：http://www.cnblogs.com/crackpotisback/p/10008213.html

版权

spaCy 并行分词

在使用spacy的时候，感觉比nltk慢了许多，一直在寻找并行化的方案，好在找到了，下面给出spaCy并行化的分词方法使用示例：

import spacy

nlp = spacy.load("en")

docs = [
    "Our dream was to bring to Shanghai a tribute event dedicated to China which tells our history and visio.",
    "It was not simply a fashion show, but something that we created especially with love and passion for China and all the people around the world who loves Dolce & Gabbana"
]

for doc in nlp.pipe(docs, batch_size=100, n_threads=3):
    print(list(doc))
    print("*" * 50)

转载于:https://www.cnblogs.com/crackpotisback/p/10008213.html