chariot是一个用于NLP模型的数据准备工具，它通过chazutsu下载数据集，scikit-learn风格的预处理流水线，并支持spaCy的多语言分词。利用Joblib并行执行预处理，提供批量格式化，支持预训练词向量。通过定义和保存预处理流水线，简化模型训练流程。
chariot
Deliver the ready-to-train data to your NLP model.
Prepare Dataset
You can prepare typical NLP datasets through the chazutsu.
Build & Run Preprocess
You can build the preprocess pipeline like scikit-learn Pipeline.
Preprocesses for each dataset column are executed in parallel by Joblib.
Multi-language text tokenization is supported by spaCy.
Format Batch
Sampling a batch from preprocessed dataset and format it to train the model (padding etc).
You can use pre-trained word vectors through the chakin.
chariot enables you to concentrate on training your model!
Install
pip install chariot
Prepare dataset
You can download various dataset by using chazutsu.
import chazutsu
最低0.47元/天 解锁文章
扫一扫
130
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
