bert-tokenization-WordpieceTokenizer
class WordpieceTokenizer(object): """Runs WordPiece tokenziation.""" def __init__(self, vocab, unk_token="[UNK]", max_input_chars_per_word=200): self.vocab = vocab self.unk_token = unk_t...
原创
2019-05-17 11:00:40 ·
1394 阅读 ·
0 评论