transformers
文章平均质量分 70
sinat_24395003
先学使用轮子,再学造轮子,再自己造轮子
展开
-
SequenceSummary
from torch import nnfrom torch.nn import Identityfrom typing import Callable,Optionalimport torchPretrainedConfig = Noneget_activation = Noneclass SequenceSummary(nn.Module): r""" Compute a single vector summary of a sequence hidden states..原创 2021-05-19 13:58:02 · 257 阅读 · 0 评论 -
自定义Conv1d和nn.Conv1d
from torch import nnimport torchclass Conv1D(nn.Module): """ 1D-convolutional layer as defined by Radford et al. for OpenAI GPT (and also used in GPT-2). Basically works like a linear layer but the weights are transposed. Args: .原创 2021-05-10 10:40:16 · 387 阅读 · 0 评论 -
transformers的beam_search
"""transformer的generation_beam_search.py中beam_search过程当decoder的输入是[N,1],N为batch_size,设置beams=k,将输入转化为[N*k,1]输入放入到decoder中生成了logits,形状为[N*k,T],T为总的token数logits和历史beam_score相加成为新的beam_score,进行topk排序,获取next_beam_scores、next_beam_index、next_beam_tokensbe.原创 2021-04-27 16:49:39 · 2163 阅读 · 2 评论 -
NoRepeatNGramLogitsProcessor的_calc_banned_ngram_tokens
#transformer.generation_logits_process NoRepeatNGramLogitsProcessor的_calc_banned_ngram_tokens目的是生成不重复的ngramimport torchfrom typing import List, Iterabledef _get_ngrams(ngram_size: int, prev_input_ids: torch.Tensor, num_hypos: int): generated_ngra..原创 2021-04-09 15:43:11 · 412 阅读 · 1 评论