icoder的博客

None起到增加维度的作用def calc_mask_score(self, attention_mask): ''' input: [b,n] output: [b,h,n,n] ''' mask_score = torch.zeros(attention_mask.size(0),self.nums_heads,attention_mask.size(1),attention_mask.size(1)) # [b,h,n,

PyTorch中向量默认为行向量！（和习惯相反）以nn.Linear()为例：Applies a linear transformation to the incoming data: y=xAT+by = xA^T + by=xAT+b