序列模型常常会用到padding(nn.functional.pad()),而padding添加的就是0.0。例如:
def train(self, input, real_val):
self.model.train()
self.optimizer.zero_grad()
input = nn.functional.pad(input, (1,0,0,0))
output = self.model(input)
......
而在计算loss的时候,并不需要计算padding部分的0,因此我们需要用到masked loss。例如:
def masked_mse(preds, labels, null_val=np.nan):
if np.isnan(null_val):
mask = ~torch.isnan(labels)
else:
mask = (labels != null_val)
mask = mask.float()
mask /= torch.mean((mask))
mask = torch.where(torch.isnan(mask), torch.zeros_like(mask), mask)
loss = (preds - labels) ** 2
loss = loss * mask
loss = torch.where(torch.isnan(loss), torch.zeros_like(loss), loss)
return torch.mean(loss)