使用背景:模型有3个输出,需要对这三个输出加权求和,于是设置可学习的参数作为权重,代码如下:
self.w1 = torch.nn.Parameter(torch.FloatTensor(1), requires_grad=True)
self.w2 = torch.nn.Parameter(torch.FloatTensor(1), requires_grad=True)
self.w3 = torch.nn.Parameter(torch.FloatTensor(1), requires_grad=True)
# initialization
self.w1.data.fill_(0.3)
self.w2.data.fill_(0.3)
self.w3.data.fill_(0.4)
def forward(self, x):
output = self.w1 * output3 + self.w2 * output4 + self.w3 * output5