多特征预测软测量LSTM

项目场景：

多特征预测软测量LSTM

例如：多特征预测软测量LSTM，根据pearson关联系数提取得到关键特征，使用LSTM网络实现软测量预测

---

问题描述

验证集loss值异常：震荡，不收敛，周期化

---

原因分析：

周期化loss

batch_size 与总数据相除不是整数

---

尝试解决：

采用L2 正则化，dropout 层，增加数据量，删除batch_size 与总数据相除后，不完整的最后一批

1.采用L2 正则化
先获得模型中所有需要辨识的参数

@L2
    for param_tuple in model.named_parameters():
        name, param = param_tuple
        print("name = ", name)
        print("-" * 100)

选择需要加入L2惩罚项的参数，在优化器中加入weight_decay参数

@L2
optimizer = torch.optim.Adam([
        {'params': (p for name, p in model.named_parameters() if 'bias' not in name), 'weight_decay': 0.0001},
        {'params': (p for name, p in model.named_parameters() if 'bias' in name)}
    ], lr=learning_rate)

    

2.添加dropout层

@dropout
class LSTMModel(nn.Module):
    def __init__(self, input_size, hidden_size, num_layers, dropout_prob=0.2):
        super(LSTMModel, self).__init__()
        self.hidden_size = hidden_size
        self.num_layers = num_layers
        self.lstm = nn.LSTM(input_size, hidden_size,num_layers, batch_first=True)
        self.dropout = nn.Dropout(dropout_prob)
        self.fc = nn.Linear(hidden_size, 1)

    def forward(self, x):
        h0 = torch.zeros(self.num_layers, x.size(0), self.hidden_size).to(x.device)
        c0 = torch.zeros(self.num_layers, x.size(0), self.hidden_size).to(x.device)
        out, _ = self.lstm(x, (h0, c0))
        out = self.dropout(out[:, -1, :])
        out = self.fc(out)
        return out

3.删除batch_size与总数据相除后，不完整的最后一批

train_loader = DataLoader(train_dataset, batch_size=batch_size, shuffle=False, drop_last=True)