短临降水预测ConvLSTM实验_基于OpenStl

共享，接受，客观，进步！

本次实验基于ConvLSTM模型进行短临降水预测，使用Github上的openstl库进行实验。Github地址，ConvLSTM的原文章地址：

[1506.04214] Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting (arxiv.org)https://arxiv.org/abs/1506.04214

nullhttps://github.com/chengtan9907/OpenSTL

创建Openstl的环境时，遇到些问题。为了节省时间，直接使用Openstl中的models中的模型，自己进行参数配置。数据集采用SEVIR，地址为：http://sevir.mit.edu/sevir-datasethttp://sevir.mit.edu/sevir-dataset

 印象中是使用亚马逊云进行下载，只使用其中的VIL数据集，下载了2017年与2018年的数据，数据格式为h5文件，大约共90GB。我是将h5文件提取出图片进行训练的。

任务说明：输入为过去10帧图像，来预测未来的10帧图像。

数据集准备：使用ConvLSTM进行训练时，我发现使用10 to 10的训练方式效果没有滑动窗的预测效果好（不排除是我的参数配置存在问题），就是使用第1~10张图像去预测第11张，也就是10 to 1 的模式，这数据集处理时需要使用一个group为11的窗口去进行数据集的截取。

模型训练：以黑盒心态来训练模型，首先搞清楚ConvLSTM的输入的size，输出的size。可以在ConvLSTM中的forward的函数来看。

可以看到在forward函数中，输入只有input_tensor，size =>[t,b,c,h,w]，以本次任务来看，t就是10，b为batch_size，c就是图像的通道，SEVIR的图像为灰度图，即只有一个通道。h,w就是图像的size。

搞清楚模型的input，就需要对模型进行初始化。模型初始化需要的参数为input_dim为输入图像的通道数，hidden_dim为lstm的隐藏层（为list形式）,kernel_size为conv中的卷积核大小（为list形式，元素为元组），num_layers为ConvLSTM的层数，该层数需要与前面两个参数len匹配。在后面会贴出对模型的具体配置。

模型：ConvLSTM由ConvLSTMCell构成，大家也可以从OpenStl上自行下载。

ConvLSTMCell:

class ConvLSTMCell(nn.Module):

    def __init__(self, input_dim, hidden_dim, kernel_size, bias):
        """
        Initialize ConvLSTM cell.

        Parameters
        ----------
        input_dim: int
            Number of channels of input tensor.
        hidden_dim: int
            Number of channels of hidden state.
        kernel_size: (int, int)
            Size of the convolutional kernel.
        bias: bool
            Whether or not to add the bias.
        """

        super(ConvLSTMCell, self).__init__()

        self.input_dim = input_dim
        self.hidden_dim = hidden_dim

        self.kernel_size = kernel_size
        self.padding = kernel_size[0] // 2, kernel_size[1] // 2
        self.bias = bias

        self.conv = nn.Conv2d(in_channels=self.input_dim + self.hidden_dim,
                              out_channels=4 * self.hidden_dim,
                              kernel_size=self.kernel_size,
                              padding=self.padding,
                              bias=self.bias)

    def forward(self, input_tensor, cur_state):
        h_cur, c_cur = cur_state

        combined = torch.cat([input_tensor, h_cur], dim=1)  # concatenate along channel axis

        combined_conv = self.conv(combined)
        cc_i, cc_f, cc_o, cc_g = torch.split(combined_conv, self.hidden_dim, dim=1)
        i = torch.sigmoid(cc_i)
        f = torch.sigmoid(cc_f)
        o = torch.sigmoid(cc_o)
        g = torch.tanh(cc_g)

        c_next = f * c_cur + i * g
        h_next = o * torch.tanh(c_next)

        return h_next, c_next

    def init_hidden(self, batch_size, image_size):
        height, width = image_size
        return (torch.zeros(batch_size, self.hidden_dim, height, width, device=self.conv.weight.device),
                torch.zeros(batch_size, self.hidden_dim, height, width, device=self.conv.weight.device))

ConvLSTM:在这里我的返回值为最后一层的隐藏状态，暴力训练的模型，因为没有真正去了解ConvLSTM的原理，这里就只算跑通罢。

class ConvLSTM(nn.Module):

    """

    Parameters:
        input_dim: Number of channels in input
        hidden_dim: Number of hidden channels
        kernel_size: Size of kernel in convolutions
        num_layers: Number of LSTM layers stacked on each other
        batch_first: Whether or not dimension 0 is the batch or not
        bias: Bias or no bias in Convolution
        return_all_layers: Return the list of computations for all layers
        Note: Will do same padding.

    Input:
        A tensor of size B, T, C, H, W or T, B, C, H, W
    Output:
        A tuple of two lists of length num_layers (or length 1 if return_all_layers is False).
            0 - layer_output_list is the list of lists of length T of each output
            1 - last_state_list is the list of last states
                    each element of the list is a tuple (h, c) for hidden state and memory
    Example:
        >> x = torch.rand((32, 10, 64, 128, 128))
        >> convlstm = ConvLSTM(64, 16, 3, 1, True, True, False)
        >> _, last_states = convlstm(x)
        >> h = last_states[0][0]  # 0 for layer index, 0 for h index
    """

    def __init__(self, input_dim, hidden_dim, kernel_size, num_layers,
                 batch_first=False, bias=True, return_all_layers=False):
        super(ConvLSTM, self).__init__()

        self._check_kernel_size_consistency(kernel_size)

        # Make sure that both `kernel_size` and `hidden_dim` are lists having len == num_layers
        kernel_size = self._extend_for_multilayer(kernel_size, num_layers)
        hidden_dim = self._extend_for_multilayer(hidden_dim, num_layers)
        if not len(kernel_size) == len(hidden_dim) == num_layers:
            raise ValueError('Inconsistent list length.')

        self.input_dim = input_dim
        self.hidden_dim = hidden_dim
        self.kernel_size = kernel_size
        self.num_layers = num_layers
        self.batch_first = batch_first
        self.bias = bias
        self.return_all_layers = return_all_layers

        cell_list = []
        for i in range(0, self.num_layers):
            cur_input_dim = self.input_dim if i == 0 else self.hidden_dim[i - 1]

            cell_list.append(ConvLSTMCell(input_dim=cur_input_dim,
                                          hidden_dim=self.hidden_dim[i],
                                          kernel_size=self.kernel_size[i],
                                          bias=self.bias))

        self.cell_list = nn.ModuleList(cell_list)

    def forward(self, input_tensor, hidden_state=None):
        """

        Parameters
        ----------
        input_tensor: todo
            5-D Tensor either of shape (t, b, c, h, w) or (b, t, c, h, w)
        hidden_state: todo
            None. todo implement stateful

        Returns
        -------
        last_state_list, layer_output
        """
        if not self.batch_first:
            # (t, b, c, h, w) -> (b, t, c, h, w)
            input_tensor = input_tensor.permute(1, 0, 2, 3, 4)

        b, _, _, h, w = input_tensor.size()

        # Implement stateful ConvLSTM
        if hidden_state is not None:
            raise NotImplementedError()
        else:
            # Since the init is done in forward. Can send image size here
            hidden_state = self._init_hidden(batch_size=b,
                                             image_size=(h, w))

        layer_output_list = []
        last_state_list = []

        seq_len = input_tensor.size(1)
        cur_layer_input = input_tensor

        for layer_idx in range(self.num_layers):

            h, c = hidden_state[layer_idx]
            output_inner = []
            # 时间步
            for t in range(seq_len):
                h, c = self.cell_list[layer_idx](input_tensor=cur_layer_input[:, t, :, :, :],
                                                 cur_state=[h, c])
                output_inner.append(h)

            layer_output = torch.stack(output_inner, dim=1)
            cur_layer_input = layer_output

            layer_output_list.append(layer_output)
            last_state_list.append([h, c])

        if not self.return_all_layers:
            layer_output_list = layer_output_list[-1:]
            last_state_list = last_state_list[-1:]

        return last_state_list[0][0]

    def _init_hidden(self, batch_size, image_size):
        init_states = []
        for i in range(self.num_layers):
            init_states.append(self.cell_list[i].init_hidden(batch_size, image_size))
        return init_states

    @staticmethod
    def _check_kernel_size_consistency(kernel_size):
        if not (isinstance(kernel_size, tuple) or
                (isinstance(kernel_size, list) and all([isinstance(elem, tuple) for elem in kernel_size]))):
            raise ValueError('`kernel_size` must be tuple or list of tuples')

    @staticmethod
    def _extend_for_multilayer(param, num_layers):
        if not isinstance(param, list):
            param = [param] * num_layers
        return param

模型的具体参数配置如下：这是一个10 to 10的，10 to 1的模型在服务器上，可以自行配置。

有了模型和数据集，就可以开始训练了，训练函数使用的是MSE_loss，优化器为Adam，学习率为1e-4。

测试：任务为输入过去10帧图像预测未来的10帧图像，而在本次实验中使用的是10 to 1的模式，所以需要构建两个数据集，一个是训练所用使用group为11的滑动窗构成，另外一个为group为20的滑动窗口构成的。在测试阶段使用0~9 to 10，然后使用1~10 to 11，循环11次。使用CSI（关键成功指数）进行评价，效果还算过得去。

大致先写这么多，欢迎大家一起交流！接受批评！

后续将会继续上传MIM、predRNN、U-net、phydnet用于短临降水预测。