pytorch（6）--深度置信网络

最新推荐文章于 2024-08-16 08:36:28 发布

ForeverYang2015

最新推荐文章于 2024-08-16 08:36:28 发布

阅读量3.7k

点赞数 2

分类专栏： pytorch 文章标签： pytorch-dbn

本文链接：https://blog.csdn.net/ap1005834/article/details/116146674

版权

本文介绍如何使用PyTorch构建深度置信网络(DBN)进行回归任务，数据维度为(N, 21)，输出为(N, 1)。网络结构包含4层RBM，训练时逐层进行，之后连接BP神经网络并进行微调，回归损失函数采用MSE。训练集MSE loss稳定在2.0-3.0。" 105837261,9032582,51单片机定时器/计数器0、1工作方式解析,"['单片机', 'C语言', '嵌入式开发']

摘要由CSDN通过智能技术生成

一、前言

本文主要使用pytorch 实现的DBN网络，用于对数据做回归，单个数据维度为（N，21），其中N为不定长，输出则为（N,1），对应N个值

DBN网络结构：

首层神经元数量输入为变量长度21，中间为RBM网络，如本篇使用的网络结构诶[128,64,32,16]，为一个4层的RBM网络结构，训练时RBM需要逐层做训练；在RBM训练后，再接上BP神经网络，再对BP网络做微调，回归损失函数使用MSE loss。

二、深度置信网络实现代码

#DBN.py
import torch
import warnings
import torch.nn as nn
import numpy as np

from RBM import RBM
from torch.utils.data import TensorDataset, DataLoader, Dataset
from torch.optim import Adam, SGD

from genCsvData import indefDataSet,DataLoader


class DBN(nn.Module):
    def __init__(self, hidden_units, visible_units=256, output_units=1, k=2,
                 learning_rate=1e-5, learning_rate_decay=False, #1e-5不可改动
                 increase_to_cd_k=False, device='cpu'):
        super(DBN, self).__init__()

        self.n_layers = len(hidden_units)
        self.rbm_layers = []
        self.rbm_nodes = []
        self.device = device
        self.is_pretrained = False
        self.is_finetune = False

        # Creating different RBM layers
        for i in range(self.n_layers):
            if i == 0:
                input_size = visible_units
            else:
                input_size = hidden_units[i - 1]
            rbm = RBM(visible_units=input_size, hidden_units=hidden_units[i],
                      k=k, learning_rate=learning_rate,
                      learning_rate_decay=learning_rate_decay,
                      increase_to_cd_k=increase_to_cd_k, device=device)

            self.rbm_layers.append(rbm)

        self.W_rec = [self.rbm_layers[i].weight for i in range(self.n_layers)]
        self.bias_rec = [self.rbm_layers[i].h_bias for i in range(self.n_layers)]

        for i in range(self.n_layers):
            self.register_parameter('W_rec%i' % i, self.W_rec[i])
            self.register_parameter('bias_rec%i' % i, self.bias_rec[i])

        self.bpnn = torch.nn.Linear(hidden_units[-1], output_units).to(self.device)    
        """ 
        self.bpnn=nn.Sequential(            #用作回归和反向微调参数
            torch.nn.Linear(32, 16),
            torch.nn.ReLU(),
            #torch.nn.Dropout(0.5),
            torch.nn.Linear(16,output_units),
        ).to(self.device)  """

    def forward(self, input_data):
        """
        running a single forward process.

        Args:
            input_data: Input data of the first RBM layer. Shape:
                [batch_size, input_length]

        Returns: Output of the last RBM hidden layer.

        """
        self.device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
        v = input_data.to(self.device)
        
        hid_output = v.clone()
        for i in range(len(self.rbm_layers)):
            hid_output, _ = self.rbm_layers[i].to_hidden(hid_output)
        output = self.bpnn(hid_output)
        return output

    def reconstruct(self, input_data):
        """
        Go forward to the last layer and then go feed backward back to the
        first layer.

        Args:
            input_data: Input data of the first RBM layer. Shape:
                [batch_size, input_length]

        Returns: Reconstructed output of the first RBM visible layer.

        """
        h = input_data.to(self.device)
        p_h = 0
        for i in range(len(self.rbm_layers)):
            # h = h.view((h.shape[0], -1))
            p_h, h = self.rbm_layers[i].to_hidden(h)

        for i in range(len(self.rbm_layers) - 1, -1, -1):
            # h = h.view((h.shape[0], -1))
            p_h, h = self.rbm_layers[i].to_visible(h)
        return p_h, h

    def pretrain(
            self, x, y, epoch=50, batch_size=10):
        """
        Train the DBN model layer by layer and fine-tuning with regression
        layer.

        Args:
            x: DBN model input data. Shape: [batch_size, input_length]
            epoch: Train epoch for each RBM.
            batch_size: DBN train batch size.

        Returns:

        """
        #hid_output_i = torch.tensor(x, dtype=torch.float, device=self.device)
        hid_output_i = x
        
        for i in range(len(self.rbm_layers)):
            print("Training rbm layer {}.".format(i + 1))