纯Python和PyTorch对比实现循环神经网络RNN及反向传播

最新推荐文章于 2022-11-24 12:21:06 发布

BrightLampCsdn

最新推荐文章于 2022-11-24 12:21:06 发布

阅读量815

点赞数

分类专栏：深度学习编程

本文链接：https://blog.csdn.net/oBrightLamp/article/details/85015387

版权

本文对比介绍了如何使用纯Python和PyTorch两种方式实现循环神经网络RNN，并详细探讨了反向传播的过程。提供了一个从基础到实践的理解RNNCell的教程链接。

摘要由CSDN通过智能技术生成

摘要

本文使用纯 Python 和 PyTorch 对比实现循环神经网络RNN及其反向传播

正文

import torch
import numpy as np


class RNNCell:
    def __init__(self, weight_ih, weight_hh,
                 bias_ih, bias_hh):
        self.weight_ih = weight_ih
        self.weight_hh = weight_hh
        self.bias_ih = bias_ih
        self.bias_hh = bias_hh

        self.x_stack = []
        self.dx_list = []
        self.dw_ih_stack = []
        self.dw_hh_stack = []
        self.db_ih_stack = []
        self.db_hh_stack = []

        self.prev_hidden_stack = []
        self.next_hidden_stack = []

        # temporary cache
        self.prev_dh = None

    def __call__(self, x, prev_hidden):
        self.x_stack.append(x)

        next_h = np.tanh(
            np.dot(x, self.weight_ih.T)
            + np.dot(prev_hidden, self.weight_hh.T)
            + self.bias_ih + self.bias_hh)

        self.prev_hidden_stack.append(prev_hidden)
        self.next_hidden_stack.append(next_h)
        # clean cache
        self.prev_dh = np.zeros(next_h.shape)
        return next_h

    def backward(self, dh):
        x = self.x_stack