Python和PyTorch对比实现affine/linear(仿射/线性)变换函数及全连接层的反向传播

摘要

本文使用纯 Python 和 PyTorch 对比实现affine/linear(仿射/线性)变换函数及其反向传播.

相关

原理和详细解释, 请参考文章 :

affine/linear(仿射/线性)变换函数详解及全连接层反向传播的梯度求导

系列文章索引 :
https://blog.csdn.net/oBrightLamp/article/details/85067981

正文

import torch
import numpy as np


class Affine:
    def __init__(self):
        self.x = None
        self.weight = None
        self.bias = None
        self.dx = None
        self.dw = None
        self.db = None

    def __call__(self, x):
        self.x = x
        out = np.dot(self.x, self.weight.T) + self.bias
        return out

    def backward(self, d_loss