【自学】深度学习入门 基于python的理论与实现 LESSON7 ＜误差反向传播法1＞

文章目录

前言

一、简单层的实现

1. 乘法层的实现

2. 加法层的实现

二、激活函数层的实现

总结

---

前言

前面使用数值微分的方法进行梯度计算，该方法简单易实现，但是计算花费时间多。本章将学习高效计算权重参数梯度的方法——误差反向传播法。

---

一、简单层的实现

这里所说的层是神经网络功能的单元。比如负责sigmoid函数的Sigmoid、负责矩阵乘积的Affine等，都是以层为单位实现。

1. 乘法层的实现

层的实现中有两个共通的方法forward（）和backward（）。

import numpy as np

class MulLayer:
    def __init__(self):
        self.x = None
        self.y = None
        
    def forward(self, x, y):
        self.x = x
        self.y = y
        out = x * y
        return out
    
    def backward(self, dout):
        dx = dout * self.y
        dy = dout * self.x
        return dx, dy

应用乘法层代码：

import numpy as np

class MulLayer:
    def __init__(self):
        self.x = None
        self.y = None
        
    def forward(self, x, y):
        self.x = x
        self.y = y
        out = x * y
        return out
    
    def backward(self, dout):
        dx = dout * self.y
        dy = dout * self.x
        return dx, dy
    
apple = 100
apple_num = 2
tax = 1.1

#layer
mul_apple_layer = MulLayer()
mul_tax_layer = MulLayer()

#forward
apple_price = mul_apple_layer.forward(apple, apple_num)
price = mul_tax_layer.forward(apple_price, tax)
print(price)

#backward
dprice = 1
dapple_price, dtax = mul_tax_layer.backward(dprice)
dapple, dapple_num = mul_apple_layer.backward(dapple_price)
print(dapple, dapple_num, dapple_price, dtax)

结果：

220.00000000000003
2.2 110.00000000000001 1.1 200

解析：

关于def __init__(self)部分的解释见链接

https://www.cnblogs.com/liruilong/p/12875515.htmlhttps://www.cnblogs.com/liruilong/p/12875515.html常见的两种类的初始化方式：

class Student:
    def __init__(self):#两者之间的区别
        self.name = None
        self.score = None
    def __init__(self, name, score):
        self.name = name
        self.score = score

第一种需要在实例化后，对属性进行赋值，第二种直接实例化时，传入相应的参数。

class Student:
    def __init__(self):#两者之间的区别
        self.name = None
        self.score = None

# def __init__(self, name, score):
# self.name = name
# self.score = score

    def print_score(self):
        print("%s score is %s" % (self.name, self.score))

    def get_grade(self):
        if self.score >= 80:
            return "A"
        elif self.score >= 70:
            return "B"
        else:
            return "C"

# student = Student("sansan", 90)
student = Student()
student.name= "sansan"
student.score = 90

# susan = Student("sunny", 78)
susan = Student()
susan.name = "sunny"
susan.score = 78

student.print_score()
susan.print_score()
print(susan.get_grade())
print(student.get_grade())
    

总结：没啥大区别

2. 加法层的实现

import numpy as np

class AddLayer:
    def __init__(self):
        pass
        
    def forward(self, x, y):
        self.x = x
        self.y = y
        out = x + y
        return out
    
    def backward(self, dout):
        dx = dout * 1
        dy = dout * 1
        return dx, dy

应用加法层和乘法层：

import numpy as np

class MulLayer:
    def __init__(self):
        self.x = None
        self.y = None
        
    def forward(self, x, y):
        self.x = x
        self.y = y
        out = x * y
        return out
    
    def backward(self, dout):
        dx = dout * self.y
        dy = dout * self.x
        return dx, dy

class AddLayer:
    def __init__(self):
        pass
        
    def forward(self, x, y):
        self.x = x
        self.y = y
        out = x + y
        return out
    
    def backward(self, dout):
        dx = dout * 1
        dy = dout * 1
        return dx, dy

  
apple = 100
apple_num = 2
orange = 150
orange_num = 3
tax = 1.1

#layer
mul_apple_layer = MulLayer()
mul_orange_layer = MulLayer()
mul_tax_layer = MulLayer()
add_layer = AddLayer()


#forward
apple_price = mul_apple_layer.forward(apple, apple_num)
orange_price = mul_orange_layer.forward(orange, orange_num)
ao_price = add_layer.forward(apple_price, orange_price)
price = mul_tax_layer.forward(ao_price, tax)
print(price)

结果：

715.0000000000001

二、激活函数层的实现

import numpy as np

class ReLU():
    def __init__(self):
        self.mask = None
        
    def forward(self, x):
        self.mask = (x <= 0)
        out = x.copy()
        out[self.mask] = 0
        
        return 0
    
    def backward(self, dout):
        dout[self.mask] = 0
        dx = dout
        
        return dx

---

总结

临时有事，明天再学！