python 反传播_RELU反向传播

在使用relu激活函数时，我在实现backprop时遇到了问题。我的模型有两个隐藏层，在两个隐藏层中有10个节点，在输出层中有一个节点(因此3个权重，3个偏差)。我的模型不适用于这个坏掉了的向后推功能。但是，该函数使用sigmoid激活函数(作为注释包含在函数中)与backprop一起工作。因此，我相信我把relu推导搞砸了。在

有人能把我推到正确的方向吗？在# The derivative of relu function is 1 if z > 0, and 0 if z <= 0

def relu_deriv(z):

z[z > 0] = 1

z[z <= 0] = 0

return z

# Handles a single backward pass through the neural network

def backward_prop(X, y, c, p):

"""

cache (c): includes activations (A) and linear transformations (Z)

params (p): includes weights (W) and biases (b)

"""

m = X.shape[1] # Number of training ex

dZ3 = c['A3'] - y

dW3 = 1/m * np.dot(dZ3,c['A2'].T)

db3 = 1/m * np.sum(dZ3, keepdims=True, axis=1)

dZ2 = np.dot(p['W3'].T, dZ3) * relu_deriv(c['A2']) # sigmoid: replace relu_deriv w/ (1-np.power(c['A2'], 2))

dW2 = 1/m * np.dot(dZ2,c['A1'].T)

db2 = 1/m * np.sum(dZ2, keepdims=True, axis=1)

dZ1 = np.dot(p['W2'].T,dZ2) * relu_deriv(c['A1']) # sigmoid: replace relu_deriv w/ (1-np.power(c['A1'], 2))

dW1 = 1/m * np.dot(dZ1,X.T)

db1 = 1/m * np.sum(dZ1, keepdims=True, axis=1)

grads = {"dW1":dW1,"db1":db1,"dW2":dW2,"db2":db2,"dW3":dW3,"db3":db3}

return grads