解读Depth Map Prediction from a Single Image using a Multi-Scale Deep Network (4)
上次把CNN卷积网络的结构作了介绍,这次仍然分析这篇博文,点击打开链接
来学习卷积神经网络的训练
训练原理:利用链式求导,计算损失函数对每一个权重的偏导数,然后根据梯度下降公式更新权重,依然是反向
传播算法
反向传播算法三个步骤:
1,前向计算每个神经元的输出值
2,反向计算每个神经元的误差项(也叫误差传递),即网络损失函数对该神经元加权输入的偏导数
3,计算每个神经元连接权重的梯度
卷积网络的训练:
1,卷积层训练:
用反向传播的原理计算filter每个权值的梯度
步长为2的情况,通过补零,转化为步长为1的情况
输入深度为D的误差传递
filter数量为N的误差传递
filter权重梯度的计算
2,Pooling层的训练:
Pooling层没有需要学习的参数,它需要做的仅仅是讲误差项传递到上一层,而没有梯度计算
写到这里,我不得不感谢这篇博客,教会我CNN的基础架构和反向传播算法
理论上大体明白了CNN网络,可以看一下CNN网络的Python实现
卷积层的初始化:
class ConvLayer(object):
def __init__(self, input_width, input_height,
channel_number, filter_width,
filter_height, filter_number,
zero_padding, stride, activator,
learning_rate):
self.input_width = input_width
self.input_height = input_height
self.channel_number = channel_number
self.filter_width = filter_width
self.filter_height = filter_height
self.filter_number = filter_number
self.zero_padding = zero_padding
self.stride = stride
self.output_width = \
ConvLayer.calculate_output_size(
self.input_width, filter_width, zero_padding,
stride)
self.output_height = \
ConvLayer.calculate_output_size(
self.input_height, filter_height, zero_padding,
stride)
self.output_array = np.zeros((self.filter_number,
self.output_height, self.output_width))
self.filters = []
for i in range(filter_number):
self.filters.append(Filter(filter_width,
filter_height, self.channel_number))
self.activator = activator
self.learning_rate = learning_rate
这段代码可以了解到:
1,参数input,channel,filter,zero_padding,stride,output的设置
2,设置filter的activator以及learning_rate
来看一下calculate_output_size()函数,
@staticmethod
def calculate_output_size(input_size,
filter_size, zero_padding, stride):
return (input_size - filter_size +
2 * zero_padding) / stride + 1
其实就是按照计算公式实现的
来看一下Filter类,
class Filter(object):
def __init__(self, width, height, depth):
self.weights = np.random.uniform(-1e-4, 1e-4,
(depth, height, width))
self.bias = 0
self.weights_grad = np.zeros(
self.weights.shape)
self.bias_grad = 0
def __repr__(self):
return 'filter weights:\n%s\nbias:\n%s' % (
repr(self.weights), repr(self.bias))
def get_weights(self):
return self.weights
def get_bias(self):
return self.bias
def update(self, learning_rate):
self.weights -= learning_rate * self.weights_grad
self.bias -= learning_rate * self.bias_grad
从Filter类代码可以了解到:
1,filter的基本参数
2,用梯度下降法更新filter权重
明日接着分析CNN的Python代码