### 4.GoogLeNet网络
Google Inception Net通常被称为Google Inception V1,在ILSVRC-2014比赛中由论文<Going deeper with convolutions>提出.
Inception V1有22层,比AlexNet的8层和VGGNet的19层还要深.参数量(500万)仅有AlexNet参数量(6000万)的1/12,但准确率远胜于AlexNet的准确率.
Inception V1降低参数量的目的:
1. 参数越多模型越庞大,需要模型学习的数据量就越大,且高质量的数据非常昂贵.
2. 参数越多,消耗的计算资源越多.
Inception V1网络的特点:
* 模型层数更深(22层),表达能力更强.
* 去除最后的全连接层,用全局平均池化层(即将图片尺寸变为1X1)来代替它.(借鉴了NIN)
* 使用Inception Module提高了参数利用效率.
![在这里插入图片描述](https://img-blog.csdnimg.cn/bb332adcabf94596940bca849a209e66.png)
Inception V2网络的特点:
* Batch Normalization 白化:使每一层的输出都规范化到N(0,1)
* 解决Interal Covariate Shift问题
* 允许较高学习率
* 取代部分Dropout
* 5X5卷积核–>2个3X3卷积核
![在这里插入图片描述](https://img-blog.csdnimg.cn/ee42559b8a9e4752990b586c19317cb6.png)
Inception V3网络的特点:
* 高效的降尺寸
* 不增加计算量
* 取消浅层的辅助分类器
* 深层辅助分类器只在训练后期有用
![在这里插入图片描述](https://img-blog.csdnimg.cn/e8024e11262c4965aef26861118045a1.png)
GoogLeNet网络结构:
![在这里插入图片描述](https://img-blog.csdnimg.cn/1a6d8b8b46e34e8ab156db3caa449cab.png)
对于我们搭建的Inception模块,所需要使用到参数有#1x1, #3x3reduce, #3x3, #5x5reduce, #5x5, poolproj,这6个参数,分别对应着所使用的卷积核个数,参数设置如下表所示:
![在这里插入图片描述](https://img-blog.csdnimg.cn/efe9fdaa55594b6bbbe4b74a891792e6.png)
**代码实现 Inception V3**
import tensorflow as tf
slim = tf.contrib.slim
trunc_normal = lambda stddev: tf.truncated_normal_initializer(0.0, stddev)
生成默认参数
def inception_v3_arg_scope(weight_decay=0.00004, # L2正则weight_decay
stddev=0.1, # 标准差
batch_norm_var_collection=‘moving_vars’):
batch_norm_params = {
‘decay’: 0.9997,
‘epsilon’:0.001,
‘updates_collections’: tf.GraphKeys.UPDATE_OPS,
‘variables_collections’:{
‘beta’: None,
‘gamma’: None,
‘moving_mean’: [batch_norm_var_collection],
‘moving_variance’: [batch_norm_var_collection],
}
}
# 提供了新的范围名称scope name
# 对slim.conv2d和slim.fully\_connected两个函数的参数自动赋值
with slim.arg_scope([slim.conv2d, slim.fully_connected],
weights_regularizer=slim.l2_regularizer(weight_decay)):
with slim.arg_scope(
[slim.conv2d], # 对卷积层的参数赋默认值
weights_initializer=tf.truncated_normal_initializer(stddev=stddev), # 权重初始化器
activation_fn=tf.nn.relu, # 激活函数用ReLU
normalizer_params=batch_norm_params) as sc: # 标准化器参数用batch\_norm\_params
return sc
inputs为输入图片数据的tensor(299x299x3),scope为包含了函数默认参数的环境
def inception_v3_base(inputs, scope=None):
# 保存某些关键节点
end_points = {}
# 定义InceptionV3的网络结构
with tf.variable_scope(scope, ‘InceptionV3’, [inputs]):
# 设置卷积/最大池化/平均池化的默认步长为1,padding模式为VALID
# 设置Inception模块组的默认参数
with slim.arg_scope([slim.conv2d, # 创建卷积层
slim.max_pool2d, # 输出的通道数
slim.avg_pool2d], # 卷积核尺寸
stride=1, # 步长
padding=‘VALID’): # padding模式
# 经3个3x3的卷积层后,输入数据(299x299x3)变为(35x35x192),空间尺寸降低,输出通道增加
net = slim.conv2d(inputs, 32, [3,3], stride=2, scope=‘Conv2d_1a_3x3’)
net = slim.conv2d(net, 32, [3, 3], scope=‘Conv2d_2a_3x3’)
net = slim.conv2d(net, 64, [3, 3], padding=‘SAME’, scope=‘Conv2d_2b_3x3’)
net = slim.max_pool2d(net, [3, 3], stride=2, scope='MaxPool\_3a\_3x3')
net = slim.conv2d(net, 80, [1, 1], scope='Conv2d\_3b\_1x1')
net = slim.conv2d(net, 192, [3, 3], scope='Conv2d\_4a\_3x3')
net = slim.max_pool2d(net, [3, 3], stride=2, scope='MaxPool\_5a\_3x3')
# 设置卷积/最大池化/平均池化的默认步长为1,padding模式为SAME
# 步长为1,padding模式为SAME,所以图像尺寸不会变,仍为35x35
with slim.arg_scope([slim.conv2d, slim.max_pool2d, slim.avg_pool2d], stride=1, padding='SAME'):
# 设置Inception Moduel名称为Mixed\_5b
with tf.variable_scope('Mixed\_5b'):
# 第1个分支:64输出通道的1x1卷积
with tf.variable_scope('Branch\_0'):
branch_0 = slim.conv2d(net, 64, [1, 1], scope='Conv2d\_0a\_1x1')
# 第2个分支:48输出通道的1x1卷积,连接64输出通道的5x5卷积
with tf.variable_scope('Branch\_1'):
branch_1 = slim.conv2d(net, 48, [1, 1], scope='Con2d\_0a\_1x1')
branch_1 = slim.conv2d(branch_1, 64, [5, 5], scope='Conv2d\_0b\_5x5')
# 第3个分支:64输出通道的1x1卷积,连接两个96输出通道的3x3卷积
with tf.variable_scope('Branch\_2'):
branch_2 = slim.conv2d(net, 64, [1, 1], scope='Conv2d\_0a\_1x1')
branch_2 = slim.conv2d(branch_2, 96, [3, 3], scope='Conv2d\_0b\_3x3')
branch_2 = slim.conv2d(branch_2, 96, [3, 3], scope='Conv2d\_0c\_3x3')
# 第4个分支:3x3的平均池化,连接32输出通道的1x1卷积
with tf.variable_scope('Branch\_3'):
branch_3 = slim.avg_pool2d(net, [3, 3], scope='AvgPool\_0a\_3x3')
branch_3 = slim.conv2d(branch_3, 32, [1, 1], scope='Conv2d\_0b\_1x1')
# 4个分支输出通道数之和=64+64+96+32=256,输出tensor为35x35x256
net = tf.concat([branch_0, branch_1, branch_2, branch_3], 3)
# 第1个Inception模块组的第2个Inception Module
with tf.variable_scope('Mixed\_5c'):
# 第1个分支:64输出通道的1x1卷积
with tf.variable_scope('Branch\_0'):
branch_0 = slim.conv2d(net, 64, [1, 1], scope='Conv2d\_0a\_1x1')
# 第2个分支:48输出通道的1x1卷积,连接64输出通道的5x5卷积
with tf.variable_scope('Branch\_1'):
branch_1 = slim.conv2d(net, 48, [1, 1], scope='Conv2d\_0b\_1x1')
branch_1 = slim.conv2d(branch_1, 64, [5, 5], scope='Conv2d\_0c\_5x5')
# 第3个分支:64输出通道的1x1卷积,连接两个96输出通道的3x3卷积
with tf.variable_scope('Branch\_2'):
branch_2 = slim.conv2d(net, 64, [1, 1], scope='Conv2d\_0a\_1x1')
branch_2 = slim.conv2d(branch_2, 96, [3, 3], scope='Conv2d\_0b\_3x3')
branch_2 = slim.conv2d(branch_2, 96, [3, 3], scope='Conv2d\_0c\_3x3')
# 第4个分支:3x3的平均池化,连接64输出通道的1x1卷积
with tf.variable_scope('Branch\_3'):
branch_3 = slim.avg_pool2d(net, [3, 3], scope='AvgPool\_0a\_3x3')
branch_3 = slim.conv2d(branch_3, 64, [1, 1], scope='Conv2d\_0b\_1x1')
# 输出tensor尺寸为35x35x288
net = tf.concat([branch_0, branch_1, branch_2, branch_3], 3)
# 第1个Inception模块组的第3个Inception Module
with tf.variable_scope('Mixed\_5d'):
# 第1个分支:64输出通道的1x1卷积
with tf.variable_scope('Branch\_0'):
branch_0 = slim.conv2d(net, 64, [1, 1], scope='Conv2d\_0a\_1x1')
# 第2个分支:48输出通道的1x1卷积,连接64输出通道的5x5卷积
with tf.variable_scope('Branch\_1'):
branch_1 = slim.conv2d(net, 48, [1, 1], scope='Conv2d\_0a\_1x1')
branch_1 = slim.conv2d(branch_1, 64, [5, 5], scope='Conv2d\_0b\_5x5')
# 第3个分支:64输出通道的1x1卷积,连接两个96输出通道的3x3卷积
with tf.variable_scope('Branch\_2'):
branch_2 = slim.conv2d(net, 64, [1, 1], scope='Conv2d\_0a\_1x1')
branch_2 = slim.conv2d(branch_2, 96, [3, 3], scope='Conv2d\_0b\_3x3')
branch_2 = slim.conv2d(branch_2, 96, [3, 3], scope='Conv2d\_0c\_3x3')
# 第4个分支:3x3的平均池化,连接64输出通道的1x1卷积
with tf.variable_scope('Branch\_3'):
branch_3 = slim.avg_pool2d(net, [3, 3], scope='AvgPool\_0a\_3x3')
branch_3 = slim.conv2d(branch_3, 64, [1, 1], scope='Conv2d\_0b\_1x1')
# 输出tensor尺寸为35x35x288
net = tf.concat([branch_0, branch_1, branch_2, branch_3], 3)
# 第2个Inception模块组
with tf.variable_scope('Mixed\_6a'):
# 第1个分支:3x3卷积,步长为2,padding模式为VALID,因此图像被压缩为17x17
with tf.variable_scope('Branch\_0'):
branch_0 = slim.conv2d(net, 384, [3, 3], stride=2 , padding='VALID', scope='Conv2d\_1a\_1x1')
# 第2个分支:64输出通道的1x1卷积,连接2个96输出通道的3x3卷积
with tf.variable_scope('Branch\_1'):
branch_1 = slim.conv2d(net, 64, [1, 1], scope='Conv2d\_0a\_1x1')
branch_1 = slim.conv2d(branch_1, 96, [3, 3], scope='Conv2d\_0b\_3x3')
# 步长为2,padding模式为VALID,因此图像被压缩为17x17
branch_1 = slim.conv2d(branch_1, 96, [3, 3], stride=2, padding='VALID', scope='Conv2d\_1a\_1x1')
# 第3个分支:3x3的最大池化层,步长为2,padding模式为VALID,因此图像被压缩为17x17x256
with tf.variable_scope('Branch\_2'):
branch_2 = slim.max_pool2d(net, [3, 3], stride=2, padding='VALID', scope='MaxPool\_1a\_3x3')
net = tf.concat([branch_0, branch_1, branch_2], 3)
# 第2个Inception模块组,包含5个Inception Module
with tf.variable_scope('Mixed\_6b'):
# 第1个分支:192输出通道的1x1卷积
with tf.variable_scope('Branch\_0'):
branch_0 = slim.conv2d(net, 192, [1, 1], scope='Conv2d\_0a\_1x1')
# 第2个分支:128输出通道的1x1卷积,接128输出通道的1x7卷积,接192输出通道的7x1卷积
with tf.variable_scope('Branch\_1'):
branch_1 = slim.conv2d(net, 128, [1, 1], scope='Conv2d\_0a\_1x1')
branch_1 = slim.conv2d(branch_1, 128, [1, 7], scope='Conv2d\_0b\_1x7')
branch_1 = slim.conv2d(branch_1, 192, [7, 1], scope='Conv2d\_0c\_7x1')
with tf.variable_scope('Branch\_2'):
branch_2 = slim.conv2d(net, 128, [1, 1], scope='Conv2d\_0a\_1x1')
branch_2 = slim.conv2d(branch_2, 128, [7, 1], scope='Conv2d\_0b\_7x1')
branch_2 = slim.conv2d(branch_2, 128, [1, 7], scope='Conv2d\_0c\_1x7')
branch_2 = slim.conv2d(branch_2, 128, [7, 1], scope='Conv2d\_0d\_7x1')
branch_2 = slim.conv2d(branch_2, 192, [1, 7], scope='Conv2d\_0e\_1x7')
with tf.variable_scope('Branch\_3'):
branch_3 = slim.avg_pool2d(net, [3, 3], scope='AvgPool\_0a\_3x3')
branch_3 = slim.conv2d(branch_3, 192, [1, 1], scope='Conv2d\_0b\_1x1')
# 输出tensor尺寸=17x17x(192+192+192+192)=17x17x768
net = tf.concat([branch_0, branch_1, branch_2, branch_3], 3)
# 经过一个Inception Module输出tensor尺寸不变,但特征相当于被精炼类一遍
# 第3个Inception模块组
with tf.variable_scope('Mixed\_6c'):
with tf.variable_scope('Branch\_0'):
branch_0 = slim.conv2d(net, 192, [1, 1], scope='Conv2d\_0a\_1x1')
with tf.variable_scope('Branch\_1'):
branch_1 = slim.conv2d(net, 160, [1, 1], scope='Conv2d\_0a\_1x1')
branch_1 = slim.conv2d(branch_1, 160, [1, 7], scope='Conv2d\_0b\_1x7')
branch_1 = slim.conv2d(branch_1, 192, [7, 1], scope='Conv2d\_0c\_7x1')
with tf.variable_scope('Branch\_2'):
branch_2 = slim.conv2d(net, 160, [1, 1], scope='Conv2d\_0a\_1x1')
branch_2 = slim.conv2d(branch_2, 160, [7, 1], scope='Conv2d\_0b\_7x1')
branch_2 = slim.conv2d(branch_2, 160, [1, 7], scope='Conv2d\_0c\_1x7')
branch_2 = slim.conv2d(branch_2, 160, [7, 1], scope='Conv2d\_0d\_7x1')
branch_2 = slim.conv2d(branch_2, 192, [1, 7], scope='Conv2d\_0e\_1x7')
with tf.variable_scope('Branch\_3'):
branch_3 = slim.avg_pool2d(net, [3, 3], scope='AvgPool\_0a\_3x3')
branch_3 = slim.conv2d(branch_3, 192, [1, 1], scope='Conv2d\_0b\_1x1')
# 输出tensor尺寸为17x17x768
net = tf.concat([branch_0, branch_1, branch_2, branch_3], 3)
# 第4个Inception模块组
with tf.variable_scope('Mixed\_6d'):
with tf.variable_scope('Branch\_0'):
branch_0 = slim.conv2d(net, 192, [1, 1], scope='Conv2d\_0a\_1x1')
with tf.variable_scope('Branch\_1'):
branch_1 = slim.conv2d(net, 160, [1, 1], scope='Conv2d\_0a\_1x1')
branch_1 = slim.conv2d(branch_1, 160, [1, 7], scope='Conv2d\_0b\_1x7')
branch_1 = slim.conv2d(branch_1, 192, [7, 1], scope='Conv2d\_0c\_7x1')
with tf.variable_scope('Branch\_2'):
branch_2 = slim.conv2d(net, 160, [1, 1], scope='Conv2d\_0a\_1x1')
branch_2 = slim.conv2d(branch_2, 160, [7, 1], scope='Conv2d\_0b\_7x1')
branch_2 = slim.conv2d(branch_2, 160, [1, 7], scope='Conv2d\_0c\_1x7')
branch_2 = slim.conv2d(branch_2, 160, [7, 1], scope='Conv2d\_0d\_7x1')
branch_2 = slim.conv2d(branch_2, 192, [1, 7], scope='Conv2d\_0e\_1x7')
with tf.variable_scope('Branch\_3'):
branch_3 = slim.avg_pool2d(net, [3, 3], scope='AvgPool\_0a\_3x3')
branch_3 = slim.conv2d(branch_3, 192, [1, 1], scope='Conv2d\_0b\_1x1')
# 输出tensor尺寸为17x17x768
net = tf.concat([branch_0, branch_1, branch_2, branch_3], 3)
# 第5个Inception模块组
with tf.variable_scope('Mixed\_6e'):
with tf.variable_scope('Branch\_0'):
branch_0 = slim.conv2d(net, 192, [1, 1], scope='Conv2d\_0a\_1x1')
with tf.variable_scope('Branch\_1'):
branch_1 = slim.conv2d(net, 192, [1, 1], scope='Conv2d\_0a\_1x1')
branch_1 = slim.conv2d(branch_1, 192, [1, 7], scope='Conv2d\_0b\_1x7')
branch_1 = slim.conv2d(branch_1, 192, [7, 1], scope='Conv2d\_0c\_7x1')
with tf.variable_scope('Branch\_2'):
branch_2 = slim.conv2d(net, 192, [1, 1], scope='Conv2d\_0a\_1x1')
branch_2 = slim.conv2d(branch_2, 192, [7, 1], scope='Conv2d\_0b\_7x1')
branch_2 = slim.conv2d(branch_2, 192, [1, 7], scope='Conv2d\_0c\_1x7')
branch_2 = slim.conv2d(branch_2, 192, [7, 1], scope='Conv2d\_0d\_7x1')
branch_2 = slim.conv2d(branch_2, 192, [1, 7], scope='Conv2d\_0e\_1x7')
with tf.variable_scope('Branch\_3'):
branch_3 = slim.avg_pool2d(net, [3, 3], scope='AvgPool\_0a\_3x3')
branch_3 = slim.conv2d(branch_3, 192, [1, 1], scope='Conv2d\_0b\_1x1')
# 输出tensor尺寸为17x17x768
net = tf.concat([branch_0, branch_1, branch_2, branch_3], 3)
# 将Mixed\_6e存储于end\_points中
end_points['Mixed\_6e'] = net
# 第3个Inception模块
# 第1个Inception模块组
with tf.variable_scope('Mixed\_7a'):
# 第1个分支:192输出通道的1x1卷积,接320输出通道的3x3卷积 步长为2
with tf.variable_scope('Branch\_0'):
branch_0 = slim.conv2d(net, 192, [1, 1], scope='Conv2d\_0a\_1x1')
branch_0 = slim.conv2d(branch_0, 320, [3, 3], stride=2, padding='VALID', scope='Conv2d\_0a\_3x3')
# 第2个分支:4个卷积层
with tf.variable_scope('Branch\_1'):
# 192输出通道的1x1卷积
branch_1 = slim.conv2d(net, 192, [1, 1], scope='Conv2d\_0a\_1x1')
# 192输出通道的1x7卷积
branch_1 = slim.conv2d(branch_1, 192, [1, 7], scope='Conv2d\_0b\_1x7')
# 192输出通道的7x1卷积
branch_1 = slim.conv2d(branch_1, 192, [7, 1], scope='Conv2d\_0c\_7x1')
# 192输出通道的3x3卷积 步长为2,输出8x8x192
branch_1 = slim.conv2d(branch_1, 192, [3, 3], stride=2, padding='VALID', scope='Conv2d\_1a\_3x3')
# 第3个分支:3x3的最大池化层,输出8x8x768
with tf.variable_scope('Branch\_2'):
branch_2 = slim.max_pool2d(net, [3, 3], stride=2, padding='VALID', scope='MaxPool\_1a\_3x3')
# 输出tensor尺寸:8x8x(320+192+768)=8x8x1280,尺寸缩小,通道数增加
net = tf.concat([branch_0, branch_1, branch_2], 3)
# 第2个Inception模块组
with tf.variable_scope('Mixed\_7b'):
# 第1个分支:320输出通道的1x1卷积
with tf.variable_scope('Branch\_0'):
branch_0 = slim.conv2d(net, 320, [1, 1], scope='Conv2d\_0a\_1x1')
# 第2个分支:384输出通道的1x1卷积
# 分支内拆分为两个分支:384输出通道的1x3卷积+384输出通道的3x1卷积
with tf.variable_scope('Branch\_1'):
branch_1 = slim.conv2d(net, 384, [1, 1], scope='Conv2d\_0a\_1x1')
branch_1 = tf.concat([
slim.conv2d(branch_1, 384, [1, 3], scope='Conv2d\_0b\_1x3'),
slim.conv2d(branch_1, 384, [3, 1], scope='Conv2d\_0b\_3x1')], 3)
# 第3个分支:448输出通道的1x1卷积,接384输出通道的3x3卷积,分支内拆分为两个分支
with tf.variable_scope('Branch\_2'):
branch_2 = slim.conv2d(net, 448, [1, 1], scope='Conv2d\_0a\_1x1')
branch_2 = slim.conv2d(branch_2, 384, [3, 3], scope='Conv2d\_0b\_3x3')
# 分支内拆分为两个分支:384输出通道的1x3卷积+384输出通道的3x1卷积
branch_2 = tf.concat([
slim.conv2d(branch_2, 384, [1, 3], scope='Conv2d\_0c\_1x3'),
slim.conv2d(branch_2, 384, [3, 1], scope='Conv2d\_0d\_3x1')], 3)
# 第4个分支:3x3的平均池化层,接192输出通道的1x1卷积,输出8x8x768
with tf.variable_scope('Branch\_3'):
branch_3 = slim.avg_pool2d(net, [3, 3], scope='AvgPool\_0a\_3x3')
branch_3 = slim.conv2d(branch_3, 192, [1, 1], scope='Conv2d\_0b\_1x1')
# 输出tensor尺寸:8x8x(320+768+768+192)=8x8x2048
net = tf.concat([branch_0, branch_1, branch_2, branch_3], 3)
# 第3个Inception模块组
with tf.variable_scope('Mixed\_7c'):
# 第1个分支:320输出通道的1x1卷积
with tf.variable_scope('Branch\_0'):
branch_0 = slim.conv2d(net, 320, [1, 1], scope='Conv2d\_0a\_1x1')
# 第2个分支:384输出通道的1x1卷积
# 分支内拆分为两个分支:384输出通道的1x3卷积+384输出通道的3x1卷积
with tf.variable_scope('Branch\_1'):
branch_1 = slim.conv2d(net, 384, [1, 1], scope='Conv2d\_0a\_1x1')
branch_1 = tf.concat([
slim.conv2d(branch_1, 384, [1, 3], scope='Conv2d\_0b\_1x3'),
slim.conv2d(branch_1, 384, [3, 1], scope='Conv2d\_0c\_3x1')], 3)
# 第3个分支:448输出通道的1x1卷积,接384输出通道的3x3卷积,分支内拆分为两个分支
with tf.variable_scope('Branch\_2'):
branch_2 = slim.conv2d(net, 448, [1, 1], scope='Conv2d\_0a\_1x1')
branch_2 = slim.conv2d(branch_2, 384, [3, 3], scope='Conv2d\_0b\_3x3')
# 分支内拆分为两个分支:384输出通道的1x3卷积+384输出通道的3x1卷积
branch_2 = tf.concat([
slim.conv2d(branch_2, 384, [1, 3], scope='Conv2d\_0c\_1x3'),
slim.conv2d(branch_2, 384, [3, 1], scope='Conv2d\_0d\_3x1')], 3)
# 第4个分支:3x3的平均池化层,接192输出通道的1x1卷积,输出8x8x768
with tf.variable_scope('Branch\_3'):
branch_3 = slim.avg_pool2d(net, [3, 3], scope='AvgPool\_0a\_3x3')
branch_3 = slim.conv2d(branch_3, 192, [1, 1], scope='Conv2d\_0b\_1x1')
# 输出tensor尺寸:8x8x(320+768+768+192)=8x8x2048
net = tf.concat([branch_0, branch_1, branch_2, branch_3], 3)
return net, end_points
全局平均池化
def inception_v3(inputs,
num_classes=1000, # 最后分类数量
is_training=True, # 是否是训练过程的标志
dropout_keep_prob=0.8, # Dropout保留节点的比例
prediction_fn=slim.softmax,# 进行分类的函数
spatial_squeeze=True, # 是否对输出进行squeeze操作,即去除维数为1的维度
reuse=None, # tf.variable_scope的reuse默认值
scope=‘InceptionV3’): # tf.variable_scope的scope默认值
with tf.variable_scope(scope, ‘InceptionV3’, [inputs, num_classes], reuse=reuse) as scope:
with slim.arg_scope([slim.batch_norm, slim.dropout], is_training=is_training):
net, end_points = inception_v3_base(inputs, scope=scope)
# 设置卷积/最大池化/平均池化的默认步长为1,padding模式为SAME
with slim.arg_scope([slim.conv2d, slim.max_pool2d, slim.avg_pool2d], stride=1, padding=‘SAME’):
aux_logits = end_points[‘Mixed_6e’]
# 辅助分类节点
with tf.variable_scope(‘AuxLogits’):
# 5x5的平均池化,步长设为3,padding模式设为VALID
aux_logits = slim.avg_pool2d(aux_logits, [5, 5], stride=3, padding=‘VALID’, scope=‘AvgPool_1a_5x5’)
aux_logits = slim.conv2d(aux_logits, 128, [1, 1], scope=‘Conv2d_1b_1x1’)
aux_logits = slim.conv2d(aux_logits,768, [5, 5], weights_initializer=trunc_normal(0.01), padding=‘VALID’, scope=‘Conv2d_2a_5x5’)
aux_logits = slim.conv2d(aux_logits, num_classes, [1, 1], activation_fn=None,
normalizer_fn=None, weights_initializer=trunc_normal(0.001), scope=‘Conv2d_2b_1x1’)
if spatial_squeeze:
# 进行squeeze操作,去除维数为1的维度
aux_logits = tf.squeeze(aux_logits, [1, 2], name=‘SpatialSqueeze’)
end_points[‘AuxLogits’] = aux_logits
# 处理正常的分类预测
with tf.variable_scope(‘Logits’):
# 8x8的平均池化层
net = slim.avg_pool2d(net, [8, 8], padding=‘VALID’, scope=‘AvgPool_1a_8x8’)
# Dropout层
net = slim.dropout(net, keep_prob=dropout_keep_prob, scope=‘Dropout_1b’)
end_points[‘PreLogits’] = net
logits = slim.conv2d(net, num_classes, [1, 1], activation_fn= None, normalizer_fn=None,scope=‘Conv2d_1c_1x1’)
if spatial_squeeze:
# 进行squeeze操作,去除维数为1的维度
logits = tf.squeeze(logits, [1, 2], name=‘SpatialSqueeze’)
# 辅助节点
end_points[‘Logits’] = logits
# 利用Softmax对结果进行分类预测
end_points[‘Predictions’] = prediction_fn(logits, scope=‘Predictions’)
return logits, end_points
import math
from datetime import datetime
import time
评估每轮计算占用的时间
输入TensorFlow的Session,需要测评的算子target,测试的名称info_string
def time_tensorflow_run(session, target, info_string):
# 定义预热轮数(忽略前10轮,不考虑显存加载等因素的影响)
num_steps_burn_in = 10
total_duration = 0.0
total_duration_squared = 0.0
for i in range(num_batches + num_steps_burn_in):
start_time = time.time()
_ = session.run(target)
# 持续时间
duration = time.time()- start_time
if i >= num_steps_burn_in:
# 只考量10轮迭代之后的计算时间
if not i % 10:
print '%s: step %d, duration = %.3f' % (datetime.now().strftime('%X'), i - num_steps_burn_in, duration)
# 记录总时间
total_duration += duration
total_duration_squared += duration \* duration
# 计算每轮迭代的平均耗时mn,和标准差sd
mn = total_duration / num_batches
vr = total_duration_squared / num_batches - mn \* mn
sd = math.sqrt(vr)
# 打印出每轮迭代耗时
print '%s: %s across %d steps, %.3f +/- %.3f sec / batch' % (datetime.now().strftime('%X'), info_string, num_batches, mn, sd)
Inception V3运行性能测试
if name == ‘__main__’:
batch_size = 32
height, width = 299, 299
inputs = tf.random_uniform((batch_size, height, width, 3))
with slim.arg_scope(inception_v3_arg_scope()):
# 传入inputs获取logits,end_points
logits, end_points = inception_v3(inputs, is_training=False)
# 初始化
init = tf.global_variables_initializer()
sess = tf.Session()
sess.run(init)
num_batches = 100
# 测试Inception V3的forward性能
time_tensorflow_run(sess, logits, ‘Forward’)
### 5.ResNet网络
ResNet是一个应用十分广泛的卷积神经网络的特征提取网络,在2016年由大名鼎鼎的何恺明(He-Kaiming)及其团队提出,他曾以第一作者身份拿过2次CVPR最佳论文奖(2009年和2016年),其中2016年CVPR最佳论文就是这个深度残差网络。
![在这里插入图片描述](https://img-blog.csdnimg.cn/5cee6fc39cca46edbd80c4ad61ec739d.png)
ResNet残差网络特点:
* 全是3X3卷积核
* 卷积步长2取代池化
* 使用BN
* 取消Max池化、全连接和Dropout
* 网络更深
各种ResNet残差网络:
![在这里插入图片描述](https://img-blog.csdnimg.cn/9cae3c3bf02545a3a7220fa548f300f1.png)
以Resnet18为例,它是由残差块堆叠而成的网络–1个卷积层+8个残差块(每个残差块有2个卷积层)+1个全连接层,如下图:
![在这里插入图片描述](https://img-blog.csdnimg.cn/169ce447c9704068881db9b2b1ea5568.png)
**代码实现 ResNet18**
#coding:utf-8
import os
os.environ[‘TF_CPP_MIN_LOG_LEVEL’] = ‘2’
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers, Sequential
#构建残差块
class BasicBlock(layers.Layer):
def \_\_init\_\_(self, filter_num, stride=1):
super(BasicBlock, self).__init__()
#卷积层(过滤器尺寸3\*3,过滤器个数filter\_num(可变),步长为stride(可变),padding为same(输出尺寸=输入尺寸/步长)
self.conv1 = layers.Conv2D(filter_num, (3, 3), strides=stride, padding='same')
#BatchNormalization标准化
self.bn1 = layers.BatchNormalization()
#激活函数选择relu
self.relu = layers.Activation('relu')
#卷积层(过滤器尺寸3\*3,过滤器个数filter\_num(可变),步长为1,padding为same(输出尺寸=输入尺寸/步长)
self.conv2 = layers.Conv2D(filter_num, (3, 3), strides=1, padding='same')
self.bn2 = layers.BatchNormalization()
#对步长进行判断,以减少参数量(如果步长等于1,则为原x;否则将x输入一个过滤器为1、步长为stride的卷积层中)
if stride != 1:
#建立一个容器
self.downsample = Sequential()
# 卷积层(过滤器尺寸1\*1,过滤器个数filter\_num(可变),步长为stride(可变));输出尺寸=向下取整((输入尺寸-过滤器尺寸)/步长)+1)
self.downsample.add(layers.Conv2D(filter_num, (1, 1), strides=stride))
else:
self.downsample = lambda x:x
#前向传播
def call(self, inputs, training=None):
# [b, h, w, c]
out = self.conv1(inputs)
out = self.bn1(out)
out = self.relu(out)
out = self.conv2(out)
out = self.bn2(out)
identity = self.downsample(inputs)
output = layers.add([out, identity])
output = tf.nn.relu(output)
return output
既有适合小白学习的零基础资料,也有适合3年以上经验的小伙伴深入学习提升的进阶课程,涵盖了95%以上大数据知识点,真正体系化!
由于文件比较多,这里只是将部分目录截图出来,全套包含大厂面经、学习笔记、源码讲义、实战项目、大纲路线、讲解视频,并且后续会持续更新
out = self.bn2(out)
identity = self.downsample(inputs)
output = layers.add([out, identity])
output = tf.nn.relu(output)
return output
[外链图片转存中…(img-bdxrvrDa-1714824035535)]
[外链图片转存中…(img-vTmtPULL-1714824035536)]
[外链图片转存中…(img-Swg3OmPG-1714824035536)]
既有适合小白学习的零基础资料,也有适合3年以上经验的小伙伴深入学习提升的进阶课程,涵盖了95%以上大数据知识点,真正体系化!
由于文件比较多,这里只是将部分目录截图出来,全套包含大厂面经、学习笔记、源码讲义、实战项目、大纲路线、讲解视频,并且后续会持续更新