Google Inception net
inception 的特点是在控制计算量和参数量的同时，分类的性能很好。共有22层（AlexNet 8层，VGGNet 19层）；参数量（500万）仅为AlexNet（6000万）的1/12，但是准确率更高。
Inception V1 降低参数量的目的有两点:
- 参数越多，模型越复杂，所需的数据量越大
- 参数越多，消耗的计算资源越多
Inception V1 特点
- 参数量少
- 去除了最后的全连接层，使用全局平均池化层【借鉴了 Network In Network 论文，简称NIN】（即将图片尺寸变为1x1）【因为全连接层参数量占90%，会引起过拟合】
- 设计了 Inception Module 提高了参数的利用率，它本身如同大网络中的小网络，可以反复堆叠在一起形成大网络，对NIN的改进是增加了分支网络，NIN主要是级联的卷积层和MLPConv层。
对于卷积层，如果要提高表达能力，主要是增加输出通道数，但可能会导致计算量增大和过拟合，每个输出通道对应的是一个过滤器filter，同一个滤波器共享参数，只能提取一类特征，因此一个输出通道只能做一种特征处理。在NIN中，MLPConv 允许在输出通道之间组合信息，拥有更强的表达能力，其基本等效于普通卷积层再连接 1*1 的卷积和ReLU 激活函数。
Inception V1 结构
共有四个分支，每个分支都使用了1*1的卷积层，它可以实现跨通道组织信息，同时可以对输出通多进行升维和降维。
卷积神经网络是一种稀疏的连接，Inception Net 的目标是找到最优的稀疏结构单元 （Inception Module)，论文中的稀疏结构基于 Hebbian 原理 ：一起发射的神经元会连在一起（cells that fire together, wire together）也就是说，当两个神经元A、B距离较近的时候，A参与了B 重复、持续的兴奋，那么会导致A作为使B兴奋的细胞。
1*1卷积将相关性很高、在同一个位置但是不同通道的特征连接在一起，这是卷积操作决定的，因为是 1*1 的卷积，所以将同一个位置的不同通道的信息进行加起来。此外还有 5*5， 3*3 的卷积。
Inception Net 共有22层，除了最后一层输出，其他中间节点的分类效果也很好，因此用到了辅助分类点（auxiliary classifiers）将中间某一层的输出作为分类，并按一个较小的权重(0.3)加到最终分类结果中。相当于做了模型的融合，同时增加了反向传播的梯度信号，也提供了额外的正则化。
Inception V2
使用两个3*3 卷积代替 5*5 的卷积，提出了 Batch Normalization 方法。解决：传统的深度神经网络学习率比较小的问题，使用了BN方法之后，学习率增大，训练时间缩短。去掉Dropout 减轻了L2正则（BN已经是正则了）；去除LRN；更彻底地对训练样本进行shuffle；减少数据增强带来的光学畸变
Inception V3
引入了 Factorization into small convolutions 将大的二维卷积拆成两个较小的一维卷积。 {7×7−−−>>1×7 { 7 × 7 − − − >> 1 × 7 and 7×1} 7 × 1 } ; 优化了Inception Module 结构；在分支中使用了分支；
- Factorization into small convolutions 很有效，可以降低参数量，减轻过拟合，增加网络非线性的表达能力
- 卷积网络从输入到输出，应该让图片尺寸逐渐减小，输出通道数逐渐增加，也就是说让空间结构简化，将空间信息转化为高阶抽象的特征信息
- Inception Module 用多个分支提取不同抽象程度的高阶特征的思路很有效，可以丰富网络的表达能力
结构
first: 卷积 —- 3x3 / 2 ——229x229x3
modules
Inception V4
结合了微软的ResNet
refer:
- 《TensorFlow 实现经典卷积神经网络》 黄文坚 唐源著
- 深入浅出——网络模型中Inception的作用与结构全解析
- 谷歌系列 :Inception v1到v4
- 神经网络 VGG、ResNet、GoogleLeNet、AlexNet等常用网络代码及预训练模型
code: Inception V3
使用 tf.contrib.slim 辅助设计
# coding:utf-8
from datetime import datetime
import time
import tensorflow as tf
import tensorflow.contrib.slim as slim
import math
# slim = tf.contrib.slim
# 产生截断的正态分布
trunc_normal = lambda stddev: tf.truncated_normal_initializer(0.0, stddev)
'''
生成网络中常用的函数的默认参数
'''
def inception_v3_arg_scope(weight_deacy=0.00004,
stddev=0.1,
batch_norm_var_collection='moving_vars'):
# batch_normilization 参数字典
batch_norm_params = {
'decay':0.997, # 衰减系数
'epsilon':0.001,
'updates_collections': tf.GraphKeys.UPDATE_OPS,
'variables_collections':{
'beta':None,
'gamma':None,
'moving_mean':[batch_norm_var_collection],
'moving_variance':[batch_norm_var_collection],
}
}
# 可以给函数的参数自动赋予某些默认值。不需要每次都重复设置参数
with slim.arg_scope([slim.conv2d, slim.fully_connected],
weights_regularizer=slim.l2_regularizer(weight_deacy)):
with slim.arg_scope([slim.conv2d],
weights_initializer=tf.truncated_normal_initializer(stddev=stddev),
activation_fn=tf.nn.relu,
normalizer_fn=slim.batch_norm,
normalizer_params=batch_norm_params) as sc:
return sc
'''
inputs: tensor, 输入图片数据，输入尺寸: [299, 299, 3]
scope: 包含了函数默认参数的环境
'''
def inception_v3_base(inputs, scope=None):
# 保存关键节点
end_points = {}
# inputs -> [299, 299, 3]
with tf.variable_scope(scope, 'InceptionV3', [inputs]):
with slim.arg_scope([slim.conv2d, slim.max_pool2d, slim.avg_pool2d],
stride=1,
padding='VALID'):
# slim.conv2d('输入的tensor', '输出的通道数', '卷积核尺寸', '步长stride', 'padding模式')
net = slim.conv2d(inputs, 32, [3, 3], stride=2, scope='Conv2d_1a_3x3')
net = slim.conv2d(net, 32, [3, 3], scope='Conv2d_2a_3x3')
net = slim.conv2d(net, 64, [3, 3], padding='SAME', scope='Conv2d_2b_3x3')
net = slim.max_pool2d(net, [3, 3], stride=2, scope='MaxPool_3a_3x3')
net = slim.conv2d(net, 80, [1, 1], scope='Conv2d_3b_1x1')
net = slim.conv2d(net, 192, [3, 3], scope='Conv2d_4a_3x3')
net = slim.max_pool2d(net, [3, 3], stride=2, scope='MaxPool_5a_3x3') # [35, 35, 192]
with slim.arg_scope([slim.conv2d, slim.max_pool2d, slim.avg_pool2d],
stride=1, padding='SAME'):
# 第一个 Inception Module
# 输出均为 [35, 35, *]
with tf.variable_scope('Mixed_5b'):
with tf.variable_scope('Branch_0'):
branch_0 = slim.conv2d(net, 64, [1, 1], scope='Conv2d_0a_1x1')
with tf.variable_scope('Branch_1'):
branch_1 = slim.conv2d(net, 48, [1, 1], scope='Conv2d_0a_1x1')
branch_1 = slim.conv2d(branch_1, 64, [5, 5], scope='Conv2d_0b_5x5')
with tf.variable_scope('Branch_2'):
branch_2 = slim.conv2d(net, 64, [1, 1], scope='Conv2d_0a_1x1')
branch_2 = slim.conv2d(branch_2, 96, [3, 3], scope='Conv2d_0b_3x3')
branch_2 = slim.conv2d(branch_2, 96, [3, 3], scope='Conv2d_0c_3x3')
with tf.variable_scope('Branch_3'):
branch_3 = slim.avg_pool2d(net, [3, 3], scope='AvgPool_0a_3x3')
branch_3 = slim.conv2d(branch_3, 32, [1, 1], scope='Conv2d_0b_1x1')
# 输出为 [35, 35, (64+64+96+32)=256]
net = tf.concat([branch_0, branch_1, branch_2, branch_3], 3)
with tf.variable_scope('Mixed_5c'):
with tf.variable_scope('Branch_0'):
branch_0 = slim.conv2d(net, 64, [1, 1], scope='Conv2d_0a_1x1')
with tf.variable_scope('Branch_1'):
branch_1 = slim.conv2d(net, 48, [1, 1], scope='Conv2d_0a_1x1') # Conv2d_0b_1x1
branch_1 = slim.conv2d(branch_1, 64, [5, 5], scope='Conv2d_0b_5x5') # Conv2d_0c_5x5
with tf.variable_scope('Branch_2'):
branch_2 = slim.conv2d(net, 64, [1, 1], scope='Conv2d_0a_1x1')
branch_2 = slim.conv2d(branch_2, 96, [3, 3], scope='Conv2d_0b_3x3')
branch_2 = slim.conv2d(branch_2, 96, [3, 3], scope='Conv2d_0c_3x3')
with tf.variable_scope('Branch_3'):
branch_3 = slim.avg_pool2d(net, [3, 3], scope='AvgPool_0a_3x3')
branch_3 = slim.conv2d(branch_3, 64, [1, 1], scope='Conv2d_0b_1x1') # 与 'Mixed_5b' 区别是输出了 64 通道，多了32
# 输出为 [35, 35, (64+64+96+64)=288]
net = tf.concat([branch_0, branch_1, branch_2, branch_3], 3)
with tf.variable_scope('Mixed_5d'):
with tf.variable_scope('Branch_0'):
branch_0 = slim.conv2d(net, 64, [1, 1], scope='Conv2d_0a_1x1')
with tf.variable_scope('Branch_1'):
branch_1 = slim.conv2d(net, 48, [1, 1], scope='Conv2d_0a_1x1')
branch_1 = slim.conv2d(branch_1, 64, [5, 5], scope='Conv2d_0b_5x5')
with tf.variable_scope('Branch_2'):
branch_2 = slim.conv2d(net, 64, [1, 1], scope='Conv2d_0a_1x1')
branch_2 = slim.conv2d(branch_2, 96, [3, 3], scope='Conv2d_0b_3x3')
branch_2 = slim.conv2d(branch_2, 96, [3, 3], scope='Conv2d_0c_3x3')
with tf.variable_scope('Branch_3'):
branch_3 = slim.avg_pool2d(net, [3, 3], scope='AvgPool_0a_3x3')
branch_3 = slim.conv2d(branch_3, 64, [1, 1], scope='Conv2d_0b_1x1')
# 输出为 [35, 35, (64+64+96+64)=288]
net = tf.concat([branch_0, branch_1, branch_2, branch_3], 3)
# 第二个 Inception Module， net->[35, 35, (64+64+96+64)=288]
with tf.variable_scope('Mixed_6a'):
# 输出 [17, 17, 384]
with tf.variable_scope('Branch_0'):
branch_0 = slim.conv2d(net, 384, [3, 3], stride=2, padding='VALID', scope='Conv2d_0a_1x1') # Conv2d_1a_1x1
# 输出 [17, 17, 96]
with tf.variable_scope('Branch_1'):
branch_1 = slim.conv2d(net, 64, [1, 1], scope='Conv2d_0a_1x1')
branch_1 = slim.conv2d(branch_1, 96, [3, 3], scope='Conv2d_0b_3x3')
branch_1 = slim.conv2d(branch_1, 96, [3, 3], stride=2, padding='VALID', scope='Conv2d_0c_1x1') # Conv2d_1a_1x1
# 输出 [17, 17, 256]
with tf.variable_scope('Branch_2'):
branch_2 = slim.max_pool2d(net, [3, 3], stride=2, padding='VALID', scope='MaxPool_1a_3x3')
# 输出 [17, 17, (384+96+256)=768]
net = tf.concat([branch_0, branch_1, branch_2], 3)
with tf.variable_scope('Mixed_6b'):
# 输出 [17, 17, 192]
with tf.variable_scope('Branch_0'):
branch_0 = slim.conv2d(net, 192, [1, 1], scope='Conv2d_0a_1x1')
# 输出 [17, 17, 192]
with tf.variable_scope('Branch_1'):
branch_1 = slim.conv2d(net, 128, [1, 1], scope='Conv2d_0a_1x1')
# 相当于一个 7*7 的卷积
branch_1 = slim.conv2d(branch_1, 128, [1, 7], scope='Conv2d_0b_1x7')
branch_1 = slim.conv2d(branch_1, 192, [7, 1], scope='Conv2d_0c_7x1')
# 输出 [17, 17, 192]
with tf.variable_scope('Branch_2'):
branch_2 = slim.conv2d(net, 128, [1, 1], scope='Conv2d_0a_1x1')
branch_2 = slim.conv2d(branch_2, 128, [7, 1], scope='Conv2d_0b_7x1')
branch_2 = slim.conv2d(branch_2, 128, [1, 7], scope='Conv2d_0c_1x7')
branch_2 = slim.conv2d(branch_2, 128, [7, 1], scope='Conv2d_0d_7x1')
branch_2 = slim.conv2d(branch_2, 192, [1, 7], scope='Conv2d_0e_1x7')
# 输出 [17, 17, 192]
with tf.variable_scope('Branch_3'):
branch_3 = slim.avg_pool2d(net, [3, 3], scope='AvgPool_0a_3x3')
branch_3 = slim.conv2d(branch_3, 192, [1, 1], scope='Conv2d_0b_1x1')
# 输出 [17, 17, (192+192+192+192)=768]
net = tf.concat([branch_0, branch_1, branch_2, branch_3], 3)
with tf.variable_scope('Mixed_6c'):
# 输出 [17, 17, 192]
with tf.variable_scope('Branch_0'):
branch_0 = slim.conv2d(net, 192, [1, 1], scope='Conv2d_0a_1x1')
# 输出 [17, 17, 192]
with tf.variable_scope('Branch_1'):
branch_1 = slim.conv2d(net, 160, [1, 1], scope='Conv2d_0a_1x1') # 与 Mixed_6b 的不同之处 128 -> 160
# 相当于一个 7*7 的卷积
branch_1 = slim.conv2d(branch_1, 160, [1, 7], scope='Conv2d_0b_1x7')
branch_1 = slim.conv2d(branch_1, 192, [7, 1], scope='Conv2d_0c_7x1')
# 输出 [17, 17, 192]
with tf.variable_scope('Branch_2'):
branch_2 = slim.conv2d(net, 128, [1, 1], scope='Conv2d_0a_1x1')
branch_2 = slim.conv2d(branch_2, 128, [7, 1], scope='Conv2d_0b_7x1')
branch_2 = slim.conv2d(branch_2, 128, [1, 7], scope='Conv2d_0c_1x7')
branch_2 = slim.conv2d(branch_2, 128, [7, 1], scope='Conv2d_0d_7x1')
branch_2 = slim.conv2d(branch_2, 192, [1, 7], scope='Conv2d_0e_1x7')
# 输出 [17, 17, 192]
with tf.variable_scope('Branch_3'):
branch_3 = slim.avg_pool2d(net, [3, 3], scope='AvgPool_0a_3x3')
branch_3 = slim.conv2d(branch_3, 192, [1, 1], scope='Conv2d_0b_1x1')
# 输出 [17, 17, (192+192+192+192)=768]
net = tf.concat([branch_0, branch_1, branch_2, branch_3], 3)
# 与 Mixed_6c 一样
with tf.variable_scope('Mixed_6d'):
# 输出 [17, 17, 192]
with tf.variable_scope('Branch_0'):
branch_0 = slim.conv2d(net, 192, [1, 1], scope='Conv2d_0a_1x1')
# 输出 [17, 17, 192]
with tf.variable_scope('Branch_1'):
branch_1 = slim.conv2d(net, 160, [1, 1], scope='Conv2d_0a_1x1')
# 相当于一个 7*7 的卷积
branch_1 = slim.conv2d(branch_1, 160, [1, 7], scope='Conv2d_0b_1x7')
branch_1 = slim.conv2d(branch_1, 192, [7, 1], scope='Conv2d_0c_7x1')
# 输出 [17, 17, 192]
with tf.variable_scope('Branch_2'):
branch_2 = slim.conv2d(net, 128, [1, 1], scope='Conv2d_0a_1x1')
branch_2 = slim.conv2d(branch_2, 128, [7, 1], scope='Conv2d_0b_7x1')
branch_2 = slim.conv2d(branch_2, 128, [1, 7], scope='Conv2d_0c_1x7')
branch_2 = slim.conv2d(branch_2, 128, [7, 1], scope='Conv2d_0d_7x1')
branch_2 = slim.conv2d(branch_2, 192, [1, 7], scope='Conv2d_0e_1x7')
# 输出 [17, 17, 192]
with tf.variable_scope('Branch_3'):
branch_3 = slim.avg_pool2d(net, [3, 3], scope='AvgPool_0a_3x3')
branch_3 = slim.conv2d(branch_3, 192, [1, 1], scope='Conv2d_0b_1x1')
# 输出 [17, 17, (192+192+192+192)=768]
net = tf.concat([branch_0, branch_1, branch_2, branch_3], 3)
# 与 Mixed_6c 一样
with tf.variable_scope('Mixed_6e'):
# 输出 [17, 17, 192]
with tf.variable_scope('Branch_0'):
branch_0 = slim.conv2d(net, 192, [1, 1], scope='Conv2d_0a_1x1')
# 输出 [17, 17, 192]
with tf.variable_scope('Branch_1'):
branch_1 = slim.conv2d(net, 160, [1, 1], scope='Conv2d_0a_1x1')
# 相当于一个 7*7 的卷积
branch_1 = slim.conv2d(branch_1, 160, [1, 7], scope='Conv2d_0b_1x7')
branch_1 = slim.conv2d(branch_1, 192, [7, 1], scope='Conv2d_0c_7x1')
# 输出 [17, 17, 192]
with tf.variable_scope('Branch_2'):
branch_2 = slim.conv2d(net, 128, [1, 1], scope='Conv2d_0a_1x1')
branch_2 = slim.conv2d(branch_2, 128, [7, 1], scope='Conv2d_0b_7x1')
branch_2 = slim.conv2d(branch_2, 128, [1, 7], scope='Conv2d_0c_1x7')
branch_2 = slim.conv2d(branch_2, 128, [7, 1], scope='Conv2d_0d_7x1')
branch_2 = slim.conv2d(branch_2, 192, [1, 7], scope='Conv2d_0e_1x7')
# 输出 [17, 17, 192]
with tf.variable_scope('Branch_3'):
branch_3 = slim.avg_pool2d(net, [3, 3], scope='AvgPool_0a_3x3')
branch_3 = slim.conv2d(branch_3, 192, [1, 1], scope='Conv2d_0b_1x1')
# 输出 [17, 17, (192+192+192+192)=768]
net = tf.concat([branch_0, branch_1, branch_2, branch_3], 3)
# 将 Mixed_6e 存储在 end_points 中，作为 Auxiliary Classifier 辅助模型的分类
end_points['Mixed_6e'] = net
# 第三个 Inception Module, net -> [17, 17, (192+192+192+192)=768]
# 从此开始，输出图片的尺寸又被缩小了，通道数在增加，tensor的总size在持续下降
with tf.variable_scope('Mixed_7a'):
# 输出 [8, 8, 320]
with tf.variable_scope('Branch_0'):
branch_0 = slim.conv2d(net, 192, [1, 1], scope='Conv2d_0a_1x1')
branch_0 = slim.conv2d(branch_0, 320, [3, 3], stride=2, padding='VALID', scope='Conv2d_1a_3x3') #Conv2d_1a_3x3
# 输出 [8, 8, 192]
with tf.variable_scope('Branch_1'):
branch_1 = slim.conv2d(net, 192, [1, 1], scope='Conv2d_0a_1x1')
# 相当于一个 7*7 的卷积
branch_1 = slim.conv2d(branch_1, 192, [1, 7], scope='Conv2d_0b_1x7')
branch_1 = slim.conv2d(branch_1, 192, [7, 1], scope='Conv2d_0c_7x1')
branch_1 = slim.conv2d(branch_1, 192, [3, 3], stride=2, padding='VALID', scope='Conv2d_1a_3x3')
# 输出 [8, 8, 768]
with tf.variable_scope('Branch_2'):
branch_2 = slim.max_pool2d(net, [3, 3], stride=2, padding='VALID', scope='MaxPool_1a_3x3') # MaxPool_1a_3x3
# 输出 [8, 8, (320+192+768)=1280]
net = tf.concat([branch_0, branch_1, branch_2], 3)
with tf.variable_scope('Mixed_7b'):
# 输出 [8, 8, 320]
with tf.variable_scope('Branch_0'):
branch_0 = slim.conv2d(net, 320, [1, 1], scope='Conv2d_0a_1x1')
# 输出 [8, 8, (384+384)=768]
with tf.variable_scope('Branch_1'):
branch_1 = slim.conv2d(net, 384, [1, 1], scope='Conv2d_0a_1x1')
branch_1 = tf.concat([
slim.conv2d(branch_1, 384, [1, 3], scope='Conv_0b_1x3'),
slim.conv2d(branch_1, 384, [3, 1], scope='Conv2d_0b_3x1')], 3)
# 输出 [8, 8, 768]
with tf.variable_scope('Branch_2'):
branch_2 = slim.conv2d(net, 448, [1, 1], scope='Conv2d_0a_1x1')
branch_2 = slim.conv2d(branch_2, 384, [3, 3], scope='Conv2d_0b_3x3')
branch_2 = tf.concat([
slim.conv2d(branch_2, 384, [1, 3], scope='Conv_0c_1x3'),
slim.conv2d(branch_2, 384, [3, 1], scope='Conv2d_0d_3x1')], 3)
# 输出 [8, 8, 192]
with tf.variable_scope('Branch_3'):
branch_3 = slim.avg_pool2d(net, [3, 3], scope='AvgPool_0a_3x3')
branch_3 = slim.conv2d(branch_3, 192, [1, 1], scope='Conv2d_0b_1x1')
# 输出 [8, 8, (320+768+768+192)=2048]
net = tf.concat([branch_0, branch_1, branch_2, branch_3], 3)
# 与 Mixed_7b 一样
with tf.variable_scope('Mixed_7c'):
# 输出 [8, 8, 320]
with tf.variable_scope('Branch_0'):
branch_0 = slim.conv2d(net, 320, [1, 1], scope='Conv2d_0a_1x1')
# 输出 [8, 8, (384+384)=768]
with tf.variable_scope('Branch_1'):
branch_1 = slim.conv2d(net, 384, [1, 1], scope='Conv2d_0a_1x1')
branch_1 = tf.concat([
slim.conv2d(branch_1, 384, [1, 3], scope='Conv_0b_1x3'),
slim.conv2d(branch_1, 384, [3, 1], scope='Conv2d_0b_3x1')], 3)
# 输出 [8, 8, 768]
with tf.variable_scope('Branch_2'):
branch_2 = slim.conv2d(net, 448, [1, 1], scope='Conv2d_0a_1x1')
branch_2 = slim.conv2d(branch_2, 384, [3, 3], scope='Conv2d_0b_3x3')
branch_2 = tf.concat([
slim.conv2d(branch_2, 384, [1, 3], scope='Conv_0c_1x3'),
slim.conv2d(branch_2, 384, [3, 1], scope='Conv2d_0d_3x1')], 3)
# 输出 [8, 8, 192]
with tf.variable_scope('Branch_3'):
branch_3 = slim.avg_pool2d(net, [3, 3], scope='AvgPool_0a_3x3')
branch_3 = slim.conv2d(branch_3, 192, [1, 1], scope='Conv2d_0b_1x1')
# 输出 [8, 8, (320+768+768+192)=2048]
net = tf.concat([branch_0, branch_1, branch_2, branch_3], 3)
return net, end_points
# 全局平均池化，Softmax, Auxiliary Logits
'''
Args:
inputs: 输入
num_classes=1000 默认分类
is_training=True 只在训练的过程中，batch normalization 和 dropout 才启动
dropout_keep_prob=0.8
prdiction_fn=slim.softmax 最后用来分类的函数
spatial_squeeze=True 是否对输出进行squeeze操作，也就是去除维度为 1 的维度 [5, 3, 1] -> [5, 3]
reuse=True 是否会对网络和 Variable 进行复用
scope='InceptionV3'
'''
def inception_v3(inputs,
num_classes=1000,
is_training=True,
dropout_keep_prob=0.8,
prdiction_fn=slim.softmax,
spatial_squeeze=True,
reuse=tf.AUTO_REUSE,
scope='InceptionV3'):
with tf.variable_scope(scope, 'InceptionsV3', [inputs, num_classes], reuse=reuse) as scope:
with slim.arg_scope([slim.batch_norm, slim.dropout], is_training=is_training):
net, end_points = inception_v3_base(inputs, scope=scope)
with slim.arg_scope([slim.conv2d, slim.max_pool2d, slim.avg_pool2d], stride=1, padding='SAME'):
# 获取 Mixed_6e
aux_logits = end_points['Mixed_6e']
with tf.variable_scope('Auxlogits'):
# [5, 5] 的池化层， [17, 17, 768] -> [5, 5, 768]
aux_logits = slim.avg_pool2d(aux_logits,
[5, 5],
stride=3,
padding='VALID',
scope='AvgPool_1a_5x5')
aux_logits = slim.conv2d(aux_logits, 128, [1, 1], scope='Conv2d_1b_1x1')
# [5, 5, 768] -> [1, 1, 768]
aux_logits = slim.conv2d(aux_logits, 768, [5, 5], weights_initializer=trunc_normal(0.01),
padding='VALID', scope='Conv2d_2a_5x5')
# [1, 1, 768] -> [1, 1, 1000]
aux_logits = slim.conv2d(aux_logits, num_classes, [1, 1], activation_fn=None,
normalizer_fn=None,
weights_initializer=trunc_normal(0.001),
padding='VALID', scope='Conv2d_2b_1x1')
if spatial_squeeze:
aux_logits = tf.squeeze(aux_logits, [1, 2], name='SpatialSqueeze')
end_points['AuxLogits'] = aux_logits
with tf.variable_scope('Logits'):
net = slim.avg_pool2d(net, [8, 8], padding='VALID', scope='AvgPool_1a_8x8')
net = slim.dropout(net, keep_prob=dropout_keep_prob, scope='Dropout_1b')
end_points['PreLogits'] = net
logits = slim.conv2d(net, num_classes, [1, 1], activation_fn=None,
normalizer_fn=None, scope='Conv2d_1c_1x1')
if spatial_squeeze:
logits = tf.squeeze(logits, [1, 2], name='SpatialSqueeze')
end_points['Logits'] = logits
end_points['Predictions'] = prdiction_fn(logits, scope='Predictions')
return logits, end_points
'''
层数，卷积核的尺寸，池化的位置，步长的大小，factorization使用的时机，分支的设计
'''
# 测试性能
num_batches = 100
batch_size = 1
height, width = 299, 299
inputs = tf.random_normal((batch_size, height, width, 3))
def time_tensorflow_run(session, target, info_string):
num_steps_burn_in = 10 # 考虑10次迭代之后的计算时间
total_duration = 0.0
total_duration_squared = 0.0
for i in range(num_batches + num_steps_burn_in):
start_time = time.time()
_ = session.run(target)
duration = time.time() - start_time
if i >= num_steps_burn_in:
if not i % 10:
print('%s: step %d, duration = %.3f' % (datetime.now(), i - num_steps_burn_in, duration))
total_duration += duration
total_duration_squared += duration * duration
mn = total_duration / num_batches
vr = total_duration_squared /num_batches - mn * mn
sd = math.sqrt(vr)
print('%s: %s across %d steps, %.3f +/- %.3f sec / batch' % (datetime.now(), info_string, num_batches, mn, sd))
with tf.device('/cpu:0'):
with slim.arg_scope(inception_v3_arg_scope()):
logits, end_points = inception_v3(inputs, is_training= False)
init = tf.global_variables_initializer()
with tf.Session() as sess:
sess.run(init)
time_tensorflow_run(sess, logits, 'Forward')
4655
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
