【Keras-Inception-resnet v2】CIFAR-10

最新推荐文章于 2023-11-21 16:11:09 发布
最新推荐文章于 2023-11-21 16:11:09 发布
阅读量2.8k 收藏 19
点赞数 3
分类专栏： PyTorch/Keras/Caffe/TensroFlow 文章标签： Keras Tensorboard Inception-resnet-v2
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/bryant_meng/article/details/85883808
版权

系列连载目录

  • 请查看博客 《Paper》 4.1 小节 【Keras】Classification in CIFAR-10 系列连载

学习借鉴

参考

代码下载

硬件

  • TITAN XP

文章目录

1 理论基础

参考【Inception-v4】《Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning》

2 Inception-resnet-v2 代码实现

[D] Why aren’t Inception-style networks successful on CIFAR-10/100?
在这里插入图片描述
在这里插入图片描述
图片来源 https://www.zhihu.com/question/50370954

2.1 Inception_resnet_v2

1）导入库，设置好超参数

import os  
os.environ["CUDA_DEVICE_ORDER"]="PCI_BUS_ID"   
os.environ["CUDA_VISIBLE_DEVICES"]="3" 

import keras
import numpy as np
import math

from keras.datasets import cifar10
from keras.layers import Conv2D, MaxPooling2D, AveragePooling2D, ZeroPadding2D, GlobalAveragePooling2D
from keras.layers import Flatten, Dense, Dropout,BatchNormalization,Activation, Convolution2D, add
from keras.models import Model
from keras.layers import Input, concatenate
from keras import optimizers, regularizers
from keras.preprocessing.image import ImageDataGenerator
from keras.initializers import he_normal
from keras.callbacks import LearningRateScheduler, TensorBoard, ModelCheckpoint

num_classes        = 10
batch_size         = 64         # 64 or 32 or other
epochs             = 300
iterations         = 782       
USE_BN=True
DROPOUT=0.2 # keep 80%
CONCAT_AXIS=3
weight_decay=1e-4
DATA_FORMAT='channels_last' # Theano:'channels_first' Tensorflow:'channels_last'

log_filepath  = './inception_resnet_v2'

2）数据预处理并设置 learning schedule

def color_preprocessing(x_train,x_test):
    x_train = x_train.astype('float32')
    x_test = x_test.astype('float32')
    mean = [125.307, 122.95, 113.865]
    std  = [62.9932, 62.0887, 66.7048]
    for i in range(3):
        x_train[:,:,:,i] = (x_train[:,:,:,i] - mean[i]) / std[i]
        x_test[:,:,:,i] = (x_test[:,:,:,i] - mean[i]) / std[i]
    return x_train, x_test

def scheduler(epoch):
    if epoch < 100:
        return 0.01
    if epoch < 200:
        return 0.001
    return 0.0001

# load data
(x_train, y_train), (x_test, y_test) = cifar10.load_data()
y_train = keras.utils.to_categorical(y_train, num_classes)
y_test  = keras.utils.to_categorical(y_test, num_classes)
x_train, x_test = color_preprocessing(x_train, x_test)

3）定义网络结构

  • Factorization into smaller convolutions
    eg：3×3 → 3×1 + 1×3
def conv_block(x, nb_filter, nb_row, nb_col, border_mode='same', subsample=(1,1), bias=False):
    x = Convolution2D(nb_filter, nb_row, nb_col, subsample=subsample, border_mode=border_mode, bias=bias,
                     init="he_normal",dim_ordering='tf',W_regularizer=regularizers.l2(weight_decay))(x)
    x = BatchNormalization(momentum=0.9, epsilon=1e-5)(x)
    x = Activation('relu')(x)
    return x
  • stem（图2右边 in paper，同 inception_v4，庞然大物）
    在这里插入图片描述
    上图中，V 表示 padding 为 valid，没有标注的表示 same， 因为数据集分辨率的不同，如下代码全部用 same 表示：
def create_stem(img_input,concat_axis):
    x = Conv2D(32,kernel_size=(3,3),strides=(2,2),padding='same',
               kernel_initializer="he_normal",kernel_regularizer=regularizers.l2(weight_decay))(img_input)
    x = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(x))
    x = Conv2D(32,kernel_size=(3,3),strides=(1,1),padding='same',
               kernel_initializer="he_normal",kernel_regularizer=regularizers.l2(weight_decay))(x)
    x = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(x))    
    x = Conv2D(64,kernel_size=(3,3),strides=(1,1),padding='same',
               kernel_initializer="he_normal",kernel_regularizer=regularizers.l2(weight_decay))(x)
    x = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(x))
    # stem1
    x_1 = Conv2D(96,kernel_size=(3,3),strides=(2,2),padding='same',
               kernel_initializer="he_normal",kernel_regularizer=regularizers.l2(weight_decay))(x)
    x_1 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(x_1))
    x_2 = MaxPooling2D(pool_size=(3,3),strides=2,padding='same',data_format=DATA_FORMAT)(x) 
    x = concatenate([x_1,x_2],axis=concat_axis)
    # stem2
    x_1 = Conv2D(64,kernel_size=(1,1),strides=(1,1),padding='same',
                 kernel_initializer="he_normal",kernel_regularizer=regularizers.l2(weight_decay))(x)
    x_1 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(x_1))
    x_1 = Conv2D(96,kernel_size=(3,3),strides=(1,1),padding='same',
                 kernel_initializer="he_normal",kernel_regularizer=regularizers.l2(weight_decay))(x_1)
    x_1 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(x_1))
    
    x_2 =  Conv2D(96,kernel_size=(1,1),strides=(1,1),padding='same',
                  kernel_initializer="he_normal",kernel_regularizer=regularizers.l2(weight_decay))(x)
    x_2 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(x_2))
    x_2 = conv_block(x_2,64,1,7)
    x_2 = conv_block(x_2,64,7,1)
    x_2 =  Conv2D(96,kernel_size=(3,3),strides=(1,1),padding='same',
                  kernel_initializer="he_normal",kernel_regularizer=regularizers.l2(weight_decay))(x_2) 
    x_2 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(x_2))
    x = concatenate([x_1,x_2],axis=concat_axis)
    # stem3
    x_1 = Conv2D(192,kernel_size=(3,3),strides=(2,2),padding='same',
                 kernel_initializer="he_normal",kernel_regularizer=regularizers.l2(weight_decay))(x)
    x_1 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(x_1))
    x_2 = MaxPooling2D(pool_size=(3,3),strides=2,padding='same',data_format=DATA_FORMAT)(x) 
    x = concatenate([x_1,x_2],axis=concat_axis)
    return x
  • inception module A（图5左边 in paper，与 inception_resnet_v1 形同神不同——形式一样，参数不一样）
    在这里插入图片描述
def inception_A(x,params,concat_axis,padding='same',data_format=DATA_FORMAT,scale_residual=False,use_bias=True,kernel_initializer="he_normal",bias_initializer='zeros',kernel_regularizer=None,bias_regularizer=None,activity_regularizer=None,kernel_constraint=None,bias_constraint=None,weight_decay=weight_decay):
    (branch1,branch2,branch3,branch4)=params
    if weight_decay:
        kernel_regularizer=regularizers.l2(weight_decay)
        bias_regularizer=regularizers.l2(weight_decay)
    else:
        kernel_regularizer=None
        bias_regularizer=None
    #1x1
    pathway1=Conv2D(filters=branch1[0],kernel_size=(1,1),strides=1,padding=padding,data_format=data_format,use_bias=use_bias,kernel_initializer=kernel_initializer,bias_initializer=bias_initializer,kernel_regularizer=kernel_regularizer,bias_regularizer=bias_regularizer,activity_regularizer=activity_regularizer,kernel_constraint=kernel_constraint,bias_constraint=bias_constraint)(x)
    pathway1 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(pathway1))
    #1x1->3x3
    pathway2=Conv2D(filters=branch2[0],kernel_size=(1,1),strides=1,padding=padding,data_format=data_format,use_bias=use_bias,kernel_initializer=kernel_initializer,bias_initializer=bias_initializer,kernel_regularizer=kernel_regularizer,bias_regularizer=bias_regularizer,activity_regularizer=activity_regularizer,kernel_constraint=kernel_constraint,bias_constraint=bias_constraint)(x)
    pathway2 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(pathway2))
    pathway2=Conv2D(filters=branch2[1],kernel_size=(3,3),strides=1,padding=padding,data_format=data_format,use_bias=use_bias,kernel_initializer=kernel_initializer,bias_initializer=bias_initializer,kernel_regularizer=kernel_regularizer,bias_regularizer=bias_regularizer,activity_regularizer=activity_regularizer,kernel_constraint=kernel_constraint,bias_constraint=bias_constraint)(pathway2)
    pathway2 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(pathway2))
    #1x1->3x3->3x3
    pathway3=Conv2D(filters=branch3[0],kernel_size=(1,1),strides=1,padding=padding,data_format=data_format,use_bias=use_bias,kernel_initializer=kernel_initializer,bias_initializer=bias_initializer,kernel_regularizer=kernel_regularizer,bias_regularizer=bias_regularizer,activity_regularizer=activity_regularizer,kernel_constraint=kernel_constraint,bias_constraint=bias_constraint)(x)
    pathway3 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(pathway3))
    pathway3=Conv2D(filters=branch3[1],kernel_size=(3,3),strides=1,padding=padding,data_format=data_format,use_bias=use_bias,kernel_initializer=kernel_initializer,bias_initializer=bias_initializer,kernel_regularizer=kernel_regularizer,bias_regularizer=bias_regularizer,activity_regularizer=activity_regularizer,kernel_constraint=kernel_constraint,bias_constraint=bias_constraint)(pathway3)
    pathway3 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(pathway3))
    pathway3=Conv2D(filters=branch3[1],kernel_size=(3,3),strides=1,padding=padding,data_format=data_format,use_bias=use_bias,kernel_initializer=kernel_initializer,bias_initializer=bias_initializer,kernel_regularizer=kernel_regularizer,bias_regularizer=bias_regularizer,activity_regularizer=activity_regularizer,kernel_constraint=kernel_constraint,bias_constraint=bias_constraint)(pathway3)
    pathway3 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(pathway3))
    # concatenate
    pathway_123 = concatenate([pathway1,pathway2,pathway3],axis=concat_axis)
    pathway_123 = Conv2D(branch4[0],kernel_size=(1,1),strides=(1,1),padding='same',activation = 'linear',
                         kernel_initializer="he_normal",kernel_regularizer=regularizers.l2(weight_decay))(pathway_123)
    if scale_residual: 
        x = Lambda(lambda p: p * 0.1)(x)
    return add([x,pathway_123])

def reduce_A(x,params,concat_axis,padding='same',data_format=DATA_FORMAT,use_bias=True,kernel_initializer="he_normal",bias_initializer='zeros',kernel_regularizer=None,bias_regularizer=None,activity_regularizer=None,kernel_constraint=None,bias_constraint=None,weight_decay=weight_decay):
    (branch1,branch2)=params
    if weight_decay:
        kernel_regularizer=regularizers.l2(weight_decay)
        bias_regularizer=regularizers.l2(weight_decay)
    else:
        kernel_regularizer=None
        bias_regularizer=None
    #1x1
    pathway1 = Conv2D(filters=branch1[0],kernel_size=(3,3),strides=2,padding=padding,data_format=data_format,use_bias=use_bias,kernel_initializer=kernel_initializer,bias_initializer=bias_initializer,kernel_regularizer=kernel_regularizer,bias_regularizer=bias_regularizer,activity_regularizer=activity_regularizer,kernel_constraint=kernel_constraint,bias_constraint=bias_constraint)(x)
    pathway1 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(pathway1))
    #1x1->3x3->3x3
    pathway2 = Conv2D(filters=branch2[0],kernel_size=(1,1),strides=1,padding=padding,data_format=data_format,use_bias=use_bias,kernel_initializer=kernel_initializer,bias_initializer=bias_initializer,kernel_regularizer=kernel_regularizer,bias_regularizer=bias_regularizer,activity_regularizer=activity_regularizer,kernel_constraint=kernel_constraint,bias_constraint=bias_constraint)(x)
    pathway2 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(pathway2))
    pathway2 = Conv2D(filters=branch2[1],kernel_size=(3,3),strides=1,padding=padding,data_format=data_format,use_bias=use_bias,kernel_initializer=kernel_initializer,bias_initializer=bias_initializer,kernel_regularizer=kernel_regularizer,bias_regularizer=bias_regularizer,activity_regularizer=activity_regularizer,kernel_constraint=kernel_constraint,bias_constraint=bias_constraint)(pathway2)
    pathway2 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(pathway2))
    pathway2 = Conv2D(filters=branch2[2],kernel_size=(3,3),strides=2,padding=padding,data_format=data_format,use_bias=use_bias,kernel_initializer=kernel_initializer,bias_initializer=bias_initializer,kernel_regularizer=kernel_regularizer,bias_regularizer=bias_regularizer,activity_regularizer=activity_regularizer,kernel_constraint=kernel_constraint,bias_constraint=bias_constraint)(pathway2)
    pathway2 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(pathway2))
    # max pooling
    pathway3 = MaxPooling2D(pool_size=(3,3),strides=2,padding=padding,data_format=DATA_FORMAT)(x)
    return concatenate([pathway1,pathway2,pathway3],axis=concat_axis)
  • inception module B（图5中间 in paper，与 inception_resnet_v1 形同神不同——形式一样，参数不一样）
    在这里插入图片描述
def inception_B(x,params,concat_axis,padding='same',data_format=DATA_FORMAT,scale_residual=False,use_bias=True,kernel_initializer="he_normal",bias_initializer='zeros',kernel_regularizer=None,bias_regularizer=None,activity_regularizer=None,kernel_constraint=None,bias_constraint=None,weight_decay=weight_decay):
    (branch1,branch2,branch3)=params
    if weight_decay:
        kernel_regularizer=regularizers.l2(weight_decay)
        bias_regularizer=regularizers.l2(weight_decay)
    else:
        kernel_regularizer=None
        bias_regularizer=None
    #1x1
    pathway1=Conv2D(filters=branch1[0],kernel_size=(1,1),strides=1,padding=padding,data_format=data_format,use_bias=use_bias,kernel_initializer=kernel_initializer,bias_initializer=bias_initializer,kernel_regularizer=kernel_regularizer,bias_regularizer=bias_regularizer,activity_regularizer=activity_regularizer,kernel_constraint=kernel_constraint,bias_constraint=bias_constraint)(x)
    pathway1 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(pathway1))
    #1x1->1x7->7x1
    pathway2=Conv2D(filters=branch2[0],kernel_size=(1,1),strides=1,padding=padding,data_format=data_format,use_bias=use_bias,kernel_initializer=kernel_initializer,bias_initializer=bias_initializer,kernel_regularizer=kernel_regularizer,bias_regularizer=bias_regularizer,activity_regularizer=activity_regularizer,kernel_constraint=kernel_constraint,bias_constraint=bias_constraint)(x)
    pathway2 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(pathway2))
    pathway2 = conv_block(pathway2,branch2[1],1,7)
    pathway2 = conv_block(pathway2,branch2[2],7,1)
    # concatenate
    pathway_12 = concatenate([pathway1,pathway2],axis=concat_axis)
    pathway_12 = Conv2D(branch3[0],kernel_size=(1,1),strides=(1,1),padding='same',activation = 'linear',
                        kernel_initializer="he_normal",kernel_regularizer=regularizers.l2(weight_decay))(pathway_12)
    if scale_residual: 
        x = Lambda(lambda p: p * 0.1)(x)
    return add([x,pathway_12])
  • reduction B（同 inception_resnet_v1）
    在这里插入图片描述
def reduce_B(x,params,concat_axis,padding='same',data_format=DATA_FORMAT,use_bias=True,kernel_initializer="he_normal",bias_initializer='zeros',kernel_regularizer=None,bias_regularizer=None,activity_regularizer=None,kernel_constraint=None,bias_constraint=None,weight_decay=weight_decay):
    (branch1,branch2,branch3)=params
    if weight_decay:
        kernel_regularizer=regularizers.l2(weight_decay)
        bias_regularizer=regularizers.l2(weight_decay)
    else:
        kernel_regularizer=None
        bias_regularizer=None
    #1x1->3x3
    pathway1 = Conv2D(filters=branch1[0],kernel_size=(1,1),strides=1,padding=padding,data_format=data_format,use_bias=use_bias,kernel_initializer=kernel_initializer,bias_initializer=bias_initializer,kernel_regularizer=kernel_regularizer,bias_regularizer=bias_regularizer,activity_regularizer=activity_regularizer,kernel_constraint=kernel_constraint,bias_constraint=bias_constraint)(x)
    pathway1 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(pathway1)) 
    pathway1 = Conv2D(filters=branch1[1],kernel_size=(3,3),strides=2,padding=padding,data_format=data_format,use_bias=use_bias,kernel_initializer=kernel_initializer,bias_initializer=bias_initializer,kernel_regularizer=kernel_regularizer,bias_regularizer=bias_regularizer,activity_regularizer=activity_regularizer,kernel_constraint=kernel_constraint,bias_constraint=bias_constraint)(pathway1)
    pathway1 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(pathway1))
    #1x1->3x3
    pathway2 = Conv2D(filters=branch2[0],kernel_size=(1,1),strides=1,padding=padding,data_format=data_format,use_bias=use_bias,kernel_initializer=kernel_initializer,bias_initializer=bias_initializer,kernel_regularizer=kernel_regularizer,bias_regularizer=bias_regularizer,activity_regularizer=activity_regularizer,kernel_constraint=kernel_constraint,bias_constraint=bias_constraint)(x)
    pathway2 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(pathway2)) 
    pathway2 = Conv2D(filters=branch2[1],kernel_size=(3,3),strides=2,padding=padding,data_format=data_format,use_bias=use_bias,kernel_initializer=kernel_initializer,bias_initializer=bias_initializer,kernel_regularizer=kernel_regularizer,bias_regularizer=bias_regularizer,activity_regularizer=activity_regularizer,kernel_constraint=kernel_constraint,bias_constraint=bias_constraint)(pathway2)
    pathway2 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(pathway2))
    #1x1->3x3->3x3
    pathway3 = Conv2D(filters=branch3[0],kernel_size=(1,1),strides=1,padding=padding,data_format=data_format,use_bias=use_bias,kernel_initializer=kernel_initializer,bias_initializer=bias_initializer,kernel_regularizer=kernel_regularizer,bias_regularizer=bias_regularizer,activity_regularizer=activity_regularizer,kernel_constraint=kernel_constraint,bias_constraint=bias_constraint)(x)
    pathway3 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(pathway3))
    pathway3 = Conv2D(filters=branch3[1],kernel_size=(3,3),strides=1,padding=padding,data_format=data_format,use_bias=use_bias,kernel_initializer=kernel_initializer,bias_initializer=bias_initializer,kernel_regularizer=kernel_regularizer,bias_regularizer=bias_regularizer,activity_regularizer=activity_regularizer,kernel_constraint=kernel_constraint,bias_constraint=bias_constraint)(pathway3)
    pathway3 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(pathway3))
    pathway3 = Conv2D(filters=branch3[2],kernel_size=(3,3),strides=2,padding=padding,data_format=data_format,use_bias=use_bias,kernel_initializer=kernel_initializer,bias_initializer=bias_initializer,kernel_regularizer=kernel_regularizer,bias_regularizer=bias_regularizer,activity_regularizer=activity_regularizer,kernel_constraint=kernel_constraint,bias_constraint=bias_constraint)(pathway3)
    pathway3 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(pathway3))
    # max pooling
    pathway4 = MaxPooling2D(pool_size=(3,3),strides=2,padding=padding,data_format=DATA_FORMAT)(x)
    return concatenate([pathway1,pathway2,pathway3,pathway4],axis=concat_axis)
  • inception module C（图5右边 in paper，与 inception_resnet_v1 形同神不同——形式一样，参数不一样）
    在这里插入图片描述
def inception_C(x,params,concat_axis,padding='same',data_format=DATA_FORMAT,scale_residual=False,use_bias=True,kernel_initializer="he_normal",bias_initializer='zeros',kernel_regularizer=None,bias_regularizer=None,activity_regularizer=None,kernel_constraint=None,bias_constraint=None,weight_decay=weight_decay):
    (branch1,branch2,branch3)=params
    if weight_decay:
        kernel_regularizer=regularizers.l2(weight_decay)
        bias_regularizer=regularizers.l2(weight_decay)
    else:
        kernel_regularizer=None
        bias_regularizer=None
    #1x1
    pathway1=Conv2D(filters=branch1[0],kernel_size=(1,1),strides=1,padding=padding,data_format=data_format,use_bias=use_bias,kernel_initializer=kernel_initializer,bias_initializer=bias_initializer,kernel_regularizer=kernel_regularizer,bias_regularizer=bias_regularizer,activity_regularizer=activity_regularizer,kernel_constraint=kernel_constraint,bias_constraint=bias_constraint)(x)
    pathway1 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(pathway1))
    #1x1->1x3->3x1
    pathway2=Conv2D(filters=branch2[0],kernel_size=(1,1),strides=1,padding=padding,data_format=data_format,use_bias=use_bias,kernel_initializer=kernel_initializer,bias_initializer=bias_initializer,kernel_regularizer=kernel_regularizer,bias_regularizer=bias_regularizer,activity_regularizer=activity_regularizer,kernel_constraint=kernel_constraint,bias_constraint=bias_constraint)(x)
    pathway2 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(pathway2))
    pathway2 = conv_block(pathway2,branch2[1],1,3)
    pathway2 = conv_block(pathway2,branch2[2],3,1)
    # concatenate
    pathway_12 = concatenate([pathway1,pathway2],axis=concat_axis)
    pathway_12 = Conv2D(branch3[0],kernel_size=(1,1),strides=(1,1),padding='same',activation = 'linear',
                        kernel_initializer="he_normal",kernel_regularizer=regularizers.l2(weight_decay))(pathway_12)
    if scale_residual: 
        x = Lambda(lambda p: p * 0.1)(x)
    return add([x,pathway_12])

4）搭建网络
用 3）中设计好的模块来搭建网络
整体结构（图6左边 in paper，同 inception_resnet_v1，不过 inception module 往往是10A，20B，10C的组合，替换5A，10B，5C的组合）
在这里插入图片描述

def create_model(img_input):
    # stem
    x = create_stem(img_input,concat_axis=CONCAT_AXIS)
    # 5 x inception_A
    for _ in range(10):
        x=inception_A(x,params=[(32,),(32,32),(32,48,64),(384,)],concat_axis=CONCAT_AXIS)
    # reduce A
    x=reduce_A(x,params=[(384,),(256,256,384)],concat_axis=CONCAT_AXIS) # 768
    # 10 x inception_B
    for _ in range(20):
        x=inception_B(x,params=[(192,),(128,160,192),(1152,)],concat_axis=CONCAT_AXIS)
    # reduce B
    x=reduce_B(x,params=[(256,384),(256,288),(256,288,320)],concat_axis=CONCAT_AXIS) # 1280
    # 5 x inception_C
    for _ in range(10):
        x=inception_C(x,params=[(192,),(192,224,256),(2144,)],concat_axis=CONCAT_AXIS)
    x=GlobalAveragePooling2D()(x)
    x=Dropout(DROPOUT)(x)
    x = Dense(num_classes,activation='softmax',kernel_initializer="he_normal",
              kernel_regularizer=regularizers.l2(weight_decay))(x)
    return x

5）生成模型

img_input=Input(shape=(32,32,3))
output = create_model(img_input)
model=Model(img_input,output)
model.summary()

Total params: 53,123,242
Trainable params: 53,066,154
Non-trainable params: 57,088

  • Inception_v4
    Total params: 41,256,266

6）开始训练

# set optimizer
sgd = optimizers.SGD(lr=.1, momentum=0.9, nesterov=True)
model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])

# set callback
tb_cb = TensorBoard(log_dir=log_filepath, histogram_freq=0)
change_lr = LearningRateScheduler(scheduler)
cbks = [change_lr,tb_cb]

# set data augmentation
datagen = ImageDataGenerator(horizontal_flip=True,
                             width_shift_range=0.125,
                             height_shift_range=0.125,
                             fill_mode='constant',cval=0.)
datagen.fit(x_train)

# start training
model.fit_generator(datagen.flow(x_train, y_train,batch_size=batch_size),
                    steps_per_epoch=iterations,
                    epochs=epochs,
                    callbacks=cbks,
                    validation_data=(x_test, y_test))
model.save('inception_resnet_v2.h5')

7）结果分析
training accuracy 和 training loss
在这里插入图片描述
在这里插入图片描述在这里插入图片描述

  • accuracy
    在这里插入图片描述
  • loss
    在这里插入图片描述
    这……多半是废了
    在这里插入图片描述

test accuracy 和 test loss
在这里插入图片描述
在这里插入图片描述在这里插入图片描述

  • accuracy
    在这里插入图片描述
  • loss
    在这里插入图片描述
    …………
    没有学到东西

2.2 Inception_resnet_v2_slim

把 Inception_resnet_v2 中 stern 结构直接替换成一个卷积，inception 结构不变，因为stern结果会把原图降到1/8的分辨率，对于 ImageNet 还行，CIFRA-10的话有些吃不消了

代码修改部分如下：

def create_stem(img_input,concat_axis):
    x = Conv2D(384,kernel_size=(3,3),strides=(1,1),padding='same',
               kernel_initializer="he_normal",kernel_regularizer=regularizers.l2(weight_decay))(img_input)
    x = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(x))  
    return x

其他部分同 Inception_resnet_v2
参数量如下（少了一些）：

Total params: 52,507,722
Trainable params: 52,451,658
Non-trainable params: 56,064

  • inception_resnet_v2
    Total params: 53,123,242
  • Inception_v4
    Total params: 41,256,266

结果分析如下：
training accuracy 和 training loss
在这里插入图片描述
在这里插入图片描述在这里插入图片描述

  • accuracy
    在这里插入图片描述
  • loss
    在这里插入图片描述

效果飙升，大网络（容量）就是猛，能在训练集上达到100%，就是很难理解一开始精度的停滞不前！大概在第 50 个 epoch 的时候，开始学东西了！先留下一个疑惑。

test accuracy 和 test loss
在这里插入图片描述
在这里插入图片描述在这里插入图片描述

  • accuracy
    在这里插入图片描述
  • loss
    在这里插入图片描述

93%+，长舒一口气，调通了

2.3 Inception_resnet_v2_slim_5105

整体结构（图6左边 in paper，同 inception_resnet_v1，不过 inception module 往往是10A，20B，10C的组合，替换5A，10B，5C的组合），这里，我们采用 5A、10B、5C 的方式
在这里插入图片描述

代码修改如下

def create_model(img_input):
    # stem
    x = create_stem(img_input,concat_axis=CONCAT_AXIS)
    # 5 x inception_A
    for _ in range(5):
        x=inception_A(x,params=[(32,),(32,32),(32,48,64),(384,)],concat_axis=CONCAT_AXIS)
    # reduce A
    x=reduce_A(x,params=[(384,),(256,256,384)],concat_axis=CONCAT_AXIS) # 768
    # 10 x inception_B
    for _ in range(10):
        x=inception_B(x,params=[(192,),(128,160,192),(1152,)],concat_axis=CONCAT_AXIS)
    # reduce B
    x=reduce_B(x,params=[(256,384),(256,288),(256,288,320)],concat_axis=CONCAT_AXIS) # 1280
    # 5 x inception_C
    for _ in range(5):
        x=inception_C(x,params=[(192,),(192,224,256),(2144,)],concat_axis=CONCAT_AXIS)
    x=GlobalAveragePooling2D()(x)
    x=Dropout(DROPOUT)(x)
    x = Dense(num_classes,activation='softmax',kernel_initializer="he_normal",
              kernel_regularizer=regularizers.l2(weight_decay))(x)
    return x

其它代码同 inception_resnet_v2_slim

参数量如下所示：

Total params: 29,692,042
Trainable params: 29,660,298
Non-trainable params: 31,744

  • inception_resnet_v2
    Total params: 53,123,242
  • inception_resnet_v2_slim
    Total params: 52,507,722
  • Inception_v4
    Total params: 41,256,266

结果分析如下：
training accuracy 和 training loss
在这里插入图片描述
在这里插入图片描述
在这里插入图片描述

  • accuracy
    在这里插入图片描述
  • loss
    在这里插入图片描述

一开始精度的停滞不前！大概在第 64 个 epoch 的时候，开始学东西了！先留下一个疑惑。

test accuracy 和 test loss
在这里插入图片描述
在这里插入图片描述在这里插入图片描述

  • accuracy
    在这里插入图片描述
  • loss
    在这里插入图片描述

大部分的时候， inception_resnet_v2_slim_5105 的效果要好一些，loss 也更小

2.4 Inception_resnet_v2_slim_5105_scale

在这里插入图片描述
论文中图7，之前 inception_resnet_v2 代码中也有这种结构
也即如下部分代码：

 if scale_residual: 
     x = Lambda(lambda p: p * 0.1)(x)

现在把这个功能激活，修改代码如下 scale_residual=True

def create_model(img_input):
    # stem
    x = create_stem(img_input,concat_axis=CONCAT_AXIS)
    # 5 x inception_A
    for _ in range(5):
        x=inception_A(x,params=[(32,),(32,32),(32,48,64),(384,)],concat_axis=CONCAT_AXIS,scale_residual=True)
    # reduce A
    x=reduce_A(x,params=[(384,),(256,256,384)],concat_axis=CONCAT_AXIS) # 768
    # 10 x inception_B
    for _ in range(10):
        x=inception_B(x,params=[(192,),(128,160,192),(1152,)],concat_axis=CONCAT_AXIS,scale_residual=True)
    # reduce B
    x=reduce_B(x,params=[(256,384),(256,288),(256,288,320)],concat_axis=CONCAT_AXIS) # 1280
    # 5 x inception_C
    for _ in range(5):
        x=inception_C(x,params=[(192,),(192,224,256),(2144,)],concat_axis=CONCAT_AXIS,scale_residual=True)
    x=GlobalAveragePooling2D()(x)
    x=Dropout(DROPOUT)(x)
    x = Dense(num_classes,activation='softmax',kernel_initializer="he_normal",
              kernel_regularizer=regularizers.l2(weight_decay))(x)
    return x

其它代码同 inception_resnet_v2_slim_5105

参数量如下所示（同 inception_resnet_v2_slim_5105）：

Total params: 29,692,042
Trainable params: 29,660,298
Non-trainable params: 31,744

  • inception_resnet_v2
    Total params: 53,123,242
  • inception_resnet_v2_slim
    Total params: 52,507,722
  • inception_resnet_v2_slim_5105
    Total params: 29,692,042
  • Inception_v4
    Total params: 41,256,266

结果分析如下：
training accuracy 和 training loss
在这里插入图片描述
在这里插入图片描述在这里插入图片描述

  • accuracy
    在这里插入图片描述
  • loss
    在这里插入图片描述

哇，终于消除了一开始精度的停滞不前的情况！

test accuracy 和 test loss
在这里插入图片描述
在这里插入图片描述 在这里插入图片描述

  • accuracy
    在这里插入图片描述
  • loss
    在这里插入图片描述
    可惜了，精度没有到 92%，有兴趣可以调调 hyper-parameters

3 总结

模型大小
在这里插入图片描述

inception 家族
在这里插入图片描述
在这里插入图片描述
在这里插入图片描述

bryant_meng
关注
  • 3
    点赞
  • 踩
  • 19
    收藏
    觉得还不错? 一键收藏
  • 2
    评论
专栏目录
Tensorflow Pretrain Inception-Resnet-V2
08-08
对Inception-Resnet-V2进行Pretrain。在Scene15数据集，亲测有效。
Tensorflow2.0 keras InceptionResNetV2代码实现
Forrest97的博客
05-16 3157
目录迁移学习自编代码定义卷积单元Stem模块InceptionRes A模块InceptionRes B模块InceptionRes C模块Reduction AReduction BBuild Inception-ResNet补充： 模型介绍请参看：博文 出处:Google AI 迁移学习 首先还是看看标准答案 import tensorflow as tf from tensorflow import keras base_model = keras.applications.Incept
第43步 深度学习图像识别：InceptionResnetV2建模（Tensorflow）
qq_30452897的博客
06-26 2747
第43步 深度学习图像识别：InceptionResnetV2建模（Tensorflow）
inception-resnet v2
Loong Cheng的博客
09-19 7623
inception-resnet v2 相对于inception-resnet v1而言，v2主要被设计来探索residual learning用于inception网络时所极尽可能带来的性能提升。因此它所用的inception 子网络并没有像v1中用的那样偷工减料。 首先下面为inception-resnet v2所使用的各个主要模块。 Inception-Resnet_v2所使用的各...
学习笔记：GoogLeNet Incepetion V2，V3
ROS学习VS图像处理
12-15 3759
上次整理了googlenet V1，V2和V3在同一篇文章里进行描述的，所以我们也在这里一起学习。 tensorflow发布了所有的模型 https://github.com/tensorflow/models/blob/master/slim/README.md#pre-trained-models 论文列表： [v1] Going Deeper wi
keras-inception-resnet-v2:使用Keras的Inception-ResNet v2模型（带有权重文件）
05-04
keras-inception-resnet-v2 使用Keras的Inception-ResNet v2模型（带有权重文件） 在python 3.6下使用tensorflow-gpu==1.15.3和Keras==2.2.5进行了测试（尽管存在很多弃用警告，因为此代码是在TF 1.15之前编写的）...
Inception-v4和Inception-ResNet（包括v1和v2）是深度卷积神经网络架构，特别设计用于图像分类任务
最新发布
03-28
根据论文中描述的架构，你需要将多个Inception模块（或Inception-ResNet模块）堆叠起来。 添加辅助分类器（可选）： 在某些Inception架构中，中间层会连接到辅助分类器。这些分类器在训练时提供额外的监督信号。 ...
inception_resnet_v2模型
06-02
在GitHub上下不下来，然后用校园网下的，传上来，给大家方便。
深度可量化：使用深度CNN和Inception-ResNet-v2（https：arxiv.orgabs1712.03400）的KerasTensorflow实现我们的论文灰度图像​​着色
02-06
我们论文的 * ， *和 * *作者的贡献相等 深度可量化是2017年Spring在DD2424中开发的一部分。 该代码使用内置和 。 如果发现有用，请考虑为其加注星标 :white_medium_star: ！ 目录 ...如果您发现Deep Koalarization...
Inception-v4, 在Keras中，起始 v4.起始 Resnet v1和 v2.zip
09-17
起始 Resnet v1和 v2 Keras中的起始使用函数API在Keras中实现 Inception-v4.起始- Resnet-v1和v2体系结构。 本文对这些体系结构的研究，在 "inception-v4.起始resnet和剩余连接对学习的影响"。模型被打印并显
keras resnet 训练自己的数据
04-30
keras resnet 迁移训练数据 和 读取数据
keras搬砖系列-inception-resnet-v2实现
Cutepingping的博客
03-15 4657
keras搬砖系列-inception-resnet-v2实现一、inception-resnet-v2是在inception v3的基础上进行演化，但是有着更高准确度，在目前的比赛中有着非常好的成绩。inception-resnet-v2的网络模型整体如下所示：其中stem部分网络结构如下所示：inception设计，并且conv也使用了7*1+1*7这种优化方式：其中inception-res...
【图像分类】【深度学习】【Pytorch版本】Inception-ResNet模型算法详解
yangyu0515的博客
11-21 982
【图像分类】【深度学习】【Pytorch版本】Inception-ResNet模型算法详解
【深度学习】Inception模型结构解析，关键词：Inception-v1、v2、v3、v4、Inception-ResNet-v1、Inception-ResNet-v2
Next_SummerAgain的博客
03-29 7325
【深度学习】Inception模型结构解析，关键词：Inception-v1、v2、v3、v4、Inception-ResNet-v1、Inception-ResNet-v2
经典卷积神经系列（Inception v1\v2\v3\v4、ResNet、ResNext、DenseNet、SENet）
weixin_42028608的博客
01-22 2504
经典卷积神经网络
Inception v1~v4和Inception-ResNet v1~v2总结
David Lee的博客
05-27 4454
Inception v1~v4和Inception-ResNet v1~v2总结
TensorFlow Keras 使用Inception-resnet-v2模型训练自己的分类数据集（含源码）
weixin_43640369的博客
07-26 4571
TensorFlow Keras 使用Inception-resnet-v2模型训练自己的分类数据集（含源码） 运行环境 TensorFlow 1.13.1 TensorFlow.Keras 2.2.4-tf 简单介绍 使用TensorFlow自带的Inception-resnet-v2模型训练自己的数据集。数据读取用的是TensorFlow自己的Dataset类，且无需转存成TFrecord...
keras inception_resnet_v2训练
jacke121的专栏
01-02 7794
keras inception_resnet_v2训练
【Keras-CNN】CIFAR-10
热门推荐
bryant_meng
07-23 1万+
参考 声明 CIFAR-10是由Alex Krizhevsky、Vinod Nair 与 Geoffrey Hinton 收集的一个用于图像识别的数据集，有10类，飞机、汽车、鸟、猫、鹿、狗、青蛙、马、船、卡车。与MNIST相比，色彩、颜色噪点较多，同一类物体大小不一、角度不同、颜色不同。 参考 【1】CIFAR-10官网 声明 声明：代码源于《Te...
keras-gpt-2
09-07
Keras-GPT-2是一个基于Keras框架的GPT-2模型实现。GPT-2是OpenAI（开放人工智能）公司于2019年发布的一种先进的自然语言处理模型，是大规模无监督训练的语言生成模型。 Keras-GPT-2是由开发者将GPT-2模型迁移到Keras框架下实现的工具。Keras是一个基于Python的深度学习库，提供了简单而高效的接口，常用于快速搭建和训练神经网络模型。通过将GPT-2模型移植到Keras框架下，可以更方便地使用和调整该模型。 Keras-GPT-2提供了一系列用于训练和生成文本的工具和函数。它的主要功能包括模型的搭建、训练和预测等。用户可以根据自己的需求，使用Keras-GPT-2生成各种类型的文本，如故事、对话或其他形式的自然语言文本。 Keras-GPT-2还可以用于各种自然语言处理任务，如机器翻译、文本摘要、情感分析等。用户可以根据自己的需求和数据特点，调整和优化Keras-GPT-2模型，以获得更好的性能和结果。 总之，Keras-GPT-2是一个基于Keras框架的GPT-2模型实现工具，便于用户使用和训练GPT-2模型，生成各种自然语言文本，并在自然语言处理任务中起到重要作用。

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论 2
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值