【Keras-Inception v3】CIFAR-10

最新推荐文章于 2023-12-24 18:04:04 发布
最新推荐文章于 2023-12-24 18:04:04 发布
阅读量1.5k 收藏 2
点赞数 2
分类专栏： PyTorch/Keras/Caffe/TensroFlow 文章标签： Keras Tensorboard Tensrorflow Inception v3 CIFAR-10
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/bryant_meng/article/details/85259193
版权

系列连载目录

  • 请查看博客 《Paper》 4.1 小节 【Keras】Classification in CIFAR-10 系列连载

学习借鉴

参考

代码下载
链接：https://pan.baidu.com/s/1ZiYlTnA4EsUeWAqj5zyKWw
提取码：vr59

硬件

  • TITAN XP

文章目录

1 理论基础

参考【BN】《Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift》
在这里插入图片描述

2 代码实现

[D] Why aren’t Inception-style networks successful on CIFAR-10/100?
在这里插入图片描述

2.1 Inception_v3

1）导入库，设置好超参数

import os  
os.environ["CUDA_DEVICE_ORDER"]="PCI_BUS_ID"   
os.environ["CUDA_VISIBLE_DEVICES"]="1" 

import keras
import numpy as np
import math

from keras.datasets import cifar10
from keras.layers import Conv2D, MaxPooling2D, AveragePooling2D, ZeroPadding2D, GlobalAveragePooling2D
from keras.layers import Flatten, Dense, Dropout,BatchNormalization,Activation, Convolution2D
from keras.models import Model
from keras.layers import Input, concatenate
from keras import optimizers, regularizers
from keras.preprocessing.image import ImageDataGenerator
from keras.initializers import he_normal
from keras.callbacks import LearningRateScheduler, TensorBoard, ModelCheckpoint

num_classes        = 10
batch_size         = 64         # 64 or 32 or other
epochs             = 300
iterations         = 782       
USE_BN=True
LRN2D_NORM = True
DROPOUT=0.2
CONCAT_AXIS=3
weight_decay=1e-4
DATA_FORMAT='channels_last' # Theano:'channels_first' Tensorflow:'channels_last'

log_filepath  = './inception_v3'

2）数据预处理并设置 learning schedule

def color_preprocessing(x_train,x_test):
    x_train = x_train.astype('float32')
    x_test = x_test.astype('float32')
    mean = [125.307, 122.95, 113.865]
    std  = [62.9932, 62.0887, 66.7048]
    for i in range(3):
        x_train[:,:,:,i] = (x_train[:,:,:,i] - mean[i]) / std[i]
        x_test[:,:,:,i] = (x_test[:,:,:,i] - mean[i]) / std[i]
    return x_train, x_test

def scheduler(epoch):
    if epoch < 100:
        return 0.01
    if epoch < 200:
        return 0.001
    return 0.0001

# load data
(x_train, y_train), (x_test, y_test) = cifar10.load_data()
y_train = keras.utils.to_categorical(y_train, num_classes)
y_test  = keras.utils.to_categorical(y_test, num_classes)
x_train, x_test = color_preprocessing(x_train, x_test)

3）定义网络结构

  • 3×3 → 3×1 + 1×3
def conv_block(x, nb_filter, nb_row, nb_col, border_mode='same', subsample=(1,1), bias=False):
    x = Convolution2D(nb_filter, nb_row, nb_col, subsample=subsample, border_mode=border_mode, bias=bias,
                     init="he_normal",dim_ordering='tf',W_regularizer=regularizers.l2(weight_decay))(x)
    x = BatchNormalization(momentum=0.9, epsilon=1e-5)(x)
    x = Activation('relu')(x)
    return x
  • Inception module 1
    average pooling
    在这里插入图片描述
def inception_module1(x,params,concat_axis,padding='same',data_format=DATA_FORMAT,use_bias=True,kernel_initializer="he_normal",bias_initializer='zeros',kernel_regularizer=None,bias_regularizer=None,activity_regularizer=None,kernel_constraint=None,bias_constraint=None,lrn2d_norm=LRN2D_NORM,weight_decay=weight_decay):
    (branch1,branch2,branch3,branch4)=params
    if weight_decay:
        kernel_regularizer=regularizers.l2(weight_decay)
        bias_regularizer=regularizers.l2(weight_decay)
    else:
        kernel_regularizer=None
        bias_regularizer=None
    #1x1
    pathway1=Conv2D(filters=branch1[0],kernel_size=(1,1),strides=1,padding=padding,data_format=data_format,use_bias=use_bias,kernel_initializer=kernel_initializer,bias_initializer=bias_initializer,kernel_regularizer=kernel_regularizer,bias_regularizer=bias_regularizer,activity_regularizer=activity_regularizer,kernel_constraint=kernel_constraint,bias_constraint=bias_constraint)(x)
    pathway1 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(pathway1))
    #1x1->3x3
    pathway2=Conv2D(filters=branch2[0],kernel_size=(1,1),strides=1,padding=padding,data_format=data_format,use_bias=use_bias,kernel_initializer=kernel_initializer,bias_initializer=bias_initializer,kernel_regularizer=kernel_regularizer,bias_regularizer=bias_regularizer,activity_regularizer=activity_regularizer,kernel_constraint=kernel_constraint,bias_constraint=bias_constraint)(x)
    pathway2 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(pathway2))
    pathway2=Conv2D(filters=branch2[1],kernel_size=(3,3),strides=1,padding=padding,data_format=data_format,use_bias=use_bias,kernel_initializer=kernel_initializer,bias_initializer=bias_initializer,kernel_regularizer=kernel_regularizer,bias_regularizer=bias_regularizer,activity_regularizer=activity_regularizer,kernel_constraint=kernel_constraint,bias_constraint=bias_constraint)(pathway2)
    pathway2 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(pathway2))
    #1x1->3x3+3x3
    pathway3=Conv2D(filters=branch3[0],kernel_size=(1,1),strides=1,padding=padding,data_format=data_format,use_bias=use_bias,kernel_initializer=kernel_initializer,bias_initializer=bias_initializer,kernel_regularizer=kernel_regularizer,bias_regularizer=bias_regularizer,activity_regularizer=activity_regularizer,kernel_constraint=kernel_constraint,bias_constraint=bias_constraint)(x)
    pathway3 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(pathway3))
    pathway3=Conv2D(filters=branch3[1],kernel_size=(3,3),strides=1,padding=padding,data_format=data_format,use_bias=use_bias,kernel_initializer=kernel_initializer,bias_initializer=bias_initializer,kernel_regularizer=kernel_regularizer,bias_regularizer=bias_regularizer,activity_regularizer=activity_regularizer,kernel_constraint=kernel_constraint,bias_constraint=bias_constraint)(pathway3)
    pathway3 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(pathway3))
    pathway3=Conv2D(filters=branch3[1],kernel_size=(3,3),strides=1,padding=padding,data_format=data_format,use_bias=use_bias,kernel_initializer=kernel_initializer,bias_initializer=bias_initializer,kernel_regularizer=kernel_regularizer,bias_regularizer=bias_regularizer,activity_regularizer=activity_regularizer,kernel_constraint=kernel_constraint,bias_constraint=bias_constraint)(pathway3)
    pathway3 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(pathway3))
    #3x3->1x1
    pathway4=AveragePooling2D(pool_size=(3,3),strides=1,padding=padding,data_format=DATA_FORMAT)(x)
    pathway4=Conv2D(filters=branch4[0],kernel_size=(1,1),strides=1,padding=padding,data_format=data_format,use_bias=use_bias,kernel_initializer=kernel_initializer,bias_initializer=bias_initializer,kernel_regularizer=kernel_regularizer,bias_regularizer=bias_regularizer,activity_regularizer=activity_regularizer,kernel_constraint=kernel_constraint,bias_constraint=bias_constraint)(pathway4)
    pathway4 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(pathway4))
    return concatenate([pathway1,pathway2,pathway3,pathway4],axis=concat_axis)
  • inception reduce 1
    max pooling
    在这里插入图片描述
    figure 10
    去掉 pathway 1 中的 1×1
def inception_reduce1(x,params,concat_axis,padding='same',data_format=DATA_FORMAT,use_bias=True,kernel_initializer="he_normal",bias_initializer='zeros',kernel_regularizer=None,bias_regularizer=None,activity_regularizer=None,kernel_constraint=None,bias_constraint=None,lrn2d_norm=LRN2D_NORM,weight_decay=weight_decay):
    (branch1,branch2)=params
    if weight_decay:
        kernel_regularizer=regularizers.l2(weight_decay)
        bias_regularizer=regularizers.l2(weight_decay)
    else:
        kernel_regularizer=None
        bias_regularizer=None
    #1x1
    pathway1 = Conv2D(filters=branch1[0],kernel_size=(3,3),strides=2,padding=padding,data_format=data_format,use_bias=use_bias,kernel_initializer=kernel_initializer,bias_initializer=bias_initializer,kernel_regularizer=kernel_regularizer,bias_regularizer=bias_regularizer,activity_regularizer=activity_regularizer,kernel_constraint=kernel_constraint,bias_constraint=bias_constraint)(x)
    pathway1 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(pathway1))
    #1x1->3x3+3x3
    pathway2 = Conv2D(filters=branch2[0],kernel_size=(1,1),strides=1,padding=padding,data_format=data_format,use_bias=use_bias,kernel_initializer=kernel_initializer,bias_initializer=bias_initializer,kernel_regularizer=kernel_regularizer,bias_regularizer=bias_regularizer,activity_regularizer=activity_regularizer,kernel_constraint=kernel_constraint,bias_constraint=bias_constraint)(x)
    pathway2 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(pathway2))
    pathway2 = Conv2D(filters=branch2[1],kernel_size=(3,3),strides=1,padding=padding,data_format=data_format,use_bias=use_bias,kernel_initializer=kernel_initializer,bias_initializer=bias_initializer,kernel_regularizer=kernel_regularizer,bias_regularizer=bias_regularizer,activity_regularizer=activity_regularizer,kernel_constraint=kernel_constraint,bias_constraint=bias_constraint)(pathway2)
    pathway2 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(pathway2))
    pathway2 = Conv2D(filters=branch2[1],kernel_size=(3,3),strides=2,padding=padding,data_format=data_format,use_bias=use_bias,kernel_initializer=kernel_initializer,bias_initializer=bias_initializer,kernel_regularizer=kernel_regularizer,bias_regularizer=bias_regularizer,activity_regularizer=activity_regularizer,kernel_constraint=kernel_constraint,bias_constraint=bias_constraint)(pathway2)
    pathway2 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(pathway2))
    #3x3->1x1
    pathway3 = MaxPooling2D(pool_size=(3,3),strides=2,padding=padding,data_format=DATA_FORMAT)(x)
    return concatenate([pathway1,pathway2,pathway3],axis=concat_axis)
  • Inception module 2
    average pooling
    在这里插入图片描述
def inception_module2(x,params,concat_axis,padding='same',data_format=DATA_FORMAT,use_bias=True,kernel_initializer="he_normal",bias_initializer='zeros',kernel_regularizer=None,bias_regularizer=None,activity_regularizer=None,kernel_constraint=None,bias_constraint=None,lrn2d_norm=LRN2D_NORM,weight_decay=weight_decay):
    (branch1,branch2,branch3,branch4)=params
    if weight_decay:
        kernel_regularizer=regularizers.l2(weight_decay)
        bias_regularizer=regularizers.l2(weight_decay)
    else:
        kernel_regularizer=None
        bias_regularizer=None
    #1x1
    pathway1=Conv2D(filters=branch1[0],kernel_size=(1,1),strides=1,padding=padding,data_format=data_format,use_bias=use_bias,kernel_initializer=kernel_initializer,bias_initializer=bias_initializer,kernel_regularizer=kernel_regularizer,bias_regularizer=bias_regularizer,activity_regularizer=activity_regularizer,kernel_constraint=kernel_constraint,bias_constraint=bias_constraint)(x)
    pathway1 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(pathway1))
    #1x1->1x7->7x1
    pathway2=Conv2D(filters=branch2[0],kernel_size=(1,1),strides=1,padding=padding,data_format=data_format,use_bias=use_bias,kernel_initializer=kernel_initializer,bias_initializer=bias_initializer,kernel_regularizer=kernel_regularizer,bias_regularizer=bias_regularizer,activity_regularizer=activity_regularizer,kernel_constraint=kernel_constraint,bias_constraint=bias_constraint)(x)
    pathway2 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(pathway2))
    pathway2 = conv_block(pathway2,branch2[1],1,7)
    pathway2 = conv_block(pathway2,branch2[2],7,1)
    #1x1->7x1->1x7->7x1->1x7
    pathway3=Conv2D(filters=branch3[0],kernel_size=(1,1),strides=1,padding=padding,data_format=data_format,use_bias=use_bias,kernel_initializer=kernel_initializer,bias_initializer=bias_initializer,kernel_regularizer=kernel_regularizer,bias_regularizer=bias_regularizer,activity_regularizer=activity_regularizer,kernel_constraint=kernel_constraint,bias_constraint=bias_constraint)(x)
    pathway3 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(pathway3))
    pathway3 = conv_block(pathway3,branch3[1],7,1)
    pathway3 = conv_block(pathway3,branch3[2],1,7)
    pathway3 = conv_block(pathway3,branch3[3],7,1)
    pathway3 = conv_block(pathway3,branch3[4],1,7)
    #3x3->1x1
    pathway4=AveragePooling2D(pool_size=(3,3),strides=1,padding=padding,data_format=DATA_FORMAT)(x)
    pathway4=Conv2D(filters=branch4[0],kernel_size=(1,1),strides=1,padding=padding,data_format=data_format,use_bias=use_bias,kernel_initializer=kernel_initializer,bias_initializer=bias_initializer,kernel_regularizer=kernel_regularizer,bias_regularizer=bias_regularizer,activity_regularizer=activity_regularizer,kernel_constraint=kernel_constraint,bias_constraint=bias_constraint)(pathway4)
    pathway4 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(pathway4))
    return concatenate([pathway1,pathway2,pathway3,pathway4],axis=concat_axis)
  • inception reduce 2
    max pooling
    在这里插入图片描述
    pathway 1：1×1（1）→3×3（2）
    pathway 2：1×1（1）→1×7（1）→7×1（1）→3×3（2）
    pathway 3：max pooling 3×3（2）
def inception_reduce2(x,params,concat_axis,padding='same',data_format=DATA_FORMAT,use_bias=True,kernel_initializer="he_normal",bias_initializer='zeros',kernel_regularizer=None,bias_regularizer=None,activity_regularizer=None,kernel_constraint=None,bias_constraint=None,lrn2d_norm=LRN2D_NORM,weight_decay=weight_decay):
    (branch1,branch2)=params
    if weight_decay:
        kernel_regularizer=regularizers.l2(weight_decay)
        bias_regularizer=regularizers.l2(weight_decay)
    else:
        kernel_regularizer=None
        bias_regularizer=None
    #1x1->3x3
    pathway1 = Conv2D(filters=branch1[0],kernel_size=(1,1),strides=1,padding=padding,data_format=data_format,use_bias=use_bias,kernel_initializer=kernel_initializer,bias_initializer=bias_initializer,kernel_regularizer=kernel_regularizer,bias_regularizer=bias_regularizer,activity_regularizer=activity_regularizer,kernel_constraint=kernel_constraint,bias_constraint=bias_constraint)(x)
    pathway1 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(pathway1)) 
    pathway1 = Conv2D(filters=branch1[1],kernel_size=(3,3),strides=2,padding=padding,data_format=data_format,use_bias=use_bias,kernel_initializer=kernel_initializer,bias_initializer=bias_initializer,kernel_regularizer=kernel_regularizer,bias_regularizer=bias_regularizer,activity_regularizer=activity_regularizer,kernel_constraint=kernel_constraint,bias_constraint=bias_constraint)(pathway1)
    pathway1 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(pathway1))
    #1x1->1x7->7x1->3x3
    pathway2 = Conv2D(filters=branch2[0],kernel_size=(1,1),strides=1,padding=padding,data_format=data_format,use_bias=use_bias,kernel_initializer=kernel_initializer,bias_initializer=bias_initializer,kernel_regularizer=kernel_regularizer,bias_regularizer=bias_regularizer,activity_regularizer=activity_regularizer,kernel_constraint=kernel_constraint,bias_constraint=bias_constraint)(x)
    pathway2 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(pathway2))
    pathway2 = conv_block(pathway2,branch2[1],1,7)
    pathway2 = conv_block(pathway2,branch2[2],7,1)
    pathway2 = Conv2D(filters=branch2[3],kernel_size=(3,3),strides=2,padding=padding,data_format=data_format,use_bias=use_bias,kernel_initializer=kernel_initializer,bias_initializer=bias_initializer,kernel_regularizer=kernel_regularizer,bias_regularizer=bias_regularizer,activity_regularizer=activity_regularizer,kernel_constraint=kernel_constraint,bias_constraint=bias_constraint)(pathway2)
    pathway2 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(pathway2))
    #3x3->1x1
    pathway3 = MaxPooling2D(pool_size=(3,3),strides=2,padding=padding,data_format=DATA_FORMAT)(x)
    return concatenate([pathway1,pathway2,pathway3],axis=concat_axis)
  • Inception module 3
    average pooling
    figure 7
    在这里插入图片描述
def inception_module3(x,params,concat_axis,padding='same',data_format=DATA_FORMAT,use_bias=True,kernel_initializer="he_normal",bias_initializer='zeros',kernel_regularizer=None,bias_regularizer=None,activity_regularizer=None,kernel_constraint=None,bias_constraint=None,lrn2d_norm=LRN2D_NORM,weight_decay=weight_decay):
    (branch1,branch2,branch3,branch4)=params
    if weight_decay:
        kernel_regularizer=regularizers.l2(weight_decay)
        bias_regularizer=regularizers.l2(weight_decay)
    else:
        kernel_regularizer=None
        bias_regularizer=None
    #1x1
    pathway1=Conv2D(filters=branch1[0],kernel_size=(1,1),strides=1,padding=padding,data_format=data_format,use_bias=use_bias,kernel_initializer=kernel_initializer,bias_initializer=bias_initializer,kernel_regularizer=kernel_regularizer,bias_regularizer=bias_regularizer,activity_regularizer=activity_regularizer,kernel_constraint=kernel_constraint,bias_constraint=bias_constraint)(x)
    pathway1 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(pathway1))
    #1x1->1x3+3x1
    pathway2=Conv2D(filters=branch2[0],kernel_size=(1,1),strides=1,padding=padding,data_format=data_format,use_bias=use_bias,kernel_initializer=kernel_initializer,bias_initializer=bias_initializer,kernel_regularizer=kernel_regularizer,bias_regularizer=bias_regularizer,activity_regularizer=activity_regularizer,kernel_constraint=kernel_constraint,bias_constraint=bias_constraint)(x)
    pathway2 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(pathway2))
    pathway2_1 = conv_block(pathway2,branch2[1],1,3)
    pathway2_2 = conv_block(pathway2,branch2[2],3,1)
    
    #1x1->3x3->1x3+3x1
    pathway3=Conv2D(filters=branch3[0],kernel_size=(1,1),strides=1,padding=padding,data_format=data_format,use_bias=use_bias,kernel_initializer=kernel_initializer,bias_initializer=bias_initializer,kernel_regularizer=kernel_regularizer,bias_regularizer=bias_regularizer,activity_regularizer=activity_regularizer,kernel_constraint=kernel_constraint,bias_constraint=bias_constraint)(x)
    pathway3 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(pathway3))
    pathway3=Conv2D(filters=branch3[1],kernel_size=(3,3),strides=1,padding=padding,data_format=data_format,use_bias=use_bias,kernel_initializer=kernel_initializer,bias_initializer=bias_initializer,kernel_regularizer=kernel_regularizer,bias_regularizer=bias_regularizer,activity_regularizer=activity_regularizer,kernel_constraint=kernel_constraint,bias_constraint=bias_constraint)(pathway3)
    pathway3 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(pathway3))
    pathway3_1 = conv_block(pathway3,branch3[2],1,3)
    pathway3_2 = conv_block(pathway3,branch3[3],3,1)
    #3x3->1x1
    pathway4=AveragePooling2D(pool_size=(3,3),strides=1,padding=padding,data_format=DATA_FORMAT)(x)
    pathway4=Conv2D(filters=branch4[0],kernel_size=(1,1),strides=1,padding=padding,data_format=data_format,use_bias=use_bias,kernel_initializer=kernel_initializer,bias_initializer=bias_initializer,kernel_regularizer=kernel_regularizer,bias_regularizer=bias_regularizer,activity_regularizer=activity_regularizer,kernel_constraint=kernel_constraint,bias_constraint=bias_constraint)(pathway4)
    pathway4 = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(pathway4))
    return concatenate([pathway1,pathway2_1,pathway2_2,pathway3_1,pathway3_2,pathway4],axis=concat_axis)

4）搭建网络
在这里插入图片描述
valid 都用的 same

def create_model(img_input):
    x = Conv2D(32,kernel_size=(3,3),strides=(2,2),padding='same',
               kernel_initializer="he_normal",kernel_regularizer=regularizers.l2(weight_decay))(img_input)
    x = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(x))
    x = Conv2D(32,kernel_size=(3,3),strides=(1,1),padding='same',
               kernel_initializer="he_normal",kernel_regularizer=regularizers.l2(weight_decay))(x)
    x = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(x))
    x = Conv2D(64,kernel_size=(3,3),strides=(1,1),padding='same',
               kernel_initializer="he_normal",kernel_regularizer=regularizers.l2(weight_decay))(x)
    x = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(x))
    x=MaxPooling2D(pool_size=(3,3),strides=2,padding='same',data_format=DATA_FORMAT)(x)
    x = Conv2D(80,kernel_size=(1,1),strides=(1,1),padding='same',
               kernel_initializer="he_normal",kernel_regularizer=regularizers.l2(weight_decay))(x)
    x = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(x))  
    x = Conv2D(192,kernel_size=(3,3),strides=(1,1),padding='same',
               kernel_initializer="he_normal",kernel_regularizer=regularizers.l2(weight_decay))(x)
    x = Activation('relu')(BatchNormalization(momentum=0.9, epsilon=1e-5)(x))    
    x=MaxPooling2D(pool_size=(3,3),strides=2,padding='same',data_format=DATA_FORMAT)(x)
    
    x=inception_module1(x,params=[(64,),(48,64),(64,96),(32,)],concat_axis=CONCAT_AXIS) #3a 256
    x=inception_module1(x,params=[(64,),(48,64),(64,96),(64,)],concat_axis=CONCAT_AXIS) #3b 288
    x=inception_module1(x,params=[(64,),(48,64),(64,96),(64,)],concat_axis=CONCAT_AXIS) #3c 288
    x=inception_reduce1(x,params=[(384,),(64,96)],concat_axis=CONCAT_AXIS) # 768
    
    x=inception_module2(x,params=[(192,),(128,128,192),(128,128,128,128,192),(192,)],concat_axis=CONCAT_AXIS) #4a 768
    x=inception_module2(x,params=[(192,),(160,160,192),(160,160,160,160,192),(192,)],concat_axis=CONCAT_AXIS) #4b 768
    x=inception_module2(x,params=[(192,),(160,160,192),(160,160,160,160,192),(192,)],concat_axis=CONCAT_AXIS) #4c 768
    x=inception_module2(x,params=[(192,),(160,160,192),(160,160,160,160,192),(192,)],concat_axis=CONCAT_AXIS) #4d 768
    x=inception_module2(x,params=[(192,),(192,192,192),(192,192,192,192,192),(192,)],concat_axis=CONCAT_AXIS) #4e 768
    x=inception_reduce2(x,params=[(192,320),(192,192,192,192)],concat_axis=CONCAT_AXIS) # 1280
    
    x=inception_module3(x,params=[(320,),(384,384,384),(448,384,384,384),(192,)],concat_axis=CONCAT_AXIS) #4e 2048
    x=inception_module3(x,params=[(320,),(384,384,384),(448,384,384,384),(192,)],concat_axis=CONCAT_AXIS) #4e 2048

    x=GlobalAveragePooling2D()(x)
    x=Dropout(DROPOUT)(x) 
    x = Dense(num_classes,activation='softmax',kernel_initializer="he_normal",
              kernel_regularizer=regularizers.l2(weight_decay))(x)
    return x

5）生成模型

img_input=Input(shape=(32,32,3))
output = create_model(img_input)
model=Model(img_input,output)
model.summary()

inception v3 参数量

Total params: 23,397,866
Trainable params: 23,359,978
Non-trainable params: 37,888

v1：Total params: 5,984,936
v2：Total params: 10,210,090

6）开始训练

# set optimizer
sgd = optimizers.SGD(lr=.1, momentum=0.9, nesterov=True)
model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])

# set callback
tb_cb = TensorBoard(log_dir=log_filepath, histogram_freq=0)
change_lr = LearningRateScheduler(scheduler)
cbks = [change_lr,tb_cb]

# set data augmentation
datagen = ImageDataGenerator(horizontal_flip=True,
                             width_shift_range=0.125,
                             height_shift_range=0.125,
                             fill_mode='constant',cval=0.)
datagen.fit(x_train)

# start training
model.fit_generator(datagen.flow(x_train, y_train,batch_size=batch_size),
                    steps_per_epoch=iterations,
                    epochs=epochs,
                    callbacks=cbks,
                    validation_data=(x_test, y_test))
model.save('inception_v3.h5')

7）结果分析
training accuracy 和 training loss
在这里插入图片描述 在这里插入图片描述 在这里插入图片描述
在这里插入图片描述在这里插入图片描述

  • accuracy
    在这里插入图片描述
  • loss
    在这里插入图片描述

因为前面的 stem 把 feature map 的分辨率降到了 4，后面 inception module 中 1x7 与 7x1 的效果可想而知，都是在处理 padding， 差，相当的差。
在这里插入图片描述

test accuracy 和 test loss
在这里插入图片描述 在这里插入图片描述 在这里插入图片描述
在这里插入图片描述在这里插入图片描述

  • accuracy
    在这里插入图片描述
  • loss
    在这里插入图片描述

…………
结果意料之中

2.2 Inception_v3_slim

把 Inception_v3 中 stern 结构直接替换成一个卷积，inception 结构不变，因为stern结果会把原图降到1/8的分辨率，对于 ImageNet 还行，CIFRA-10的话有些吃不消了

  • 调整网络结构
def create_model(img_input):
    x = Conv2D(192,kernel_size=(3,3),strides=(1,1),padding='same',
               kernel_initializer="he_normal",kernel_regularizer=regularizers.l2(weight_decay))(img_input)
    
    x=inception_module1(x,params=[(64,),(48,64),(64,96),(32,)],concat_axis=CONCAT_AXIS) #3a 256
    x=inception_module1(x,params=[(64,),(48,64),(64,96),(64,)],concat_axis=CONCAT_AXIS) #3b 288
    x=inception_module1(x,params=[(64,),(48,64),(64,96),(64,)],concat_axis=CONCAT_AXIS) #3c 288
    x=inception_reduce1(x,params=[(384,),(64,96)],concat_axis=CONCAT_AXIS) # 768
    
    x=inception_module2(x,params=[(192,),(128,128,192),(128,128,128,128,192),(192,)],concat_axis=CONCAT_AXIS) #4a 768
    x=inception_module2(x,params=[(192,),(160,160,192),(160,160,160,160,192),(192,)],concat_axis=CONCAT_AXIS) #4b 768
    x=inception_module2(x,params=[(192,),(160,160,192),(160,160,160,160,192),(192,)],concat_axis=CONCAT_AXIS) #4c 768
    x=inception_module2(x,params=[(192,),(160,160,192),(160,160,160,160,192),(192,)],concat_axis=CONCAT_AXIS) #4d 768
    x=inception_module2(x,params=[(192,),(192,192,192),(192,192,192,192,192),(192,)],concat_axis=CONCAT_AXIS) #4e 768
    x=inception_reduce2(x,params=[(192,320),(192,192,192,192)],concat_axis=CONCAT_AXIS) # 1280
    
    x=inception_module3(x,params=[(320,),(384,384,384),(448,384,384,384),(192,)],concat_axis=CONCAT_AXIS) #4e 2048
    x=inception_module3(x,params=[(320,),(384,384,384),(448,384,384,384),(192,)],concat_axis=CONCAT_AXIS) #4e 2048

    x=GlobalAveragePooling2D()(x)
    x=Dropout(DROPOUT)(x) 
    x = Dense(num_classes,activation='softmax',kernel_initializer="he_normal",
              kernel_regularizer=regularizers.l2(weight_decay))(x)
    return x

参数量如下：

Total params: 23,229,370
Trainable params: 23,192,282
Non-trainable params: 37,088

v1：Total params: 5,984,936
v2：Total params: 10,210,090
v3：Total params: 23,397,866

结果分析如下：

training accuracy 和 training loss
在这里插入图片描述 在这里插入图片描述 在这里插入图片描述 在这里插入图片描述

在这里插入图片描述 在这里插入图片描述

  • accuracy
    在这里插入图片描述

  • loss
    在这里插入图片描述

test accuracy 和 test loss
在这里插入图片描述 在这里插入图片描述 在这里插入图片描述 在这里插入图片描述
在这里插入图片描述 在这里插入图片描述

  • accuracy
    在这里插入图片描述
  • loss
    在这里插入图片描述

OK，精度 94%+，还行，hyper parameters 么有怎么调整

2.3 Inception_v3_slim_352

2.2 小节中的 inception module 1，2，3 内部重复的模块还有些细微的参数变动，现在将他们 inception module 重复的模块参数统一起来
修改如下

def create_model(img_input):
    x = Conv2D(192,kernel_size=(3,3),strides=(1,1),padding='same',
               kernel_initializer="he_normal",kernel_regularizer=regularizers.l2(weight_decay))(img_input)
    
    # inception 3a-3c 288
    for _ in range(3):
        x=inception_module1(x,params=[(64,),(48,64),(64,96),(64,)],concat_axis=CONCAT_AXIS)
    x=inception_reduce1(x,params=[(384,),(64,96)],concat_axis=CONCAT_AXIS) # 768
    # inception 4a-4e 768
    for _ in range(5):
        x=inception_module2(x,params=[(192,),(128,128,192),(128,128,128,128,192),(192,)],concat_axis=CONCAT_AXIS) 
    x=inception_reduce2(x,params=[(192,320),(192,192,192,192)],concat_axis=CONCAT_AXIS) # 1280
    # inception 5a-5b 2048
    for _ in range(2):
        x=inception_module3(x,params=[(320,),(384,384,384),(448,384,384,384),(192,)],concat_axis=CONCAT_AXIS)
    x=GlobalAveragePooling2D()(x)
    x=Dropout(DROPOUT)(x) 
    x = Dense(num_classes,activation='softmax',kernel_initializer="he_normal",
              kernel_regularizer=regularizers.l2(weight_decay))(x)
    return x

参数量：

Total params: 21,215,770
Trainable params: 21,180,538
Non-trainable params: 35,232

v1：Total params: 5,984,936
v2：Total params: 10,210,090
v3：Total params: 23,397,866
v3_slim：Total params: 23,229,370

结果分析如下：

training accuracy 和 training loss
在这里插入图片描述 在这里插入图片描述

在这里插入图片描述在这里插入图片描述

  • accuracy
    在这里插入图片描述
  • loss
    在这里插入图片描述

test accuracy 和 test loss
在这里插入图片描述 在这里插入图片描述
在这里插入图片描述在这里插入图片描述

  • accuracy
    在这里插入图片描述
  • loss
    在这里插入图片描述

差别不大，inception module 内部参数递增效果要比都一样的好一点点

2.4 inception_v3_stem_slim

把 stem 的 stride 都改为1
在这里插入图片描述
直接把 stem 替换成一个卷积结果要好一些

2.5 纵向对比

在这里插入图片描述
在这里插入图片描述

3 总结

精度最高的
模型大小
在这里插入图片描述
在这里插入图片描述

bryant_meng
关注
专栏目录
tensorflow2 keras inception cifar10训练代码分享
qq_40575024的博客
05-03 309
tensorflow2 keras inception cifar10训练代码分享 代码分享： import tensorflow as tf import os import numpy as np from matplotlib import pyplot as plt from tensorflow.keras.layers import Conv2D, BatchNormalization...
关于CIFAR-10图像分类总结
MirrorN的博客
05-08 1万+
之前一段时间学习深度学习在CIFAR-10数据集上图像分类问题，在此做一点点总结，关于这个问题，网络上已经有很多优秀的资源了，因此，主要是汇总一些博客等资料。 关于图像分类算法的总结 很多网站都转载了这篇文章，文章比较了五种图像分类算法，传统机器学习以及深度学习的方法都有涉及，了解图像分类可以看一看 原文：Image Classification in 5 Methods 翻译等可见：图像分类 |...
inception-v3-cifar10:初始v3数字10
05-01
盗梦空间-v3-cifar10 基于 。 安装 拉Docker镜像 $ docker pull tensorflow/tensorflow:1.7.0-py3 $ docker run -it --name {docker-name} tensorflow/tensorflow:1.7.0-py3 /bin/bash 拉GitHub存储库 ~# apt-get update ~# apt-get install git ~# git clone https://github.com/PJunhyuk/inception-v3-cifar10 下载数据集 ~/inception-v3-cifar10# python download_and_convert_data.py --dataset_name=cifar10 --dataset_dir="/tmp/data/cifar10" 用法
pytorch实现InceptionV3模型，Cifar10数据集测试
josenxiao的博客
09-28 4510
1、GoogleNet的InceptionV3结构 Google的InceptionV3版本以前的模型，有着以下改进部分： 前期模型构造，自己在V1、V2版本中并没有加入BN，加快模型收敛，这是考虑模型更深更广，训练难度大，而且也容易出现差错，自己也考虑在V2、V1模型加入BN，具体的测试对比和代码会将其放在github上。 # 自己封装的BN+ReLU模型 # cl...
【Keras-Inception v3】CIFAR-10-附件资源
03-02
【Keras-Inception v3】CIFAR-10-附件资源
Inception网络 运行在Cifar10 测试集87.88% Tensorflow 2.1 小白从代码实践中 理解
Harry的博客
04-01 574
环境 tensorflow 2.1 最好用GPU 模型 Inception 训练数据 Cifar10 或者 Cifar 100 训练集上准确率：93%左右 验证集上准确率：88%左右 测试集上准确率：87.88% 训练时间在GPU上：一小时多 权重大小：3.98 MB 原理介绍 图像中突出部分的大小差别很大。例如，狗的图像可以是以下任意情况。每张图像中狗所占区域都是不同的。 从左到右：狗占据图像...
keras安装与配置指南_Keras-快速指南
cunzai1985的博客
09-24 3885
Keras-快速指南 (Keras - Quick Guide) Advertisements 广告 Previous Page 上一页 Next Page 下一页 Keras-简介 (Keras - Introduction ) Deep learning is one of the major subfield of machine learning fr...
[深度学习]使用torchvison经典模型训练cifar10(AlexNet,VGG,Inception,ResNet等)
sinat_33487968的博客
11-02 7474
代码是参考pytorch上面的官方教程，教程给出了一个简单了卷积网络训练，前半部分是卷积，后半部分全连接的类型，用来分类cifar10. https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html#sphx-glr-beginner-blitz-cifar10-tutorial-py 而我想看看torchvison里经典模...
CIFAR-10:使用预先训练的VGG-16，Resnet和Inception网络模型对CIFAR-10数据集中的图像进行分类
04-08
CIFAR-10 使用VGG-16，Resnet和Inception net，模型对CIFAR-10数据集的图像进行分类，以对不同的对象（例如汽车，狗等）进行分类。
pytorch-cifar100：在cifar100上实践（ResNet，DenseNet，VGG，GoogleNet，InceptionV3，InceptionV4，Inception-ResNetv2，Xception，Resnet In Resnet，ResNext，ShuffleNet，ShuffleNetv2，MobileNet，MobileNetv2，SqueezeNet，NasNet，Residual Attention Network，SE WideResNet）
02-05
皮托奇·西法尔100 pytorch在cifar100上练习 要求 这是我的实验资料 python3.6 pytorch1.6.0 + cu101 张量板2.2.2（可选） 用法 1.输入目录 $ cd pytorch-cifar100 2.数据集 我将使用来自torchvision的cifar100数据集，因为它更方便，但我还将示例代码保留了用于在数据集文件夹中编写您自己的数据集模块的示例，以作为人们不知道如何编写它的示例。 3.运行tensorbard（可选） 安装张量板 $ pip install tensorboard $ mkdir runs Run tensorboard
谷歌的inception_v3模型.zip
11-30
经过，终于将它下载下来，现在将它分享出来。基于谷歌inception v3训练好的模型。包含： inflating: imagenet_comp_graph_label_strings.txt inflating: LICENSE inflating: tensorflow_inception_graph.pb
计算生成任务fid的代码
01-09
计算生成任务fid的代码
MindSpore社区活动：InceptionV3和CIFAR-10的奇遇
最新发布
longvoyage的博客
12-24 1836
1.mindspore社区InceptionV3模型是谷歌Inception系列里面的第三代模型，相比于其它神经网络模型，Inception网络最大的特点在于将卷积核组合在一起，建立了一个多分支结构，使得网络能够并行地计算。Inception网络架构的优点更高的表现力：Inception网络具有更高的表现力，即可以在相同的计算资源下获得更好的分类效果。并行计算：通过并行计算，不同分支的计算可以在不同的GPU上进行，并且可以有效地活用多个GPU的计算资源。
使用深度学习的inception卷积神经网络训练CIFAR10数据集代码实现
繁华三千东流水不舍昼夜的博客
09-18 1028
使用深度学习的inception卷积神经网络训练CIFAR10数据集代码实现
InceptionV3迁移学习训练并预测
gulingfengze的博客
04-12 7768
迁移学习的内容这里就不说它了，这里有篇文章推荐给大家：数据不够怎么训练深度学习模型？不妨试试迁移学习 。内容基本上来源《TensorFlow:实战Google深度学习框架》和这篇博文【TensorFlow】迁移学习（使用Inception-v3） ，这里对三位作者表示感谢！ 1.数据集与模型下载 数据集下载 inceptionV3模型 ，密码：ilab 2.训练代码 import...
python 线性回归 优化_从头开始使用梯度下降优化在Python中实现多元线性回归(代码)...
weixin_39800918的博客
12-08 280
import matplotlib.pyplot as pltimport numpy as npdef hypothesis(theta, X, n):h = np.ones((X.shape[0],1))theta = theta.reshape(1,n+1)for i in range(0,X.shape[0]):h[i] = float(np.matmul(theta, X[i]))h =...
【Keras-Xception】CIFAR-10
bryant_meng
01-15 1855
系列连载目录 请查看博客 《Paper》 4.1 小节 【Keras】Classification in CIFAR-10 系列连载 学习借鉴 github：BIGBALLON/cifar-10-cnn 知乎专栏：写给妹子的深度学习教程 Xception Caffe 代码：https://github.com/soeaver/caffe-model/edit/master/cls/incep...
用Inception-V3模型进行图像分类
热门推荐
xingwei_09的博客
01-25 2万+
Inception-V3模型简介 本例使用预训练好的深度神经网络Inception-v3模型来进行图像分类。Inception-v3模型在一台配有 8 Tesla K40 GPUs，大概价值$30,000的野兽级计算机上训练了几个星期，因此不可能在一台普通的PC上训练。我们将会下载预训练好的Inception模型，然后用它来做图像分类。 Inception-v3模型
深度学习框架Keras-2.4.1正式发布
资源摘要信息:"Keras-2.4.1.tar.gz 是一个深度学习库Keras的源代码压缩包，版本为2.4.1。Keras是一个高层神经网络API，能够以TensorFlow、Microsoft Cognitive Toolkit、Theano或PlaidML作为后端运行。该版本的Keras...
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值