图像分类 keras 训练自己的数据集

最新推荐文章于 2024-05-27 17:46:54 发布

置顶东东就是我

最新推荐文章于 2024-05-27 17:46:54 发布

阅读量3.7k

点赞数 10

分类专栏：图像分类文章标签： fine-tuning

本文链接：https://blog.csdn.net/qq_33228039/article/details/102951440

版权

图像分类专栏收录该内容

10 篇文章 0 订阅

订阅专栏

2.2 加载imagenet训练好的权重，修改网络后几层。

1.构建数据集

1.1数据增强

train_datagen = ImageDataGenerator(
            rescale=1. / 255,
            shear_range=0.2,
            zoom_range=0.2,
            rotation_range=90,
            width_shift_range=0.2,
            height_shift_range=0.2,
            horizontal_flip=True
        )

参数解释：

# featurewise_center: Boolean.对输入的图片每个通道减去每个通道对应均值。
# samplewise_center: Boolan.每张图片减去样本均值, 使得每个样本均值为0。
# featurewise_std_normalization(): Boolean()
# samplewise_std_normalization(): Boolean()
# zca_epsilon(): Default
# zca_whitening: Boolean.去除样本之间的相关性
# rotation_range(): 旋转范围
# width_shift_range(): 水平平移范围
# height_shift_range(): 垂直平移范围
# shear_range(): float, 透视变换的范围
# zoom_range(): 缩放范围
# fill_mode: 填充模式, constant, nearest, reflect
# cval: fill_mode == 'constant'
# 的时候填充值
# horizontal_flip(): 水平反转
# vertical_flip(): 垂直翻转
# preprocessing_function(): user提供的处理函数
# data_format(): channels_first或者channels_last
# validation_split(): 多少数据用于验证集

1.2根据batch_size构建训练集

        train_generator = train_datagen.flow_from_directory(
            train_data_dir,
            target_size=(img_rows, img_cols),
            batch_size=batch_size,
            class_mode='binary')

参数解释：

# class_mode: "categorical", "binary", "sparse"或None之一. 默认为"categorical. 该参数决定了返回的标签数组的形式, "categorical"会返回2D的one-hot编码标签,"binary"返回1D的二值标签."sparse"返回1D的整数标签,如果为None则不返回任何标签, 生成器将仅仅生成batch数据

如果是二分类class_model=binary,如果是多分类class_model=categorical

2.构建模型

2.1.不加载权重，从头训练

model = MobileNetV2(weights=None, classes=2, input_shape=(64,64,3))
# 定义训练参数
# 采用随机梯度下降法，学习率初始值0.1,动量参数为0.9,学习率衰减值为1e-6,确定使用Nesterov动量
sgd = SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)
# 配置模型学习过程，目标函数为categorical_crossentropy：亦称作多类的对数损失，注意使用该目标函数时，需要将标签转化为形如(nb_samples, nb_classes)的二值序列
model.compile(optimizer=sgd, loss='categorical_crossentropy', metrics=['accuracy'])

参数解释

optimizer：优化器，Adam，rmsprop(root mean square prop）、adagrad等优化器，

loss：计算损失，这里用的是交叉熵损失，binary_crossentropy（二分类）categorical_crossentropy（多分类）

metrics: 列表，包含评估模型在训练和测试时的性能的指标，典型用法是metrics=[‘accuracy’]。

2.2 加载imagenet训练好的权重，修改网络后几层。

        Inp = Input((img_rows, img_cols, color))
        base_model = VGG19(weights='imagenet', include_top=False, pooling=None, input_shape=(img_rows, img_cols, color),
                           classes=nb_classes)

        top_model = Sequential()
        top_model.add(Flatten(input_shape=base_model.output_shape[1:]))  # model.output_shape[1:])
        top_model.add(Dense(256, activation='relu'))
        top_model.add(Dropout(0.8))
        top_model.add(Dense(nb_classes, activation='sigmoid'))

        # 训练模型
        model = Model(inputs=base_model.input, outputs=top_model(base_model.output))

        # 冻结base_model所有层，这样就可以正确获得bottleneck特征
        for layer in base_model.layers:
            layer.trainable = False

        sgd = SGD(lr=lr, decay=decay, momentum=momentum, nesterov=True)
        model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])

base_model 就是把vgg19全连接层去掉后用于提取特征的网络层，采用imagenet训练过的权重，

top_model 就是把提取的特征，用于分类，自己构建的分类网络

冻结 base_model层，只训练我们的分类网络

记住，冻结一定要放在model后面和compile前面

3.训练

    hist = model.fit_generator(
        train_generator,
        validation_data=validation_generator,
        steps_per_epoch=count1 // batch,
        validation_steps=count2 // batch,
        epochs=epochs,
        callbacks=[earlystop])

    df = pd.DataFrame.from_dict(hist.history)
    df.to_csv('model/hist.csv', encoding='utf-8', index=False)

参照批次训练，减少内存的使用

4.测试

from keras.preprocessing import image
import numpy as np
import os
from keras.applications.mobilenetv2 import decode_predictions, preprocess_input
from keras.applications.mobilenetv2 import MobileNetV2
from keras.layers import Conv2D, Reshape, Activation,Dropout
from keras.models import Model
# 预测测试
import time
path = "C:\\classi\\validation\\A\\"
classes=['error','ok']
# model = MobileNetV2((32, 32, 3), include_top=False, alpha=1.0, classes=2)
model = MobileNetV2(input_shape=(64,64,3),weights=None,classes=2)
model.load_weights('best_MobileNetV2_2019.7.23.h5')
print(model.summary())
for i in os.listdir(path):
    img_path=path+i
    img = image.load_img(img_path, target_size=(64, 64))
    # mobilenetv2
    t0 = time.time()
    x = image.img_to_array(img) / 255
    x = np.expand_dims(x, axis=0)
    preds = model.predict(x)

    t1 = time.time()
    # print('mobilenetv2 Predicted:', decode_predictions(preds, top=1))
    print('class name=%s' %(classes[int(np.argmax(preds,axis=1))])+'||probe= %.3f '%(np.max(preds)))
    print('times:', t1 - t0)

5.完整的代码

5.1 train

# -*- coding: utf-8 -*-
# @Time    : 2019/11/5 11:13
# @Author  : Don
# @File    : train_inceptionv3.py
# @Software: PyCharm
# -*- coding: utf-8 -*-
import os
from keras.utils import plot_model
from keras.applications.resnet50 import ResNet50
from keras.applications.vgg19 import VGG19
from keras.applications.inception_v3 import InceptionV3
from keras.layers import Dense, Flatten, GlobalAveragePooling2D, Input, regularizers, BatchNormalization, Dropout
from keras.models import Model, load_model, Sequential
from keras.optimizers import SGD
from keras.preprocessing.image import ImageDataGenerator
import matplotlib.pyplot as plt
from keras.regularizers import l2
from keras.applications.mobilenetv2 import MobileNetV2

class PowerTransferMode:
    # 数据准备
    def DataGen(self, dir_path, img_row, img_col, batch_size, is_train):
        if is_train:
            datagen = ImageDataGenerator(rescale=1. / 255,
                                         zoom_range=0.25, rotation_range=15.,
                                         channel_shift_range=25., width_shift_range=0.02, height_shift_range=0.02,
                                         horizontal_flip=True, fill_mode='constant')
        else:
            datagen = ImageDataGenerator(rescale=1. / 255)

        generator = datagen.flow_from_directory(
            dir_path, target_size=(img_row, img_col),
            batch_size=batch_size,
            # class_mode='binary',
            shuffle=is_train)

        return generator

    # ResNet模型
    def ResNet50_model(self, lr=0.005, decay=1e-6, momentum=0.9, nb_classes=2, img_rows=197, img_cols=197, RGB=True,
                       is_plot_model=False):
        color = 3 if RGB else 1

        Inp = Input((img_rows, img_cols, color))
        print(Inp)
        base_model = ResNet50(weights='imagenet', include_top=False,
                              input_shape=(img_rows, img_cols, color),
                              classes=nb_classes)

        x = base_model(Inp)
        x = GlobalAveragePooling2D(name='average_pool')(x)
        # x = Flatten(name='flatten')(x)
        x = Dense(2048, activation='relu', kernel_regularizer=regularizers.l2(0.0001), )(x)
        x = BatchNormalization()(x)
        x = Dense(1024, activation='relu', kernel_regularizer=regularizers.l2(0.0001))(x)
        x = BatchNormalization(name='bn_fc_01')(x)
        predictions = Dense(nb_classes, activation='softmax')(x)

        # 训练模型
        model = Model(inputs=Inp, outputs=predictions)

        # 冻结base_model所有层，这样就可以正确获得bottleneck特征
        for layer in base_model.layers:
            layer.trainable = False

        sgd = SGD(lr=lr, decay=decay, momentum=momentum, nesterov=True)
        model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])

        # 绘制模型
        if is_plot_model:
            plot_model(model, to_file='resnet50_model.png', show_shapes=True)

        return model

    # VGG模型
    def VGG19_model(self, lr=0.005, decay=1e-6, momentum=0.9, nb_classes=2, img_rows=197, img_cols=197, RGB=True,
                    is_plot_model=False):
        color = 3 if RGB else 1
        Inp = Input((img_rows, img_cols, color))
        base_model = VGG19(weights='imagenet', include_top=False, pooling=None, input_shape=(img_rows, img_cols, color),
                           classes=nb_classes)

        top_model = Sequential()
        top_model.add(Flatten(input_shape=base_model.output_shape[1:]))  # model.output_shape[1:])
        top_model.add(Dense(256, activation='relu'))
        top_model.add(Dropout(0.8))
        top_model.add(Dense(nb_classes, activation='sigmoid'))

        # 训练模型
        model = Model(inputs=base_model.input, outputs=top_model(base_model.output))

        # 冻结base_model所有层，这样就可以正确获得bottleneck特征
        for layer in base_model.layers:
            layer.trainable = False

        sgd = SGD(lr=lr, decay=decay, momentum=momentum, nesterov=True)
        model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])

        # 绘图
        if is_plot_model:
            plot_model(model, to_file='vgg19_model.png', show_shapes=True)

        return model

    # InceptionV3模型
    def InceptionV3_model(self, lr=0.005, decay=1e-6, momentum=0.9, nb_classes=2, img_rows=197, img_cols=197,
                          RGB=True,
                          is_plot_model=False):
        color = 3 if RGB else 1

        Inp = Input((img_rows, img_cols, color))
        print(Inp)
        base_model = InceptionV3(weights='imagenet', include_top=False,
                                 input_shape=(img_rows, img_cols, color),
                                 classes=nb_classes)

        x = base_model(Inp)
        x = GlobalAveragePooling2D(name='average_pool')(x)
        # x = Flatten(name='flatten')(x)
        x = Dense(2048, activation='relu', kernel_regularizer=regularizers.l2(0.0001), )(x)
        x = BatchNormalization()(x)
        x = Dense(1024, activation='relu', kernel_regularizer=regularizers.l2(0.0001))(x)
        x = BatchNormalization(name='bn_fc_01')(x)
        predictions = Dense(nb_classes, activation='softmax')(x)

        # 训练模型
        model = Model(inputs=Inp, outputs=predictions)

        # 冻结base_model所有层，这样就可以正确获得bottleneck特征
        for layer in base_model.layers:
            layer.trainable = False

        sgd = SGD(lr=lr, decay=decay, momentum=momentum, nesterov=True)
        model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])

        # 绘图
        # if is_plot_model:
        # plot_model(model, to_file='inception_v3_model.png', show_shapes=True)

        return model

    # MobileNetv2模型
    def MobileNetv2_model(self, lr=0.005, decay=1e-6, momentum=0.9, nb_classes=2, img_rows=197, img_cols=197,
                          RGB=True,
                          is_plot_model=False):
        color = 3 if RGB else 1

        Inp = Input((img_rows, img_cols, color))
        print(Inp)
        base_model = MobileNetV2(weights='imagenet', include_top=False,
                                 input_shape=(img_rows, img_cols, color),
                                 classes=nb_classes)

        x = base_model(Inp)
        x = GlobalAveragePooling2D(name='average_pool')(x)
        # x = Flatten(name='flatten')(x)
        x = Dense(2048, activation='relu', kernel_regularizer=regularizers.l2(0.0001), )(x)
        x = BatchNormalization()(x)
        x = Dense(1024, activation='relu', kernel_regularizer=regularizers.l2(0.0001))(x)
        x = BatchNormalization(name='bn_fc_01')(x)
        predictions = Dense(nb_classes, activation='softmax')(x)

        # 训练模型
        model = Model(inputs=Inp, outputs=predictions)

        # 冻结base_model所有层，这样就可以正确获得bottleneck特征
        for layer in base_model.layers:
            layer.trainable = False

        sgd = SGD(lr=lr, decay=decay, momentum=momentum, nesterov=True)
        model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])

        # 绘图
        if is_plot_model:
            plot_model(model, to_file='Mobilenet_v2_model.png', show_shapes=True)

        return model

    # 训练模型
    def train_model(self, model, epochs, train_generator, steps_per_epoch, validation_generator, validation_steps,
                    model_url, is_load_model=False):
        # 载入模型
        if is_load_model and os.path.exists(model_url):
            model = load_model(model_url)

        history_ft = model.fit_generator(
            train_generator,
            steps_per_epoch=steps_per_epoch,
            epochs=epochs,
            validation_data=validation_generator,
            validation_steps=validation_steps)
        # 模型保存
        model.save(model_url, overwrite=True)
        return history_ft

    # 画图
    def plot_training(self, history):
        acc = history.history['acc']
        val_acc = history.history['val_acc']
        loss = history.history['loss']
        val_loss = history.history['val_loss']
        epochs = range(len(acc))
        plt.plot(epochs, acc, 'b-')
        plt.plot(epochs, val_acc, 'r')
        plt.title('Training and validation accuracy')
        plt.figure()
        plt.plot(epochs, loss, 'b-')
        plt.plot(epochs, val_loss, 'r-')
        plt.title('Training and validation loss')
        plt.show()


if __name__ == '__main__':
    image_size = 75
    batch_size = 32

    transfer = PowerTransferMode()

    # 得到数据
    train_generator = transfer.DataGen('C:\\classi\\train', image_size, image_size, batch_size, True)
    validation_generator = transfer.DataGen('C:\\classi\\validation', image_size, image_size, batch_size, False)

    # VGG19
    model = transfer.VGG19_model(nb_classes=2, img_rows=image_size, img_cols=image_size, is_plot_model=False)
    history_ft = transfer.train_model(model, 100, train_generator, 50, validation_generator, 20,
                                      'vgg19_model_weights.h5', is_load_model=False)

    # ResNet50
    # model = transfer.ResNet50_model(nb_classes=2, img_rows=image_size, img_cols=image_size, is_plot_model=False)
    # history_ft = transfer.train_model(model, 100, train_generator, 50, validation_generator, 20,
    #                                   'resnet50_model_weights.h5', is_load_model=False)

    # InceptionV3
    # model = transfer.InceptionV3_model(nb_classes=2, img_rows=image_size, img_cols=image_size, is_plot_model=True)
    # history_ft = transfer.train_model(model, 100, train_generator, 50, validation_generator, 20,
    #                                   'inception_v3_model_weights.h5', is_load_model=False)

    # MobileNetv2
    # model = transfer.MobileNetv2_model(nb_classes=2, img_rows=image_size, img_cols=image_size, is_plot_model=True)
    # history_ft = transfer.train_model(model, 100, train_generator, 50, validation_generator, 20,
    #                                   'mobileNetv2_model_weights.h5', is_load_model=False)

    transfer.plot_training(history_ft)

5.2 predict

# -*- coding: utf-8 -*-
# @Time    : 2019/10/30 14:25
# @Author  : Don
# @File    : predict.py
# @Software: PyCharm
from keras.preprocessing import image
import numpy as np
import os
from keras.applications.mobilenetv2 import decode_predictions, preprocess_input
from keras.applications.mobilenetv2 import MobileNetV2
from keras.layers import Conv2D, Reshape, Activation,Dropout
from keras.models import Model
# 预测测试
import time
path = "C:\\classi\\validation\\A\\"
classes=['error','ok']
# model = MobileNetV2((32, 32, 3), include_top=False, alpha=1.0, classes=2)
model = MobileNetV2(input_shape=(64,64,3),weights=None,classes=2)
model.load_weights('best_MobileNetV2_2019.7.23.h5')
print(model.summary())
for i in os.listdir(path):
    img_path=path+i
    img = image.load_img(img_path, target_size=(64, 64))
    # mobilenetv2
    t0 = time.time()
    x = image.img_to_array(img) / 255
    x = np.expand_dims(x, axis=0)
    preds = model.predict(x)

    t1 = time.time()
    # print('mobilenetv2 Predicted:', decode_predictions(preds, top=1))
    print('class name=%s' %(classes[int(np.argmax(preds,axis=1))])+'||probe= %.3f '%(np.max(preds)))
    print('times:', t1 - t0)

参考：https://blog.csdn.net/pengdali/article/details/79050662

东东就是我

关注

10
点赞
踩
52

收藏

觉得还不错? 一键收藏
1
评论
图像分类 keras 训练自己的数据集

目录1.构建数据集 1.1数据增强参数解释：1.2根据batch_size构建训练集参数解释：2.构建模型2.1.不加载权重，从头训练参数解释2.2 加载imagenet训练好的权重，修改网络后几层。3.训练4.测试5.完整的代码5.1 train5.2 predict1.构建数据集 1.1数据增强train_datagen...
复制链接

扫一扫

专栏目录