T10 数据增强学习记录

最新推荐文章于 2024-05-18 16:03:12 发布

月亮泡面=

最新推荐文章于 2024-05-18 16:03:12 发布

阅读量935

点赞数 14

文章标签：学习

本文链接：https://blog.csdn.net/weixin_45103399/article/details/134693769

版权

🍨 本文为🔗365天深度学习训练营中的学习记录博客
🍖 原作者：K同学啊 | 接辅导、项目定制
🚀 文章来源：K同学的学习圈子

任务：

数据增强提高acc
方式1：将数据增强嵌入model
方式2：在dataset中进行数据增强

我的环境：

语言环境：python 3.8
编译器：jupyter notebook
深度学习环境：Tensorflow(CPU)

一、数据处理工作

1. 数据导入

import os,PIL,pathlib
import matplotlib.pyplot as plt
# 支持中文
plt.rcParams['font.sans-serif'] = ['SimHei']  # 用来正常显示中文标签
plt.rcParams['axes.unicode_minus'] = False  # 用来正常显示负号

import numpy             as np
from tensorflow          import keras
from tensorflow.keras    import layers,models
import tensorflow as tf

data_dir = "D:/BaiduNetdiskDownload/datasets/t10"
data_dir = pathlib.Path(data_dir)

data_dir

WindowsPath('D:/BaiduNetdiskDownload/datasets/t10')

2. 数据查看

用glob（）获取所有文件并存入列表中。

这里* 的意思为获取所有文件，/,/*,/**区别如下

因此*/*的意思则为获取文件夹下的所有文件及它们的子文件。

image_count = len(list(data_dir.glob('*/*')))
print('图片总数为：',image_count)

图片总数为： 3400

im = list(data_dir.glob('cat/*'))
PIL.Image.open(str(im[0]))

在这里插入图片描述

3. 加载数据

使用image_dataset_from_directory方法将磁盘中的数据加载到tf.data.Dataset中

batch_size = 32
img_height = 224
img_width = 224

tf.keras.preprocessing.image_dataset_from_directory 介绍

directory: 数据所在目录。如果标签是inferred（默认），则它应该包含子目录，每个目录包含一个类的图像。否则，将忽略目录结构。
labels: inferred（标签从目录结构生成），或者是整数标签的列表/元组，其大小与目录中找到的图像文件的数量相同。标签应根据图像文件路径的字母顺序排序（通过Python中的os.walk(directory)获得）。
label_mode:
int：标签将被编码成整数（使用的损失函数应为：sparse_categorical_crossentropy loss）。
categorical：标签将被编码为分类向量（使用的损失函数应为：categorical_crossentropy loss）。
binary：意味着标签（只能有2个）被编码为值为0或1的float32标量（例如：binary_crossentropy）。
None：（无标签）。
class_names: 仅当labels为inferred时有效。这是类名称的明确列表（必须与子目录的名称匹配）。用于控制类的顺序（否则使用字母数字顺序）。
color_mode: grayscale、rgb、rgba之一。默认值：rgb。图像将被转换为1、3或者4通道。
batch_size: 数据批次的大小。默认值：32
image_size: 从磁盘读取数据后将其重新调整大小。默认：（256，256）。由于管道处理的图像批次必须具有相同的大小，因此该参数必须提供。
shuffle: 是否打乱数据。默认值：True。如果设置为False,则按字母数字顺序对数据进行排序。
seed: 用于shuffle和转换的可选随机种子。
validation_split: 0和1之间的可选浮点数，可保留一部分数据用于验证。
subset: training或validation之一。仅在设置validation_split时使用。
interpolation: 字符串,当调整图像大小时使用的插值方法。默认为：bilinear。支持bilinear, nearest, bicubic, area, lanczos3, lanczos5, gaussian, mitchellcubic。
follow_links: 是否访问符号链接指向的子目录。默认：False。

train_ds = tf.keras.preprocessing.image_dataset_from_directory(
    data_dir,
    validation_split=0.3,
    subset="training",
    seed=12,
    image_size=(img_height, img_width),
    batch_size=batch_size)

Found 3400 files belonging to 2 classes.
Using 2380 files for training.

# class_names 按字母顺序对应目录名称，作为数据集标签
class_names = train_ds.class_names
print(class_names)

['cat', 'dog']

val_ds = tf.keras.preprocessing.image_dataset_from_directory(
    data_dir,
    validation_split=0.3,
    subset="validation",
    seed=12,
    image_size=(img_height, img_width),
    batch_size=batch_size)

Found 3400 files belonging to 2 classes.
Using 1020 files for validation.

val_batches = tf.data.experimental.cardinality(val_ds)
test_ds     = val_ds.take(val_batches // 5)
val_ds      = val_ds.skip(val_batches // 5)

print('Number of validation batches: %d' % tf.data.experimental.cardinality(val_ds))
print('Number of test batches: %d' % tf.data.experimental.cardinality(test_ds))

Number of validation batches: 26
Number of test batches: 6

5. 数据大小显示

for image_batch, labels_batch in train_ds:
    print(image_batch.shape)
    print(labels_batch.shape)
    break

(32, 224, 224, 3)
(32,)

image_batch：32张图片，图片形状为224 * 224 * 3（3代表彩色）。

label_batch：标签对应32张图片。

6. 配置数据集

shuffle()：打乱数据

prefetch()：预取数据，加速运行。

cache()：将数据缓存到cache中，加速运行。

AUTOTUNE = tf.data.AUTOTUNE

def preprocess_image(image,label):
    return (image/255.0,label)

# 归一化处理
train_ds = train_ds.map(preprocess_image, num_parallel_calls=AUTOTUNE)
val_ds   = val_ds.map(preprocess_image, num_parallel_calls=AUTOTUNE)
test_ds  = test_ds.map(preprocess_image, num_parallel_calls=AUTOTUNE)

train_ds = train_ds.cache().prefetch(buffer_size=AUTOTUNE)
val_ds   = val_ds.cache().prefetch(buffer_size=AUTOTUNE)

plt.figure(figsize=(15, 10))

for images, labels in train_ds.take(1):
    for i in range(8):
        ax = plt.subplot(5, 8, i + 1)

        plt.imshow(images[i])
        plt.title(class_names[labels[i]])
        
        plt.axis("off")

在这里插入图片描述

二、数据增强

1. 对图像进行数据增强——增强方式：旋转。

tf.keras.layers.experimental.preprocessing.RandomFlip：水平和垂直随机翻转每个图像。
tf.keras.layers.experimental.preprocessing.RandomRotation：随机旋转每个图像（按照某个弧度）。

data_augmentation = tf.keras.Sequential([
  tf.keras.layers.experimental.preprocessing.RandomFlip("horizontal_and_vertical"),
  tf.keras.layers.experimental.preprocessing.RandomRotation(0.2),# 按照0.2的弧度
])

image = tf.expand_dims(images[i],0) #把数据添加到batch中

plt.figure(figsize=(8, 8))
for i in range(9):
    augmented_image = data_augmentation(image)
    ax = plt.subplot(3, 3, i + 1)
    plt.imshow(augmented_image[0])
    plt.axis("off")

在这里插入图片描述

2. 方法1：将数据增强嵌入model

该方法的优点是可以获得GPU加速，但只有模型训练（.fit）时增强，在模型评估及预测时未增强。

model = tf.keras.Sequential([
  data_augmentation, #加入数据增强
  layers.Conv2D(16, 3, padding='same', activation='relu'),
  layers.MaxPooling2D(),
  layers.Conv2D(32, 3, padding='same', activation='relu'),
  layers.MaxPooling2D(),
  layers.Conv2D(64, 3, padding='same', activation='relu'),
  layers.MaxPooling2D(),
  layers.Flatten(),
  layers.Dense(128, activation='relu'),
  layers.Dense(len(class_names))
])

model.compile(optimizer='adam',
              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
              metrics=['accuracy'])

epochs=20
history = model.fit(
  train_ds,
  validation_data=val_ds,
  epochs=epochs
)

Epoch 1/20
75/75 [==============================] - 67s 871ms/step - loss: 0.8782 - accuracy: 0.5055 - val_loss: 0.6828 - val_accuracy: 0.6002
Epoch 2/20
75/75 [==============================] - 58s 775ms/step - loss: 0.6079 - accuracy: 0.6542 - val_loss: 0.6295 - val_accuracy: 0.6703
Epoch 3/20
75/75 [==============================] - 57s 760ms/step - loss: 0.4496 - accuracy: 0.7887 - val_loss: 0.3734 - val_accuracy: 0.8333
Epoch 4/20
75/75 [==============================] - 58s 776ms/step - loss: 0.3423 - accuracy: 0.8454 - val_loss: 0.2597 - val_accuracy: 0.8925
Epoch 5/20
75/75 [==============================] - 58s 773ms/step - loss: 0.3045 - accuracy: 0.8689 - val_loss: 0.2426 - val_accuracy: 0.9022
Epoch 6/20
75/75 [==============================] - 58s 776ms/step - loss: 0.2246 - accuracy: 0.9055 - val_loss: 0.2001 - val_accuracy: 0.9106
Epoch 7/20
75/75 [==============================] - 58s 777ms/step - loss: 0.2231 - accuracy: 0.9059 - val_loss: 0.2381 - val_accuracy: 0.9022
Epoch 8/20
75/75 [==============================] - 59s 782ms/step - loss: 0.2153 - accuracy: 0.9122 - val_loss: 0.1748 - val_accuracy: 0.9179
Epoch 9/20
75/75 [==============================] - 59s 785ms/step - loss: 0.2061 - accuracy: 0.9172 - val_loss: 0.1837 - val_accuracy: 0.9263
Epoch 10/20
75/75 [==============================] - 59s 785ms/step - loss: 0.1596 - accuracy: 0.9429 - val_loss: 0.1536 - val_accuracy: 0.9348
Epoch 11/20
75/75 [==============================] - 58s 774ms/step - loss: 0.1639 - accuracy: 0.9353 - val_loss: 0.1681 - val_accuracy: 0.9312
Epoch 12/20
75/75 [==============================] - 58s 774ms/step - loss: 0.1504 - accuracy: 0.9399 - val_loss: 0.1676 - val_accuracy: 0.9300
Epoch 13/20
75/75 [==============================] - 58s 774ms/step - loss: 0.1355 - accuracy: 0.9521 - val_loss: 0.1858 - val_accuracy: 0.9263
Epoch 14/20
75/75 [==============================] - 58s 777ms/step - loss: 0.1422 - accuracy: 0.9466 - val_loss: 0.1290 - val_accuracy: 0.9505
Epoch 15/20
75/75 [==============================] - 58s 779ms/step - loss: 0.1261 - accuracy: 0.9500 - val_loss: 0.1601 - val_accuracy: 0.9396
Epoch 16/20
75/75 [==============================] - 58s 775ms/step - loss: 0.1474 - accuracy: 0.9437 - val_loss: 0.1296 - val_accuracy: 0.9589
Epoch 17/20
75/75 [==============================] - 59s 781ms/step - loss: 0.1174 - accuracy: 0.9563 - val_loss: 0.1430 - val_accuracy: 0.9420
Epoch 18/20
75/75 [==============================] - 59s 780ms/step - loss: 0.1085 - accuracy: 0.9588 - val_loss: 0.1240 - val_accuracy: 0.9505
Epoch 19/20
75/75 [==============================] - 59s 781ms/step - loss: 0.1054 - accuracy: 0.9618 - val_loss: 0.1101 - val_accuracy: 0.9589
Epoch 20/20
75/75 [==============================] - 60s 801ms/step - loss: 0.1035 - accuracy: 0.9605 - val_loss: 0.1186 - val_accuracy: 0.9589

loss, acc = model.evaluate(test_ds)
print("Accuracy", acc)

6/6 [==============================] - 1s 138ms/step - loss: 0.1012 - accuracy: 0.9583
Accuracy 0.9583333134651184

如果不进行数据增强：

model = tf.keras.Sequential([
  #data_augmentation, #不加数据增强
  layers.Conv2D(16, 3, padding='same', activation='relu'),
  layers.MaxPooling2D(),
  layers.Conv2D(32, 3, padding='same', activation='relu'),
  layers.MaxPooling2D(),
  layers.Conv2D(64, 3, padding='same', activation='relu'),
  layers.MaxPooling2D(),
  layers.Flatten(),
  layers.Dense(128, activation='relu'),
  layers.Dense(len(class_names))
])
model.compile(optimizer='adam',
              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
              metrics=['accuracy'])
epochs=20
history = model.fit(
  train_ds,
  validation_data=val_ds,
  epochs=epochs
)

Epoch 1/20
75/75 [==============================] - 56s 740ms/step - loss: 0.5363 - accuracy: 0.7311 - val_loss: 0.2373 - val_accuracy: 0.9022
Epoch 2/20
75/75 [==============================] - 54s 718ms/step - loss: 0.1449 - accuracy: 0.9441 - val_loss: 0.1765 - val_accuracy: 0.9215
Epoch 3/20
75/75 [==============================] - 56s 741ms/step - loss: 0.0705 - accuracy: 0.9731 - val_loss: 0.1427 - val_accuracy: 0.9481
Epoch 4/20
75/75 [==============================] - 55s 732ms/step - loss: 0.0390 - accuracy: 0.9878 - val_loss: 0.1472 - val_accuracy: 0.9481
Epoch 5/20
75/75 [==============================] - 54s 725ms/step - loss: 0.0284 - accuracy: 0.9916 - val_loss: 0.1429 - val_accuracy: 0.9517
Epoch 6/20
75/75 [==============================] - 54s 718ms/step - loss: 0.0227 - accuracy: 0.9933 - val_loss: 0.1295 - val_accuracy: 0.9457
Epoch 7/20
75/75 [==============================] - 54s 718ms/step - loss: 0.0283 - accuracy: 0.9887 - val_loss: 0.1596 - val_accuracy: 0.9577
Epoch 8/20
75/75 [==============================] - 54s 722ms/step - loss: 0.0284 - accuracy: 0.9887 - val_loss: 0.0976 - val_accuracy: 0.9710
Epoch 9/20
75/75 [==============================] - 54s 716ms/step - loss: 0.0172 - accuracy: 0.9933 - val_loss: 0.1854 - val_accuracy: 0.9444
Epoch 10/20
75/75 [==============================] - 54s 714ms/step - loss: 0.0191 - accuracy: 0.9933 - val_loss: 0.3522 - val_accuracy: 0.8998
Epoch 11/20
75/75 [==============================] - 55s 730ms/step - loss: 0.0103 - accuracy: 0.9962 - val_loss: 0.0961 - val_accuracy: 0.9758
Epoch 12/20
75/75 [==============================] - 54s 720ms/step - loss: 0.0098 - accuracy: 0.9966 - val_loss: 0.0934 - val_accuracy: 0.9734
Epoch 13/20
75/75 [==============================] - 54s 717ms/step - loss: 9.1417e-04 - accuracy: 1.0000 - val_loss: 0.0797 - val_accuracy: 0.9783
Epoch 14/20
75/75 [==============================] - 54s 716ms/step - loss: 2.3197e-04 - accuracy: 1.0000 - val_loss: 0.0771 - val_accuracy: 0.9819
Epoch 15/20
75/75 [==============================] - 54s 721ms/step - loss: 1.3945e-04 - accuracy: 1.0000 - val_loss: 0.0770 - val_accuracy: 0.9807
Epoch 16/20
75/75 [==============================] - 54s 715ms/step - loss: 9.3226e-05 - accuracy: 1.0000 - val_loss: 0.0773 - val_accuracy: 0.9831
Epoch 17/20
75/75 [==============================] - 54s 716ms/step - loss: 6.8843e-05 - accuracy: 1.0000 - val_loss: 0.0773 - val_accuracy: 0.9831
Epoch 18/20
75/75 [==============================] - 54s 713ms/step - loss: 5.4362e-05 - accuracy: 1.0000 - val_loss: 0.0777 - val_accuracy: 0.9831
Epoch 19/20
75/75 [==============================] - 54s 715ms/step - loss: 4.3219e-05 - accuracy: 1.0000 - val_loss: 0.0782 - val_accuracy: 0.9831
Epoch 20/20
75/75 [==============================] - 54s 714ms/step - loss: 3.5648e-05 - accuracy: 1.0000 - val_loss: 0.0789 - val_accuracy: 0.9831

loss, acc = model.evaluate(test_ds)
print("Accuracy", acc)

6/6 [==============================] - 1s 140ms/step - loss: 0.1829 - accuracy: 0.9688
Accuracy 0.96875

3. 方法2：在dataset数据集中进行数据增强

batch_size = 32
AUTOTUNE = tf.data.AUTOTUNE

def prepare(ds):
    ds = ds.map(lambda x, y: (data_augmentation(x, training=True), y), num_parallel_calls=AUTOTUNE)
    return ds

train_ds = prepare(train_ds)

model = tf.keras.Sequential([
  #data_augmentation, 
  layers.Conv2D(16, 3, padding='same', activation='relu'),
  layers.MaxPooling2D(),
  layers.Conv2D(32, 3, padding='same', activation='relu'),
  layers.MaxPooling2D(),
  layers.Conv2D(64, 3, padding='same', activation='relu'),
  layers.MaxPooling2D(),
  layers.Flatten(),
  layers.Dense(128, activation='relu'),
  layers.Dense(len(class_names))
])
model.compile(optimizer='adam',
              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
              metrics=['accuracy'])
epochs=20
history = model.fit(
  train_ds,
  validation_data=val_ds,
  epochs=epochs
)

Epoch 1/20
75/75 [==============================] - 57s 744ms/step - loss: 0.7650 - accuracy: 0.5534 - val_loss: 0.6590 - val_accuracy: 0.5797
Epoch 2/20
75/75 [==============================] - 57s 745ms/step - loss: 0.5115 - accuracy: 0.7445 - val_loss: 0.4208 - val_accuracy: 0.7923
Epoch 3/20
75/75 [==============================] - 57s 745ms/step - loss: 0.3788 - accuracy: 0.8290 - val_loss: 0.3610 - val_accuracy: 0.8213
Epoch 4/20
75/75 [==============================] - 57s 749ms/step - loss: 0.2959 - accuracy: 0.8777 - val_loss: 0.2563 - val_accuracy: 0.8792
Epoch 5/20
75/75 [==============================] - 58s 767ms/step - loss: 0.2268 - accuracy: 0.9071 - val_loss: 0.2398 - val_accuracy: 0.8973
Epoch 6/20
75/75 [==============================] - 58s 760ms/step - loss: 0.2006 - accuracy: 0.9223 - val_loss: 0.1889 - val_accuracy: 0.9191
Epoch 7/20
75/75 [==============================] - 58s 759ms/step - loss: 0.1659 - accuracy: 0.9370 - val_loss: 0.1504 - val_accuracy: 0.9336
Epoch 8/20
75/75 [==============================] - 58s 761ms/step - loss: 0.1656 - accuracy: 0.9332 - val_loss: 0.1585 - val_accuracy: 0.9372
Epoch 9/20
75/75 [==============================] - 58s 763ms/step - loss: 0.1443 - accuracy: 0.9416 - val_loss: 0.1599 - val_accuracy: 0.9372
Epoch 10/20
75/75 [==============================] - 58s 764ms/step - loss: 0.1095 - accuracy: 0.9630 - val_loss: 0.1650 - val_accuracy: 0.9324
Epoch 11/20
75/75 [==============================] - 62s 815ms/step - loss: 0.1336 - accuracy: 0.9513 - val_loss: 0.1585 - val_accuracy: 0.9372
Epoch 12/20
75/75 [==============================] - 61s 804ms/step - loss: 0.1204 - accuracy: 0.9508 - val_loss: 0.1401 - val_accuracy: 0.9481
Epoch 13/20
75/75 [==============================] - 58s 762ms/step - loss: 0.1069 - accuracy: 0.9630 - val_loss: 0.1282 - val_accuracy: 0.9517
Epoch 14/20
75/75 [==============================] - 57s 754ms/step - loss: 0.0984 - accuracy: 0.9664 - val_loss: 0.1110 - val_accuracy: 0.9614
Epoch 15/20
75/75 [==============================] - 58s 758ms/step - loss: 0.1111 - accuracy: 0.9618 - val_loss: 0.1611 - val_accuracy: 0.9408
Epoch 16/20
75/75 [==============================] - 59s 779ms/step - loss: 0.1044 - accuracy: 0.9626 - val_loss: 0.1590 - val_accuracy: 0.9360
Epoch 17/20
75/75 [==============================] - 58s 756ms/step - loss: 0.0999 - accuracy: 0.9588 - val_loss: 0.1301 - val_accuracy: 0.9469
Epoch 18/20
75/75 [==============================] - 58s 765ms/step - loss: 0.0843 - accuracy: 0.9706 - val_loss: 0.1642 - val_accuracy: 0.9336
Epoch 19/20
75/75 [==============================] - 58s 757ms/step - loss: 0.0881 - accuracy: 0.9664 - val_loss: 0.1428 - val_accuracy: 0.9444
Epoch 20/20
75/75 [==============================] - 58s 762ms/step - loss: 0.0834 - accuracy: 0.9710 - val_loss: 0.1364 - val_accuracy: 0.9493

loss, acc = model.evaluate(test_ds)
print("Accuracy", acc)

6/6 [==============================] - 1s 116ms/step - loss: 0.0723 - accuracy: 0.9740
Accuracy 0.9739583134651184

test_ds准确率对比：

无数据增强： 0.96875
在model.fit中数据增强：0.95833
在dataset中数据增强： 0.97396
在dataset中的数据增强是有效的，在model.fit中的增强效果没有应用到model.evaluate中？后续再查资料确定一下。

三、自定义增强函数

import random
# 这是大家可以自由发挥的一个地方
def aug_img(image):
    seed = (random.randint(0,9), 0)
    # 随机改变图像对比度
    stateless_random_brightness = tf.image.stateless_random_contrast(image, lower=0.1, upper=1.0, seed=seed)
    return stateless_random_brightness

image = tf.expand_dims(images[3]*255, 0)
print("Min and max pixel values:", image.numpy().min(), image.numpy().max())

Min and max pixel values: 0.0 255.0

plt.figure(figsize=(8, 8))
for i in range(9):
    augmented_image = aug_img(image)
    ax = plt.subplot(3, 3, i + 1)
    plt.imshow(augmented_image[0].numpy().astype("uint8"))

    plt.axis("off")

在这里插入图片描述

将aug_img应用于图像预处理preprocess_image：（重新加载了一次数据集）

train_ds = tf.keras.preprocessing.image_dataset_from_directory(
    data_dir,
    validation_split=0.3,
    subset="training",
    seed=12,
    image_size=(img_height, img_width),
    batch_size=batch_size)

Found 3400 files belonging to 2 classes.
Using 2380 files for training.

val_ds = tf.keras.preprocessing.image_dataset_from_directory(
    data_dir,
    validation_split=0.3,
    subset="validation",
    seed=12,
    image_size=(img_height, img_width),
    batch_size=batch_size)

Found 3400 files belonging to 2 classes.
Using 1020 files for validation.

class_names = train_ds.class_names
print(class_names)

['cat', 'dog']

val_batches = tf.data.experimental.cardinality(val_ds)
test_ds     = val_ds.take(val_batches // 5)
val_ds      = val_ds.skip(val_batches // 5)

print('Number of validation batches: %d' % tf.data.experimental.cardinality(val_ds))
print('Number of test batches: %d' % tf.data.experimental.cardinality(test_ds))

Number of validation batches: 26
Number of test batches: 6

AUTOTUNE = tf.data.AUTOTUNE

def preprocess_image(image,label):
    image = image/255.0
    image = aug_img(image)
    return (image,label)

# 归一化处理
train_ds = train_ds.map(preprocess_image, num_parallel_calls=AUTOTUNE)
val_ds   = val_ds.map(preprocess_image, num_parallel_calls=AUTOTUNE)
test_ds  = test_ds.map(preprocess_image, num_parallel_calls=AUTOTUNE)

train_ds = train_ds.cache().prefetch(buffer_size=AUTOTUNE)
val_ds   = val_ds.cache().prefetch(buffer_size=AUTOTUNE)

展示进行了随机对比度处理后的图片情况：

plt.figure(figsize=(15, 10))

for images, labels in train_ds.take(1):
    for i in range(8):
        ax = plt.subplot(5, 8, i + 1)

        plt.imshow(images[i])
        plt.title(class_names[labels[i]])
        
        plt.axis("off")

在这里插入图片描述

train_ds = prepare(train_ds) #加入旋转

展示加入旋转后的图片情况：

plt.figure(figsize=(15, 10))

for images, labels in train_ds.take(1):
    for i in range(8):
        ax = plt.subplot(5, 8, i + 1)

        plt.imshow(images[i])
        plt.title(class_names[labels[i]])
        
        plt.axis("off")

在这里插入图片描述

model = tf.keras.Sequential([
  #data_augmentation, 
  layers.Conv2D(16, 3, padding='same', activation='relu'),
  layers.MaxPooling2D(),
  layers.Conv2D(32, 3, padding='same', activation='relu'),
  layers.MaxPooling2D(),
  layers.Conv2D(64, 3, padding='same', activation='relu'),
  layers.MaxPooling2D(),
  layers.Flatten(),
  layers.Dense(128, activation='relu'),
  layers.Dense(len(class_names))
])
model.compile(optimizer='adam',
              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
              metrics=['accuracy'])
epochs=20
history = model.fit(
  train_ds,
  validation_data=val_ds,
  epochs=epochs
)

Epoch 1/20
75/75 [==============================] - 65s 831ms/step - loss: 0.7219 - accuracy: 0.4962 - val_loss: 0.6898 - val_accuracy: 0.5519
Epoch 2/20
75/75 [==============================] - 57s 752ms/step - loss: 0.6890 - accuracy: 0.5290 - val_loss: 0.6407 - val_accuracy: 0.6099
Epoch 3/20
75/75 [==============================] - 58s 769ms/step - loss: 0.6082 - accuracy: 0.6819 - val_loss: 0.8206 - val_accuracy: 0.6244
Epoch 4/20
75/75 [==============================] - 58s 761ms/step - loss: 0.5350 - accuracy: 0.7391 - val_loss: 0.7123 - val_accuracy: 0.7089
Epoch 5/20
75/75 [==============================] - 58s 759ms/step - loss: 0.4415 - accuracy: 0.7983 - val_loss: 0.7518 - val_accuracy: 0.7246
Epoch 6/20
75/75 [==============================] - 60s 784ms/step - loss: 0.3982 - accuracy: 0.8172 - val_loss: 0.6058 - val_accuracy: 0.8019
Epoch 7/20
75/75 [==============================] - 59s 769ms/step - loss: 0.3607 - accuracy: 0.8395 - val_loss: 0.5616 - val_accuracy: 0.8056
Epoch 8/20
75/75 [==============================] - 59s 778ms/step - loss: 0.3516 - accuracy: 0.8534 - val_loss: 0.4924 - val_accuracy: 0.8237
Epoch 9/20
75/75 [==============================] - 57s 747ms/step - loss: 0.3328 - accuracy: 0.8597 - val_loss: 0.4675 - val_accuracy: 0.8394
Epoch 10/20
75/75 [==============================] - 57s 745ms/step - loss: 0.3170 - accuracy: 0.8630 - val_loss: 0.5189 - val_accuracy: 0.8333
Epoch 11/20
75/75 [==============================] - 57s 753ms/step - loss: 0.2900 - accuracy: 0.8731 - val_loss: 0.3477 - val_accuracy: 0.8623
Epoch 12/20
75/75 [==============================] - 57s 748ms/step - loss: 0.2923 - accuracy: 0.8786 - val_loss: 0.4257 - val_accuracy: 0.8623
Epoch 13/20
75/75 [==============================] - 57s 749ms/step - loss: 0.2618 - accuracy: 0.8945 - val_loss: 0.3925 - val_accuracy: 0.8720
Epoch 14/20
75/75 [==============================] - 58s 762ms/step - loss: 0.2414 - accuracy: 0.9067 - val_loss: 0.3254 - val_accuracy: 0.8925
Epoch 15/20
75/75 [==============================] - 57s 746ms/step - loss: 0.2258 - accuracy: 0.9105 - val_loss: 0.3228 - val_accuracy: 0.8925
Epoch 16/20
75/75 [==============================] - 56s 739ms/step - loss: 0.2292 - accuracy: 0.9113 - val_loss: 0.3775 - val_accuracy: 0.8865
Epoch 17/20
75/75 [==============================] - 57s 748ms/step - loss: 0.2103 - accuracy: 0.9181 - val_loss: 0.2988 - val_accuracy: 0.9022
Epoch 18/20
75/75 [==============================] - 57s 743ms/step - loss: 0.2040 - accuracy: 0.9139 - val_loss: 0.2662 - val_accuracy: 0.9106
Epoch 19/20
75/75 [==============================] - 57s 751ms/step - loss: 0.1726 - accuracy: 0.9345 - val_loss: 0.2835 - val_accuracy: 0.9058
Epoch 20/20
75/75 [==============================] - 57s 751ms/step - loss: 0.1910 - accuracy: 0.9256 - val_loss: 0.2658 - val_accuracy: 0.9094

loss, acc = model.evaluate(test_ds)
print("Accuracy", acc)

6/6 [==============================] - 1s 121ms/step - loss: 0.1690 - accuracy: 0.9323
Accuracy 0.9322916865348816

感觉调整对比度的数据增强方法可能对于低对比度的测试数据集效果会更好，但让图片本身的特征不那么明显了，可能多训练几轮效果会更好。

四、总结

数据增强的位置：
1. 在数据集中对图像进行处理。
2. 在model.fit中嵌入数据增强。
数据增强的方式：
1. 调整旋转角度：

data_augmentation = tf.keras.Sequential([
  tf.keras.layers.experimental.preprocessing.RandomFlip("horizontal_and_vertical"),
  tf.keras.layers.experimental.preprocessing.RandomRotation(0.2),# 按照0.2的弧度
])
def prepare(ds):
    ds = ds.map(lambda x, y: (data_augmentation(x, training=True), y), num_parallel_calls=AUTOTUNE)
    return ds

train_ds = prepare(train_ds)

调整对比度：

def aug_img(image):
    seed = (random.randint(0,9), 0)
    # 随机改变图像对比度
    stateless_random_brightness = tf.image.stateless_random_contrast(image, lower=0.1, upper=1.0, seed=seed)
    return stateless_random_brightness

月亮泡面=

关注

14
点赞
踩
18

收藏

觉得还不错? 一键收藏
1
评论
T10 数据增强学习记录

这是大家可以自由发挥的一个地方# 随机改变图像对比度将aug_img应用于图像预处理preprocess_image：（重新加载了一次数据集）data_dir,seed=12,data_dir,seed=12,# 归一化处理train_ds = prepare(train_ds) #加入旋转])epochs=20train_ds,Epoch 1/20Epoch 2/20Epoch 3/20Epoch 4/20Epoch 5/20Epoch 6/20。
复制链接

扫一扫