- 🍨 本文为🔗365天深度学习训练营 中的学习记录博客
- 🍦 参考文章：365天深度学习训练营-第P1周：实现mnist手写数字识别（训练营内部成员可读）
- 🍖 原作者：K同学啊|接辅导、项目定制
目录
1.1 Keras 模块中的 layers 和 models
1.3 image_dataset_from_directory 函数
知识点说明：
一、Keras:
Keras 是一个高级神经网络 API，用于构建、训练和测试深度学习模型。它的设计重点是使神经网络的构建和实验变得更加简单、快速和可扩展。
Keras 提供了一种直观、模块化和可扩展的接口，允许开发者快速构建各种类型的神经网络模型，包括卷积神经网络 (CNN)、循环神经网络 (RNN)、深度前馈网络 (DNN) 等。它也支持多种深度学习任务，如分类、回归、聚类、生成对抗网络 (GAN) 等。
Keras 为开发者提供了丰富的功能，包括：
-
简单易用: Keras 提供了简单直观的 API，允许开发者通过简单的函数调用来构建模型、添加层、定义损失函数、选择优化算法等。
-
模块化: 可以通过简单的组合层来创建复杂的模型，每个层都可以看作是一个模块，可以随意组合。
-
灵活性: Keras 允许用户使用 TensorFlow、Microsoft Cognitive Toolkit (CNTK) 或 Theano 作为后端，提供了跨平台、多后端的支持。
-
易于扩展: 可以轻松扩展 Keras，添加自定义组件、损失函数、层或者初始化策略，以满足特定需求。
-
高效训练: 支持 GPU 和多 GPU 训练，充分利用硬件资源进行高效训练。
Keras 最初由 François Chollet 开发，并作为独立项目。在 TensorFlow 2.0 之后，Keras 被整合到 TensorFlow 中，成为 TensorFlow 的官方高级 API，这也使得 TensorFlow 更加易于使用和理解。
1.1 Keras 模块中的 layers 和 models
这两个模块是 Keras 提供的重要组件，用于构建深度学习模型。
-
layers: 这个模块包含了构建神经网络模型所需的各种层，如全连接层 (Dense)、卷积层 (Conv2D)、循环层 (LSTM,GRU) 等。通过使用这些层，可以在模型中逐层添加不同类型的神经网络层，从而构建出所需的网络结构。 -
models: 这个模块包含了构建和训练模型的类和函数。其中，Sequential类允许按顺序堆叠各种层来构建模型，而Model类则允许定义更加复杂的模型结构，包括多输入、多输出等情况。
通过导入这两个模块，可以使用其中的类和函数来构建神经网络模型，定义层次结构，编译模型，进行训练和评估等操作。
1.2 preprocessing
在 Keras 中，keras.preprocessing 模块提供了一系列用于数据预处理和数据增强的工具，以帮助您准备输入数据，使其适合用于训练神经网络模型。这些功能通常用于图像、文本等类型的数据。
1.3 image_dataset_from_directory 函数
image_dataset_from_directory 函数是 TensorFlow 的 Keras API 中用于从目录加载图像数据集的函数。它允许您轻松地从目录结构中加载图像数据，然后创建一个数据集对象，以便用于模型的训练和评估。
1.3.1 subset 参数
在 image_dataset_from_directory 函数中，subset 参数是用于指定加载数据集的子集的选项。该参数允许您选择加载整个数据集、训练集或验证集。
具体来说，subset 参数可以接受以下几种值：
-
"training": 选择加载训练集。这将加载指定目录下的训练集图像。 -
"validation": 选择加载验证集。这将加载指定目录下的验证集图像。 -
None或不指定: 如果不指定subset参数或指定为None，则默认加载整个数据集，包括训练集和验证集。
1.3.2 validation_split 参数
image_dataset_from_directory 函数默认情况下会将数据集分成训练集和验证集，而不会分成训练集、测试集和验证集。一般的划分是按照指定的比例将数据划分成训练集和验证集。
这个比例可以通过参数 validation_split 来指定，默认为 0.2，即 80% 的数据用于训练，20% 的数据用于验证。
1.3.3 class_names 属性
通过tf.keras.preprocessing.image_dataset_from_directory创建数据集
通过 train_dataset.class_names 输出数据集的标签，可以在创建数据集后直接访问 class_names 属性。
这些标签通常对应于数据集目录中的子目录名，每个子目录代表一个类别。
1.3.4 图像对应的标签（labels）
使用 tf.keras.preprocessing.image_dataset_from_directory 创建数据集时，默认情况下，图像对应的标签（labels）是整数，而且这些整数会自动从 0 开始递增，对应于子目录的索引值。
具体来说：
-
对于每个子目录，函数会将其视为一个类别，标签从 0 开始递增，每个子目录对应一个整数标签。
-
在创建数据集时，可以通过参数
labels设置标签的处理方式：-
"inferred": 默认值，会自动为每个子目录分配一个整数标签，从 0 开始递增。 -
"categorical": 将标签视为分类的 one-hot 编码。 -
"binary": 将标签视为二进制分类的 one-hot 编码。 -
自定义函数: 通过传递一个函数来自定义标签的处理方式。
-
示例代码：
import tensorflow as tf
from tensorflow.keras.preprocessing import image_dataset_from_directory
# 定义图像数据集目录
data_dir = "path/to/data"
# 创建图像数据集
# labels="inferred" 表示自动推断整数标签
train_dataset = image_dataset_from_directory(
data_dir,
labels="inferred", # 自动推断标签
validation_split=0.2, # 20% 数据用于验证
subset="training", # 只加载训练集
seed=1337, # 随机种子，用于数据划分的随机性
image_size=(224, 224), # 图像尺寸
batch_size=32 # 批次大小
)
# 显示第一个批次的图像和标签
for images, labels in train_dataset.take(1):
print("Images shape:", images.shape)
print("Labels:", labels)在这个示例中，我们使用 image_dataset_from_directory 创建了一个图像数据集，并输出了第一个批次的图像和标签。标签是整数，对应于子目录的索引值。
1.4 Input 类
Input 是 Keras 中用于定义模型输入的类。它用于创建一个张量作为模型的输入节点，作为模型的起始点。
基本语法：
from tensorflow.keras.layers import Input
input_tensor = Input(shape=(height, width, channels))其中：
-
shape=(height, width, channels)指定了输入张量的形状，height表示高度，width表示宽度，channels表示通道数。 -
input_tensor是创建的输入张量对象，可以作为模型的输入。
Input 类实际上是一个函数，它返回一个张量对象，这个张量可以用作模型的输入。Input 也可以用于定义模型的输入层，通常作为模型的第一层。
示例用法：
from tensorflow.keras.layers import Input
# 定义输入张量
input_tensor = Input(shape=(128, 128, 3))
# 可以将这个输入张量作为模型的输入
# ...
# 使用定义的输入张量作为模型的第一层
# model = Model(inputs=input_tensor, ...)在示例中，首先使用 Input 创建了一个输入张量 input_tensor，指定了输入张量的形状为 (128, 128, 3)。接下来可以将这个输入张量用作模型的输入，也可以作为模型的第一层。
二、pyplot:
plt.figure(figsize=(10, 4)) 是一个 Matplotlib 函数，用于创建一个指定尺寸的新图像窗口。它的参数 figsize 用于设置图像的宽度和高度，以英寸为单位。
-
figsize=(10, 4)指定了图像的宽度为 10 英寸，高度为 4 英寸。
通常情况下，我们会在创建图像窗口后，使用其他 Matplotlib 函数来向图像窗口中添加子图、绘制图形等。
例如，您可以在此图像窗口中添加子图并绘制折线图：
import matplotlib.pyplot as plt
# 创建图像窗口
plt.figure(figsize=(10, 4))
# 添加子图
plt.subplot(1, 2, 1) # 1 行 2 列，第 1 个子图
plt.plot([1, 2, 3, 4], [10, 20, 15, 25])
# 添加子图
plt.subplot(1, 2, 2) # 1 行 2 列，第 2 个子图
plt.plot([1, 2, 3, 4], [30, 25, 20, 35])
# 显示图像窗口
plt.show()在这个示例中，我们创建了一个尺寸为 (10, 4) 的图像窗口，并向其中添加了两个子图，分别绘制了折线图。然后通过 plt.show() 函数显示图像窗口。
三、map() 函数:
map() 函数是 Python 中常用的函数之一，用于对可迭代对象（如列表、元组、集合等）中的每个元素应用一个指定的函数，并返回一个包含结果的迭代器。在 TensorFlow 数据处理中，map() 函数常用于对数据集中的元素进行预处理。
在 TensorFlow 中，map() 函数通常用于数据集对象（如 tf.data.Dataset）中。它可以将一个函数应用于数据集中的每个元素（如图像、标签等），实现对数据集的预处理、变换等操作。
基本语法如下：
mapped_dataset = original_dataset.map(lambda x: your_function(x))其中，original_dataset 是原始数据集对象，your_function 是你希望应用于数据集中每个元素的函数。这个函数会接收数据集中的一个元素作为输入，对其进行处理并返回处理后的结果。
示例用法：
import tensorflow as tf
# 假设有一个数据集 dataset 包含图像和标签
# 定义一个预处理函数
def preprocess_image(image):
# 在这里可以进行图像预处理操作，例如归一化、调整尺寸等
return image
# 将预处理函数应用于数据集中的图像
processed_dataset = dataset.map(lambda x: preprocess_image(x))
# 打印处理后的图像
for image in processed_dataset:
print(image)在这个示例中，preprocess_image 函数是一个自定义的图像预处理函数，map() 函数将这个预处理函数应用于数据集中的每个图像。
四、VGG-16网络结构说明
VGG-16（Visual Geometry Group-16）是由牛津大学的Visual Geometry Group团队在2014年提出的一个经典的卷积神经网络架构，特别适用于图像识别任务。VGG-16的结构相对简单而且非常深，它的核心是深度堆叠的卷积层。
结构说明：
-
13个卷积层（Convolutional Layer），分别用
blockX_convX表示
-
3个全连接层（Fully connected Layer），分别用
fcX与predictions表示
-
5个池化层（Pool layer），分别用
blockX_pool表示
VGG-16包含了16个隐藏层（13个卷积层和3个全连接层），故称为VGG-16
以下是VGG-16的网络结构详细说明：
-
输入层：
-
输入层接受图像数据，通常为RGB三通道图像。
-
-
卷积层：
-
VGG-16的卷积部分由13个卷积层组成，这些卷积层使用3x3的卷积核，步幅为1，填充为1，保持输入图像的尺寸不变。
-
卷积层之后通常接ReLU激活函数，增加非线性特征。
-
-
池化层：
-
在每个卷积块的后面加入池化层，使用2x2的最大池化操作，步幅为2，将特征图的尺寸减半。
-
-
全连接层：
-
在卷积部分之后有3个全连接层，每个全连接层包含4096个神经元，使用ReLU激活函数。
-
-
输出层：
-
输出层是一个具有类别数目的全连接层，通常使用softmax激活函数进行多类别分类。
-
整体网络结构如下：
Input -> [Conv2D -> ReLU -> Conv2D -> ReLU -> MaxPooling] * 5 -> Flatten -> Dense -> ReLU -> Dense -> ReLU -> Dense -> Softmax
VGG-16的特点是网络非常深，使用小尺寸卷积核和池化层，以及重复性的结构。这种设计可以有效地提取图像的特征，对于图像识别任务取得了很好的性能。
代码与运行结果：
import tensorflow as tf
gpus = tf.config.list_physical_devices("GPU")
if gpus:
gpu0 = gpus[0]
# experimental 在 TensorFlow 中是一个命名空间（namespace），用于包含一些实验性的、可能会发生变化或在未来版本中可能会更改的功能和接口。
tf.config.experimental.set_memory_growth(gpu0, True)
tf.config.set_visible_devices([gpu0], "GPU")
gpusfrom tensorflow import keras
from tensorflow.keras import layers, models
import numpy as np
import matplotlib.pyplot as plt
import os, pathlib, PIL
data_dir = './49-data/'
data_dir = pathlib.Path(data_dir)
image_count = len(list(data_dir.glob("*/*.png")))
print(type(image_count))
print("图片总数为：" , image_count)<class 'int'>
图片总数为： 1200batch_size = 32
img_height = 224
img_width = 224
train_ds = tf.keras.preprocessing.image_dataset_from_directory(
data_dir,
batch_size=batch_size,
image_size=(img_height, img_width),
seed=123,
validation_split=0.2,
subset="training"
)
val_ds = tf.keras.preprocessing.image_dataset_from_directory(
data_dir,
batch_size=batch_size,
image_size=(img_height, img_width),
seed=123,
validation_split=0.2,
subset="validation"
)
class_names = train_ds.class_names
print("数据集标签：", class_names)Found 1200 files belonging to 4 classes.
Using 960 files for training.
Found 1200 files belonging to 4 classes.
Using 240 files for validation.
数据集标签： ['Dark', 'Green', 'Light', 'Medium']plt.figure(figsize=(10, 4))
for images, labels in train_ds.take(1):
for i in range(8):
ax = plt.subplot(2, 4, i+1)
plt.imshow(images[i].numpy().astype("uint8"))
plt.title(class_names[labels[i]])
plt.axis("off")
for image_batch, labels_batch in train_ds:
print(image_batch.shape)
print(labels_batch.shape)
break
AUTOTUNE = tf.data.experimental.AUTOTUNE
train_ds = train_ds.cache().shuffle(1000).prefetch(buffer_size=AUTOTUNE)
val_ds = val_ds.cache().prefetch(buffer_size=AUTOTUNE)
normalization_layer = layers.experimental.preprocessing.Rescaling(1./255)
# 将训练集 train_ds 和验证集 val_ds 中的图像数据通过 map 函数，应用归一化层进行归一化处理。
train_ds = train_ds.map(lambda x, y:(normalization_layer(x), y))
val_ds = val_ds.map(lambda x, y:(normalization_layer(x), y))
# 通过 next(iter(val_ds)) 取得一个批次的图像数据，并输出该批次中第一张图像的最小值和最大值。
image_batch, labels_batch = next(iter(val_ds))
first_image = image_batch[0]
print(np.min(first_image), np.max(first_image)) 
0.0 1.0from tensorflow.keras import layers, models, Input
from tensorflow.keras.models import Model
from tensorflow.keras.layers import Conv2D, MaxPooling2D, Dense, Flatten, Dropout
"""
代码中构建了VGG16模型的各个层，按照VGG16的结构将卷积层、池化层和全连接层依次连接起来。
整个VGG16模型的结构分为5个卷积块（block），每个卷积块包含一到多个卷积层和一个池化层。
在最后的全连接层之前，通过Flatten将最后一个卷积块的输出展平，然后接上两个全连接层，并最终输出预测类别的 softmax 层。
最后，使用 Model 类将输入张量和输出张量连接起来，构建模型并返回。
"""
def VGG16_1(nb_classes, input_shape):
input_tensor = Input(shape=input_shape)
# 1st block
x = Conv2D(64, (3,3), activation='relu', padding='same',name='block1_conv1')(input_tensor)
x = Conv2D(64, (3,3), activation='relu', padding='same',name='block1_conv2')(x)
x = MaxPooling2D((2,2), strides=(2,2), name = 'block1_pool')(x)
# 2nd block
x = Conv2D(128, (3,3), activation='relu', padding='same',name='block2_conv1')(x)
x = Conv2D(128, (3,3), activation='relu', padding='same',name='block2_conv2')(x)
x = MaxPooling2D((2,2), strides=(2,2), name = 'block2_pool')(x)
# 3rd block
x = Conv2D(256, (3,3), activation='relu', padding='same',name='block3_conv1')(x)
x = Conv2D(256, (3,3), activation='relu', padding='same',name='block3_conv2')(x)
x = Conv2D(256, (3,3), activation='relu', padding='same',name='block3_conv3')(x)
x = MaxPooling2D((2,2), strides=(2,2), name = 'block3_pool')(x)
# 4th block
x = Conv2D(512, (3,3), activation='relu', padding='same',name='block4_conv1')(x)
x = Conv2D(512, (3,3), activation='relu', padding='same',name='block4_conv2')(x)
x = Conv2D(512, (3,3), activation='relu', padding='same',name='block4_conv3')(x)
x = MaxPooling2D((2,2), strides=(2,2), name = 'block4_pool')(x)
# 5th block
x = Conv2D(512, (3,3), activation='relu', padding='same',name='block5_conv1')(x)
x = Conv2D(512, (3,3), activation='relu', padding='same',name='block5_conv2')(x)
x = Conv2D(512, (3,3), activation='relu', padding='same',name='block5_conv3')(x)
x = MaxPooling2D((2,2), strides=(2,2), name = 'block5_pool')(x)
# full connection
x = Flatten()(x)
x = Dense(4096, activation='relu', name='fc1')(x)
x = Dense(4096, activation='relu', name='fc2')(x)
output_tensor = Dense(nb_classes, activation='softmax', name='predictions')(x)
model = Model(input_tensor, output_tensor)
return model
model_1=VGG16_1(len(class_names), (img_width, img_height, 3))
model_1.summary()Model: "model"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) [(None, 224, 224, 3)] 0
_________________________________________________________________
block1_conv1 (Conv2D) (None, 224, 224, 64) 1792
_________________________________________________________________
block1_conv2 (Conv2D) (None, 224, 224, 64) 36928
_________________________________________________________________
block1_pool (MaxPooling2D) (None, 112, 112, 64) 0
_________________________________________________________________
block2_conv1 (Conv2D) (None, 112, 112, 128) 73856
_________________________________________________________________
block2_conv2 (Conv2D) (None, 112, 112, 128) 147584
_________________________________________________________________
block2_pool (MaxPooling2D) (None, 56, 56, 128) 0
_________________________________________________________________
block3_conv1 (Conv2D) (None, 56, 56, 256) 295168
_________________________________________________________________
block3_conv2 (Conv2D) (None, 56, 56, 256) 590080
_________________________________________________________________
block3_conv3 (Conv2D) (None, 56, 56, 256) 590080
_________________________________________________________________
block3_pool (MaxPooling2D) (None, 28, 28, 256) 0
_________________________________________________________________
block4_conv1 (Conv2D) (None, 28, 28, 512) 1180160
_________________________________________________________________
block4_conv2 (Conv2D) (None, 28, 28, 512) 2359808
_________________________________________________________________
block4_conv3 (Conv2D) (None, 28, 28, 512) 2359808
_________________________________________________________________
block4_pool (MaxPooling2D) (None, 14, 14, 512) 0
_________________________________________________________________
block5_conv1 (Conv2D) (None, 14, 14, 512) 2359808
_________________________________________________________________
block5_conv2 (Conv2D) (None, 14, 14, 512) 2359808
_________________________________________________________________
block5_conv3 (Conv2D) (None, 14, 14, 512) 2359808
_________________________________________________________________
block5_pool (MaxPooling2D) (None, 7, 7, 512) 0
_________________________________________________________________
flatten (Flatten) (None, 25088) 0
_________________________________________________________________
fc1 (Dense) (None, 4096) 102764544
_________________________________________________________________
fc2 (Dense) (None, 4096) 16781312
_________________________________________________________________
predictions (Dense) (None, 4) 16388
=================================================================
Total params: 134,276,932
Trainable params: 134,276,932
Non-trainable params: 0
_________________________________________________________________from tensorflow.keras import layers, models, Input
from tensorflow.keras import regularizers
from tensorflow.keras.models import Model
from tensorflow.keras.layers import Conv2D, MaxPooling2D, Dense, Flatten, Dropout, BatchNormalization
# 在VGG16中，为减少过拟合风险，通常的做法是在全连接层之后添加 Dropout 层，因为全连接层的参数数量通常比卷积层多，因此容易过拟合。
def VGG16_2(nb_classes, input_shape):
input_tensor = Input(shape=input_shape)
# 1st block
x = Conv2D(64, (3,3), activation='relu', padding='same',name='block1_conv1')(input_tensor)
x = BatchNormalization()(x) # 添加BN层
x = Conv2D(64, (3,3), activation='relu', padding='same',name='block1_conv2')(x)
x = BatchNormalization()(x) # 添加BN层
x = MaxPooling2D((2,2), strides=(2,2), name = 'block1_pool')(x)
# 2nd block
x = Conv2D(128, (3,3), activation='relu', padding='same',name='block2_conv1')(x)
x = BatchNormalization()(x) # 添加BN层
x = Conv2D(128, (3,3), activation='relu', padding='same',name='block2_conv2')(x)
x = BatchNormalization()(x) # 添加BN层
x = MaxPooling2D((2,2), strides=(2,2), name = 'block2_pool')(x)
# 3rd block
x = Conv2D(256, (3,3), activation='relu', padding='same',name='block3_conv1')(x)
x = BatchNormalization()(x) # 添加BN层
x = Conv2D(256, (3,3), activation='relu', padding='same',name='block3_conv2')(x)
x = BatchNormalization()(x) # 添加BN层
x = Conv2D(256, (3,3), activation='relu', padding='same',name='block3_conv3')(x)
x = BatchNormalization()(x) # 添加BN层
x = MaxPooling2D((2,2), strides=(2,2), name = 'block3_pool')(x)
# 4th block
x = Conv2D(512, (3,3), activation='relu', padding='same',name='block4_conv1')(x)
x = BatchNormalization()(x) # 添加BN层
x = Conv2D(512, (3,3), activation='relu', padding='same',name='block4_conv2')(x)
x = BatchNormalization()(x) # 添加BN层
x = Conv2D(512, (3,3), activation='relu', padding='same',name='block4_conv3')(x)
x = BatchNormalization()(x) # 添加BN层
x = MaxPooling2D((2,2), strides=(2,2), name = 'block4_pool')(x)
# 5th block
x = Conv2D(512, (3,3), activation='relu', padding='same',name='block5_conv1')(x)
x = BatchNormalization()(x) # 添加BN层
x = Conv2D(512, (3,3), activation='relu', padding='same',name='block5_conv2')(x)
x = BatchNormalization()(x) # 添加BN层
x = Conv2D(512, (3,3), activation='relu', padding='same',name='block5_conv3')(x)
x = BatchNormalization()(x) # 添加BN层
x = MaxPooling2D((2,2), strides=(2,2), name = 'block5_pool')(x)
# full connection
x = Flatten()(x)
x = Dense(4096, activation='relu', kernel_regularizer=regularizers.l2(0.1), name='fc1')(x)
x = Dropout(0.4)(x) # 添加Dropout层
x = Dense(4096, activation='relu', kernel_regularizer=regularizers.l2(0.1), name='fc2')(x)
x = Dropout(0.4)(x) # 添加Dropout层
output_tensor = Dense(nb_classes, activation='softmax', name='predictions')(x)
model = Model(input_tensor, output_tensor)
return model
model_2=VGG16_2(len(class_names), (img_width, img_height, 3))
model_2.summary()Model: "model_1"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_2 (InputLayer) [(None, 224, 224, 3)] 0
_________________________________________________________________
block1_conv1 (Conv2D) (None, 224, 224, 64) 1792
_________________________________________________________________
batch_normalization (BatchNo (None, 224, 224, 64) 256
_________________________________________________________________
block1_conv2 (Conv2D) (None, 224, 224, 64) 36928
_________________________________________________________________
batch_normalization_1 (Batch (None, 224, 224, 64) 256
_________________________________________________________________
block1_pool (MaxPooling2D) (None, 112, 112, 64) 0
_________________________________________________________________
block2_conv1 (Conv2D) (None, 112, 112, 128) 73856
_________________________________________________________________
batch_normalization_2 (Batch (None, 112, 112, 128) 512
_________________________________________________________________
block2_conv2 (Conv2D) (None, 112, 112, 128) 147584
_________________________________________________________________
batch_normalization_3 (Batch (None, 112, 112, 128) 512
_________________________________________________________________
block2_pool (MaxPooling2D) (None, 56, 56, 128) 0
_________________________________________________________________
block3_conv1 (Conv2D) (None, 56, 56, 256) 295168
_________________________________________________________________
batch_normalization_4 (Batch (None, 56, 56, 256) 1024
_________________________________________________________________
block3_conv2 (Conv2D) (None, 56, 56, 256) 590080
_________________________________________________________________
batch_normalization_5 (Batch (None, 56, 56, 256) 1024
_________________________________________________________________
block3_conv3 (Conv2D) (None, 56, 56, 256) 590080
_________________________________________________________________
batch_normalization_6 (Batch (None, 56, 56, 256) 1024
_________________________________________________________________
block3_pool (MaxPooling2D) (None, 28, 28, 256) 0
_________________________________________________________________
block4_conv1 (Conv2D) (None, 28, 28, 512) 1180160
_________________________________________________________________
batch_normalization_7 (Batch (None, 28, 28, 512) 2048
_________________________________________________________________
block4_conv2 (Conv2D) (None, 28, 28, 512) 2359808
_________________________________________________________________
batch_normalization_8 (Batch (None, 28, 28, 512) 2048
_________________________________________________________________
block4_conv3 (Conv2D) (None, 28, 28, 512) 2359808
_________________________________________________________________
batch_normalization_9 (Batch (None, 28, 28, 512) 2048
_________________________________________________________________
block4_pool (MaxPooling2D) (None, 14, 14, 512) 0
_________________________________________________________________
block5_conv1 (Conv2D) (None, 14, 14, 512) 2359808
_________________________________________________________________
batch_normalization_10 (Batc (None, 14, 14, 512) 2048
_________________________________________________________________
block5_conv2 (Conv2D) (None, 14, 14, 512) 2359808
_________________________________________________________________
batch_normalization_11 (Batc (None, 14, 14, 512) 2048
_________________________________________________________________
block5_conv3 (Conv2D) (None, 14, 14, 512) 2359808
_________________________________________________________________
batch_normalization_12 (Batc (None, 14, 14, 512) 2048
_________________________________________________________________
block5_pool (MaxPooling2D) (None, 7, 7, 512) 0
_________________________________________________________________
flatten_1 (Flatten) (None, 25088) 0
_________________________________________________________________
fc1 (Dense) (None, 4096) 102764544
_________________________________________________________________
dropout (Dropout) (None, 4096) 0
_________________________________________________________________
fc2 (Dense) (None, 4096) 16781312
_________________________________________________________________
dropout_1 (Dropout) (None, 4096) 0
_________________________________________________________________
predictions (Dense) (None, 4) 16388
=================================================================
Total params: 134,293,828
Trainable params: 134,285,380
Non-trainable params: 8,448
_________________________________________________________________# 设置初始学习率
initial_learning_rate = 1e-4
lr_schedule = tf.keras.optimizers.schedules.ExponentialDecay(
initial_learning_rate,
decay_steps=30, # 敲黑板！！！这里是指 steps，不是指epochs
decay_rate=0.92, # lr经过一次衰减就会变成 decay_rate*lr
staircase=True)
# 设置优化器
opt = tf.keras.optimizers.Adam(learning_rate=initial_learning_rate)model_1.compile(optimizer=opt,
loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
metrics=['accuracy'])
model_2.compile(optimizer=opt,
loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
metrics=['accuracy'])epochs = 20
history1 = model_1.fit(
train_ds,
validation_data=val_ds,
epochs=epochs
)Epoch 1/20
/root/miniconda3/lib/python3.8/site-packages/tensorflow/python/keras/backend.py:4929: UserWarning: "`sparse_categorical_crossentropy` received `from_logits=True`, but the `output` argument was produced by a sigmoid or softmax activation and thus does not represent logits. Was this intended?"
warnings.warn(
2023-09-22 18:46:37.337736: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudnn.so.8
2023-09-22 18:46:38.004925: I tensorflow/stream_executor/cuda/cuda_dnn.cc:359] Loaded cuDNN version 8101
2023-09-22 18:46:39.011395: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcublas.so.11
2023-09-22 18:46:39.611609: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcublasLt.so.11
2023-09-22 18:46:41.417131: I tensorflow/stream_executor/cuda/cuda_blas.cc:1838] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once.
30/30 [==============================] - 11s 163ms/step - loss: 1.3867 - accuracy: 0.2594 - val_loss: 1.3879 - val_accuracy: 0.2125
Epoch 2/20
30/30 [==============================] - 4s 122ms/step - loss: 1.2142 - accuracy: 0.3854 - val_loss: 1.0887 - val_accuracy: 0.6000
Epoch 3/20
30/30 [==============================] - 4s 128ms/step - loss: 0.8916 - accuracy: 0.5323 - val_loss: 0.8058 - val_accuracy: 0.4667
Epoch 4/20
30/30 [==============================] - 4s 122ms/step - loss: 0.6227 - accuracy: 0.6656 - val_loss: 0.4256 - val_accuracy: 0.8083
Epoch 5/20
30/30 [==============================] - 4s 123ms/step - loss: 0.5242 - accuracy: 0.7344 - val_loss: 0.4505 - val_accuracy: 0.8167
Epoch 6/20
30/30 [==============================] - 4s 123ms/step - loss: 0.2728 - accuracy: 0.8917 - val_loss: 0.2606 - val_accuracy: 0.8958
Epoch 7/20
30/30 [==============================] - 4s 123ms/step - loss: 0.1560 - accuracy: 0.9417 - val_loss: 0.0720 - val_accuracy: 0.9792
Epoch 8/20
30/30 [==============================] - 4s 124ms/step - loss: 0.1485 - accuracy: 0.9479 - val_loss: 0.1295 - val_accuracy: 0.9667
Epoch 9/20
30/30 [==============================] - 4s 129ms/step - loss: 0.0707 - accuracy: 0.9677 - val_loss: 0.0676 - val_accuracy: 0.9708
Epoch 10/20
30/30 [==============================] - 4s 128ms/step - loss: 0.2462 - accuracy: 0.9198 - val_loss: 0.0822 - val_accuracy: 0.9750
Epoch 11/20
30/30 [==============================] - 4s 127ms/step - loss: 0.0530 - accuracy: 0.9865 - val_loss: 0.0618 - val_accuracy: 0.9875
Epoch 12/20
30/30 [==============================] - 4s 123ms/step - loss: 0.0396 - accuracy: 0.9896 - val_loss: 0.1289 - val_accuracy: 0.9625
Epoch 13/20
30/30 [==============================] - 4s 128ms/step - loss: 0.0367 - accuracy: 0.9823 - val_loss: 0.0433 - val_accuracy: 0.9833
Epoch 14/20
30/30 [==============================] - 4s 128ms/step - loss: 0.0202 - accuracy: 0.9937 - val_loss: 0.0438 - val_accuracy: 0.9875
Epoch 15/20
30/30 [==============================] - 4s 123ms/step - loss: 0.1125 - accuracy: 0.9677 - val_loss: 0.0786 - val_accuracy: 0.9583
Epoch 16/20
30/30 [==============================] - 4s 127ms/step - loss: 0.0297 - accuracy: 0.9917 - val_loss: 0.0496 - val_accuracy: 0.9917
Epoch 17/20
30/30 [==============================] - 4s 125ms/step - loss: 0.0281 - accuracy: 0.9885 - val_loss: 0.0545 - val_accuracy: 0.9833
Epoch 18/20
30/30 [==============================] - 4s 127ms/step - loss: 0.0109 - accuracy: 0.9969 - val_loss: 0.0626 - val_accuracy: 0.9750
Epoch 19/20
30/30 [==============================] - 4s 123ms/step - loss: 0.0066 - accuracy: 0.9990 - val_loss: 0.0940 - val_accuracy: 0.9708
Epoch 20/20
30/30 [==============================] - 4s 128ms/step - loss: 0.1222 - accuracy: 0.9604 - val_loss: 0.5158 - val_accuracy: 0.7958epochs2 = 35
history2 = model_2.fit(
train_ds,
validation_data=val_ds,
epochs=epochs2
)Epoch 1/35
30/30 [==============================] - 7s 167ms/step - loss: 676.6680 - accuracy: 0.7135 - val_loss: 312.3109 - val_accuracy: 0.1375
Epoch 2/35
30/30 [==============================] - 5s 156ms/step - loss: 194.1897 - accuracy: 0.9375 - val_loss: 128.6329 - val_accuracy: 0.3208
Epoch 3/35
30/30 [==============================] - 5s 151ms/step - loss: 86.9935 - accuracy: 0.9542 - val_loss: 67.4843 - val_accuracy: 0.3208
Epoch 4/35
30/30 [==============================] - 5s 151ms/step - loss: 50.3708 - accuracy: 0.9688 - val_loss: 44.4434 - val_accuracy: 0.3208
Epoch 5/35
30/30 [==============================] - 5s 155ms/step - loss: 33.0698 - accuracy: 0.9812 - val_loss: 32.6868 - val_accuracy: 0.3208
Epoch 6/35
30/30 [==============================] - 5s 150ms/step - loss: 23.3255 - accuracy: 0.9885 - val_loss: 26.1045 - val_accuracy: 0.3208
Epoch 7/35
30/30 [==============================] - 5s 151ms/step - loss: 17.2402 - accuracy: 0.9875 - val_loss: 18.7858 - val_accuracy: 0.3208
Epoch 8/35
30/30 [==============================] - 5s 150ms/step - loss: 13.1128 - accuracy: 0.9885 - val_loss: 14.8672 - val_accuracy: 0.3208
Epoch 9/35
30/30 [==============================] - 5s 153ms/step - loss: 10.1714 - accuracy: 0.9927 - val_loss: 13.0882 - val_accuracy: 0.3208
Epoch 10/35
30/30 [==============================] - 5s 155ms/step - loss: 8.0155 - accuracy: 0.9979 - val_loss: 11.9238 - val_accuracy: 0.3208
Epoch 11/35
30/30 [==============================] - 5s 151ms/step - loss: 6.4128 - accuracy: 0.9927 - val_loss: 9.7622 - val_accuracy: 0.3208
Epoch 12/35
30/30 [==============================] - 5s 151ms/step - loss: 5.1555 - accuracy: 0.9979 - val_loss: 8.7582 - val_accuracy: 0.3208
Epoch 13/35
30/30 [==============================] - 5s 152ms/step - loss: 4.1962 - accuracy: 0.9927 - val_loss: 7.3595 - val_accuracy: 0.3708
Epoch 14/35
30/30 [==============================] - 5s 151ms/step - loss: 3.4347 - accuracy: 0.9969 - val_loss: 7.7884 - val_accuracy: 0.3458
Epoch 15/35
30/30 [==============================] - 5s 154ms/step - loss: 2.8145 - accuracy: 0.9969 - val_loss: 5.2008 - val_accuracy: 0.5208
Epoch 16/35
30/30 [==============================] - 5s 151ms/step - loss: 2.3259 - accuracy: 0.9979 - val_loss: 4.7135 - val_accuracy: 0.5542
Epoch 17/35
30/30 [==============================] - 5s 151ms/step - loss: 1.9294 - accuracy: 0.9979 - val_loss: 4.2373 - val_accuracy: 0.5542
Epoch 18/35
30/30 [==============================] - 5s 151ms/step - loss: 1.6064 - accuracy: 1.0000 - val_loss: 4.1891 - val_accuracy: 0.5458
Epoch 19/35
30/30 [==============================] - 5s 151ms/step - loss: 1.3526 - accuracy: 0.9969 - val_loss: 3.8042 - val_accuracy: 0.5542
Epoch 20/35
30/30 [==============================] - 5s 153ms/step - loss: 1.1269 - accuracy: 0.9990 - val_loss: 2.7082 - val_accuracy: 0.5667
Epoch 21/35
30/30 [==============================] - 5s 151ms/step - loss: 0.9437 - accuracy: 1.0000 - val_loss: 2.4145 - val_accuracy: 0.6250
Epoch 22/35
30/30 [==============================] - 5s 153ms/step - loss: 0.7999 - accuracy: 0.9979 - val_loss: 1.5690 - val_accuracy: 0.7750
Epoch 23/35
30/30 [==============================] - 5s 151ms/step - loss: 0.6953 - accuracy: 0.9948 - val_loss: 2.3244 - val_accuracy: 0.7417
Epoch 24/35
30/30 [==============================] - 5s 153ms/step - loss: 0.5907 - accuracy: 1.0000 - val_loss: 1.3375 - val_accuracy: 0.7875
Epoch 25/35
30/30 [==============================] - 5s 153ms/step - loss: 0.4955 - accuracy: 0.9990 - val_loss: 0.7102 - val_accuracy: 0.9208
Epoch 26/35
30/30 [==============================] - 5s 151ms/step - loss: 0.4236 - accuracy: 1.0000 - val_loss: 0.7070 - val_accuracy: 0.8917
Epoch 27/35
30/30 [==============================] - 5s 151ms/step - loss: 0.3568 - accuracy: 1.0000 - val_loss: 0.4382 - val_accuracy: 0.9708
Epoch 28/35
30/30 [==============================] - 5s 151ms/step - loss: 0.3099 - accuracy: 0.9990 - val_loss: 0.3680 - val_accuracy: 0.9750
Epoch 29/35
30/30 [==============================] - 5s 153ms/step - loss: 0.2692 - accuracy: 1.0000 - val_loss: 0.3228 - val_accuracy: 0.9750
Epoch 30/35
30/30 [==============================] - 5s 150ms/step - loss: 0.2317 - accuracy: 1.0000 - val_loss: 0.2719 - val_accuracy: 0.9792
Epoch 31/35
30/30 [==============================] - 5s 151ms/step - loss: 0.2004 - accuracy: 1.0000 - val_loss: 0.2169 - val_accuracy: 0.9875
Epoch 32/35
30/30 [==============================] - 5s 154ms/step - loss: 0.1789 - accuracy: 1.0000 - val_loss: 0.1726 - val_accuracy: 0.9958
Epoch 33/35
30/30 [==============================] - 5s 151ms/step - loss: 0.1562 - accuracy: 1.0000 - val_loss: 0.1551 - val_accuracy: 0.9958
Epoch 34/35
30/30 [==============================] - 5s 152ms/step - loss: 0.1366 - accuracy: 1.0000 - val_loss: 0.1729 - val_accuracy: 0.9792
Epoch 35/35
30/30 [==============================] - 5s 152ms/step - loss: 0.1214 - accuracy: 1.0000 - val_loss: 0.1287 - val_accuracy: 0.9917acc = history1.history['accuracy']
val_acc = history1.history['val_accuracy']
loss = history1.history['loss']
val_loss = history1.history['val_loss']
epochs_range = range(epochs)
plt.figure(figsize=(12, 4))
plt.subplot(1, 2, 1)
plt.plot(epochs_range, acc, label='Training Accuracy')
plt.plot(epochs_range, val_acc, label='Validation Accuracy')
plt.legend(loc='lower right')
plt.title('Training and Validation Accuracy')
plt.subplot(1, 2, 2)
plt.plot(epochs_range, loss, label='Training Loss')
plt.plot(epochs_range, val_loss, label='Validation Loss')
plt.legend(loc='upper right')
plt.title('Training and Validation Loss')
plt.show()
acc = history2.history['accuracy']
val_acc = history2.history['val_accuracy']
loss = history2.history['loss']
val_loss = history2.history['val_loss']
epochs_range = range(epochs2)
plt.figure(figsize=(12, 4))
plt.subplot(1, 2, 1)
plt.plot(epochs_range, acc, label='Training Accuracy')
plt.plot(epochs_range, val_acc, label='Validation Accuracy')
plt.legend(loc='lower right')
plt.title('Training and Validation Accuracy')
plt.subplot(1, 2, 2)
plt.plot(epochs_range, loss, label='Training Loss')
plt.plot(epochs_range, val_loss, label='Validation Loss')
plt.legend(loc='upper right')
plt.title('Training and Validation Loss')
plt.show() 
2955
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
