纹理识别分类

OUC-STU-LCX

已于 2024-06-15 01:16:18 修改

阅读量685

点赞数 8

文章标签：分类数据挖掘人工智能

于 2024-06-10 21:20:02 首次发布

本文链接：https://blog.csdn.net/tsgof/article/details/139580705

版权

本文探讨了一种结合深度学习与传统机器学习的方法，对图像纹理进行分类。实验涉及CNN对原图像和傅里叶图谱的分类，以及多层感知机对灰度共生矩阵和颜色直方图的分类。通过对多种特征的提取和模型训练，实现了模型的性能优化和评估。

摘要由CSDN通过智能技术生成

一、实验要求：

设计算法自动识别图像中的纹理并对其进行分类。

1. 利用所学知识，分析不同类属纹理图像之间的特性。

2. 设计算法自动识别并对不同类属的纹理图像进行分类，使同一类别的纹理图像尽可能地被分类到同一类别。

3. 阐述原理，分析得到的结果与实际之间的差异原因，总结整理报告。

二、实验方案：

2.1 数据准备

使用爬虫爬取1600余张动物纹理、植物纹理和地质纹理图片，按照3:1:1的比例放到train1、train2、test三个文件夹中。

train1是CNN卷积神经网络、CNN卷积神经网络分类傅里叶图谱、多层感知机分类灰度共生矩阵和多层感知机分类颜色直方图的训练集和验证集。
train2是CNN卷积神经网络、CNN卷积神经网络分类傅里叶图谱、多层感知机分类灰度共生矩阵和多层感知机分类颜色直方图的测试集，同时也是堆叠模型的训练集和验证集。
test是CNN卷积神经网络、CNN卷积神经网络分类傅里叶图谱、多层感知机分类灰度共生矩阵、多层感知机分类颜色直方图的另一个测试集，同时也是堆叠模型的测试集。
为了方便比较，规定train2是平均四个模型预测结果算法的训练集和验证集，test是其测试集。
pic
├── train1
│ ├── plant_texture
│ ├── animal_texture
│ └── earth_texture
├── train2
│ ├── plant_texture
│ ├── animal_texture
│ └── earth_texture
└── test
├── plant_texture
├── animal_texture
└── earth_texture

2.2 CNN卷积神经网络分类原图像

通过深度学习中的卷积神经网络（CNN）对图像进行分类

1. 导入库和设置

首先导入必要的库，如NumPy、Pandas、Seaborn、Matplotlib、OpenCV、Keras Tuner、TensorFlow等，用于数据处理、可视化和模型构建。
设置图像的宽度和高度均为128像素，并启用内联matplotlib以直接在Jupyter Notebook中显示图像。

2. 数据预处理

get_data函数：读取数据集文件夹下的所有图像，将它们调整到统一尺寸，进行归一化处理后存储为numpy数组。
调用get_data函数加载图像数据，并打印其形状及类别名称。
load_data函数：进一步组织数据为DataFrame格式，便于后续操作，并统计各类别的数量，绘制柱状图和饼状图展示类别分布。
显示部分样本图像以直观了解数据。

在这里插入图片描述

3. 数据编码与分割

使用pd.get_dummies进行独热编码，将类别标签转换为数值形式。
利用train_test_split从原始数据中分割出训练集和测试集，比例为80%训练，20%测试。

4. 模型构建与训练

基础模型构建：定义一个简单的CNN模型，包含卷积层、激活函数、最大池化层、展平层以及全连接层。使用SGD优化器和交叉熵损失函数进行编译。
训练模型并绘制训练过程中的准确率和损失曲线。

5. 模型评估

对测试集进行预测，计算混淆矩阵和分类报告来评估模型性能。

6. 超参数调优

使用Keras Tuner的RandomSearch方法进行超参数优化，定义了一个可变结构的CNN模型，其中超参数包括卷积层数量、每层的过滤器数量和学习率。
执行超参数搜索，选择最佳配置。
使用找到的最佳超参数重新构建模型并训练，再次评估模型性能，包括准确率和损失曲线。
最后，保存最佳模型至本地，并输出最终的混淆矩阵和分类报告。

CNN卷积神经网络在验证集上的效果

	precision	recall	f1-score	support
animal_texture	0.80	0.71	0.75	56
earth_texture	0.74	0.82	0.78	61
plant_texture	0.92	0.90	0.91	63
accuracy			0 .82	180
macro avg	0.82	0.81	0.81	180
weighted avg	0.82	0.82	0.82	180

2.3 CNN卷积神经网络分类傅里叶图谱

利用卷积神经网络（CNN）通过图像的傅里叶图谱对图像进行分类
除数据预处理外，与2.1同

数据预处理

定义get_data函数来读取并预处理图像数据：调整图像尺寸为128x128，应用快速傅里叶变换(FFT)并计算幅度谱，然后将图像归一化到0-1之间，并存储为numpy数组。
调用get_data函数加载图像数据和对应的标签。
统计类别数量并展示数据分布。

在这里插入图片描述

CNN卷积神经网络分类傅里叶图谱在验证集上的效果

	precision	recall	f1-score	support
animal_texture	0.52	0.77	0.62	56
earth_texture	0.79	0.25	0.38	61
plant_texture	0.75	0.94	0.83	63
accuracy			0.65	180
macro avg	0.69	0.65	0.61	180
weighted avg	0.69	0.65	0.61	180

2.4 多层感知机分类灰度共生矩阵

使用GLCM（灰度共生矩阵）特征和深度学习模型（Keras与TensorFlow后端）对图像纹理进行分类。

1. 数据加载与预处理

定义图像加载函数：load_and_preprocess_images函数用于从数据集文件夹加载图像，并将其转换为灰度图像。这里，三个不同的文件夹分别代表了三种不同的纹理类别（动物、地球、植物纹理）。
数据收集：通过调用上述函数，从指定路径加载图像数据并分配相应的标签（0, 1, 2），将图像和标签分别存储在全局变量images和labels中。

2. 特征提取

GLCM特征计算：preprocess_image_for_prediction函数对单个图像计算GLCM特征，包括不同距离和角度下的最大概率、对比度、相关性、均匀性、能量和熵。这些特征能够很好地描述图像的纹理信息。
特征集合：遍历所有预处理后的图像，应用preprocess_image_for_prediction函数，将提取的所有特征整合到glcm_features列表中。

3. 数据准备

数据标准化：使用StandardScaler对特征进行标准化，确保所有特征具有相同尺度。
数据分割：利用train_test_split将数据集划分为训练集和测试集，比例为80%训练，20%测试。
标签编码：将类别标签使用to_categorical进行one-hot编码，以便神经网络模型可以处理多分类问题。

4. 模型构建与训练

构建神经网络：创建一个Sequential模型，包含多层全连接层（Dense），使用ReLU激活函数，最后一层使用softmax激活函数以输出类别概率。
编译与训练：模型使用‘categorical_crossentropy’作为损失函数，Adam优化器，经过23轮训练，每批处理16个样本，并在训练过程中使用10%的数据作为验证集。

5. 性能评估

评估模型：在测试集上评估模型，打印出损失和准确率。
分类报告与混淆矩阵：利用 sklearn.metrics.classification_report 函数生成分类报告，提供每个类别的精确度（Precision）、召回率（Recall）、F1分数及支持度（Support）。

6. 模型与预处理器的持久化

保存Scaler：使用joblib.dump保存标准化器对象，以便在未来对新数据进行同样的预处理。
保存模型：使用Keras的.save方法保存训练好的模型，便于后续直接加载和使用。

多层感知机分类灰度共生矩阵在验证集上的效果

	precision	recall	f1-score	support
animal_texture	0.72	0.53	0.61	55
earth_texture	0.66	0.77	0.71	64
plant_texture	0.76	0.82	0.79	61
accuracy			0.71	180
macro avg	0.70	0.70	0.70	180
weighted avg	0.71	0.71	0.71	180

2.5 多层感知机分类颜色直方图

使用颜色直方图特征和深度学习模型（Keras与TensorFlow后端）对图像纹理进行分类。
除特征提取外与2.3同

特征提取

直方图计算：对于每个图像，计算每个颜色通道的直方图。通过cv2.calcHist()函数计算直方图，然后展平这些直方图得到一维特征向量。
特征归一化：计算得到的直方图特征归一化，将其总和变为1，确保所有图像的特征具有相同的比例，避免因不同图像亮度或对比度差异导致的偏斜。

在这里插入图片描述

多层感知机分类颜色直方图在验证集上的效果

	precision	recall	f1-score	support
animal_texture	0.90	0.96	0.93	55
earth_texture	1.00	1.00	1.00	64
plant_texture	0.96	0.90	0.93	61
accuracy			0.96	180
macro avg	0.95	0.96	0.95	180
weighted avg	0.96	0.96	0.96	180

2.6 简单平均预测结果模型

1. 导入必要的库和模块

导入了如numpy, pandas, seaborn, matplotlib, cv2（OpenCV用于图像处理）, keras_tuner, tensorflow, sklearn等库，以及相关模块，用于模型训练、图像处理和数据分析。

2. 定义图像预处理函数

preprocess_image_for_gray_prediction: 用于灰度图像的预处理。读取图像并转换为灰度，然后计算不同距离和角度下的灰度共生矩阵(GLCM)特征，包括最大概率、对比度、相关性、同质性、能量和熵。
preprocess_image_for_color_prediction: 用于彩色图像的预处理。读取图像后分离各通道，计算每个通道的直方图，并进行归一化处理，使直方图之和为1。

3. 准备验证数据集

定义图像加载函数：load_and_preprocess_images函数用于从数据集文件夹加载图像，并将其转换为灰度图像。这里，三个不同的文件夹分别代表了三种不同的纹理类别（动物、地球、植物纹理）。
数据收集：通过调用上述函数，从指定路径加载图像数据并分配相应的标签（0, 1, 2），将图像和标签分别存储在全局变量images和labels中。

4. 加载模型

加载了两个基于卷积神经网络（CNN）的模型（cnn_base_model 和 cnn_FLY_model），用于原始图像和其频谱图像的预测。
加载了两个基于传统特征的机器学习模型（gray_model 和 color_model），分别处理灰度共生矩阵特征和颜色直方图特征。
使用pandas库分别记录并保存它们的预测结果到csv文件中，便于后续模型堆叠训练。

5. 特征提取与模型预测

遍历每张图片，首先对原始图像和其频谱图像使用CNN模型进行预测，然后分别提取并处理图像的灰度共生矩阵特征和颜色直方图特征，对这两个特征集应用相应的预处理（标准化）后，用对应的模型进行预测。
使用pandas库分别记录并保存它们的预测结果到csv文件中，便于后续模型堆叠训练。
将所有模型的预测结果相加，取平均来得到最终的预测类别。

6. 结果评估

收集真正的标签和预测的标签，使用classification_report生成详细的分类报告，包括精度、召回率、F1分数等指标，评估各个模型和取平均预测结果模型在验证集上的表现。

CNN卷积神经网络在测试集上的效果

	precision	recall	f1-score	support
animal_texture	0.63	0.54	0.58	109
earth_texture	0.53	0.74	0.62	106
plant_texture	0.96	0.79	0.86	137
accuracy			0 .70	352
macro avg	0.71	0.69	0.69	352
weighted avg	0.73	0.70	0.70	352

CNN卷积神经网络分类傅里叶图谱在测试集上的效果

	precision	recall	f1-score	support
animal_texture	0.47	0.74	0.58	109
earth_texture	0.75	0.08	0.15	106
plant_texture	0.69	0.85	0.76	137
accuracy			0 .59	352
macro avg	0.64	0.56	0.50	352
weighted avg	0.64	0.59	0.52	352

多层感知机分类灰度共生矩阵在测试集上的效果

	precision	recall	f1-score	support
animal_texture	0.76	0.79	0.77	109
earth_texture	0.76	0.79	0.78	106
plant_texture	0.82	0.77	0.80	137
accuracy			0.78	352
macro avg	0.78	0.79	0.78	352
weighted avg	0.79	0.78	0.78	352

多层感知机分类颜色直方图在测试集上的效果

	precision	recall	f1-score	support
animal_texture	0.79	0.98	0.88	109
earth_texture	1.00	0.99	1.00	106
plant_texture	0.98	0.80	0.88	137
accuracy			0.91	352
macro avg	0.92	0.93	0.92	352
weighted avg	0.93	0.91	0.92	352

平均四个模型的预测结果在验证集上的效果

	precision	recall	f1-score	support
animal_texture	0.88	0.97	0.93	109
earth_texture	0.98	0.97	0.98	106
plant_texture	0.98	0.91	0.95	137
accuracy			0.95	352
macro avg	0.95	0.95	0.95	352
weighted avg	0.95	0.95	0.95	352

2.7 模型堆叠

整合不同类型的模型（深度学习与传统机器学习
方法）以及多模态特征（原始图像、频谱图像、灰度特征、颜色特征）来进行图像分类任务。即训练一个多层感知机来对于前面四个模型的输出结果进行处理。

1. 导入所需库和模块

导入NumPy、skimage、sklearn、tensorflow、cv2、os、joblib、pandas等必要的Python库，分别用于数据处理、模型构建、图像处理、文件操作和结果保存等功能。

2. 数据读取与整合

通过pd.read_csv()函数读取四个模型的预测结果（model1_predictions.csv, model2_predictions.csv, model3_predictions.csv, model4_predictions.csv）到DataFrame中，并将它们横向合并成一个特征矩阵ensemble_features。
从labels.csv文件中读取真实的标签数据，存储在变量labels中。

3. 数据预处理

将特征矩阵转换为NumPy数组格式，便于后续处理。
使用StandardScaler对特征进行标准化处理，以消除量纲影响，提高模型训练效率。
将标签数据转化为one-hot编码形式，以适应多分类任务的需求。
划分训练集和测试集，其中测试集占20%，并设置随机种子以确保实验的可复现性。

4. 构建深度学习模型

使用Keras（集成于TensorFlow中）创建一个顺序模型，该模型包含多层全连接层（Dense层），使用ReLU激活函数，最后一层使用softmax激活函数以输出类别概率。
模型编译，指定损失函数（‘categorical_crossentropy’）、优化器（Adam）和评估指标（accuracy）。
训练模型，设置20个epochs和每批20个样本的批量大小，验证分割设为0.2。

5. 模型评估

在测试集上评估模型，输出测试损失和准确率。

6. 结果分析

使用classification_report生成包含精度、召回率、F1分数等评估指标的报告，并通过给预测类别和真实类别赋予实际意义（如动物、地球、植物纹理），使报告更具可读性。

7. 模型与预处理器的保存

使用joblib.dump()保存标准化器实例，以便在未来对新数据进行同样的预处理。
使用model.save()保存训练好的模型，方便后续部署或进一步调用。

堆叠模型在验证集上的效果

	precision	recall	f1-score	support
animal_texture	0.94	1.00	0.97	30
earth_texture	1.00	1.00	1.00	19
plant_texture	1.00	0.91	0.95	22
accuracy			0.97	300
macro avg	0.98	0.97	0.97	300
weighted avg	0.97	0.97	0.97	300

2.8 最终测试

CNN卷积神经网络在另一测试集上的效果

	precision	recall	f1-score	support
animal_texture	0.59	0.60	0.59	100
earth_texture	0.58	0.56	0.57	100
plant_texture	0.85	0.86	0.86	100
accuracy			0.67	300
macro avg	0.67	0.67	0.67	300
weighted avg	0.67	0.67	0.67	300

CNN卷积神经网络分类傅里叶图谱在另一测试集上的效果

	precision	recall	f1-score	support
animal_texture	0.47	0.70	0.56	100
earth_texture	0.78	0.07	0.13	100
plant_texture	0.64	0.92	0.76	100
accuracy			0.56	300
macro avg	0.63	0.56	0.48	300
weighted avg	0.63	0.56	0.48	300

多层感知机分类灰度共生矩阵在另一测试集上的效果

	precision	recall	f1-score	support
animal_texture	0.81	0.76	0.78	100
earth_texture	0.74	0.75	0.75	100
plant_texture	0.76	0.80	0.78	100
accuracy			0.77	300
macro avg	0.77	0.77	0.77	300
weighted avg	0.77	0.77	0.77	300

多层感知机分类颜色直方图在另一测试集上的效果

	precision	recall	f1-score	support
animal_texture	0.81	0.95	0.87	100
earth_texture	1.00	0.99	0.99	100
plant_texture	0.94	0.78	0.85	100
accuracy			0.91	300
macro avg	0.91	0.91	0.91	300
weighted avg	0.91	0.91	0.91	300

平均四个模型的预测结果在测试集上的效果

	precision	recall	f1-score	support
animal_texture	0.90	0.94	0.92	100
earth_texture	0.98	0.92	0.95	100
plant_texture	0.90	0.92	0.91	100
accuracy			0.93	300
macro avg	0.93	0.93	0.93	300
weighted avg	0.93	0.93	0.93	300

堆叠模型在测试集上的效果

	precision	recall	f1-score	support
animal_texture	0.94	0.94	0.94	100
earth_texture	1.00	0.99	0.99	100
plant_texture	0.94	0.95	0.95	100
accuracy			0.96	300
macro avg	0.96	0.96	0.96	300
weighted avg	0.96	0.96	0.96	300

三、实验讨论：

CNN对原始图像的局限性：尽管卷积神经网络(CNN)在许多图像识别任务中表现出色，但它们可能不足以捕捉到所有类型的纹理细节，尤其是在纹理相似但实际类别不同的情况下。例如，地质纹理与动物纹理可能因形状和分布上的相似性而导致分类混淆。
傅里叶频谱的不适用性：尝试将图像转换为傅里叶频谱并用CNN处理，效果不佳，说明传统的图像频谱分析方法可能不直接适用于复杂的纹理分类任务。这可能是由于CNN难以直接从频谱图的数值差异中提取有效的分类特征，尤其是在纹理重复性高且频率分布相近的情况下。
灰度共生矩阵的价值：通过计算灰度共生矩阵(GLCM)，可以捕捉到图像纹理的局部统计信息，这对于区分细微的纹理差异至关重要。这种方法能够提供关于纹理分布、对比度、均匀性等多方面的量化信息，从而帮助模型更精细地分类。
颜色特征的重要性：颜色直方图作为颜色分布的统计表示，在纹理分类中也扮演着重要角色，尤其是当不同类别的图像在颜色上有显著差异时。利用颜色信息可以为分类任务提供有力的补充线索。
多模态融合的优势：将不同特征提取方法（原始图像、频谱图像、灰度特征、颜色特征）的模型结果进行集成，通过简单平均或其他融合策略，可以显著提高分类系统的鲁棒性和准确性。这种方法能够综合不同视角的信息，减少单一模型的偏见或局限性，提高整体性能。
模型堆叠的小幅度优化：利用多层感知机学习不同特征提取方法（原始图像、频谱图像、灰度特征、颜色特征）的模型结果，可以挖掘多模态模型提供的潜在信息，能进一步提高分类系统的鲁棒性和准确性。可以更准确地利用不同分类模型提供的信息，减少分类精度差的模型提供误导信息的可能性。

四、实验总结

通过完成这个综合性的实验，我学到了以下几个关键点：

深度学习模型应用：理解了如何使用卷积神经网络(CNN)进行图像分类，从基本的模型搭建、训练、评估到超参数调优的全过程。CNN能够自动学习图像中的局部特征，适用于纹理识别这类任务。
图像预处理技术：掌握了不同的图像预处理技巧，包括尺寸调整、归一化、傅里叶变换和灰度共生矩阵(GLCM)的计算、颜色直方图的提取等。这些技术对于提高模型性能至关重要。
特征工程：学习了如何从原始图像数据中提取有意义的特征，比如通过GLCM和颜色直方图，这些特征能够有效表征图像的纹理特性和颜色分布，对于非深度学习模型尤其重要。
模型融合与多模态学习：实践了如何结合多种模型和特征类型来提升分类效果，通过模型组合策略，实现对图像的综合分析，模型堆叠进一步提高分类精度。
问题解决与创新思维：面对不同纹理分类的挑战，学会了如何灵活应用和调整不同的技术和方法，培养了面对复杂问题时的创新思维和解决方案设计能力。

综上所述，这次实验不仅深化了我对深度学习、图像处理和特征提取技术的理解，还提升了我在数据科学项目从设计到实施再到评估的全流程能力。

附录代码

CNN卷积神经网络分类原图像

import numpy as np
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
%matplotlib inline
import os
import cv2
import random
import keras_tuner as kt
from tensorflow import keras
import tensorflow as tf
from pathlib import Path
from tensorflow.keras import layers
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Conv2D, MaxPooling2D, GlobalAveragePooling2D
from tensorflow.keras.layers import Dense, Dropout, Activation, Flatten
from tensorflow.keras.optimizers import SGD
from sklearn.metrics import classification_report

w = 128
h = 128
#import image data
def get_data(folder):
  image_raw = []
  label_raw = []

  for i in os.listdir(folder):
    for j in os.listdir(os.path.join(folder,i)):
      path = os.path.join(folder, i, j)
      image = cv2.imread(path)
      image = cv2.resize(image, (w, h), interpolation = cv2.INTER_AREA)
      f_img = np.fft.fft2(image)
      fshift = np.fft.fftshift(f_img)
      magnitude_spectrum = 20 * np.log(np.abs(fshift)+1)
      image = np.array(magnitude_spectrum)
      image = image.astype('float32')
      image /= 255
      image_raw.append(image)
      label_raw.append(i)
  image_raw = np.array(image_raw)
  return image_raw, label_raw
image_raw, label_raw = get_data(r"F:\pic\train1")
print('Shape of image data: ', image_raw.shape)
#get unique class name
label_names = np.unique(label_raw)
print(label_names)
print(os.listdir(r"F:\pic\train1"))
data_dir = Path(r"F:\pic\train1")
data_dir
def load_data():
    animal_texture_dir = data_dir /'animal_texture'
    plant_texture_dir = data_dir / 'plant_texture'
    earth_texture_dir = data_dir / 'earth_texture'
    
    #Get list of all image
    animal = animal_texture_dir.glob('*.png')
    plant = plant_texture_dir.glob('*.png')
    earth = earth_texture_dir.glob('*.png')
   
    
    img_data = []
    img_label = []
    for img in animal:
   
        img_data.append(img)
        img_label.append('animal')
    for img in earth:
        
        img_data.append(img)
        img_label.append('earth')
    for img in plant:
       
        img_data.append(img)
        img_label.append('plant')
    
    
    df = pd.DataFrame(img_data)
    df.columns = ['Images']
    df['Labels'] = img_label
    df = df.sample(frac=1).reset_index(drop=True)
    return df
img_data = load_data()
img_data.head()
plt.figure(figsize = (15,6))
plt.title('Matrials count')
plt.bar(img_data['Labels'].value_counts().index, img_data['Labels'].value_counts().values)
plt.show()
img_data['Labels'].value_counts().plot.pie(figsize=(5,5))
plt.show()
plt.figure(figsize=(15,15))
for i in range(30):
    ax = plt.subplot(5,6,i+1)
    img = plt.imread(img_data["Images"][i])
    plt.title(img_data["Labels"][i])
    plt.imshow(img)
    plt.axis('off')
#One hot encoding on label name
label_names = pd.DataFrame(label_names)
label_raw = pd.DataFrame(label_raw)

label_names = pd.get_dummies(label_names)
y_label = pd.get_dummies(label_raw)

from sklearn.preprocessing import LabelBinarizer
one_hot = LabelBinarizer().fit(label_names)
y_label = one_hot.transform(y_label)
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(image_raw, y_label, test_size = 0.2, random_state = 0)
print('Number of training set =', X_train.shape[0])
print('Number of testing set =', X_test.shape[0])
model = Sequential()
model.add(Conv2D(100, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Flatten())
model.add(Dense(len(label_names)))
model.add(Activation('softmax'))

model.compile(loss='categorical_crossentropy',
              optimizer='sgd',
              metrics=['accuracy'])
epochs= 21
batch_size= 4
result = model.fit(X_train, y_train,
              batch_size=batch_size,
              epochs=epochs,
              validation_data=(X_test, y_test),
              shuffle=True)
plt.figure(figsize=[20,8])
plt.plot(result.history['accuracy'])
plt.plot(result.history['val_accuracy'])
plt.title('Epoch VS Model Accuracy', size=25, pad=20)
plt.ylabel('Accuracy', size=15)
plt.xlabel('Epoch', size=15)
plt.legend(['train', 'test'], loc='upper left')


plt.figure(figsize=[20,8])
plt.plot(result.history['loss'])
plt.plot(result.history['val_loss'])
plt.title('Epoch VS Model Loss', size=25, pad=20)
plt.ylabel('Loss', size=15)
plt.xlabel('Epoch', size=15)
plt.legend(['train', 'test'], loc='upper right')
plt.show()
prediction = model.predict(X_test)
prediction_ = np.argmax(prediction, axis=1)
y_test_ = np.argmax(y_test, axis=1)
print(tf.math.confusion_matrix(y_test_, prediction_))
print(classification_report(y_test_, prediction_ , target_names= label_names))
def build_model(hp):
    inputs = keras.Input(shape=(128,128,3))
    x = inputs
    for i in range(hp.Int("cnn_layers", 1, 3)):
        x = layers.Conv2D(
            hp.Int(f"filters_{i}", 32, 128, step=32), #128
            kernel_size=(3, 3),
            activation="relu",)(x)
        x = layers.MaxPooling2D(pool_size=(2, 2))(x)
    x = layers.Flatten()(x)
    Dropout(0.2),
    outputs = layers.Dense(units=3, activation="softmax")(x)
    model = keras.Model(inputs=inputs, outputs=outputs)
    hp_learning_rate = hp.Choice('learning_rate', values=[0.01, 0.001, 0.0001])
    # Compile the model.
    model.compile(
        loss="categorical_crossentropy", 
        optimizer= keras.optimizers.Adam(learning_rate=hp_learning_rate),
        metrics=["accuracy"])
    return model
hp = kt.HyperParameters()
model = build_model(hp)
model(X_train[:])
model.summary()
tuner = kt.RandomSearch(
    build_model,
    max_trials=10,
    overwrite=True,
    objective="val_accuracy",
    directory="/tmp/tb")
Early = tf.keras.callbacks.EarlyStopping(monitor='val_loss', patience=3)

#Control the learning rate
from keras.callbacks import ReduceLROnPlateau
learning_rate_reduction = ReduceLROnPlateau(monitor='val_accuracy',
                                            patience=2,
                                            verbose=1,
                                            factor=0.5,
                                            min_lr=0.0001)
callback = [Early,learning_rate_reduction] #]
tuner.search(
    X_train,
    y_train,
    validation_split=0.2,
    epochs=20,
    callbacks=callback,
    validation_data= (X_test, y_test)
)
best_hps=tuner.get_best_hyperparameters(num_trials=1)[0]
hypermodel = tuner.hypermodel.build(best_hps)
BATCH_SIZE = 128
EPOCH = 100
result_hp = hypermodel.fit(X_train, y_train,
                           epochs=EPOCH,
                           batch_size = BATCH_SIZE,
                           callbacks = callback,
                           validation_data= (X_test, y_test),
                           shuffle = True)
#Plot accuracy and loss on training set and test set

plt.figure(figsize=[20,8])
plt.title('Epoch VS Model Accuracy', size = 20)
plt.plot(result_hp.history['accuracy'])
plt.plot(result_hp.history['val_accuracy'])
plt.xlabel('Epoch', size = 15)
plt.ylabel('Accuracy', size = 15)
plt.legend(['Train', 'Test'], loc = 'upper left')
plt.show()

plt.figure(figsize=[20,8])
plt.title('Epoch VS Model Loss', size = 20)
plt.plot(result_hp.history['loss'])
plt.plot(result_hp.history['val_loss'])
plt.xlabel('Epoch', size = 15)
plt.ylabel('Loss', size = 15)
plt.legend(['Train', 'Test'], loc = 'upper right')
plt.show()
prediction = hypermodel.predict(X_test) 
prediction_hp =np.argmax(prediction, axis=1)
y_test_hp =np.argmax(y_test, axis=1)
print(tf.math.confusion_matrix(y_test_hp, prediction_hp))\
hypermodel.save('cnn_FLY_best_model.keras.keras')
print(classification_report(y_test_hp, prediction_hp, target_names=label_names))

CNN卷积神经网络分类傅里叶图谱

import numpy as np
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
%matplotlib inline
import os
import cv2
import random
import keras_tuner as kt
from tensorflow import keras
import tensorflow as tf
from pathlib import Path
from tensorflow.keras import layers
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Conv2D, MaxPooling2D, GlobalAveragePooling2D
from tensorflow.keras.layers import Dense, Dropout, Activation, Flatten
from tensorflow.keras.optimizers import SGD
from sklearn.metrics import classification_report

w = 128
h = 128

def get_data(folder):
  image_raw = []
  label_raw = []

  for i in os.listdir(folder):
    for j in os.listdir(os.path.join(folder,i)):
      path = os.path.join(folder, i, j)
      image = cv2.imread(path)
      image = cv2.resize(image, (w, h), interpolation = cv2.INTER_AREA)
      image = image.astype('float32')
      image /= 255
      image_raw.append(image)
      label_raw.append(i)
  image_raw = np.array(image_raw)
  return image_raw, label_raw
image_raw, label_raw = get_data(r"F:\pic\train1")
print('Shape of image data: ', image_raw.shape)
#get unique class name
label_names = np.unique(label_raw)
print(label_names)
print(os.listdir(r"F:\pic\train1"))
data_dir = Path(r"F:\pic\train1")
data_dir
def load_data():
    animal_texture_dir = data_dir /'animal_texture'
    plant_texture_dir = data_dir / 'plant_texture'
    earth_texture_dir = data_dir / 'earth_texture'
    
    #Get list of all image
    animal = animal_texture_dir.glob('*.png')
    plant = plant_texture_dir.glob('*.png')
    earth = earth_texture_dir.glob('*.png')
   
    
    img_data = []
    img_label = []
    for img in animal:
   
        img_data.append(img)
        img_label.append('animal')
    for img in earth:
        
        img_data.append(img)
        img_label.append('earth')
    for img in plant:
       
        img_data.append(img)
        img_label.append('plant')
    
    
    df = pd.DataFrame(img_data)
    df.columns = ['Images']
    df['Labels'] = img_label
    df = df.sample(frac=1).reset_index(drop=True)
    return df
img_data = load_data()
img_data.head()
plt.figure(figsize = (15,6))
plt.title('Matrials count')
plt.bar(img_data['Labels'].value_counts().index, img_data['Labels'].value_counts().values)
plt.show()
img_data['Labels'].value_counts().plot.pie(figsize=(5,5))
plt.show()
plt.figure(figsize=(15,15))
for i in range(30):
    ax = plt.subplot(5,6,i+1)
    img = plt.imread(img_data["Images"][i])
    plt.title(img_data["Labels"][i])
    plt.imshow(img)
    plt.axis('off')
#One hot encoding on label name
label_names = pd.DataFrame(label_names)
label_raw = pd.DataFrame(label_raw)

label_names = pd.get_dummies(label_names)
y_label = pd.get_dummies(label_raw)

from sklearn.preprocessing import LabelBinarizer
one_hot = LabelBinarizer().fit(label_names)
y_label = one_hot.transform(y_label)
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(image_raw, y_label, test_size = 0.2, random_state = 0)
print('Number of training set =', X_train.shape[0])
print('Number of testing set =', X_test.shape[0])
model = Sequential()
model.add(Conv2D(100, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Flatten())
model.add(Dense(len(label_names)))
model.add(Activation('softmax'))

model.compile(loss='categorical_crossentropy',
              optimizer='sgd',
              metrics=['accuracy'])
epochs= 21
batch_size= 4
result = model.fit(X_train, y_train,
              batch_size=batch_size,
              epochs=epochs,
              validation_data=(X_test, y_test),
              shuffle=True)
plt.figure(figsize=[20,8])
plt.plot(result.history['accuracy'])
plt.plot(result.history['val_accuracy'])
plt.title('Epoch VS Model Accuracy', size=25, pad=20)
plt.ylabel('Accuracy', size=15)
plt.xlabel('Epoch', size=15)
plt.legend(['train', 'test'], loc='upper left')


plt.figure(figsize=[20,8])
plt.plot(result.history['loss'])
plt.plot(result.history['val_loss'])
plt.title('Epoch VS Model Loss', size=25, pad=20)
plt.ylabel('Loss', size=15)
plt.xlabel('Epoch', size=15)
plt.legend(['train', 'test'], loc='upper right')
plt.show()
prediction = model.predict(X_test)
prediction_ = np.argmax(prediction, axis=1)
y_test_ = np.argmax(y_test, axis=1)
print(tf.math.confusion_matrix(y_test_, prediction_))
print(classification_report(y_test_, prediction_ , target_names= label_names))
def build_model(hp):
    inputs = keras.Input(shape=(128,128,3))
    x = inputs
    for i in range(hp.Int("cnn_layers", 1, 3)):
        x = layers.Conv2D(
            hp.Int(f"filters_{i}", 32, 128, step=32), #128
            kernel_size=(3, 3),
            activation="relu",)(x)
        x = layers.MaxPooling2D(pool_size=(2, 2))(x)
    x = layers.Flatten()(x)
    Dropout(0.2),
    outputs = layers.Dense(units=3, activation="softmax")(x)
    model = keras.Model(inputs=inputs, outputs=outputs)
    hp_learning_rate = hp.Choice('learning_rate', values=[0.01, 0.001, 0.0001])
    # Compile the model.
    model.compile(
        loss="categorical_crossentropy", 
        optimizer= keras.optimizers.Adam(learning_rate=hp_learning_rate),
        metrics=["accuracy"])
    return model
hp = kt.HyperParameters()
model = build_model(hp)
model(X_train[:])
model.summary()
tuner = kt.RandomSearch(
    build_model,
    max_trials=10,
    overwrite=True,
    objective="val_accuracy",
    directory="/tmp/tb")
Early = tf.keras.callbacks.EarlyStopping(monitor='val_loss', patience=3)

#Control the learning rate
from keras.callbacks import ReduceLROnPlateau
learning_rate_reduction = ReduceLROnPlateau(monitor='val_accuracy',
                                            patience=2,
                                            verbose=1,
                                            factor=0.5,
                                            min_lr=0.0001)
callback = [Early,learning_rate_reduction] #]
tuner.search(
    X_train,
    y_train,
    validation_split=0.2,
    epochs=20,
    callbacks=callback,
    validation_data= (X_test, y_test)
)
best_hps=tuner.get_best_hyperparameters(num_trials=1)[0]
hypermodel = tuner.hypermodel.build(best_hps)
BATCH_SIZE = 128
EPOCH = 100
result_hp = hypermodel.fit(X_train, y_train,
                           epochs=EPOCH,
                           batch_size = BATCH_SIZE,
                           callbacks = callback,
                           validation_data= (X_test, y_test),
                           shuffle = True)
#Plot accuracy and loss on training set and test set

plt.figure(figsize=[20,8])
plt.title('Epoch VS Model Accuracy', size = 20)
plt.plot(result_hp.history['accuracy'])
plt.plot(result_hp.history['val_accuracy'])
plt.xlabel('Epoch', size = 15)
plt.ylabel('Accuracy', size = 15)
plt.legend(['Train', 'Test'], loc = 'upper left')
plt.show()

plt.figure(figsize=[20,8])
plt.title('Epoch VS Model Loss', size = 20)
plt.plot(result_hp.history['loss'])
plt.plot(result_hp.history['val_loss'])
plt.xlabel('Epoch', size = 15)
plt.ylabel('Loss', size = 15)
plt.legend(['Train', 'Test'], loc = 'upper right')
plt.show()
prediction = hypermodel.predict(X_test) 
prediction_hp =np.argmax(prediction, axis=1)
y_test_hp =np.argmax(y_test, axis=1)
print(tf.math.confusion_matrix(y_test_hp, prediction_hp))\
hypermodel.save('cnn_base_best_model.keras')
print(classification_report(y_test_hp, prediction_hp, target_names=label_names))

多层感知机分类灰度共生矩阵

import numpy as np  
from skimage.feature import graycomatrix, graycoprops  
from sklearn.model_selection import train_test_split  
from sklearn.preprocessing import StandardScaler  
from tensorflow.keras.models import Sequential  
from tensorflow.keras.layers import Dense  
from tensorflow.keras.utils import to_categorical  
from tensorflow.keras.optimizers import Adam  
import cv2
import os
import joblib
from scikeras.wrappers import KerasClassifier, KerasRegressor
from sklearn.model_selection import GridSearchCV
import tensorflow as tf

 
images = list() 
labels = list()
def load_and_preprocess_images(folder_path,label):     
    for filename in os.listdir(folder_path):  
        if filename.endswith(".jpg") or filename.endswith(".png"):  
            file_path = os.path.join(folder_path, filename) 
            img=cv2.imread(file_path,cv2.IMREAD_GRAYSCALE)
            global images
            global labels
            images.append(img)  
            labels.append(label)
load_and_preprocess_images(r'F:pic\train1\animal_texture',0)
load_and_preprocess_images(r'F:pic\train1\earth_texture',1)
load_and_preprocess_images(r'F:pic\train1\plant_texture',2)

def preprocess_image_for_prediction(img):
    glcm_distances = [1,2,3,4,5,6,7,8,9,10]
    glcm_angles = [0, 45, 135, 90]
    features = []
    for distance in glcm_distances:
        for angle in glcm_angles:
            glcm = graycomatrix(img, distances=[distance], angles=[angle], levels=256, symmetric=True, normed=True)
            max_prob = np.max(glcm)
            contrast = graycoprops(glcm, 'contrast')[0, 0]
            correlation = graycoprops(glcm, 'correlation')[0, 0]
            homogeneity = graycoprops(glcm, 'homogeneity')[0, 0]
            energy = graycoprops(glcm, 'energy')[0, 0]
            entropy = -np.sum(glcm * np.log2(glcm + np.finfo(float).eps))
            features.extend([max_prob, contrast, correlation, homogeneity, energy, entropy])
    return np.array(features)


glcm_features = []  
for image in images: 
    glcm_features.append(preprocess_image_for_prediction(image))
     
X = np.array(glcm_features)  
y = np.array(labels)  
  
scaler = StandardScaler()  
X_scaled = scaler.fit_transform(X)  
y_categorical = to_categorical(y)  
  
X_train, X_test, y_train, y_test = train_test_split(X_scaled, y_categorical, test_size=0.2, random_state=76)  

model = Sequential()  
model.add(Dense(256, input_dim=X_train.shape[1], activation='relu'))  
model.add(Dense(128, activation='relu'))  
model.add(Dense(64, activation='relu')) 
model.add(Dense(y_categorical.shape[1], activation='softmax'))  
  
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])  
  
model.fit(X_train, y_train, epochs=23, batch_size=16, validation_split=0.2)  
  
loss, accuracy = model.evaluate(X_test, y_test)  
print(f'Test loss: {loss:.4f}')  
print(f'Test accuracy: {accuracy:.4f}')


from sklearn.metrics import classification_report, confusion_matrix


class_names = {0: 'animal_texture', 1: 'earth_texture', 2: 'plant_texture'}
y_pred = model.predict(X_test)
y_pred_classes = np.argmax(y_pred, axis=1)
y_test_classes = np.argmax(y_test, axis=1)
y_pred_class_names = [class_names[pred] for pred in y_pred_classes]
y_true_class_names = [class_names[label] for label in y_test_classes]


report_with_names = classification_report(y_true_class_names, y_pred_class_names, target_names=class_names.values())
print(report_with_names)

scaler_filename = 'gray_scaler.save'
joblib.dump(scaler, scaler_filename)
print("Scaler saved.")
model.save('gray_classifier_model.h5')
print("Model saved.")

多层感知机分类颜色直方图

import numpy as np  
from skimage.feature import graycomatrix, graycoprops  
from sklearn.model_selection import train_test_split  
from sklearn.preprocessing import StandardScaler  
from tensorflow.keras.models import Sequential  
from tensorflow.keras.layers import Dense  
from tensorflow.keras.utils import to_categorical  
from tensorflow.keras.optimizers import Adam  
import cv2
import os
import joblib
from scikeras.wrappers import KerasClassifier, KerasRegressor
from sklearn.model_selection import GridSearchCV
import tensorflow as tf
from itertools import chain
  
images = list() 
labels = list()
def load_and_preprocess_images(folder_path,label):     
    for filename in os.listdir(folder_path):  
        if filename.endswith(".jpg") or filename.endswith(".png"):  # 你可以根据需要添加其他图片格式  
            file_path = os.path.join(folder_path, filename) 
            img=cv2.imread(file_path)
                
            global images
            global labels
            images.append(img)  
            labels.append(label)
load_and_preprocess_images(r'F:pic\train1\animal_texture',0)
load_and_preprocess_images(r'F:pic\train1\earth_texture',1)
load_and_preprocess_images(r'F:pic\train1\plant_texture',2)

def preprocess_image_for_prediction(img):
    features = []
    chans = cv2.split(img)
    for chan in chans:
        hist = cv2.calcHist([chan],[0],None,[256],[0,256])
        hist = list(chain.from_iterable(hist))
        features.extend(hist)
    features=np.array(features)
    features_sum = features.sum()
        
    features_nor= features / features_sum
    return np.array(features)


glcm_features = []  
for image in images: 
    glcm_features.append(preprocess_image_for_prediction(image))


X = np.array(glcm_features)  
y = np.array(labels)  
  
 
scaler = StandardScaler()  
X_scaled = scaler.fit_transform(X)  
y_categorical = to_categorical(y)  
  

X_train, X_test, y_train, y_test = train_test_split(X_scaled, y_categorical, test_size=0.2, random_state=76)  


model = Sequential()  
model.add(Dense(256, input_dim=X_train.shape[1], activation='relu'))  
model.add(Dense(128, activation='relu'))  
model.add(Dense(64, activation='relu')) 
model.add(Dense(y_categorical.shape[1], activation='softmax'))  
  

model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])  
  
model.fit(X_train, y_train, epochs=23, batch_size=16, validation_split=0.2)  
  
loss, accuracy = model.evaluate(X_test, y_test)  
print(f'Test loss: {loss:.4f}')  
print(f'Test accuracy: {accuracy:.4f}')

from sklearn.metrics import classification_report, confusion_matrix

class_names = {0: 'animal_texture', 1: 'earth_texture', 2: 'plant_texture'}

y_pred = model.predict(X_test)
y_pred_classes = np.argmax(y_pred, axis=1)
y_test_classes = np.argmax(y_test, axis=1)
y_pred_class_names = [class_names[pred] for pred in y_pred_classes]
y_true_class_names = [class_names[label] for label in y_test_classes]


report_with_names = classification_report(y_true_class_names, y_pred_class_names, target_names=class_names.values())
print(report_with_names)


scaler_filename = 'color_scaler.save'
joblib.dump(scaler, scaler_filename)
print("Scaler saved.")

model.save('color_classifier_model.h5')
print("Model saved.")

测试四个模型并保存结果

import numpy as np
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
%matplotlib inline
import os
import cv2
import random
import keras_tuner as kt
from tensorflow import keras
import tensorflow as tf
from pathlib import Path
from tensorflow.keras import layers
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Conv2D, MaxPooling2D, GlobalAveragePooling2D
from tensorflow.keras.layers import Dense, Dropout, Activation, Flatten
from tensorflow.keras.optimizers import SGD
from sklearn.metrics import classification_report
from tensorflow.keras.models import load_model
import joblib
from skimage.feature import graycomatrix, graycoprops 
from itertools import chain
from sklearn.metrics import classification_report
subfolders = ['animal_texture', 'earth_texture','plant_texture']
def preprocess_image_for_gray_prediction(image_path):
    img = cv2.imread(image_path, cv2.IMREAD_GRAYSCALE)
    glcm_distances = [1,2,3,4,5,6,7,8,9,10]
    glcm_angles = [0, 45, 135, 90]
    features = []
    
    for distance in glcm_distances:
        for angle in glcm_angles:
            glcm = graycomatrix(img, distances=[distance], angles=[angle], levels=256, symmetric=True, normed=True)
            max_prob = np.max(glcm)
            contrast = graycoprops(glcm, 'contrast')[0, 0]
            correlation = graycoprops(glcm, 'correlation')[0, 0]
            homogeneity = graycoprops(glcm, 'homogeneity')[0, 0]
            energy = graycoprops(glcm, 'energy')[0, 0]
            entropy = -np.sum(glcm * np.log2(glcm + np.finfo(float).eps))
            features.extend([max_prob, contrast, correlation, homogeneity, energy, entropy])
    
    return np.array(features)


def preprocess_image_for_color_prediction(image_path):
    
    img = cv2.imread(image_path)
    
    features = []
    chans = cv2.split(img)
    for chan in chans:
        hist = cv2.calcHist([chan],[0],None,[256],[0,256])
        hist = list(chain.from_iterable(hist))
        features.extend(hist)
    features=np.array(features)
    features_sum = features.sum()
    features_nor= features / features_sum
    return np.array(features)
    

true_labels = []
predicted_labels = []
labels = [0, 1, 2]
images_and_labels = []
model1_predicted_labels = []
model2_predicted_labels = []
model3_predicted_labels = []
model4_predicted_labels = []


def load_and_preprocess_images(folder_path,label):     
    for filename in os.listdir(folder_path):  
        if filename.endswith(".jpg") or filename.endswith(".png"):  
            file_path = os.path.join(folder_path, filename) 
            images_and_labels.append((file_path, label))


load_and_preprocess_images(r'F:pic\train2\animal_texture',0)
load_and_preprocess_images(r'F:pic\train2\earth_texture',1)
load_and_preprocess_images(r'F:pic\train2\plant_texture',2)


random.shuffle(images_and_labels)
w=128
h=128


cnn_base_model = keras.models.load_model(r'cnn_base_best_model.keras')
cnn_FLY_model = keras.models.load_model(r'cnn_FLY_best_model.keras')

gray_model = load_model('gray_classifier_model.h5')
scaler_filename1 = 'gray_scaler.save'
gray_scaler = joblib.load(scaler_filename1)

color_model = load_model('color_classifier_model.h5')
scaler_filename2 = 'color_scaler.save'
color_scaler = joblib.load(scaler_filename2)


columns = ['prediction_1', 'prediction_2', 'prediction_3']

for i in range(4):
    pd.DataFrame(columns=columns).to_csv('model{}_predictions.csv'.format(i+1), index=False)

for img_path,label in images_and_labels:
    img=cv2.imread(img_path)
    image = cv2.resize(img, (w, h), interpolation = cv2.INTER_AREA)
    image = np.array(image)
    image = image.astype('float32')
    image /= 255
    img = np.expand_dims(image, axis=0)
    prediction1 = cnn_base_model.predict(img)
    predictions=prediction1
    print(prediction1)
    model1_prediction=np.argmax(prediction1)
    model1_predicted_labels.append(model1_prediction)
    df_prediction1 = pd.DataFrame(prediction1)
    df_prediction1.to_csv('model1_predictions.csv',mode='a',header=False,index=False)
    
    img=cv2.imread(img_path)
    image = cv2.resize(img, (w, h), interpolation = cv2.INTER_AREA)
    f_img = np.fft.fft2(image)
    fshift = np.fft.fftshift(f_img)
    magnitude_spectrum = 20 * np.log(np.abs(fshift)+1)
    image = np.array(magnitude_spectrum)
    image = image.astype('float32')
    image /= 255
    img = np.expand_dims(image, axis=0)
    prediction2 =cnn_FLY_model.predict(img)
    model2_prediction=np.argmax(prediction2)
    model2_predicted_labels.append(model2_prediction)
    predictions+=prediction2
    df_prediction2 = pd.DataFrame(prediction2)
    df_prediction2.to_csv('model2_predictions.csv',mode='a', header=False,index=False)
    
    image_features = preprocess_image_for_gray_prediction(img_path)
    image_features_scaled = gray_scaler.transform(np.array([image_features]))
    prediction3 =gray_model.predict(image_features_scaled)
    model3_prediction=np.argmax(prediction3)
    model3_predicted_labels.append(model3_prediction)
    predictions+=prediction3
    df_prediction3 = pd.DataFrame(prediction3)
    df_prediction3.to_csv('model3_predictions.csv',mode='a',header=False, index=False)

    image_features = preprocess_image_for_color_prediction(img_path)
    image_features_scaled = color_scaler.transform(np.array([image_features]))
    prediction4 =color_model.predict(image_features_scaled)
    model4_prediction=np.argmax(prediction4)
    model4_predicted_labels.append(model4_prediction)
    predictions+=prediction4
    df_prediction4 = pd.DataFrame(prediction4)
    df_prediction4.to_csv('model4_predictions.csv',mode='a',header=False, index=False)

    predicted_class_index = np.argmax(predictions)
    print(label,predicted_class_index)
    
    true_labels.append(label)
    predicted_labels.append(predicted_class_index)
df_labels=pd.DataFrame(true_labels,columns=['label'])
df_labels.to_csv('labels.csv',index=False)


model1_report = classification_report(true_labels, model1_predicted_labels, target_names=subfolders)
print(model1_report)
model2_report = classification_report(true_labels, model2_predicted_labels, target_names=subfolders)
print(model2_report)
model3_report = classification_report(true_labels, model3_predicted_labels, target_names=subfolders)
print(model3_report)
model4_report = classification_report(true_labels, model4_predicted_labels, target_names=subfolders)
print(model4_report)


report = classification_report(true_labels, predicted_labels, target_names=subfolders)
print(report)

训练堆叠模型

import numpy as np  
from skimage.feature import graycomatrix, graycoprops  
from sklearn.model_selection import train_test_split  
from sklearn.preprocessing import StandardScaler  
from tensorflow.keras.models import Sequential  
from tensorflow.keras.layers import Dense  
from tensorflow.keras.utils import to_categorical  
from tensorflow.keras.optimizers import Adam  
import cv2
import os
import joblib
from scikeras.wrappers import KerasClassifier, KerasRegressor
from sklearn.model_selection import GridSearchCV
import tensorflow as tf
import pandas as pd
df1 = pd.read_csv('model1_predictions.csv')
df2 = pd.read_csv('model2_predictions.csv')
df3 = pd.read_csv('model3_predictions.csv')
df4 = pd.read_csv('model4_predictions.csv')
ensemble_features = pd.concat([df1, df2,df3, df4],axis=1)

labels = pd.read_csv('labels.csv')['label'] #

X = np.array(ensemble_features)  
y = np.array(labels)  
  
scaler = StandardScaler()  
X_scaled = scaler.fit_transform(X)  
y_categorical = to_categorical(y)  
  
X_train, X_test, y_train, y_test = train_test_split(X_scaled, y_categorical, test_size=0.2, random_state=76)  

model = Sequential()  
model.add(Dense(64, input_dim=X_train.shape[1], activation='relu'))  
model.add(Dense(32, activation='relu'))  
model.add(Dense(16, activation='relu')) 
model.add(Dense(y_categorical.shape[1], activation='softmax'))  
  
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])  
  
model.fit(X_train, y_train, epochs=20, batch_size=20, validation_split=0.2)  
  
loss, accuracy = model.evaluate(X_test, y_test)  
print(f'Test loss: {loss:.4f}')  
print(f'Test accuracy: {accuracy:.4f}')


from sklearn.metrics import classification_report, confusion_matrix


class_names = {0: 'animal_texture', 1: 'earth_texture', 2: 'plant_texture'}
y_pred = model.predict(X_test)
y_pred_classes = np.argmax(y_pred, axis=1)
y_test_classes = np.argmax(y_test, axis=1)
y_pred_class_names = [class_names[pred] for pred in y_pred_classes]
y_true_class_names = [class_names[label] for label in y_test_classes]


report_with_names = classification_report(y_true_class_names, y_pred_class_names, target_names=class_names.values())
print(report_with_names)

scaler_filename = 'final_scaler.save'
joblib.dump(scaler, scaler_filename)
print("Scaler saved.")
model.save('final_classifier_model.h5')
print("Model saved.")

最终测试

import numpy as np
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
%matplotlib inline
import os
import cv2
import random
import keras_tuner as kt
from tensorflow import keras
import tensorflow as tf
from pathlib import Path
from tensorflow.keras import layers
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Conv2D, MaxPooling2D, GlobalAveragePooling2D
from tensorflow.keras.layers import Dense, Dropout, Activation, Flatten
from tensorflow.keras.optimizers import SGD
from sklearn.metrics import classification_report
from tensorflow.keras.models import load_model
import joblib
from skimage.feature import graycomatrix, graycoprops 
from itertools import chain
from sklearn.metrics import classification_report
import pandas as pd
def preprocess_image_for_gray_prediction(image_path):
    img = cv2.imread(image_path, cv2.IMREAD_GRAYSCALE)
    glcm_distances = [1,2,3,4,5,6,7,8,9,10]
    glcm_angles = [0, 45, 135, 90]
    features = []
    
    for distance in glcm_distances:
        for angle in glcm_angles:
            glcm = graycomatrix(img, distances=[distance], angles=[angle], levels=256, symmetric=True, normed=True)
            max_prob = np.max(glcm)
            contrast = graycoprops(glcm, 'contrast')[0, 0]
            correlation = graycoprops(glcm, 'correlation')[0, 0]
            homogeneity = graycoprops(glcm, 'homogeneity')[0, 0]
            energy = graycoprops(glcm, 'energy')[0, 0]
            entropy = -np.sum(glcm * np.log2(glcm + np.finfo(float).eps))
            features.extend([max_prob, contrast, correlation, homogeneity, energy, entropy])
    
    return np.array(features)


def preprocess_image_for_color_prediction(image_path):
    
    img = cv2.imread(image_path)
    
    features = []
    chans = cv2.split(img)
    for chan in chans:
        hist = cv2.calcHist([chan],[0],None,[256],[0,256])
        hist = list(chain.from_iterable(hist))
        features.extend(hist)
    features=np.array(features)
    features_sum = features.sum()
    features_nor= features / features_sum
    return np.array(features)
    

true_labels = []
final_predicted_labels= []
predicted_labels = []
model1_predicted_labels = []
model2_predicted_labels = []
model3_predicted_labels = []
model4_predicted_labels = []

labels = [0, 1, 2]
images_and_labels = []

def load_and_preprocess_images(folder_path,label):     
    for filename in os.listdir(folder_path):  
        if filename.endswith(".jpg") or filename.endswith(".png"):  
            file_path = os.path.join(folder_path, filename) 
            images_and_labels.append((file_path, label))


load_and_preprocess_images(r'F:pic\test\animal_texture',0)
load_and_preprocess_images(r'F:pic\test\earth_texture',1)
load_and_preprocess_images(r'F:pic\test\plant_texture',2)

random.shuffle(images_and_labels)
w=128
h=128


cnn_base_model = keras.models.load_model(r'cnn_base_best_model.keras')
cnn_FLY_model = keras.models.load_model(r'cnn_FLY_best_model.keras')

gray_model = load_model('gray_classifier_model.h5')
scaler_filename1 = 'gray_scaler.save'
gray_scaler = joblib.load(scaler_filename1)

color_model = load_model('color_classifier_model.h5')
scaler_filename2 = 'color_scaler.save'
color_scaler = joblib.load(scaler_filename2)

final_model = load_model('final_classifier_model.h5')
scaler_filename2 = 'final_scaler.save'
final_scaler = joblib.load(scaler_filename2)

for img_path,label in images_and_labels:
  
    img=cv2.imread(img_path)
    image = cv2.resize(img, (w, h), interpolation = cv2.INTER_AREA)
    image = np.array(image)
    image = image.astype('float32')
    image /= 255
    img = np.expand_dims(image, axis=0)
    prediction1 = cnn_base_model.predict(img)
    model1_prediction=np.argmax(prediction1)
    model1_predicted_labels.append(model1_prediction)
    predictions=prediction1
   
    
    img=cv2.imread(img_path)
    image = cv2.resize(img, (w, h), interpolation = cv2.INTER_AREA)
    f_img = np.fft.fft2(image)
    fshift = np.fft.fftshift(f_img)
    magnitude_spectrum = 20 * np.log(np.abs(fshift)+1)
    image = np.array(magnitude_spectrum)
    image = image.astype('float32')
    image /= 255
    img = np.expand_dims(image, axis=0)
    prediction2 =cnn_FLY_model.predict(img)
    model2_prediction=np.argmax(prediction2)
    model2_predicted_labels.append(model2_prediction)
    predictions=predictions+prediction2
    
    
    
    image_features = preprocess_image_for_gray_prediction(img_path)
    image_features_scaled = gray_scaler.transform(np.array([image_features]))
    prediction3 =gray_model.predict(image_features_scaled)
    model3_prediction=np.argmax(prediction3)
    model3_predicted_labels.append(model3_prediction)
    predictions=predictions+prediction3
    


    image_features = preprocess_image_for_color_prediction(img_path)
    image_features_scaled = color_scaler.transform(np.array([image_features]))
    prediction4 =color_model.predict(image_features_scaled)
    model4_prediction=np.argmax(prediction4)
    model4_predicted_labels.append(model4_prediction)
    predictions=predictions+prediction4


    prediction_sum=np.concatenate((prediction1.flatten(),prediction2.flatten(),prediction3.flatten(),prediction4.flatten()))
    print(prediction_sum)
    predictions_final=final_model.predict(final_scaler.transform(prediction_sum.reshape(1,-1)))
    final_prediction=np.argmax(predictions_final)
    final_predicted_labels.append(final_prediction)
    
    predicted_class_index = np.argmax(predictions)
    true_labels.append(label)
    predicted_labels.append(predicted_class_index)
    

model1_report = classification_report(true_labels, model1_predicted_labels, target_names=subfolders)
print(model1_report)
model2_report = classification_report(true_labels, model2_predicted_labels, target_names=subfolders)
print(model2_report)
model3_report = classification_report(true_labels, model3_predicted_labels, target_names=subfolders)
print(model3_report)
model4_report = classification_report(true_labels, model4_predicted_labels, target_names=subfolders)
print(model4_report)


report1 = classification_report(true_labels, predicted_labels, target_names=subfolders)
print(report1)

report2 = classification_report(true_labels, final_predicted_labels, target_names=subfolders)
print(report2)