tensorflow2学习（三）：双卷积神经网络（CNN3D）高光谱图像分类

参考文章双卷积池化结构的3D-CNN高光谱遥感影像分类方法，将双卷积神经网络（CNN3D）应用与高光谱图像HSI分类，记录有亲自实现的Python源代码，环境：TF2+Python3+cuda10.1+CuDNN7.6 .

tensorflow2学习（三）：双卷积神经网络（CNN3D）高光谱图像分类

一、参考资料

1. Conv3D参数
2. Keras的网络层介绍
3. keras使用入门及3D卷积神经网络资源
4 .Keras源码实例
5. 双卷积池化结构的3D-CNN高光谱遥感影像分类方法

二、网络结构

针对数据集Indian-Pines数据集设计的网络部分结构（Chanel_Last）：

Input	N * 200 * 19 * 19 * 1
Conv3D	N * 40 * 17 * 17 * 16
Conv3D(16, kernel_size=(3, 3, 3), input_shape=(200, 19, 19, 1), strides=(5, 1, 1))
Conv3D	N * 40 * 17 * 17 * 16
Conv3D(16, kernel_size=(3, 3, 3), padding=‘same’)
MaxPooling	N * 20 * 8 * 8 * 16
MaxPooling3D(pool_size=2)
Conv3D	N * 20 * 8 * 8 * 32
Conv3D(32, kernel_size=(3, 3, 3), padding=‘same’))
Conv3D	N * 20 * 8 * 8 * 32
Conv3D(32, kernel_size=(3, 3, 3), padding=‘same’)
MaxPooling	N * 10 * 4 * 4 * 32
MaxPooling3D(pool_size=2)
Conv3D	N * 10 * 4 * 4 * 64
Conv3D(64, kernel_size=(3, 3, 3), padding=‘same’))
Conv3D	N * 10 * 4 * 4 * 64
Conv3D(64, kernel_size=(3, 3, 3), padding=‘same’)
MaxPooling	N * 5 * 2 * 2 * 64
MaxPooling3D(pool_size=2)

详细结构可以参考代码部分。

三、代码

import os
from tensorflow.keras.layers import Conv3D, MaxPooling3D, Dropout, Dense, Flatten, Activation, BatchNormalization
import tensorflow as tf

from DataLoad import loadData

path_image = 'Indian-pines/Indian_pines_corrected.mat'
path_label = 'Indian-pines/Indian_pines_gt.mat'
key_image = 'indian_pines_corrected'
key_label = 'indian_pines_gt'
window_size = 19
input_size = 200
X_train_new, X_val_new, y_train_new, y_val_new, train_samples = loadData(path_image, path_label, key_image, key_label)
batch_size = 16
model = tf.keras.Sequential([
    Conv3D(16, kernel_size=(3, 3, 3), input_shape=(200, 19, 19, 1), strides=(5, 1, 1)),  # 17
    BatchNormalization(),
    Activation(tf.nn.relu),
    Conv3D(16, kernel_size=(3, 3, 3), padding='same'),
    BatchNormalization(),
    Activation(tf.nn.relu),
    MaxPooling3D(pool_size=2),  # 8
    Conv3D(32, kernel_size=(3, 3, 3), padding='same'),
    BatchNormalization(),
    Activation(tf.nn.relu),
    Conv3D(32, kernel_size=(3, 3, 3), padding='same'),
    BatchNormalization(),
    Activation(tf.nn.relu),
    MaxPooling3D(pool_size=2),  # 4
    Conv3D(64, kernel_size=(3, 3, 3), padding='same'),
    BatchNormalization(),
    Activation(tf.nn.relu),
    Conv3D(64, kernel_size=(3, 3, 3), padding='same'),
    BatchNormalization(),
    Activation(tf.nn.relu),
    MaxPooling3D(pool_size=2),  # 2
    Flatten(),
    Dense(128),
    BatchNormalization(),
    Activation(tf.nn.relu),
    Dropout(0.5),
    Dense(16, activation=tf.nn.softmax)]
)
epoch = 100
tf.keras.backend.set_learning_phase(True)
model.compile(
    optimizer=tf.keras.optimizers.RMSprop(0.0003),
    loss=tf.keras.losses.categorical_crossentropy,
    metrics=[tf.keras.metrics.categorical_accuracy])
model_dir = 'model/CNN6_IP2'
model_file = 'model_weights'
model_saved_path = model_dir + '/' + model_file
is_load_model = input('Would you like load the existed model weights if it exist ? y/n\n')
if is_load_model == 'y':
    model.load_weights(model_saved_path)
    print('An existed model_weight table has been gotten...')
else:
    print('A new model will be gotten...')
print('*****************************************')
epoch = int(input('Please input epoch : '))
if epoch < 0:
    epoch = 0
print('*****************************************')
if epoch != 0:
    hist = model.fit(
        X_train_new,
        y_train_new,
        batch_size=batch_size,
        epochs=epoch,
        shuffle=True
    )
loss, acc = model.evaluate(
    X_val_new,
    y_val_new,
    batch_size=batch_size
)
print('train: val', y_train_new.shape, y_val_new.shape)
print('acc:', acc)
if not os.path.exists(model_dir):
    os.mkdir(model_dir)
model.save_weights(model_saved_path)
print('model_weights have been saved at ' + model_saved_path)
print(model.summary())

import numpy as np
import tensorflow as tf
from scipy.io import loadmat
from sklearn.model_selection import train_test_split


def loadData(path_image='Indian-pines/Indian_pines_corrected.mat',
             path_label='Indian-pines/Indian_pines_gt.mat',
             key_image='indian_pines_corrected',
             key_label='indian_pines_gt',
             window_size=19, input_size=200):
    mat = loadmat(path_image)
    # print(mat.keys())
    features = mat[key_image]
    features_shape = features.shape

    mat_labels = loadmat(path_label)
    labels = mat_labels[key_label]
    labels = np.reshape(labels, (-1, 1))
    f2 = np.zeros((labels.shape[0], window_size, window_size, input_size), dtype='float32')

    parameter_b = np.array([i - window_size // 2 for i in range(window_size)])
    for i in range(features_shape[0]):
        for j in range(features_shape[1]):
            index = i * features_shape[1] + j
            for p in range(window_size):
                for q in range(window_size):
                    f2[index][p][q] = \
                        features[(i + parameter_b[p]) % features_shape[0]][(j + parameter_b[q]) % features_shape[1]]

    index = np.where(np.reshape(labels, (-1)) == 0)
    labels = np.delete(labels, index, axis=0)
    # print(np.unique(labels))
    tmp = np.unique(labels)
    for i in range(len(labels)):
        labels[i][0] = np.where(tmp == labels[i][0])[0][0]
    f2 = np.delete(f2, index, axis=0)
    labels = np.reshape(labels, (-1))
    dataset = [f2, labels]
    (X, Y) = (dataset[0], dataset[1])  # -1,19,19,200
    X = X.swapaxes(1, 3)
    print('X_shape: ', X.shape)
    train_set = X[:, :, :, :, np.newaxis]
    print('TrainSet_shape: ', train_set.shape)
    classes = 16
    Y = tf.keras.utils.to_categorical(Y, classes)
    train_set = train_set.astype('float32')
    train_set -= np.mean(train_set)
    train_set /= np.max(train_set)
    # Split the data
    X_train_new, X_val_new, y_train_new, y_val_new = train_test_split(train_set, Y, test_size=0.8, random_state=4)
    return X_train_new, X_val_new, y_train_new, y_val_new, labels.shape[0]