一维卷积神经网络直接读取mat文件并进行数据增强和归一化

  众多师弟师妹在使用一维卷积神经网络对凯斯西储，哥伦比亚等大学开源的轴承故障数据集进行故障诊断时，不知道该怎么把Matlab文件中的数据直接导进伸进网络进行训练。
   我之前也不会，我一直用matlab打开数据然后黏贴到EXCEL再进行数据增强和归一化，最终保存成CSV格式。在训练神经网络的时候直接读取归一化好的数据进行训练，虽然这样比较直观，而且只做一次数据集就可以满足所有形式神经网络的训练。
   但是，如果要研究自己发论文所提出来的神经网络模型的泛化能力、抗噪声能力、超参数寻优等问题的时候可能要不停的重新制作加噪声的数据集等等，这时候就比较麻烦。
   因此，本人自己编写了一段数据处理的代码，包括归一化，代码如下所示：    

众多师弟师妹在使用一维卷积神经网络对凯斯西储，哥伦比亚等大学开源的轴承故障数据集进行故障诊断时，不知道该怎么把Matlab文件中的数据直接导进伸进网络进行训练。
我之前也不会，我一直用matlab打开数据然后黏贴到EXCEL再进行数据增强和归一化，最终保存成CSV格式。在训练神经网络的时候直接读取归一化好的数据进行训练，虽然这样比较直观，而且只做一次数据集就可以满足所有形式神经网络的训练。
但是，如果要研究自己发论文所提出来的神经网络模型的泛化能力、抗噪声能力、超参数寻优等问题的时候可能要不停的重新制作加噪声的数据集等等，这时候就比较麻烦。
因此，本人自己编写了一段数据处理的代码，包括归一化，代码如下所示：

# -*- coding: utf-8 -*-
"""
Created on Tue Oct  5 14:21:26 2021

@author: Hebei University of Engineering
"""
import numpy as np
import pandas as pd
import keras
from keras.models import Sequential
from keras.layers import LSTM,Input,add,Dense,AveragePooling1D, Activation, Flatten, Convolution1D, Dropout,MaxPooling1D,BatchNormalization,GlobalAveragePooling1D,ZeroPadding1D
import tensorflow as tf#将tensorflow命名为tf
from keras.models import Sequential#导入序列函数
from keras.wrappers.scikit_learn import KerasClassifier#导入分类标签显示工具
from keras.utils import np_utils,plot_model#导入独热吗部分编辑函数
from sklearn.model_selection import cross_val_score,train_test_split,KFold#导入独热吗部分编辑函数
from sklearn.preprocessing import LabelEncoder#导入独热吗部分编辑函数
from keras.models import load_model#导入保存模型工具
from keras.models import model_from_json#导入保存模型工具
import matplotlib.pyplot as plt#导入画图工具
from sklearn.metrics import confusion_matrix#导入混淆矩阵,这段代码用不上
import itertools#这个我忘记了
from keras import layers#导入层
from keras.optimizers import SGD#导入优化器
import scipy.io as scio
import os
os.environ["CUDA_VISIBLE_DEVICES"] = "0"
data1 = scio.loadmat('C:/Users/1/Desktop/大论文作图/mydata1')['DA09'] 
data2 = scio.loadmat('C:/Users/1/Desktop/大论文作图/mydata2')['DA09']
data3 = scio.loadmat('C:/Users/1/Desktop/大论文作图/mydata3')['DA09']
data4 = scio.loadmat('C:/Users/1/Desktop/大论文作图/mydata4')['DA09'] 
data5 = scio.loadmat('C:/Users/1/Desktop/大论文作图/mydata5')['DA09']
data6 = scio.loadmat('C:/Users/1/Desktop/大论文作图/mydata6')['DA09']
data7 = scio.loadmat('C:/Users/1/Desktop/大论文作图/mydata7')['DA09']
data8 = scio.loadmat('C:/Users/1/Desktop/大论文作图/mydata8')['DA09']
data9 = scio.loadmat('C:/Users/1/Desktop/大论文作图/mydata9')['DA09']


 
#由于凯斯西储大学的轴承故障数据集（字典格式）中包含多个键['X097_DE_time']
#L=list()
 
def ber(data,windowlen,buchang):
    p=len(data)
    L=list()
    q=(p-windowlen)
    for i in range(q):
        if  int(i) % int(buchang) == 0:
            a=data[i:i+windowlen]          
            L.append(a)
    return L

def data(data,windowlen,buchang,biaoqian):
    data=ber(data,windowlen,buchang)
    data=np.array(data)
    x,y,z=data.shape
    c=data.reshape(x,y)
    for i in range(x):
        c1=c[i,:]
        pingjunzhi=np.mean(c1)
        c2=c1-pingjunzhi
        c3=max(abs(c2))
        c1=c2/c3
        c[i,:]=c1
    k=np.zeros((x,1))+biaoqian
    KK=np.hstack([c,k])
    print(k)
    print(KK.shape)
    return KK
    
yb1=data(data1,1024,24,1)
yb2=data(data2,1024,24,2)
yb3=data(data3,1024,24,3)
yb4=data(data4,1024,24,4)
yb5=data(data5,1024,24,5)
yb6=data(data6,1024,60,6)
yb7=data(data7,1024,72,7)
yb8=data(data8,1024,72,8)
yb9=data(data9,1024,72,9)

L3=np.vstack([yb1,yb2,yb3,yb4,yb5,yb6,yb7,yb8,yb9])
X = np.expand_dims(L3[:, 0:1024].astype(float), axis=2)#每行的1-1024列作为训练数据
Y = L3[:, 1024]#每行的第1025列作为训练标签

encoder = LabelEncoder()
Y_encoded = encoder.fit_transform(Y)
Y_onehot = np_utils.to_categorical(Y_encoded)
 
#划分数据集
X_train, X_test, Y_train, Y_test = train_test_split(X, Y_onehot, test_size=0.3, random_state=0)
 

seed = 7
np.random.seed(seed)
 
def Conv2d_BN(x, nb_filter,kernel_size, strides=1, padding='same',name=None):
    if name is not None:
        bn_name = name + '_bn'
        conv_name = name + '_conv'
    else:
        bn_name = None
        conv_name = None
 
    x = Convolution1D(nb_filter,kernel_size,padding=padding,strides=strides,activation='tanh',name=conv_name)(x)
    x = MaxPooling1D(pool_size=2,strides=1,padding='same')(x)
    x = BatchNormalization(axis=1,name=bn_name)(x)
    return x
 
def Conv_Block(inpt,nb_filter,kernel_size,strides=1, with_conv_shortcut=False):
    x = Conv2d_BN(inpt,nb_filter=nb_filter,kernel_size=1,strides=strides,padding='same')
    x = Conv2d_BN(x, nb_filter=nb_filter, kernel_size=3, padding='same')
    x = Conv2d_BN(x, nb_filter=nb_filter, kernel_size=1, padding='same')
    if with_conv_shortcut:
        shortcut = Conv2d_BN(inpt,nb_filter=nb_filter,strides=strides,kernel_size=kernel_size)
        x = add([x,shortcut])
        return x
    else:
        x = add([x,inpt])
        return x
 
inpt = Input(shape=(1024,1))
x = ZeroPadding1D(3)(inpt)
x = Dropout(0.5)(x)
x = Conv2d_BN(x,nb_filter=16,kernel_size=64,strides=8,padding='same')
x = Conv_Block(x,nb_filter=32,kernel_size=32,strides=1,with_conv_shortcut=True)
x = Conv_Block(x,nb_filter=64,kernel_size=3,strides=1,with_conv_shortcut=True)
x = Conv_Block(x,nb_filter=64,kernel_size=3,strides=1,with_conv_shortcut=True)
x = Conv_Block(x,nb_filter=64,kernel_size=3,strides=1,with_conv_shortcut=True)
x = Conv_Block(x,nb_filter=64,kernel_size=3,strides=1,with_conv_shortcut=True)
x = Conv_Block(x,nb_filter=64,kernel_size=3,strides=1,with_conv_shortcut=True)
x = Flatten()(x)
x = BatchNormalization(axis=-1, momentum=0.99, epsilon=0.001, center=True, scale=True, beta_initializer='zeros', gamma_initializer='ones', moving_mean_initializer='zeros', moving_variance_initializer='ones', beta_regularizer=None, gamma_regularizer=None, beta_constraint=None, gamma_constraint=None)(x) 
x=Dense(100,activation='tanh')(x)
x = BatchNormalization(axis=-1, momentum=0.99, epsilon=0.001, center=True, scale=True, beta_initializer='zeros', gamma_initializer='ones', moving_mean_initializer='zeros', moving_variance_initializer='ones', beta_regularizer=None, gamma_regularizer=None, beta_constraint=None, gamma_constraint=None)(x) 
x=Dropout(0.5)(x)
x=Dense(60,activation='tanh')(x)
x = BatchNormalization(axis=-1, momentum=0.99, epsilon=0.001, center=True, scale=True, beta_initializer='zeros', gamma_initializer='ones', moving_mean_initializer='zeros', moving_variance_initializer='ones', beta_regularizer=None, gamma_regularizer=None, beta_constraint=None, gamma_constraint=None)(x) 
x=Dropout(0.5)(x)
x = Dense(9,activation='softmax')(x)

from keras.models import Model 
model = Model(inputs=inpt,outputs=x)
model.summary()

model.compile(loss='categorical_crossentropy',optimizer='adam', metrics=['accuracy'])

from keras.callbacks import TensorBoard
import time
import keras
model_name = "模型名-{}".format(int(time.time()))
tensorboard = TensorBoard(log_dir='logs'.format(model_name),write_images='Ture')
tensorboard=keras.callbacks.TensorBoard(histogram_freq=1)
history=model.fit(X_train, Y_train,epochs=600,validation_data=(X_test, Y_test),batch_size=512,callbacks=[tensorboard])
#训练批量大小和批次
#history=model.fit(X_train, Y_train,epochs=100,validation_data=(X_test, Y_test),batch_size=64)

#model.save('C:/Users/1/Desktop/加速度传感器4.h5')
model_json =model.to_json()
with open('C:/Users/1/Desktop/大论文作图/传感器9', 'w')as json_file:
    json_file.write(model_json)# 权重不在json中,只保存网络结构
model.save_weights('C:/Users/1/Desktop/大论文作图/传感器9.json.h5')
#画出损失函数图
import matplotlib.pyplot as plt
plt.rcParams['font.sans-serif'] = ['SimHei']
plt.plot(history.history['loss'],'_',label='train_loss')
plt.plot(history.history['val_loss'],':',label='val_loss')
plt.plot(history.history['accuracy'],'.',label='train_acc')
plt.plot(history.history['val_accuracy'],'r',label='val_acc')
plt.show()

los=history.history['val_loss']
Aos=history.history['val_accuracy']
a=history.history['loss']
b=los=history.history['val_loss']
print(b)

有人要问没看见给形成的数据进行打乱顺序，这一点不用担心，在代码的第90行，划分数据集的时候是随机选择并打乱顺序的。

省刊
核心
EI
SCI
师弟师妹赶紧发吧