深度残差网络+自适应参数化ReLU激活函数：调参记录14

最新推荐文章于 2024-09-19 20:35:10 发布
最新推荐文章于 2024-09-19 20:35:10 发布
阅读量298 收藏
点赞数
文章标签： 深度学习 神经网络 机器学习
原文链接：https://blog.csdn.net/dangqing1988/article/details/105831139
版权

这次再尝试解决过拟合，把残差模块的数量减少到2个，自适应参数化ReLU激活函数里面第一个全连接层的权重数量，减少为之前的1/8，批量大小设置为1000（主要是为了省时间）。

自适应参数化ReLU激活函数的基本原理如下：
在这里插入图片描述
Keras程序如下：

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
Created on Tue Apr 14 04:17:45 2020
Implemented using TensorFlow 1.10.0 and Keras 2.2.1

Minghang Zhao, Shisheng Zhong, Xuyun Fu, Baoping Tang, Shaojiang Dong, Michael Pecht,
Deep Residual Networks with Adaptively Parametric Rectifier Linear Units for Fault Diagnosis, 
IEEE Transactions on Industrial Electronics, 2020,  DOI: 10.1109/TIE.2020.2972458 

@author: Minghang Zhao
"""

from __future__ import print_function
import keras
import numpy as np
from keras.datasets import cifar10
from keras.layers import Dense, Conv2D, BatchNormalization, Activation, Minimum
from keras.layers import AveragePooling2D, Input, GlobalAveragePooling2D, Concatenate, Reshape
from keras.regularizers import l2
from keras import backend as K
from keras.models import Model
from keras import optimizers
from keras.preprocessing.image import ImageDataGenerator
from keras.callbacks import LearningRateScheduler
K.set_learning_phase(1)

# The data, split between train and test sets
(x_train, y_train), (x_test, y_test) = cifar10.load_data()

# Noised data
x_train = x_train.astype('float32') / 255.
x_test = x_test.astype('float32') / 255.
x_test = x_test-np.mean(x_train)
x_train = x_train-np.mean(x_train)
print('x_train shape:', x_train.shape)
print(x_train.shape[0], 'train samples')
print(x_test.shape[0], 'test samples')

# convert class vectors to binary class matrices
y_train = keras.utils.to_categorical(y_train, 10)
y_test = keras.utils.to_categorical(y_test, 10)

# Schedule the learning rate, multiply 0.1 every 1500 epoches
def scheduler(epoch):
    if epoch % 1500 == 0 and epoch != 0:
        lr = K.get_value(model.optimizer.lr)
        K.set_value(model.optimizer.lr, lr * 0.1)
        print("lr changed to {}".format(lr * 0.1))
    return K.get_value(model.optimizer.lr)

# An adaptively parametric rectifier linear unit (APReLU)
def aprelu(inputs):
    # get the number of channels
    channels = inputs.get_shape().as_list()[-1]
    # get a zero feature map
    zeros_input = keras.layers.subtract([inputs, inputs])
    # get a feature map with only positive features
    pos_input = Activation('relu')(inputs)
    # get a feature map with only negative features
    neg_input = Minimum()([inputs,zeros_input])
    # define a network to obtain the scaling coefficients
    scales_p = GlobalAveragePooling2D()(pos_input)
    scales_n = GlobalAveragePooling2D()(neg_input)
    scales = Concatenate()([scales_n, scales_p])
    scales = Dense(channels//8, activation='linear', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(scales)
    scales = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(scales)
    scales = Activation('relu')(scales)
    scales = Dense(channels, activation='linear', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(scales)
    scales = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(scales)
    scales = Activation('sigmoid')(scales)
    scales = Reshape((1,1,channels))(scales)
    # apply a paramtetric relu
    neg_part = keras.layers.multiply([scales, neg_input])
    return keras.layers.add([pos_input, neg_part])

# Residual Block
def residual_block(incoming, nb_blocks, out_channels, downsample=False,
                   downsample_strides=2):
    
    residual = incoming
    in_channels = incoming.get_shape().as_list()[-1]
    
    for i in range(nb_blocks):
        
        identity = residual
        
        if not downsample:
            downsample_strides = 1
        
        residual = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(residual)
        residual = aprelu(residual)
        residual = Conv2D(out_channels, 3, strides=(downsample_strides, downsample_strides), 
                          padding='same', kernel_initializer='he_normal', 
                          kernel_regularizer=l2(1e-4))(residual)
        
        residual = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(residual)
        residual = aprelu(residual)
        residual = Conv2D(out_channels, 3, padding='same', kernel_initializer='he_normal', 
                          kernel_regularizer=l2(1e-4))(residual)
        
        # Downsampling
        if downsample_strides > 1:
            identity = AveragePooling2D(pool_size=(1,1), strides=(2,2))(identity)
            
        # Zero_padding to match channels
        if in_channels != out_channels:
            zeros_identity = keras.layers.subtract([identity, identity])
            identity = keras.layers.concatenate([identity, zeros_identity])
            in_channels = out_channels
        
        residual = keras.layers.add([residual, identity])
    
    return residual


# define and train a model
inputs = Input(shape=(32, 32, 3))
net = Conv2D(16, 3, padding='same', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(inputs)
# net = residual_block(net, 3, 16, downsample=False)
net = residual_block(net, 1, 32, downsample=True)
# net = residual_block(net, 2, 32, downsample=False)
net = residual_block(net, 1, 64, downsample=True)
# net = residual_block(net, 2, 64, downsample=False)
net = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(net)
net = Activation('relu')(net)
net = GlobalAveragePooling2D()(net)
outputs = Dense(10, activation='softmax', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(net)
model = Model(inputs=inputs, outputs=outputs)
sgd = optimizers.SGD(lr=0.1, decay=0., momentum=0.9, nesterov=True)
model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])

# data augmentation
datagen = ImageDataGenerator(
    # randomly rotate images in the range (deg 0 to 180)
    rotation_range=30,
    # Range for random zoom
    zoom_range = 0.2,
    # shear angle in counter-clockwise direction in degrees
    shear_range = 30,
    # randomly flip images
    horizontal_flip=True,
    # randomly shift images horizontally
    width_shift_range=0.125,
    # randomly shift images vertically
    height_shift_range=0.125)

reduce_lr = LearningRateScheduler(scheduler)
# fit the model on the batches generated by datagen.flow().
model.fit_generator(datagen.flow(x_train, y_train, batch_size=1000),
                    validation_data=(x_test, y_test), epochs=5000, 
                    verbose=1, callbacks=[reduce_lr], workers=4)

# get results
K.set_learning_phase(0)
DRSN_train_score = model.evaluate(x_train, y_train, batch_size=1000, verbose=0)
print('Train loss:', DRSN_train_score[0])
print('Train accuracy:', DRSN_train_score[1])
DRSN_test_score = model.evaluate(x_test, y_test, batch_size=1000, verbose=0)
print('Test loss:', DRSN_test_score[0])
print('Test accuracy:', DRSN_test_score[1])

实验结果如下（为了方便看，删掉了一部分等号）：

Epoch 2000/5000
50/50 [=======] - 10s 191ms/step - loss: 0.5083 - acc: 0.8658 - val_loss: 0.5190 - val_acc: 0.8644
Epoch 2001/5000
50/50 [=======] - 10s 194ms/step - loss: 0.5102 - acc: 0.8651 - val_loss: 0.5203 - val_acc: 0.8656
Epoch 2002/5000
50/50 [=======] - 10s 192ms/step - loss: 0.5073 - acc: 0.8659 - val_loss: 0.5245 - val_acc: 0.8612
Epoch 2003/5000
50/50 [=======] - 10s 190ms/step - loss: 0.5105 - acc: 0.8646 - val_loss: 0.5181 - val_acc: 0.8636
Epoch 2004/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5080 - acc: 0.8661 - val_loss: 0.5217 - val_acc: 0.8631
Epoch 2005/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5074 - acc: 0.8641 - val_loss: 0.5237 - val_acc: 0.8614
Epoch 2006/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5060 - acc: 0.8651 - val_loss: 0.5241 - val_acc: 0.8641
Epoch 2007/5000
50/50 [=======] - 10s 190ms/step - loss: 0.5096 - acc: 0.8651 - val_loss: 0.5185 - val_acc: 0.8660
Epoch 2008/5000
50/50 [=======] - 9s 190ms/step - loss: 0.5053 - acc: 0.8686 - val_loss: 0.5186 - val_acc: 0.8624
Epoch 2009/5000
50/50 [=======] - 10s 191ms/step - loss: 0.5057 - acc: 0.8670 - val_loss: 0.5208 - val_acc: 0.8636
Epoch 2010/5000
50/50 [=======] - 10s 190ms/step - loss: 0.5102 - acc: 0.8653 - val_loss: 0.5214 - val_acc: 0.8614
Epoch 2011/5000
50/50 [=======] - 9s 188ms/step - loss: 0.5091 - acc: 0.8651 - val_loss: 0.5211 - val_acc: 0.8629
Epoch 2012/5000
50/50 [=======] - 9s 187ms/step - loss: 0.5069 - acc: 0.8672 - val_loss: 0.5221 - val_acc: 0.8616
Epoch 2013/5000
50/50 [=======] - 9s 184ms/step - loss: 0.5098 - acc: 0.8652 - val_loss: 0.5241 - val_acc: 0.8625
Epoch 2014/5000
50/50 [=======] - 9s 187ms/step - loss: 0.5056 - acc: 0.8666 - val_loss: 0.5188 - val_acc: 0.8623
Epoch 2015/5000
50/50 [=======] - 9s 189ms/step - loss: 0.5050 - acc: 0.8672 - val_loss: 0.5237 - val_acc: 0.8621
Epoch 2016/5000
50/50 [=======] - 10s 193ms/step - loss: 0.5057 - acc: 0.8667 - val_loss: 0.5207 - val_acc: 0.8607
Epoch 2017/5000
50/50 [=======] - 10s 192ms/step - loss: 0.5092 - acc: 0.8642 - val_loss: 0.5172 - val_acc: 0.8637
Epoch 2018/5000
50/50 [=======] - 9s 190ms/step - loss: 0.5062 - acc: 0.8671 - val_loss: 0.5265 - val_acc: 0.8612
Epoch 2019/5000
50/50 [=======] - 9s 188ms/step - loss: 0.5112 - acc: 0.8648 - val_loss: 0.5256 - val_acc: 0.8617
Epoch 2020/5000
50/50 [=======] - 9s 185ms/step - loss: 0.5071 - acc: 0.8663 - val_loss: 0.5241 - val_acc: 0.8622
Epoch 2021/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5079 - acc: 0.8660 - val_loss: 0.5212 - val_acc: 0.8627
Epoch 2022/5000
50/50 [=======] - 9s 185ms/step - loss: 0.5058 - acc: 0.8667 - val_loss: 0.5227 - val_acc: 0.8600
Epoch 2023/5000
50/50 [=======] - 9s 189ms/step - loss: 0.5070 - acc: 0.8661 - val_loss: 0.5259 - val_acc: 0.8608
Epoch 2024/5000
50/50 [=======] - 10s 193ms/step - loss: 0.5053 - acc: 0.8663 - val_loss: 0.5219 - val_acc: 0.8606
Epoch 2025/5000
50/50 [=======] - 10s 191ms/step - loss: 0.5106 - acc: 0.8655 - val_loss: 0.5205 - val_acc: 0.8625
Epoch 2026/5000
50/50 [=======] - 10s 191ms/step - loss: 0.5090 - acc: 0.8649 - val_loss: 0.5221 - val_acc: 0.8610
Epoch 2027/5000
50/50 [=======] - 9s 189ms/step - loss: 0.5103 - acc: 0.8648 - val_loss: 0.5242 - val_acc: 0.8631
Epoch 2028/5000
50/50 [=======] - 9s 189ms/step - loss: 0.5051 - acc: 0.8663 - val_loss: 0.5253 - val_acc: 0.8596
Epoch 2029/5000
50/50 [=======] - 9s 185ms/step - loss: 0.5091 - acc: 0.8654 - val_loss: 0.5237 - val_acc: 0.8612
Epoch 2030/5000
50/50 [=======] - 9s 189ms/step - loss: 0.5076 - acc: 0.8654 - val_loss: 0.5197 - val_acc: 0.8627
Epoch 2031/5000
50/50 [=======] - 9s 189ms/step - loss: 0.5058 - acc: 0.8657 - val_loss: 0.5226 - val_acc: 0.8625
Epoch 2032/5000
50/50 [=======] - 10s 194ms/step - loss: 0.5078 - acc: 0.8669 - val_loss: 0.5225 - val_acc: 0.8639
Epoch 2033/5000
50/50 [=======] - 10s 193ms/step - loss: 0.5101 - acc: 0.8643 - val_loss: 0.5195 - val_acc: 0.8632
Epoch 2034/5000
50/50 [=======] - 10s 191ms/step - loss: 0.5088 - acc: 0.8665 - val_loss: 0.5237 - val_acc: 0.8634
Epoch 2035/5000
50/50 [=======] - 10s 191ms/step - loss: 0.5091 - acc: 0.8644 - val_loss: 0.5182 - val_acc: 0.8652
Epoch 2036/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5090 - acc: 0.8658 - val_loss: 0.5199 - val_acc: 0.8615
Epoch 2037/5000
50/50 [=======] - 9s 187ms/step - loss: 0.5056 - acc: 0.8670 - val_loss: 0.5256 - val_acc: 0.8600
Epoch 2038/5000
50/50 [=======] - 9s 187ms/step - loss: 0.5057 - acc: 0.8665 - val_loss: 0.5261 - val_acc: 0.8600
Epoch 2039/5000
50/50 [=======] - 9s 190ms/step - loss: 0.5088 - acc: 0.8647 - val_loss: 0.5242 - val_acc: 0.8625
Epoch 2040/5000
50/50 [=======] - 10s 192ms/step - loss: 0.5068 - acc: 0.8662 - val_loss: 0.5223 - val_acc: 0.8620
Epoch 2041/5000
50/50 [=======] - 10s 191ms/step - loss: 0.5068 - acc: 0.8655 - val_loss: 0.5189 - val_acc: 0.8594
Epoch 2042/5000
50/50 [=======] - 10s 191ms/step - loss: 0.5070 - acc: 0.8661 - val_loss: 0.5244 - val_acc: 0.8613
Epoch 2043/5000
50/50 [=======] - 9s 187ms/step - loss: 0.5008 - acc: 0.8683 - val_loss: 0.5238 - val_acc: 0.8603
Epoch 2044/5000
50/50 [=======] - 9s 189ms/step - loss: 0.5100 - acc: 0.8652 - val_loss: 0.5233 - val_acc: 0.8609
Epoch 2045/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5007 - acc: 0.8692 - val_loss: 0.5206 - val_acc: 0.8651
Epoch 2046/5000
50/50 [=======] - 9s 188ms/step - loss: 0.5043 - acc: 0.8662 - val_loss: 0.5246 - val_acc: 0.8623
Epoch 2047/5000
50/50 [=======] - 10s 190ms/step - loss: 0.5084 - acc: 0.8644 - val_loss: 0.5211 - val_acc: 0.8620
Epoch 2048/5000
50/50 [=======] - 10s 192ms/step - loss: 0.5068 - acc: 0.8658 - val_loss: 0.5206 - val_acc: 0.8635
Epoch 2049/5000
50/50 [=======] - 10s 194ms/step - loss: 0.5033 - acc: 0.8669 - val_loss: 0.5230 - val_acc: 0.8589
Epoch 2050/5000
50/50 [=======] - 10s 192ms/step - loss: 0.5096 - acc: 0.8653 - val_loss: 0.5217 - val_acc: 0.8610
Epoch 2051/5000
50/50 [=======] - 10s 191ms/step - loss: 0.5081 - acc: 0.8651 - val_loss: 0.5238 - val_acc: 0.8600
Epoch 2052/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5061 - acc: 0.8665 - val_loss: 0.5258 - val_acc: 0.8617
Epoch 2053/5000
50/50 [=======] - 9s 187ms/step - loss: 0.5036 - acc: 0.8683 - val_loss: 0.5233 - val_acc: 0.8606
Epoch 2054/5000
50/50 [=======] - 9s 187ms/step - loss: 0.5075 - acc: 0.8652 - val_loss: 0.5242 - val_acc: 0.8599
Epoch 2055/5000
50/50 [=======] - 9s 189ms/step - loss: 0.5105 - acc: 0.8656 - val_loss: 0.5201 - val_acc: 0.8621
Epoch 2056/5000
50/50 [=======] - 10s 191ms/step - loss: 0.5064 - acc: 0.8664 - val_loss: 0.5264 - val_acc: 0.8586
Epoch 2057/5000
50/50 [=======] - 10s 192ms/step - loss: 0.5049 - acc: 0.8655 - val_loss: 0.5225 - val_acc: 0.8621
Epoch 2058/5000
50/50 [=======] - 10s 190ms/step - loss: 0.5083 - acc: 0.8657 - val_loss: 0.5218 - val_acc: 0.8648
Epoch 2059/5000
50/50 [=======] - 9s 188ms/step - loss: 0.5118 - acc: 0.8657 - val_loss: 0.5233 - val_acc: 0.8627
Epoch 2060/5000
50/50 [=======] - 9s 187ms/step - loss: 0.5037 - acc: 0.8668 - val_loss: 0.5183 - val_acc: 0.8618
Epoch 2061/5000
50/50 [=======] - 9s 185ms/step - loss: 0.5092 - acc: 0.8661 - val_loss: 0.5207 - val_acc: 0.8642
Epoch 2062/5000
50/50 [=======] - 9s 188ms/step - loss: 0.5089 - acc: 0.8649 - val_loss: 0.5175 - val_acc: 0.8625
Epoch 2063/5000
50/50 [=======] - 10s 190ms/step - loss: 0.5056 - acc: 0.8667 - val_loss: 0.5175 - val_acc: 0.8635
Epoch 2064/5000
50/50 [=======] - 10s 192ms/step - loss: 0.5078 - acc: 0.8659 - val_loss: 0.5217 - val_acc: 0.8602
Epoch 2065/5000
50/50 [=======] - 10s 192ms/step - loss: 0.5065 - acc: 0.8685 - val_loss: 0.5195 - val_acc: 0.8627
Epoch 2066/5000
50/50 [=======] - 10s 190ms/step - loss: 0.5099 - acc: 0.8660 - val_loss: 0.5223 - val_acc: 0.8608
Epoch 2067/5000
50/50 [=======] - 10s 190ms/step - loss: 0.5048 - acc: 0.8663 - val_loss: 0.5188 - val_acc: 0.8629
Epoch 2068/5000
50/50 [=======] - 9s 187ms/step - loss: 0.5047 - acc: 0.8659 - val_loss: 0.5176 - val_acc: 0.8611
Epoch 2069/5000
50/50 [=======] - 9s 185ms/step - loss: 0.5055 - acc: 0.8680 - val_loss: 0.5182 - val_acc: 0.8636
Epoch 2070/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5076 - acc: 0.8659 - val_loss: 0.5235 - val_acc: 0.8619
Epoch 2071/5000
50/50 [=======] - 9s 189ms/step - loss: 0.5100 - acc: 0.8664 - val_loss: 0.5185 - val_acc: 0.8630
Epoch 2072/5000
50/50 [=======] - 10s 192ms/step - loss: 0.5052 - acc: 0.8663 - val_loss: 0.5214 - val_acc: 0.8611
Epoch 2073/5000
50/50 [=======] - 10s 192ms/step - loss: 0.5052 - acc: 0.8662 - val_loss: 0.5250 - val_acc: 0.8630
Epoch 2074/5000
50/50 [=======] - 10s 193ms/step - loss: 0.5054 - acc: 0.8668 - val_loss: 0.5162 - val_acc: 0.8650
Epoch 2075/5000
50/50 [=======] - 9s 189ms/step - loss: 0.5081 - acc: 0.8647 - val_loss: 0.5238 - val_acc: 0.8598
Epoch 2076/5000
50/50 [=======] - 9s 189ms/step - loss: 0.5075 - acc: 0.8647 - val_loss: 0.5237 - val_acc: 0.8609
Epoch 2077/5000
50/50 [=======] - 9s 185ms/step - loss: 0.5127 - acc: 0.8642 - val_loss: 0.5209 - val_acc: 0.8662
Epoch 2078/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5063 - acc: 0.8655 - val_loss: 0.5192 - val_acc: 0.8639
Epoch 2079/5000
50/50 [=======] - 9s 189ms/step - loss: 0.5113 - acc: 0.8650 - val_loss: 0.5193 - val_acc: 0.8613
Epoch 2080/5000
50/50 [=======] - 10s 191ms/step - loss: 0.5087 - acc: 0.8660 - val_loss: 0.5199 - val_acc: 0.8623
Epoch 2081/5000
50/50 [=======] - 10s 191ms/step - loss: 0.5097 - acc: 0.8648 - val_loss: 0.5187 - val_acc: 0.8611
Epoch 2082/5000
50/50 [=======] - 10s 191ms/step - loss: 0.5059 - acc: 0.8674 - val_loss: 0.5229 - val_acc: 0.8608
Epoch 2083/5000
50/50 [=======] - 10s 191ms/step - loss: 0.5100 - acc: 0.8641 - val_loss: 0.5251 - val_acc: 0.8599
Epoch 2084/5000
50/50 [=======] - 9s 187ms/step - loss: 0.5098 - acc: 0.8645 - val_loss: 0.5195 - val_acc: 0.8631
Epoch 2085/5000
50/50 [=======] - 9s 187ms/step - loss: 0.5023 - acc: 0.8680 - val_loss: 0.5185 - val_acc: 0.8638
Epoch 2086/5000
50/50 [=======] - 9s 187ms/step - loss: 0.5077 - acc: 0.8660 - val_loss: 0.5249 - val_acc: 0.8628
Epoch 2087/5000
50/50 [=======] - 9s 189ms/step - loss: 0.5076 - acc: 0.8645 - val_loss: 0.5219 - val_acc: 0.8599
Epoch 2088/5000
50/50 [=======] - 9s 189ms/step - loss: 0.5074 - acc: 0.8665 - val_loss: 0.5228 - val_acc: 0.8633
Epoch 2089/5000
50/50 [=======] - 10s 191ms/step - loss: 0.5064 - acc: 0.8658 - val_loss: 0.5219 - val_acc: 0.8626
Epoch 2090/5000
50/50 [=======] - 10s 191ms/step - loss: 0.5064 - acc: 0.8673 - val_loss: 0.5207 - val_acc: 0.8626
Epoch 2091/5000
50/50 [=======] - 9s 188ms/step - loss: 0.5064 - acc: 0.8673 - val_loss: 0.5229 - val_acc: 0.8616
Epoch 2092/5000
50/50 [=======] - 9s 190ms/step - loss: 0.5055 - acc: 0.8670 - val_loss: 0.5236 - val_acc: 0.8613
Epoch 2093/5000
50/50 [=======] - 9s 187ms/step - loss: 0.5075 - acc: 0.8657 - val_loss: 0.5197 - val_acc: 0.8627
Epoch 2094/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5082 - acc: 0.8664 - val_loss: 0.5228 - val_acc: 0.8602
Epoch 2095/5000
50/50 [=======] - 9s 189ms/step - loss: 0.5072 - acc: 0.8672 - val_loss: 0.5258 - val_acc: 0.8618
Epoch 2096/5000
50/50 [=======] - 10s 192ms/step - loss: 0.5099 - acc: 0.8641 - val_loss: 0.5190 - val_acc: 0.8631
Epoch 2097/5000
50/50 [=======] - 10s 192ms/step - loss: 0.5069 - acc: 0.8662 - val_loss: 0.5212 - val_acc: 0.8603
Epoch 2098/5000
50/50 [=======] - 10s 191ms/step - loss: 0.5080 - acc: 0.8656 - val_loss: 0.5203 - val_acc: 0.8611
Epoch 2099/5000
50/50 [=======] - 10s 192ms/step - loss: 0.5051 - acc: 0.8663 - val_loss: 0.5177 - val_acc: 0.8604
Epoch 2100/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5059 - acc: 0.8643 - val_loss: 0.5206 - val_acc: 0.8633
Epoch 2101/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5081 - acc: 0.8651 - val_loss: 0.5215 - val_acc: 0.8622
Epoch 2102/5000
50/50 [=======] - 9s 188ms/step - loss: 0.5060 - acc: 0.8655 - val_loss: 0.5183 - val_acc: 0.8645
Epoch 2103/5000
50/50 [=======] - 10s 190ms/step - loss: 0.5062 - acc: 0.8650 - val_loss: 0.5229 - val_acc: 0.8610
Epoch 2104/5000
50/50 [=======] - 10s 192ms/step - loss: 0.5060 - acc: 0.8664 - val_loss: 0.5250 - val_acc: 0.8595
Epoch 2105/5000
50/50 [=======] - 10s 194ms/step - loss: 0.5072 - acc: 0.8653 - val_loss: 0.5233 - val_acc: 0.8634
Epoch 2106/5000
50/50 [=======] - 10s 193ms/step - loss: 0.5073 - acc: 0.8645 - val_loss: 0.5187 - val_acc: 0.8626
Epoch 2107/5000
50/50 [=======] - 9s 187ms/step - loss: 0.5070 - acc: 0.8644 - val_loss: 0.5202 - val_acc: 0.8636
Epoch 2108/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5075 - acc: 0.8648 - val_loss: 0.5222 - val_acc: 0.8609
Epoch 2109/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5049 - acc: 0.8649 - val_loss: 0.5200 - val_acc: 0.8623
Epoch 2110/5000
50/50 [=======] - 9s 188ms/step - loss: 0.5025 - acc: 0.8691 - val_loss: 0.5246 - val_acc: 0.8613
Epoch 2111/5000
50/50 [=======] - 10s 190ms/step - loss: 0.5086 - acc: 0.8646 - val_loss: 0.5212 - val_acc: 0.8598
Epoch 2112/5000
50/50 [=======] - 10s 191ms/step - loss: 0.5071 - acc: 0.8658 - val_loss: 0.5213 - val_acc: 0.8636
Epoch 2113/5000
50/50 [=======] - 10s 193ms/step - loss: 0.5063 - acc: 0.8657 - val_loss: 0.5229 - val_acc: 0.8599
Epoch 2114/5000
50/50 [=======] - 9s 190ms/step - loss: 0.5070 - acc: 0.8652 - val_loss: 0.5194 - val_acc: 0.8629
Epoch 2115/5000
50/50 [=======] - 10s 191ms/step - loss: 0.5089 - acc: 0.8648 - val_loss: 0.5208 - val_acc: 0.8608
Epoch 2116/5000
50/50 [=======] - 9s 188ms/step - loss: 0.5021 - acc: 0.8666 - val_loss: 0.5199 - val_acc: 0.8595
Epoch 2117/5000
50/50 [=======] - 9s 188ms/step - loss: 0.5027 - acc: 0.8682 - val_loss: 0.5228 - val_acc: 0.8613
Epoch 2118/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5099 - acc: 0.8648 - val_loss: 0.5247 - val_acc: 0.8596
Epoch 2119/5000
50/50 [=======] - 10s 190ms/step - loss: 0.5082 - acc: 0.8667 - val_loss: 0.5223 - val_acc: 0.8598
Epoch 2120/5000
50/50 [=======] - 10s 191ms/step - loss: 0.5107 - acc: 0.8639 - val_loss: 0.5200 - val_acc: 0.8608
Epoch 2121/5000
50/50 [=======] - 10s 191ms/step - loss: 0.5075 - acc: 0.8651 - val_loss: 0.5216 - val_acc: 0.8601
Epoch 2122/5000
50/50 [=======] - 10s 191ms/step - loss: 0.5030 - acc: 0.8646 - val_loss: 0.5156 - val_acc: 0.8647
Epoch 2123/5000
50/50 [=======] - 9s 187ms/step - loss: 0.5031 - acc: 0.8668 - val_loss: 0.5280 - val_acc: 0.8588
Epoch 2124/5000
50/50 [=======] - 9s 187ms/step - loss: 0.5066 - acc: 0.8663 - val_loss: 0.5184 - val_acc: 0.8622
Epoch 2125/5000
50/50 [=======] - 9s 187ms/step - loss: 0.5025 - acc: 0.8660 - val_loss: 0.5189 - val_acc: 0.8624
Epoch 2126/5000
50/50 [=======] - 9s 189ms/step - loss: 0.5038 - acc: 0.8676 - val_loss: 0.5244 - val_acc: 0.8605
Epoch 2127/5000
50/50 [=======] - 10s 191ms/step - loss: 0.5106 - acc: 0.8634 - val_loss: 0.5201 - val_acc: 0.8634
Epoch 2128/5000
50/50 [=======] - 10s 193ms/step - loss: 0.5068 - acc: 0.8653 - val_loss: 0.5264 - val_acc: 0.8617
Epoch 2129/5000
50/50 [=======] - 10s 193ms/step - loss: 0.5054 - acc: 0.8635 - val_loss: 0.5238 - val_acc: 0.8604
Epoch 2130/5000
50/50 [=======] - 9s 189ms/step - loss: 0.5082 - acc: 0.8652 - val_loss: 0.5238 - val_acc: 0.8599
Epoch 2131/5000
50/50 [=======] - 10s 191ms/step - loss: 0.5055 - acc: 0.8659 - val_loss: 0.5204 - val_acc: 0.8627
Epoch 2132/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5059 - acc: 0.8656 - val_loss: 0.5225 - val_acc: 0.8619
Epoch 2133/5000
50/50 [=======] - 9s 189ms/step - loss: 0.5044 - acc: 0.8671 - val_loss: 0.5210 - val_acc: 0.8617
Epoch 2134/5000
50/50 [=======] - 9s 187ms/step - loss: 0.5066 - acc: 0.8676 - val_loss: 0.5216 - val_acc: 0.8609
Epoch 2135/5000
50/50 [=======] - 10s 191ms/step - loss: 0.5031 - acc: 0.8667 - val_loss: 0.5206 - val_acc: 0.8612
Epoch 2136/5000
50/50 [=======] - 10s 190ms/step - loss: 0.5058 - acc: 0.8651 - val_loss: 0.5200 - val_acc: 0.8621
Epoch 2137/5000
50/50 [=======] - 10s 192ms/step - loss: 0.5065 - acc: 0.8653 - val_loss: 0.5189 - val_acc: 0.8634
Epoch 2138/5000
50/50 [=======] - 10s 191ms/step - loss: 0.5080 - acc: 0.8654 - val_loss: 0.5173 - val_acc: 0.8638
Epoch 2139/5000
50/50 [=======] - 9s 188ms/step - loss: 0.5008 - acc: 0.8676 - val_loss: 0.5229 - val_acc: 0.8609
Epoch 2140/5000
50/50 [=======] - 9s 188ms/step - loss: 0.5061 - acc: 0.8642 - val_loss: 0.5203 - val_acc: 0.8622
Epoch 2141/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5077 - acc: 0.8655 - val_loss: 0.5212 - val_acc: 0.8614
Epoch 2142/5000
50/50 [=======] - 9s 187ms/step - loss: 0.5080 - acc: 0.8662 - val_loss: 0.5197 - val_acc: 0.8609
Epoch 2143/5000
50/50 [=======] - 9s 189ms/step - loss: 0.5042 - acc: 0.8669 - val_loss: 0.5261 - val_acc: 0.8592
Epoch 2144/5000
50/50 [=======] - 10s 192ms/step - loss: 0.5115 - acc: 0.8645 - val_loss: 0.5163 - val_acc: 0.8638
Epoch 2145/5000
50/50 [=======] - 10s 193ms/step - loss: 0.5032 - acc: 0.8663 - val_loss: 0.5202 - val_acc: 0.8608
Epoch 2146/5000
50/50 [=======] - 10s 192ms/step - loss: 0.5061 - acc: 0.8647 - val_loss: 0.5189 - val_acc: 0.8614
Epoch 2147/5000
50/50 [=======] - 10s 191ms/step - loss: 0.5073 - acc: 0.8665 - val_loss: 0.5224 - val_acc: 0.8612
Epoch 2148/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5074 - acc: 0.8647 - val_loss: 0.5217 - val_acc: 0.8605
Epoch 2149/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5077 - acc: 0.8646 - val_loss: 0.5221 - val_acc: 0.8611
Epoch 2150/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5079 - acc: 0.8641 - val_loss: 0.5205 - val_acc: 0.8609
Epoch 2151/5000
50/50 [=======] - 10s 191ms/step - loss: 0.5036 - acc: 0.8665 - val_loss: 0.5173 - val_acc: 0.8648
Epoch 2152/5000
50/50 [=======] - 10s 193ms/step - loss: 0.5104 - acc: 0.8642 - val_loss: 0.5295 - val_acc: 0.8571
Epoch 2153/5000
50/50 [=======] - 10s 192ms/step - loss: 0.5030 - acc: 0.8656 - val_loss: 0.5240 - val_acc: 0.8589
Epoch 2154/5000
50/50 [=======] - 10s 192ms/step - loss: 0.5046 - acc: 0.8664 - val_loss: 0.5217 - val_acc: 0.8620
Epoch 2155/5000
50/50 [=======] - 9s 188ms/step - loss: 0.5104 - acc: 0.8644 - val_loss: 0.5199 - val_acc: 0.8598
Epoch 2156/5000
50/50 [=======] - 9s 187ms/step - loss: 0.5042 - acc: 0.8657 - val_loss: 0.5159 - val_acc: 0.8643
Epoch 2157/5000
50/50 [=======] - 9s 185ms/step - loss: 0.5019 - acc: 0.8681 - val_loss: 0.5242 - val_acc: 0.8617
Epoch 2158/5000
50/50 [=======] - 9s 188ms/step - loss: 0.5101 - acc: 0.8635 - val_loss: 0.5187 - val_acc: 0.8628
Epoch 2159/5000
50/50 [=======] - 9s 190ms/step - loss: 0.5064 - acc: 0.8645 - val_loss: 0.5147 - val_acc: 0.8623
Epoch 2160/5000
50/50 [=======] - 10s 191ms/step - loss: 0.5034 - acc: 0.8663 - val_loss: 0.5175 - val_acc: 0.8623
Epoch 2161/5000
50/50 [=======] - 10s 191ms/step - loss: 0.5051 - acc: 0.8670 - val_loss: 0.5236 - val_acc: 0.8593
Epoch 2162/5000
50/50 [=======] - 9s 189ms/step - loss: 0.5020 - acc: 0.8666 - val_loss: 0.5165 - val_acc: 0.8641
Epoch 2163/5000
50/50 [=======] - 9s 189ms/step - loss: 0.5059 - acc: 0.8657 - val_loss: 0.5250 - val_acc: 0.8592
Epoch 2164/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5047 - acc: 0.8649 - val_loss: 0.5237 - val_acc: 0.8613
Epoch 2165/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5071 - acc: 0.8654 - val_loss: 0.5203 - val_acc: 0.8624
Epoch 2166/5000
50/50 [=======] - 9s 185ms/step - loss: 0.5001 - acc: 0.8673 - val_loss: 0.5168 - val_acc: 0.8634
Epoch 2167/5000
50/50 [=======] - 9s 188ms/step - loss: 0.5043 - acc: 0.8654 - val_loss: 0.5185 - val_acc: 0.8608
Epoch 2168/5000
50/50 [=======] - 10s 193ms/step - loss: 0.5071 - acc: 0.8649 - val_loss: 0.5225 - val_acc: 0.8629
Epoch 2169/5000
50/50 [=======] - 10s 192ms/step - loss: 0.5060 - acc: 0.8655 - val_loss: 0.5231 - val_acc: 0.8597
Epoch 2170/5000
50/50 [=======] - 10s 191ms/step - loss: 0.5072 - acc: 0.8665 - val_loss: 0.5236 - val_acc: 0.8621
Epoch 2171/5000
50/50 [=======] - 9s 187ms/step - loss: 0.5050 - acc: 0.8676 - val_loss: 0.5193 - val_acc: 0.8618
Epoch 2172/5000
50/50 [=======] - 9s 187ms/step - loss: 0.5016 - acc: 0.8674 - val_loss: 0.5173 - val_acc: 0.8625
Epoch 2173/5000
50/50 [=======] - 9s 184ms/step - loss: 0.5027 - acc: 0.8657 - val_loss: 0.5197 - val_acc: 0.8621
Epoch 2174/5000
50/50 [=======] - 9s 185ms/step - loss: 0.5039 - acc: 0.8668 - val_loss: 0.5183 - val_acc: 0.8637
Epoch 2175/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5056 - acc: 0.8670 - val_loss: 0.5257 - val_acc: 0.8614
Epoch 2176/5000
50/50 [=======] - 9s 189ms/step - loss: 0.5045 - acc: 0.8651 - val_loss: 0.5171 - val_acc: 0.8641
Epoch 2177/5000
50/50 [=======] - 9s 190ms/step - loss: 0.5045 - acc: 0.8661 - val_loss: 0.5215 - val_acc: 0.8637
Epoch 2178/5000
50/50 [=======] - 9s 190ms/step - loss: 0.5073 - acc: 0.8629 - val_loss: 0.5189 - val_acc: 0.8600
Epoch 2179/5000
50/50 [=======] - 9s 190ms/step - loss: 0.5054 - acc: 0.8658 - val_loss: 0.5240 - val_acc: 0.8620
Epoch 2180/5000
50/50 [=======] - 9s 185ms/step - loss: 0.5089 - acc: 0.8648 - val_loss: 0.5281 - val_acc: 0.8587
Epoch 2181/5000
50/50 [=======] - 9s 185ms/step - loss: 0.5047 - acc: 0.8660 - val_loss: 0.5253 - val_acc: 0.8614
Epoch 2182/5000
50/50 [=======] - 9s 185ms/step - loss: 0.5057 - acc: 0.8654 - val_loss: 0.5213 - val_acc: 0.8594
Epoch 2183/5000
50/50 [=======] - 9s 188ms/step - loss: 0.5088 - acc: 0.8657 - val_loss: 0.5214 - val_acc: 0.8608
Epoch 2184/5000
50/50 [=======] - 9s 189ms/step - loss: 0.5038 - acc: 0.8663 - val_loss: 0.5253 - val_acc: 0.8584
Epoch 2185/5000
50/50 [=======] - 9s 190ms/step - loss: 0.5070 - acc: 0.8658 - val_loss: 0.5223 - val_acc: 0.8584
Epoch 2186/5000
50/50 [=======] - 10s 190ms/step - loss: 0.5065 - acc: 0.8650 - val_loss: 0.5250 - val_acc: 0.8602
Epoch 2187/5000
50/50 [=======] - 9s 185ms/step - loss: 0.5032 - acc: 0.8669 - val_loss: 0.5148 - val_acc: 0.8636
Epoch 2188/5000
50/50 [=======] - 9s 184ms/step - loss: 0.5025 - acc: 0.8668 - val_loss: 0.5234 - val_acc: 0.8595
Epoch 2189/5000
50/50 [=======] - 9s 184ms/step - loss: 0.5077 - acc: 0.8638 - val_loss: 0.5249 - val_acc: 0.8600
Epoch 2190/5000
50/50 [=======] - 9s 185ms/step - loss: 0.5090 - acc: 0.8635 - val_loss: 0.5256 - val_acc: 0.8606
Epoch 2191/5000
50/50 [=======] - 9s 188ms/step - loss: 0.5072 - acc: 0.8648 - val_loss: 0.5211 - val_acc: 0.8605
Epoch 2192/5000
50/50 [=======] - 9s 187ms/step - loss: 0.5054 - acc: 0.8649 - val_loss: 0.5203 - val_acc: 0.8608
Epoch 2193/5000
50/50 [=======] - 9s 189ms/step - loss: 0.5024 - acc: 0.8666 - val_loss: 0.5207 - val_acc: 0.8618
Epoch 2194/5000
50/50 [=======] - 9s 190ms/step - loss: 0.5049 - acc: 0.8652 - val_loss: 0.5231 - val_acc: 0.8608
Epoch 2195/5000
50/50 [=======] - 9s 188ms/step - loss: 0.5061 - acc: 0.8658 - val_loss: 0.5225 - val_acc: 0.8606
Epoch 2196/5000
50/50 [=======] - 9s 185ms/step - loss: 0.5104 - acc: 0.8637 - val_loss: 0.5259 - val_acc: 0.8589
Epoch 2197/5000
50/50 [=======] - 9s 184ms/step - loss: 0.5042 - acc: 0.8657 - val_loss: 0.5193 - val_acc: 0.8633
Epoch 2198/5000
50/50 [=======] - 9s 185ms/step - loss: 0.5036 - acc: 0.8664 - val_loss: 0.5178 - val_acc: 0.8663
Epoch 2199/5000
50/50 [=======] - 9s 187ms/step - loss: 0.5066 - acc: 0.8675 - val_loss: 0.5172 - val_acc: 0.8633
Epoch 2200/5000
50/50 [=======] - 9s 188ms/step - loss: 0.5095 - acc: 0.8641 - val_loss: 0.5165 - val_acc: 0.8624
Epoch 2201/5000
50/50 [=======] - 10s 190ms/step - loss: 0.5089 - acc: 0.8634 - val_loss: 0.5183 - val_acc: 0.8635
Epoch 2202/5000
50/50 [=======] - 9s 189ms/step - loss: 0.5069 - acc: 0.8649 - val_loss: 0.5179 - val_acc: 0.8624
Epoch 2203/5000
50/50 [=======] - 9s 184ms/step - loss: 0.5116 - acc: 0.8632 - val_loss: 0.5259 - val_acc: 0.8591
Epoch 2204/5000
50/50 [=======] - 9s 187ms/step - loss: 0.5066 - acc: 0.8659 - val_loss: 0.5187 - val_acc: 0.8611
Epoch 2205/5000
50/50 [=======] - 9s 181ms/step - loss: 0.5052 - acc: 0.8655 - val_loss: 0.5187 - val_acc: 0.8618
Epoch 2206/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5067 - acc: 0.8650 - val_loss: 0.5182 - val_acc: 0.8619
Epoch 2207/5000
50/50 [=======] - 9s 185ms/step - loss: 0.5091 - acc: 0.8644 - val_loss: 0.5177 - val_acc: 0.8624
Epoch 2208/5000
50/50 [=======] - 10s 192ms/step - loss: 0.5038 - acc: 0.8661 - val_loss: 0.5194 - val_acc: 0.8607
Epoch 2209/5000
50/50 [=======] - 10s 190ms/step - loss: 0.5070 - acc: 0.8640 - val_loss: 0.5176 - val_acc: 0.8623
Epoch 2210/5000
50/50 [=======] - 10s 190ms/step - loss: 0.5076 - acc: 0.8637 - val_loss: 0.5210 - val_acc: 0.8609
Epoch 2211/5000
50/50 [=======] - 9s 189ms/step - loss: 0.5018 - acc: 0.8666 - val_loss: 0.5191 - val_acc: 0.8606
Epoch 2212/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5055 - acc: 0.8661 - val_loss: 0.5209 - val_acc: 0.8609
Epoch 2213/5000
50/50 [=======] - 9s 185ms/step - loss: 0.5056 - acc: 0.8640 - val_loss: 0.5210 - val_acc: 0.8596
Epoch 2214/5000
50/50 [=======] - 9s 184ms/step - loss: 0.5034 - acc: 0.8672 - val_loss: 0.5182 - val_acc: 0.8618
Epoch 2215/5000
50/50 [=======] - 9s 187ms/step - loss: 0.5066 - acc: 0.8649 - val_loss: 0.5201 - val_acc: 0.8603
Epoch 2216/5000
50/50 [=======] - 9s 189ms/step - loss: 0.5060 - acc: 0.8652 - val_loss: 0.5190 - val_acc: 0.8597
Epoch 2217/5000
50/50 [=======] - 10s 190ms/step - loss: 0.5062 - acc: 0.8649 - val_loss: 0.5234 - val_acc: 0.8583
Epoch 2218/5000
50/50 [=======] - 10s 192ms/step - loss: 0.5044 - acc: 0.8663 - val_loss: 0.5167 - val_acc: 0.8638
Epoch 2219/5000
50/50 [=======] - 9s 185ms/step - loss: 0.5063 - acc: 0.8647 - val_loss: 0.5253 - val_acc: 0.8575
Epoch 2220/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5096 - acc: 0.8649 - val_loss: 0.5245 - val_acc: 0.8585
Epoch 2221/5000
50/50 [=======] - 9s 184ms/step - loss: 0.5083 - acc: 0.8634 - val_loss: 0.5212 - val_acc: 0.8588
Epoch 2222/5000
50/50 [=======] - 9s 187ms/step - loss: 0.5060 - acc: 0.8656 - val_loss: 0.5202 - val_acc: 0.8603
Epoch 2223/5000
50/50 [=======] - 9s 188ms/step - loss: 0.5032 - acc: 0.8669 - val_loss: 0.5179 - val_acc: 0.8603
Epoch 2224/5000
50/50 [=======] - 9s 190ms/step - loss: 0.5051 - acc: 0.8658 - val_loss: 0.5224 - val_acc: 0.8608
Epoch 2225/5000
50/50 [=======] - 10s 191ms/step - loss: 0.5022 - acc: 0.8663 - val_loss: 0.5204 - val_acc: 0.8615
Epoch 2226/5000
50/50 [=======] - 9s 189ms/step - loss: 0.5044 - acc: 0.8668 - val_loss: 0.5194 - val_acc: 0.8603
Epoch 2227/5000
50/50 [=======] - 9s 188ms/step - loss: 0.5066 - acc: 0.8658 - val_loss: 0.5194 - val_acc: 0.8612
Epoch 2228/5000
50/50 [=======] - 9s 183ms/step - loss: 0.5091 - acc: 0.8642 - val_loss: 0.5199 - val_acc: 0.8590
Epoch 2229/5000
50/50 [=======] - 9s 185ms/step - loss: 0.5023 - acc: 0.8648 - val_loss: 0.5202 - val_acc: 0.8637
Epoch 2230/5000
50/50 [=======] - 9s 185ms/step - loss: 0.5041 - acc: 0.8646 - val_loss: 0.5196 - val_acc: 0.8622
Epoch 2231/5000
50/50 [=======] - 9s 189ms/step - loss: 0.5014 - acc: 0.8668 - val_loss: 0.5193 - val_acc: 0.8612
Epoch 2232/5000
50/50 [=======] - 9s 188ms/step - loss: 0.5031 - acc: 0.8663 - val_loss: 0.5227 - val_acc: 0.8614
Epoch 2233/5000
50/50 [=======] - 10s 192ms/step - loss: 0.5037 - acc: 0.8661 - val_loss: 0.5181 - val_acc: 0.8610
Epoch 2234/5000
50/50 [=======] - 10s 192ms/step - loss: 0.5067 - acc: 0.8666 - val_loss: 0.5257 - val_acc: 0.8587
Epoch 2235/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5068 - acc: 0.8649 - val_loss: 0.5163 - val_acc: 0.8618
Epoch 2236/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5071 - acc: 0.8639 - val_loss: 0.5234 - val_acc: 0.8599
Epoch 2237/5000
50/50 [=======] - 9s 185ms/step - loss: 0.5042 - acc: 0.8648 - val_loss: 0.5147 - val_acc: 0.8623
Epoch 2238/5000
50/50 [=======] - 9s 185ms/step - loss: 0.5053 - acc: 0.8652 - val_loss: 0.5217 - val_acc: 0.8588
Epoch 2239/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5070 - acc: 0.8636 - val_loss: 0.5205 - val_acc: 0.8624
Epoch 2240/5000
50/50 [=======] - 10s 192ms/step - loss: 0.5024 - acc: 0.8658 - val_loss: 0.5219 - val_acc: 0.8639
Epoch 2241/5000
50/50 [=======] - 10s 191ms/step - loss: 0.5069 - acc: 0.8651 - val_loss: 0.5211 - val_acc: 0.8619
Epoch 2242/5000
50/50 [=======] - 9s 188ms/step - loss: 0.5040 - acc: 0.8658 - val_loss: 0.5152 - val_acc: 0.8622
Epoch 2243/5000
50/50 [=======] - 9s 188ms/step - loss: 0.5036 - acc: 0.8651 - val_loss: 0.5222 - val_acc: 0.8599
Epoch 2244/5000
50/50 [=======] - 9s 185ms/step - loss: 0.5068 - acc: 0.8635 - val_loss: 0.5177 - val_acc: 0.8611
Epoch 2245/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5059 - acc: 0.8650 - val_loss: 0.5247 - val_acc: 0.8600
Epoch 2246/5000
50/50 [=======] - 9s 184ms/step - loss: 0.5041 - acc: 0.8661 - val_loss: 0.5194 - val_acc: 0.8609
Epoch 2247/5000
50/50 [=======] - 9s 188ms/step - loss: 0.5058 - acc: 0.8652 - val_loss: 0.5250 - val_acc: 0.8607
Epoch 2248/5000
50/50 [=======] - 10s 191ms/step - loss: 0.5072 - acc: 0.8649 - val_loss: 0.5225 - val_acc: 0.8623
Epoch 2249/5000
50/50 [=======] - 10s 191ms/step - loss: 0.5071 - acc: 0.8650 - val_loss: 0.5214 - val_acc: 0.8609
Epoch 2250/5000
50/50 [=======] - 10s 192ms/step - loss: 0.5040 - acc: 0.8660 - val_loss: 0.5235 - val_acc: 0.8594
Epoch 2251/5000
50/50 [=======] - 9s 188ms/step - loss: 0.5052 - acc: 0.8667 - val_loss: 0.5213 - val_acc: 0.8615
Epoch 2252/5000
50/50 [=======] - 9s 187ms/step - loss: 0.5055 - acc: 0.8664 - val_loss: 0.5178 - val_acc: 0.8604
Epoch 2253/5000
50/50 [=======] - 9s 184ms/step - loss: 0.5074 - acc: 0.8666 - val_loss: 0.5228 - val_acc: 0.8605
Epoch 2254/5000
50/50 [=======] - 9s 187ms/step - loss: 0.5065 - acc: 0.8657 - val_loss: 0.5181 - val_acc: 0.8640
Epoch 2255/5000
50/50 [=======] - 9s 187ms/step - loss: 0.5033 - acc: 0.8668 - val_loss: 0.5194 - val_acc: 0.8625
Epoch 2256/5000
50/50 [=======] - 9s 190ms/step - loss: 0.5065 - acc: 0.8644 - val_loss: 0.5209 - val_acc: 0.8599
Epoch 2257/5000
50/50 [=======] - 9s 190ms/step - loss: 0.5064 - acc: 0.8645 - val_loss: 0.5248 - val_acc: 0.8588
Epoch 2258/5000
50/50 [=======] - 9s 188ms/step - loss: 0.5070 - acc: 0.8645 - val_loss: 0.5210 - val_acc: 0.8633
Epoch 2259/5000
50/50 [=======] - 9s 189ms/step - loss: 0.5039 - acc: 0.8642 - val_loss: 0.5231 - val_acc: 0.8633
Epoch 2260/5000
50/50 [=======] - 9s 185ms/step - loss: 0.5078 - acc: 0.8641 - val_loss: 0.5208 - val_acc: 0.8609
Epoch 2261/5000
50/50 [=======] - 9s 185ms/step - loss: 0.5001 - acc: 0.8675 - val_loss: 0.5218 - val_acc: 0.8633
Epoch 2262/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5063 - acc: 0.8653 - val_loss: 0.5183 - val_acc: 0.8602
Epoch 2263/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5039 - acc: 0.8665 - val_loss: 0.5264 - val_acc: 0.8582
Epoch 2264/5000
50/50 [=======] - 10s 190ms/step - loss: 0.5058 - acc: 0.8660 - val_loss: 0.5223 - val_acc: 0.8605
Epoch 2265/5000
50/50 [=======] - 9s 190ms/step - loss: 0.5027 - acc: 0.8659 - val_loss: 0.5194 - val_acc: 0.8629
Epoch 2266/5000
50/50 [=======] - 9s 189ms/step - loss: 0.5042 - acc: 0.8663 - val_loss: 0.5171 - val_acc: 0.8610
Epoch 2267/5000
50/50 [=======] - 9s 185ms/step - loss: 0.5032 - acc: 0.8662 - val_loss: 0.5224 - val_acc: 0.8594
Epoch 2268/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5089 - acc: 0.8636 - val_loss: 0.5156 - val_acc: 0.8617
Epoch 2269/5000
50/50 [=======] - 9s 183ms/step - loss: 0.5065 - acc: 0.8629 - val_loss: 0.5170 - val_acc: 0.8615
Epoch 2270/5000
50/50 [=======] - 9s 185ms/step - loss: 0.5052 - acc: 0.8653 - val_loss: 0.5179 - val_acc: 0.8612
Epoch 2271/5000
50/50 [=======] - 9s 187ms/step - loss: 0.5058 - acc: 0.8646 - val_loss: 0.5214 - val_acc: 0.8610
Epoch 2272/5000
50/50 [=======] - 9s 188ms/step - loss: 0.5054 - acc: 0.8655 - val_loss: 0.5210 - val_acc: 0.8616
Epoch 2273/5000
50/50 [=======] - 10s 196ms/step - loss: 0.5026 - acc: 0.8650 - val_loss: 0.5182 - val_acc: 0.8612
Epoch 2274/5000
50/50 [=======] - 10s 194ms/step - loss: 0.5045 - acc: 0.8653 - val_loss: 0.5181 - val_acc: 0.8621
Epoch 2275/5000
50/50 [=======] - 9s 187ms/step - loss: 0.5019 - acc: 0.8674 - val_loss: 0.5220 - val_acc: 0.8617
Epoch 2276/5000
50/50 [=======] - 9s 185ms/step - loss: 0.5060 - acc: 0.8633 - val_loss: 0.5245 - val_acc: 0.8592
Epoch 2277/5000
50/50 [=======] - 9s 182ms/step - loss: 0.5051 - acc: 0.8664 - val_loss: 0.5211 - val_acc: 0.8593
Epoch 2278/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5048 - acc: 0.8640 - val_loss: 0.5253 - val_acc: 0.8605
Epoch 2279/5000
50/50 [=======] - 9s 189ms/step - loss: 0.5082 - acc: 0.8631 - val_loss: 0.5198 - val_acc: 0.8642
Epoch 2280/5000
50/50 [=======] - 10s 192ms/step - loss: 0.5044 - acc: 0.8660 - val_loss: 0.5190 - val_acc: 0.8618
Epoch 2281/5000
50/50 [=======] - 10s 192ms/step - loss: 0.5053 - acc: 0.8660 - val_loss: 0.5215 - val_acc: 0.8586
Epoch 2282/5000
50/50 [=======] - 10s 191ms/step - loss: 0.5032 - acc: 0.8647 - val_loss: 0.5280 - val_acc: 0.8604
Epoch 2283/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5086 - acc: 0.8635 - val_loss: 0.5234 - val_acc: 0.8564
Epoch 2284/5000
50/50 [=======] - 9s 187ms/step - loss: 0.5015 - acc: 0.8669 - val_loss: 0.5199 - val_acc: 0.8601
Epoch 2285/5000
50/50 [=======] - 9s 183ms/step - loss: 0.5049 - acc: 0.8642 - val_loss: 0.5206 - val_acc: 0.8607
Epoch 2286/5000
50/50 [=======] - 9s 187ms/step - loss: 0.5046 - acc: 0.8644 - val_loss: 0.5229 - val_acc: 0.8590
Epoch 2287/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5042 - acc: 0.8663 - val_loss: 0.5194 - val_acc: 0.8618
Epoch 2288/5000
50/50 [=======] - 9s 188ms/step - loss: 0.5001 - acc: 0.8680 - val_loss: 0.5266 - val_acc: 0.8598
Epoch 2289/5000
50/50 [=======] - 10s 191ms/step - loss: 0.5044 - acc: 0.8660 - val_loss: 0.5180 - val_acc: 0.8626
Epoch 2290/5000
50/50 [=======] - 9s 189ms/step - loss: 0.5053 - acc: 0.8638 - val_loss: 0.5227 - val_acc: 0.8600
Epoch 2291/5000
50/50 [=======] - 9s 189ms/step - loss: 0.5054 - acc: 0.8645 - val_loss: 0.5171 - val_acc: 0.8617
Epoch 2292/5000
50/50 [=======] - 9s 182ms/step - loss: 0.5035 - acc: 0.8667 - val_loss: 0.5225 - val_acc: 0.8587
Epoch 2293/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5067 - acc: 0.8657 - val_loss: 0.5227 - val_acc: 0.8621
Epoch 2294/5000
50/50 [=======] - 9s 185ms/step - loss: 0.4980 - acc: 0.8681 - val_loss: 0.5199 - val_acc: 0.8593
Epoch 2295/5000
50/50 [=======] - 9s 188ms/step - loss: 0.5036 - acc: 0.8653 - val_loss: 0.5232 - val_acc: 0.8627
Epoch 2296/5000
50/50 [=======] - 10s 190ms/step - loss: 0.5046 - acc: 0.8648 - val_loss: 0.5222 - val_acc: 0.8598
Epoch 2297/5000
50/50 [=======] - 10s 191ms/step - loss: 0.5052 - acc: 0.8657 - val_loss: 0.5163 - val_acc: 0.8621
Epoch 2298/5000
50/50 [=======] - 9s 189ms/step - loss: 0.5079 - acc: 0.8636 - val_loss: 0.5138 - val_acc: 0.8651
Epoch 2299/5000
50/50 [=======] - 9s 189ms/step - loss: 0.5021 - acc: 0.8646 - val_loss: 0.5184 - val_acc: 0.8637
Epoch 2300/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5071 - acc: 0.8634 - val_loss: 0.5206 - val_acc: 0.8629
Epoch 2301/5000
50/50 [=======] - 9s 182ms/step - loss: 0.5054 - acc: 0.8657 - val_loss: 0.5240 - val_acc: 0.8620
Epoch 2302/5000
50/50 [=======] - 9s 185ms/step - loss: 0.5034 - acc: 0.8658 - val_loss: 0.5235 - val_acc: 0.8608
Epoch 2303/5000
50/50 [=======] - 9s 187ms/step - loss: 0.5092 - acc: 0.8627 - val_loss: 0.5228 - val_acc: 0.8604
Epoch 2304/5000
50/50 [=======] - 10s 191ms/step - loss: 0.5026 - acc: 0.8664 - val_loss: 0.5142 - val_acc: 0.8619
Epoch 2305/5000
50/50 [=======] - 9s 190ms/step - loss: 0.5031 - acc: 0.8655 - val_loss: 0.5167 - val_acc: 0.8627
Epoch 2306/5000
50/50 [=======] - 9s 188ms/step - loss: 0.5065 - acc: 0.8655 - val_loss: 0.5212 - val_acc: 0.8631
Epoch 2307/5000
50/50 [=======] - 9s 190ms/step - loss: 0.5036 - acc: 0.8675 - val_loss: 0.5190 - val_acc: 0.8609
Epoch 2308/5000
50/50 [=======] - 9s 184ms/step - loss: 0.5017 - acc: 0.8668 - val_loss: 0.5247 - val_acc: 0.8598
Epoch 2309/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5043 - acc: 0.8639 - val_loss: 0.5142 - val_acc: 0.8641
Epoch 2310/5000
50/50 [=======] - 9s 185ms/step - loss: 0.5070 - acc: 0.8637 - val_loss: 0.5193 - val_acc: 0.8622
Epoch 2311/5000
50/50 [=======] - 9s 188ms/step - loss: 0.5043 - acc: 0.8650 - val_loss: 0.5229 - val_acc: 0.8637
Epoch 2312/5000
50/50 [=======] - 9s 189ms/step - loss: 0.5068 - acc: 0.8645 - val_loss: 0.5190 - val_acc: 0.8607
Epoch 2313/5000
50/50 [=======] - 10s 190ms/step - loss: 0.5041 - acc: 0.8653 - val_loss: 0.5195 - val_acc: 0.8620
Epoch 2314/5000
50/50 [=======] - 10s 191ms/step - loss: 0.5037 - acc: 0.8640 - val_loss: 0.5208 - val_acc: 0.8613
Epoch 2315/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5041 - acc: 0.8652 - val_loss: 0.5194 - val_acc: 0.8625
Epoch 2316/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5075 - acc: 0.8632 - val_loss: 0.5149 - val_acc: 0.8620
Epoch 2317/5000
50/50 [=======] - 9s 185ms/step - loss: 0.5038 - acc: 0.8657 - val_loss: 0.5223 - val_acc: 0.8617
Epoch 2318/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5053 - acc: 0.8643 - val_loss: 0.5190 - val_acc: 0.8634
Epoch 2319/5000
50/50 [=======] - 9s 187ms/step - loss: 0.5028 - acc: 0.8662 - val_loss: 0.5242 - val_acc: 0.8596
Epoch 2320/5000
50/50 [=======] - 9s 189ms/step - loss: 0.5017 - acc: 0.8669 - val_loss: 0.5195 - val_acc: 0.8606
Epoch 2321/5000
50/50 [=======] - 10s 191ms/step - loss: 0.5055 - acc: 0.8646 - val_loss: 0.5220 - val_acc: 0.8609
Epoch 2322/5000
50/50 [=======] - 10s 190ms/step - loss: 0.5034 - acc: 0.8655 - val_loss: 0.5179 - val_acc: 0.8606
Epoch 2323/5000
50/50 [=======] - 9s 189ms/step - loss: 0.5033 - acc: 0.8668 - val_loss: 0.5193 - val_acc: 0.8605
Epoch 2324/5000
50/50 [=======] - 9s 183ms/step - loss: 0.5032 - acc: 0.8663 - val_loss: 0.5221 - val_acc: 0.8610
Epoch 2325/5000
50/50 [=======] - 9s 183ms/step - loss: 0.5066 - acc: 0.8649 - val_loss: 0.5202 - val_acc: 0.8610
Epoch 2326/5000
50/50 [=======] - 9s 184ms/step - loss: 0.5042 - acc: 0.8660 - val_loss: 0.5222 - val_acc: 0.8597
Epoch 2327/5000
50/50 [=======] - 9s 187ms/step - loss: 0.5091 - acc: 0.8655 - val_loss: 0.5203 - val_acc: 0.8598
Epoch 2328/5000
50/50 [=======] - 9s 188ms/step - loss: 0.5090 - acc: 0.8636 - val_loss: 0.5196 - val_acc: 0.8603
Epoch 2329/5000
50/50 [=======] - 10s 192ms/step - loss: 0.5030 - acc: 0.8658 - val_loss: 0.5255 - val_acc: 0.8602
Epoch 2330/5000
50/50 [=======] - 9s 188ms/step - loss: 0.5063 - acc: 0.8650 - val_loss: 0.5205 - val_acc: 0.8606
Epoch 2331/5000
50/50 [=======] - 9s 188ms/step - loss: 0.5038 - acc: 0.8659 - val_loss: 0.5256 - val_acc: 0.8584
Epoch 2332/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5077 - acc: 0.8642 - val_loss: 0.5186 - val_acc: 0.8618
Epoch 2333/5000
50/50 [=======] - 9s 183ms/step - loss: 0.5051 - acc: 0.8643 - val_loss: 0.5204 - val_acc: 0.8586
Epoch 2334/5000
50/50 [=======] - 9s 187ms/step - loss: 0.5008 - acc: 0.8670 - val_loss: 0.5186 - val_acc: 0.8607
Epoch 2335/5000
50/50 [=======] - 9s 185ms/step - loss: 0.5044 - acc: 0.8664 - val_loss: 0.5204 - val_acc: 0.8623
Epoch 2336/5000
50/50 [=======] - 10s 191ms/step - loss: 0.5059 - acc: 0.8646 - val_loss: 0.5191 - val_acc: 0.8616
Epoch 2337/5000
50/50 [=======] - 10s 191ms/step - loss: 0.5013 - acc: 0.8675 - val_loss: 0.5189 - val_acc: 0.8629
Epoch 2338/5000
50/50 [=======] - 9s 189ms/step - loss: 0.5069 - acc: 0.8645 - val_loss: 0.5244 - val_acc: 0.8630
Epoch 2339/5000
50/50 [=======] - 10s 190ms/step - loss: 0.5085 - acc: 0.8638 - val_loss: 0.5189 - val_acc: 0.8609
Epoch 2340/5000
50/50 [=======] - 9s 184ms/step - loss: 0.5009 - acc: 0.8653 - val_loss: 0.5210 - val_acc: 0.8609
Epoch 2341/5000
50/50 [=======] - 9s 187ms/step - loss: 0.5008 - acc: 0.8657 - val_loss: 0.5144 - val_acc: 0.8628
Epoch 2342/5000
50/50 [=======] - 9s 184ms/step - loss: 0.5017 - acc: 0.8661 - val_loss: 0.5251 - val_acc: 0.8606
Epoch 2343/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5069 - acc: 0.8646 - val_loss: 0.5204 - val_acc: 0.8608
Epoch 2344/5000
50/50 [=======] - 9s 189ms/step - loss: 0.4997 - acc: 0.8675 - val_loss: 0.5203 - val_acc: 0.8651
Epoch 2345/5000
50/50 [=======] - 9s 189ms/step - loss: 0.5015 - acc: 0.8672 - val_loss: 0.5220 - val_acc: 0.8610
Epoch 2346/5000
50/50 [=======] - 10s 192ms/step - loss: 0.5005 - acc: 0.8656 - val_loss: 0.5200 - val_acc: 0.8620
Epoch 2347/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5070 - acc: 0.8636 - val_loss: 0.5230 - val_acc: 0.8607
Epoch 2348/5000
50/50 [=======] - 9s 189ms/step - loss: 0.5018 - acc: 0.8657 - val_loss: 0.5175 - val_acc: 0.8611
Epoch 2349/5000
50/50 [=======] - 9s 182ms/step - loss: 0.5042 - acc: 0.8656 - val_loss: 0.5244 - val_acc: 0.8603
Epoch 2350/5000
50/50 [=======] - 9s 187ms/step - loss: 0.5062 - acc: 0.8656 - val_loss: 0.5182 - val_acc: 0.8614
Epoch 2351/5000
50/50 [=======] - 9s 184ms/step - loss: 0.5053 - acc: 0.8642 - val_loss: 0.5183 - val_acc: 0.8611
Epoch 2352/5000
50/50 [=======] - 10s 192ms/step - loss: 0.5094 - acc: 0.8640 - val_loss: 0.5202 - val_acc: 0.8589
Epoch 2353/5000
50/50 [=======] - 10s 191ms/step - loss: 0.5077 - acc: 0.8648 - val_loss: 0.5176 - val_acc: 0.8630
Epoch 2354/5000
50/50 [=======] - 9s 189ms/step - loss: 0.5082 - acc: 0.8635 - val_loss: 0.5217 - val_acc: 0.8620
Epoch 2355/5000
50/50 [=======] - 9s 189ms/step - loss: 0.5024 - acc: 0.8656 - val_loss: 0.5174 - val_acc: 0.8627
Epoch 2356/5000
50/50 [=======] - 9s 185ms/step - loss: 0.5014 - acc: 0.8653 - val_loss: 0.5185 - val_acc: 0.8615
Epoch 2357/5000
50/50 [=======] - 9s 183ms/step - loss: 0.5070 - acc: 0.8650 - val_loss: 0.5207 - val_acc: 0.8630
Epoch 2358/5000
50/50 [=======] - 9s 182ms/step - loss: 0.5031 - acc: 0.8648 - val_loss: 0.5190 - val_acc: 0.8649
Epoch 2359/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5080 - acc: 0.8630 - val_loss: 0.5172 - val_acc: 0.8636
Epoch 2360/5000
50/50 [=======] - 9s 189ms/step - loss: 0.5041 - acc: 0.8661 - val_loss: 0.5220 - val_acc: 0.8597
Epoch 2361/5000
50/50 [=======] - 10s 191ms/step - loss: 0.5029 - acc: 0.8654 - val_loss: 0.5226 - val_acc: 0.8602
Epoch 2362/5000
50/50 [=======] - 9s 188ms/step - loss: 0.4963 - acc: 0.8706 - val_loss: 0.5257 - val_acc: 0.8605
Epoch 2363/5000
50/50 [=======] - 9s 189ms/step - loss: 0.4999 - acc: 0.8673 - val_loss: 0.5233 - val_acc: 0.8563
Epoch 2364/5000
50/50 [=======] - 9s 184ms/step - loss: 0.5031 - acc: 0.8658 - val_loss: 0.5160 - val_acc: 0.8612
Epoch 2365/5000
50/50 [=======] - 9s 184ms/step - loss: 0.5002 - acc: 0.8673 - val_loss: 0.5206 - val_acc: 0.8605
Epoch 2366/5000
50/50 [=======] - 9s 185ms/step - loss: 0.5039 - acc: 0.8649 - val_loss: 0.5190 - val_acc: 0.8608
Epoch 2367/5000
50/50 [=======] - 9s 185ms/step - loss: 0.5082 - acc: 0.8639 - val_loss: 0.5184 - val_acc: 0.8623
Epoch 2368/5000
50/50 [=======] - 9s 189ms/step - loss: 0.5051 - acc: 0.8658 - val_loss: 0.5175 - val_acc: 0.8611
Epoch 2369/5000
50/50 [=======] - 10s 191ms/step - loss: 0.5079 - acc: 0.8635 - val_loss: 0.5172 - val_acc: 0.8612
Epoch 2370/5000
50/50 [=======] - 9s 189ms/step - loss: 0.5056 - acc: 0.8647 - val_loss: 0.5243 - val_acc: 0.8609
Epoch 2371/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5034 - acc: 0.8654 - val_loss: 0.5149 - val_acc: 0.8632
Epoch 2372/5000
50/50 [=======] - 9s 185ms/step - loss: 0.5055 - acc: 0.8625 - val_loss: 0.5204 - val_acc: 0.8613
Epoch 2373/5000
50/50 [=======] - 9s 183ms/step - loss: 0.5015 - acc: 0.8663 - val_loss: 0.5233 - val_acc: 0.8620
Epoch 2374/5000
50/50 [=======] - 9s 185ms/step - loss: 0.5034 - acc: 0.8649 - val_loss: 0.5199 - val_acc: 0.8624
Epoch 2375/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5043 - acc: 0.8667 - val_loss: 0.5198 - val_acc: 0.8643
Epoch 2376/5000
50/50 [=======] - 10s 190ms/step - loss: 0.5060 - acc: 0.8655 - val_loss: 0.5159 - val_acc: 0.8641
Epoch 2377/5000
50/50 [=======] - 10s 190ms/step - loss: 0.5025 - acc: 0.8661 - val_loss: 0.5171 - val_acc: 0.8600
Epoch 2378/5000
50/50 [=======] - 9s 190ms/step - loss: 0.5058 - acc: 0.8643 - val_loss: 0.5229 - val_acc: 0.8596
Epoch 2379/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5067 - acc: 0.8649 - val_loss: 0.5204 - val_acc: 0.8585
Epoch 2380/5000
50/50 [=======] - 9s 185ms/step - loss: 0.5039 - acc: 0.8648 - val_loss: 0.5211 - val_acc: 0.8616
Epoch 2381/5000
50/50 [=======] - 9s 186ms/step - loss: 0.5031 - acc: 0.8650 - val_loss: 0.5184 - val_acc: 0.8609
Epoch 2382/5000
50/50 [=======] - 9s 185ms/step - loss: 0.5069 - acc: 0.8644 - val_loss: 0.5213 - val_acc: 0.8594
Epoch 2383/5000
50/50 [=======] - 9s 189ms/step - loss: 0.5071 - acc: 0.8639 - val_loss: 0.5227 - val_acc: 0.8589
Epoch 2384/5000
50/50 [=======] - 9s 189ms/step - loss: 0.5073 - acc: 0.8663 - val_loss: 0.5190 - val_acc: 0.8623
Epoch 2385/5000
50/50 [=======] - 10s 193ms/step - loss: 0.5056 - acc: 0.8662 - val_loss: 0.5171 - val_acc: 0.8633
Epoch 2386/5000
50/50 [=======] - 9s 188ms/step - loss: 0.5086 - acc: 0.8638 - val_loss: 0.5144 - val_acc: 0.8637
Epoch 2387/5000
50/50 [=======] - 9s 188ms/step - loss: 0.4997 - acc: 0.8667 - val_loss: 0.5166 - val_acc: 0.8624
Epoch 2388/5000
50/50 [=======] - 9s 184ms/step - loss: 0.4999 - acc: 0.8663 - val_loss: 0.5198 - val_acc: 0.8641
Epoch 2389/5000
50/50 [=======] - 9s 185ms/step - loss: 0.5057 - acc: 0.8638 - val_loss: 0.5206 - val_acc: 0.8614
Epoch 2390/5000
50/50 [=======] - 9s 185ms/step - loss: 0.5030 - acc: 0.8666 - val_loss: 0.5173 - val_acc: 0.8608
Epoch 2391/5000
50/50 [=======] - 9s 188ms/step - loss: 0.5021 - acc: 0.8650 - val_loss: 0.5237 - val_acc: 0.8592
Epoch 2392/5000
50/50 [=======] - 9s 190ms/step - loss: 0.5059 - acc: 0.8660 - val_loss: 0.5201 - val_acc: 0.8632
Epoch 2393/5000
50/50 [=======] - 10s 192ms/step - loss: 0.5029 - acc: 0.8650 - val_loss: 0.5208 - val_acc: 0.8596
Epoch 2394/5000
50/50 [=======] - 10s 191ms/step - loss: 0.5014 - acc: 0.8664 - val_loss: 0.5260 - val_acc: 0.8575
Epoch 2395/5000
50/50 [=======] - 9s 185ms/step - loss: 0.5004 - acc: 0.8677 - val_loss: 0.5158 - val_acc: 0.8635
Epoch 2396/5000
50/50 [=======] - 9s 187ms/step - loss: 0.5001 - acc: 0.8665 - val_loss: 0.5234 - val_acc: 0.8596
Epoch 2397/5000
50/50 [=======] - 9s 182ms/step - loss: 0.5038 - acc: 0.8644 - val_loss: 0.5157 - val_acc: 0.8631
Epoch 2398/5000
50/50 [=======] - 9s 187ms/step - loss: 0.5047 - acc: 0.8655 - val_loss: 0.5172 - val_acc: 0.8619
Epoch 2399/5000
50/50 [=======] - 9s 190ms/step - loss: 0.5061 - acc: 0.8647 - val_loss: 0.5225 - val_acc: 0.8602
Epoch 2400/5000
50/50 [=======] - 10s 191ms/step - loss: 0.5054 - acc: 0.8632 - val_loss: 0.5205 - val_acc: 0.8616
Epoch 2401/5000
50/50 [=======] - 10s 193ms/step - loss: 0.5061 - acc: 0.8649 - val_loss: 0.5271 - val_acc: 0.8580

程序还没运行完，看这样子，估计是欠拟合了。

Minghang Zhao, Shisheng Zhong, Xuyun Fu, Baoping Tang, Shaojiang Dong, Michael Pecht, Deep Residual Networks with Adaptively Parametric Rectifier Linear Units for Fault Diagnosis, IEEE Transactions on Industrial Electronics, 2020, DOI: 10.1109/TIE.2020.2972458

https://ieeexplore.ieee.org/document/8998530

————————————————
版权声明：本文为CSDN博主「dangqing1988」的原创文章，遵循CC 4.0 BY-SA版权协议，转载请附上原文出处链接及本声明。
原文链接：https://blog.csdn.net/dangqing1988/article/details/105831139

ABCD_202020
关注
深度残差网络+自适应参数化ReLU激活函数（调参记录14）
dangqing1988的博客
04-29 523
这次再尝试解决过拟合，把残差模块的数量减少到2个，自适应参数化ReLU激活函数里面第一个全连接层的权重数量，减少为之前的1/4，批量大小设置为1000（主要是为了省时间）。 自适应参数化ReLU激活函数的基本原理如下： Keras程序如下： #!/usr/bin/env python3 # -*- coding: utf-8 -*- """ Created on Tue Apr 14 04:17...
ResNet(残差网络)之残差模块
热门推荐
tigerlib的博客
11-10 4万+
2015年何凯明提出的152层ResNet，获得了ILSVRC比赛图像识别的冠军（top1误差3.6%），同时也使得卷积神经网络有了真正的“深度”。随着网络深度的增加，训练变得愈加困难，这主要是因为在基于随机梯度下降的网络训练过程中，误差信号的多层反向传播非常容易引发“梯度弥散”（梯度过小会使回传的训练误差信号极其微弱）或者“梯度爆炸”（梯度过大导致模型出现NaN）的现象。目前一些特殊的权重初始...
深度残差网络+自适应参数化ReLU激活函数：调参记录18（Cifar10~94.28%）
ABCD_202020的博客
05-05 1220
本文将残差模块的数量增加到27个。其实之前也这样做过，现在的区别在于，自适应参数化ReLU激活函数中第一个全连接层中的神经元个数设置成了特征通道数量的1/16。同样是在Cifar10数据集上进行测试。 自适应参数化ReLU激活函数的基本原理如下： Keras代码如下： #!/usr/bin/env python3 # -*- coding: utf-8 -*- """ Created on Tu...
对池化层、ReLU函数、全连接层的理解
weixin_40396948的博客
02-23 2万+
一、pooling层的作用      pooling主要是在用于图像处理的卷积神经网络中，但随着深层神经网络的发展，pooling相关技术在其他领域，其他结构的神经网络中也越来越受关注。      卷积神经网络中的卷积层是对图像的一个邻域进行卷积得到图像的邻域特征，亚采样层就是使用pooling技术将小邻域内的特征点整合得到新的特征。pooling确实起到了整合特征的作用。       pooli...
深度残差网络+自适应参数化ReLU激活函数：调参记录13
ABCD_202020的博客
05-04 451
从以往的调参结果来看，过拟合是最主要的问题。本文在调参记录12的基础上，将层数减少，减到9个残差模块，再试一次。 自适应参数化ReLU激活函数原理如下： Keras程序如下： #!/usr/bin/env python3 # -*- coding: utf-8 -*- """ Created on Tue Apr 14 04:17:45 2020 Implemented using Tensor...
深度残差网络+自适应参数化ReLU激活函数：调参记录9
ABCD_202020的博客
05-04 403
本文在调参记录6的基础上，继续调整超参数，测试Adaptively Parametric ReLU（APReLU）激活函数在Cifar10图像集上的效果。 深度残差网络+自适应参数化ReLU激活函数（调参记录6） https://blog.csdn.net/dangqing1988/article/details/105628681 自适应参数化ReLU激活函数的基本原理见下图： 在Keras里...
深度残差网络+自适应参数化ReLU激活函数：调参记录12
ABCD_202020的博客
05-04 989
本文在调参记录10的基础上，在数据增强部分添加了zoom_range = 0.2，将训练迭代次数增加到5000个epoch，批量大小改成了625，测试Adaptively Parametric ReLU（APReLU）激活函数在Cifar10图像集上的效果。 Adaptively Parametric ReLU（APReLU）激活函数的原理如下： Keras程序： #!/usr/bin/env ...
深度残差网络+自适应参数化ReLU激活函数：调参记录3
ABCD_202020的博客
04-27 438
续上一篇： 深度残差网络+自适应参数化ReLU激活函数（调参记录2） https://blog.csdn.net/dangqing1988/article/details/105595917 本文继续测试深度残差网络和自适应参数化ReLU激活函数在Cifar10图像集上的表现，残差模块仍然是27个，卷积核的个数分别增加到16个、32个和64个，迭代次数从1000个epoch减到了500个epoch...
深度残差网络+自适应参数化ReLU激活函数：调参记录11
ABCD_202020的博客
05-04 1676
本文在调参记录10的基础上，将残差模块的数量从27个增加到60个，测试采用Adaptively Parametric ReLU（APReLU）激活函数的深度残差网络，在Cifar10图像集上的效果。 Keras程序如下： #!/usr/bin/env python3 # -*- coding: utf-8 -*- """ Created on Tue Apr 14 04:17:45 2020 I...
深度残差网络+自适应参数化ReLU激活函数
07-28
深度残差网络（Deep Residual Network）是一种深度学习网络结构，通过引入残差连接来解决深层网络训练过程中的梯度消失...深度残差网络和自适应参数化ReLU激活函数可以结合使用，以提高深度学习模型的性能和训练效果。
深度残差网络+自适应参数化ReLU激活函数（调参记录16）
dangqing1988的博客
04-30 464
在调参记录15的基础上，将第一个残差模块的卷积核数量，从16个增加到32个，同时将自适应参数化ReLU激活函数中第一个全连接层的神经元个数改成原先的1/16，继续测试其在Cifar10数据集上的效果。 自适应参数化ReLU激活函数的基本原理如下： Keras程序： #!/usr/bin/env python3 # -*- coding: utf-8 -*- """ Created on Tue ...
深度残差网络+自适应参数化ReLU激活函数（调参记录4）
dangqing1988的博客
04-19 1897
续上一篇： 深度残差网络+自适应参数化ReLU激活函数（调参记录3） https://blog.csdn.net/dangqing1988/article/details/105601313 本文在深度残差网络中采用了自适应参数化ReLU激活函数，继续测试其在Cifar10图像集上的效果。与上一篇不同的是，这次修改了残差模块里面的结构，原先是两个3×3的卷积层，现在改成了1×1→3×3→1×1的瓶...
深度残差网络+自适应参数化ReLU激活函数（调参记录2）
dangqing1988的博客
04-18 517
续上一篇 https://blog.csdn.net/dangqing1988/article/details/105590515 本文依然是测试深度残差网络+自适应参数化ReLU激活函数，残差模块的数量增加到了27个，其他保持不变，卷积核的个数依然是8个、16个到32个，继续测试在Cifar10数据集上的效果。 其中，自适应参数化ReLU激活函数是Parametric ReLU的一种改进，基本原...
【深度学习】深度学习模型的加密及解密方案及源码
利哥AI实例探险
09-18 1149
本文主要根据自己遇到的情况，例如：对于yolo或paddle训练的模型文件，对外使用，不想要别人拿到我的模型文件随意乱用，此时就涉及到对模型文件进行加密与解密 深度学习模型的加密保护非常重要，尤其在商业应用场景下。常见的模型加密方法包括模型文件加密、加密硬件加密、授权管理和自定义解密机制。下面介绍几种常用方法及其原理。
深度学习02-pytorch-03-张量的数值计算
最新发布
weixin_41645791的博客
09-19 447
张量是多维数组的泛化，广泛应用于深度学习和数值计算。通过基本的算术操作、线性代数运算、维度操作等，我们可以高效地处理大规模数据。同时，PyTorch 还支持自动微分和 GPU 加速，这使得它在深度学习领域非常流行。
深度学习速通系列:TextCNN介绍
Ven%的博客
09-18 604
例如，使用PyTorch实现TextCNN进行中文文本分类的案例中，首先需要对中文文本进行分词和词向量转换，然后构建TextCNN模型，包括卷积层、池化层和分类层。它将卷积神经网络（CNN）应用于文本数据，通过使用不同大小的卷积核来提取文本中的局部特征，类似于捕捉不同长度的n-gram信息，从而有效地捕捉局部相关性。在实际案例中，可以通过配置文件设置训练参数，如批量大小、学习率、优化器类型等，然后使用训练脚本开始训练过程。TextCNN适用于各种文本分类任务，如情感分析、主题分类、垃圾邮件检测等。
深度学习自编码器 - 去噪自编码器篇
绎岚科技的博客
09-18 1224
在深度学习的广阔天地中，自编码器作为一种强大的无监督学习工具，通过重构输入数据的方式，不仅实现了数据的有效压缩，还探索了数据的内在表示。而去噪自编码器（Denoising Autoencoder, DAE），作为自编码器的一个变种，更是以其独特的去噪能力，在众多应用中脱颖而出。去噪自编码器在训练过程中，会故意向输入数据中添加噪声，然后迫使模型从这种被“污染”的数据中恢复出原始的纯净数据。这一过程不仅锻炼了模型的鲁棒性，还促进了模型学习到更加稳健和本质的数据特征，从而提高了其在噪声环境下的泛化能力。
深度学习02-pytorch-08-自动微分模块
weixin_41645791的博客
09-19 673
PyTorch 的自动微分机制通过autograd实现，用户只需要将张量的设置为True，在执行反向传播时，PyTorch 会自动计算张量的梯度。通过自动构建计算图，autograd能够跟踪张量上的所有操作，动态计算梯度，极大地方便了深度学习模型的训练。
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值