深度残差网络+自适应参数化ReLU激活函数（调参记录20）Cifar10~94.17%

最新推荐文章于 2021-05-08 22:57:14 发布
最新推荐文章于 2021-05-08 22:57:14 发布
阅读量1k 收藏 1
点赞数
文章标签： 深度学习 神经网络 机器学习
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/dangqing1988/article/details/106128478
版权

在之前的调参记录18中，是将深度残差网络ResNet中的所有ReLU都替换成了自适应参数化ReLU（Adaptively Parametric ReLU，APReLU）。

由于APReLU的输入特征图与输出特征图的尺寸是完全一致的，所以APReLU可以被嵌入到神经网络的任意部分。

本文将APReLU放在每个残差模块的第二个卷积层之后。这种结构与Squeeze-and-Excitation Network是非常相似的，其区别在于APReLU额外地包含了非线性变换。

同时，迭代次数也从5000个epoch减少到了500个epoch。时间耗不起。

APReLU激活函数的原理如下图所示：
在这里插入图片描述
整体代码如下：

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
Created on Tue Apr 14 04:17:45 2020
Implemented using TensorFlow 1.10.0 and Keras 2.2.1

Minghang Zhao, Shisheng Zhong, Xuyun Fu, Baoping Tang, Shaojiang Dong, Michael Pecht,
Deep Residual Networks with Adaptively Parametric Rectifier Linear Units for Fault Diagnosis, 
IEEE Transactions on Industrial Electronics, 2020,  DOI: 10.1109/TIE.2020.2972458 

@author: Minghang Zhao
"""

from __future__ import print_function
import keras
import numpy as np
from keras.datasets import cifar10
from keras.layers import Dense, Conv2D, BatchNormalization, Activation, Minimum
from keras.layers import AveragePooling2D, Input, GlobalAveragePooling2D, Concatenate, Reshape
from keras.regularizers import l2
from keras import backend as K
from keras.models import Model
from keras import optimizers
from keras.preprocessing.image import ImageDataGenerator
from keras.callbacks import LearningRateScheduler
K.set_learning_phase(1)

# The data, split between train and test sets
(x_train, y_train), (x_test, y_test) = cifar10.load_data()

# Noised data
x_train = x_train.astype('float32') / 255.
x_test = x_test.astype('float32') / 255.
x_test = x_test-np.mean(x_train)
x_train = x_train-np.mean(x_train)
print('x_train shape:', x_train.shape)
print(x_train.shape[0], 'train samples')
print(x_test.shape[0], 'test samples')

# convert class vectors to binary class matrices
y_train = keras.utils.to_categorical(y_train, 10)
y_test = keras.utils.to_categorical(y_test, 10)

# Schedule the learning rate, multiply 0.1 every 150 epoches
def scheduler(epoch):
    if epoch % 150 == 0 and epoch != 0:
        lr = K.get_value(model.optimizer.lr)
        K.set_value(model.optimizer.lr, lr * 0.1)
        print("lr changed to {}".format(lr * 0.1))
    return K.get_value(model.optimizer.lr)

# An adaptively parametric rectifier linear unit (APReLU)
def aprelu(inputs):
    # get the number of channels
    channels = inputs.get_shape().as_list()[-1]
    # get a zero feature map
    zeros_input = keras.layers.subtract([inputs, inputs])
    # get a feature map with only positive features
    pos_input = Activation('relu')(inputs)
    # get a feature map with only negative features
    neg_input = Minimum()([inputs,zeros_input])
    # define a network to obtain the scaling coefficients
    scales_p = GlobalAveragePooling2D()(pos_input)
    scales_n = GlobalAveragePooling2D()(neg_input)
    scales = Concatenate()([scales_n, scales_p])
    scales = Dense(channels//16, activation='linear', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(scales)
    scales = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(scales)
    scales = Activation('relu')(scales)
    scales = Dense(channels, activation='linear', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(scales)
    scales = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(scales)
    scales = Activation('sigmoid')(scales)
    scales = Reshape((1,1,channels))(scales)
    # apply a paramtetric relu
    neg_part = keras.layers.multiply([scales, neg_input])
    return keras.layers.add([pos_input, neg_part])

# Residual Block
def residual_block(incoming, nb_blocks, out_channels, downsample=False,
                   downsample_strides=2):
    
    residual = incoming
    in_channels = incoming.get_shape().as_list()[-1]
    
    for i in range(nb_blocks):
        
        identity = residual
        
        if not downsample:
            downsample_strides = 1
        
        residual = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(residual)
        residual = Activation('relu')(residual)
        residual = Conv2D(out_channels, 3, strides=(downsample_strides, downsample_strides), 
                          padding='same', kernel_initializer='he_normal', 
                          kernel_regularizer=l2(1e-4))(residual)
        
        residual = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(residual)
        residual = Activation('relu')(residual)
        residual = Conv2D(out_channels, 3, padding='same', kernel_initializer='he_normal', 
                          kernel_regularizer=l2(1e-4))(residual)
        
        residual = aprelu(residual)
        
        # Downsampling
        if downsample_strides > 1:
            identity = AveragePooling2D(pool_size=(1,1), strides=(2,2))(identity)
            
        # Zero_padding to match channels
        if in_channels != out_channels:
            zeros_identity = keras.layers.subtract([identity, identity])
            identity = keras.layers.concatenate([identity, zeros_identity])
            in_channels = out_channels
        
        residual = keras.layers.add([residual, identity])
    
    return residual


# define and train a model
inputs = Input(shape=(32, 32, 3))
net = Conv2D(16, 3, padding='same', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(inputs)
net = residual_block(net, 9, 32, downsample=False)
net = residual_block(net, 1, 32, downsample=True)
net = residual_block(net, 8, 32, downsample=False)
net = residual_block(net, 1, 64, downsample=True)
net = residual_block(net, 8, 64, downsample=False)
net = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(net)
net = Activation('relu')(net)
net = GlobalAveragePooling2D()(net)
outputs = Dense(10, activation='softmax', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(net)
model = Model(inputs=inputs, outputs=outputs)
sgd = optimizers.SGD(lr=0.1, decay=0., momentum=0.9, nesterov=True)
model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])

# data augmentation
datagen = ImageDataGenerator(
    # randomly rotate images in the range (deg 0 to 180)
    rotation_range=30,
    # Range for random zoom
    zoom_range = 0.2,
    # shear angle in counter-clockwise direction in degrees
    shear_range = 30,
    # randomly flip images
    horizontal_flip=True,
    # randomly shift images horizontally
    width_shift_range=0.125,
    # randomly shift images vertically
    height_shift_range=0.125)

reduce_lr = LearningRateScheduler(scheduler)
# fit the model on the batches generated by datagen.flow().
model.fit_generator(datagen.flow(x_train, y_train, batch_size=100),
                    validation_data=(x_test, y_test), epochs=500, 
                    verbose=1, callbacks=[reduce_lr], workers=4)

# get results
K.set_learning_phase(0)
DRSN_train_score = model.evaluate(x_train, y_train, batch_size=100, verbose=0)
print('Train loss:', DRSN_train_score[0])
print('Train accuracy:', DRSN_train_score[1])
DRSN_test_score = model.evaluate(x_test, y_test, batch_size=100, verbose=0)
print('Test loss:', DRSN_test_score[0])
print('Test accuracy:', DRSN_test_score[1])

实验结果如下：

Using TensorFlow backend.
x_train shape: (50000, 32, 32, 3)
50000 train samples
10000 test samples
Epoch 1/500
70s 140ms/step - loss: 2.5742 - acc: 0.4145 - val_loss: 2.1224 - val_acc: 0.5538
Epoch 2/500
52s 104ms/step - loss: 2.0566 - acc: 0.5631 - val_loss: 1.7678 - val_acc: 0.6502
Epoch 3/500
52s 104ms/step - loss: 1.7697 - acc: 0.6317 - val_loss: 1.5134 - val_acc: 0.7114
Epoch 4/500
52s 103ms/step - loss: 1.5790 - acc: 0.6694 - val_loss: 1.3269 - val_acc: 0.7508
Epoch 5/500
52s 104ms/step - loss: 1.4270 - acc: 0.6971 - val_loss: 1.2040 - val_acc: 0.7703
Epoch 6/500
52s 104ms/step - loss: 1.3109 - acc: 0.7165 - val_loss: 1.1187 - val_acc: 0.7809
Epoch 7/500
52s 104ms/step - loss: 1.2249 - acc: 0.7302 - val_loss: 1.0393 - val_acc: 0.7919
Epoch 8/500
52s 103ms/step - loss: 1.1457 - acc: 0.7482 - val_loss: 0.9639 - val_acc: 0.8084
Epoch 9/500
52s 104ms/step - loss: 1.0931 - acc: 0.7555 - val_loss: 0.9324 - val_acc: 0.8130
Epoch 10/500
52s 104ms/step - loss: 1.0418 - acc: 0.7693 - val_loss: 0.9043 - val_acc: 0.8138
Epoch 11/500
52s 104ms/step - loss: 1.0049 - acc: 0.7747 - val_loss: 0.8600 - val_acc: 0.8281
Epoch 12/500
51s 102ms/step - loss: 0.9835 - acc: 0.7796 - val_loss: 0.8364 - val_acc: 0.8288
Epoch 13/500
51s 103ms/step - loss: 0.9480 - acc: 0.7878 - val_loss: 0.7905 - val_acc: 0.8432
Epoch 14/500
52s 104ms/step - loss: 0.9246 - acc: 0.7906 - val_loss: 0.7895 - val_acc: 0.8400
Epoch 15/500
52s 103ms/step - loss: 0.9067 - acc: 0.7967 - val_loss: 0.7709 - val_acc: 0.8451
Epoch 16/500
52s 103ms/step - loss: 0.8928 - acc: 0.8000 - val_loss: 0.7728 - val_acc: 0.8447
Epoch 17/500
52s 104ms/step - loss: 0.8787 - acc: 0.8042 - val_loss: 0.7842 - val_acc: 0.8367
Epoch 18/500
52s 104ms/step - loss: 0.8644 - acc: 0.8095 - val_loss: 0.7588 - val_acc: 0.8489
Epoch 19/500
52s 103ms/step - loss: 0.8531 - acc: 0.8111 - val_loss: 0.7626 - val_acc: 0.8427
Epoch 20/500
52s 103ms/step - loss: 0.8463 - acc: 0.8140 - val_loss: 0.7256 - val_acc: 0.8620
Epoch 21/500
52s 103ms/step - loss: 0.8402 - acc: 0.8158 - val_loss: 0.7499 - val_acc: 0.8502
Epoch 22/500
52s 103ms/step - loss: 0.8347 - acc: 0.8170 - val_loss: 0.7154 - val_acc: 0.8629
Epoch 23/500
52s 103ms/step - loss: 0.8262 - acc: 0.8219 - val_loss: 0.7074 - val_acc: 0.8625
Epoch 24/500
51s 102ms/step - loss: 0.8251 - acc: 0.8211 - val_loss: 0.7165 - val_acc: 0.8601
Epoch 25/500
51s 102ms/step - loss: 0.8166 - acc: 0.8250 - val_loss: 0.7233 - val_acc: 0.8564
Epoch 26/500
51s 102ms/step - loss: 0.8090 - acc: 0.8266 - val_loss: 0.7401 - val_acc: 0.8481
Epoch 27/500
51s 102ms/step - loss: 0.8049 - acc: 0.8290 - val_loss: 0.6991 - val_acc: 0.8658
Epoch 28/500
51s 102ms/step - loss: 0.8037 - acc: 0.8302 - val_loss: 0.7159 - val_acc: 0.8630
Epoch 29/500
51s 102ms/step - loss: 0.7977 - acc: 0.8327 - val_loss: 0.7189 - val_acc: 0.8614
Epoch 30/500
51s 102ms/step - loss: 0.7968 - acc: 0.8334 - val_loss: 0.7030 - val_acc: 0.8706
Epoch 31/500
51s 102ms/step - loss: 0.7956 - acc: 0.8339 - val_loss: 0.6863 - val_acc: 0.8764
Epoch 32/500
51s 102ms/step - loss: 0.7875 - acc: 0.8377 - val_loss: 0.7160 - val_acc: 0.8647
Epoch 33/500
51s 102ms/step - loss: 0.7874 - acc: 0.8368 - val_loss: 0.7002 - val_acc: 0.8704
Epoch 34/500
51s 102ms/step - loss: 0.7917 - acc: 0.8357 - val_loss: 0.6829 - val_acc: 0.8783
Epoch 35/500
51s 102ms/step - loss: 0.7808 - acc: 0.8423 - val_loss: 0.7057 - val_acc: 0.8685
Epoch 36/500
51s 103ms/step - loss: 0.7795 - acc: 0.8413 - val_loss: 0.7044 - val_acc: 0.8710
Epoch 37/500
52s 103ms/step - loss: 0.7768 - acc: 0.8429 - val_loss: 0.6871 - val_acc: 0.8743
Epoch 38/500
52s 103ms/step - loss: 0.7763 - acc: 0.8418 - val_loss: 0.6995 - val_acc: 0.8687
Epoch 39/500
52s 103ms/step - loss: 0.7665 - acc: 0.8459 - val_loss: 0.6907 - val_acc: 0.8736
Epoch 40/500
52s 103ms/step - loss: 0.7751 - acc: 0.8428 - val_loss: 0.6947 - val_acc: 0.8717
Epoch 41/500
52s 103ms/step - loss: 0.7691 - acc: 0.8456 - val_loss: 0.6960 - val_acc: 0.8709
Epoch 42/500
52s 103ms/step - loss: 0.7704 - acc: 0.8460 - val_loss: 0.6690 - val_acc: 0.8851
Epoch 43/500
52s 103ms/step - loss: 0.7641 - acc: 0.8484 - val_loss: 0.6825 - val_acc: 0.8792
Epoch 44/500
52s 103ms/step - loss: 0.7630 - acc: 0.8499 - val_loss: 0.6765 - val_acc: 0.8781
Epoch 45/500
52s 103ms/step - loss: 0.7583 - acc: 0.8502 - val_loss: 0.6955 - val_acc: 0.8724
Epoch 46/500
52s 103ms/step - loss: 0.7599 - acc: 0.8493 - val_loss: 0.6750 - val_acc: 0.8773
Epoch 47/500
51s 102ms/step - loss: 0.7617 - acc: 0.8497 - val_loss: 0.6925 - val_acc: 0.8757
Epoch 48/500
51s 102ms/step - loss: 0.7589 - acc: 0.8501 - val_loss: 0.6848 - val_acc: 0.8775
Epoch 49/500
51s 102ms/step - loss: 0.7572 - acc: 0.8506 - val_loss: 0.6834 - val_acc: 0.8767
Epoch 50/500
51s 103ms/step - loss: 0.7512 - acc: 0.8523 - val_loss: 0.6908 - val_acc: 0.8742
Epoch 51/500
51s 102ms/step - loss: 0.7553 - acc: 0.8523 - val_loss: 0.6987 - val_acc: 0.8705
Epoch 52/500
51s 103ms/step - loss: 0.7501 - acc: 0.8523 - val_loss: 0.7237 - val_acc: 0.8665
Epoch 53/500
52s 103ms/step - loss: 0.7544 - acc: 0.8535 - val_loss: 0.6973 - val_acc: 0.8747
Epoch 54/500
52s 103ms/step - loss: 0.7467 - acc: 0.8561 - val_loss: 0.6836 - val_acc: 0.8814
Epoch 55/500
52s 103ms/step - loss: 0.7490 - acc: 0.8544 - val_loss: 0.6914 - val_acc: 0.8767
Epoch 56/500
52s 103ms/step - loss: 0.7451 - acc: 0.8568 - val_loss: 0.6881 - val_acc: 0.8803
Epoch 57/500
52s 103ms/step - loss: 0.7488 - acc: 0.8554 - val_loss: 0.6867 - val_acc: 0.8786
Epoch 58/500
52s 103ms/step - loss: 0.7465 - acc: 0.8576 - val_loss: 0.6845 - val_acc: 0.8759
Epoch 59/500
52s 103ms/step - loss: 0.7443 - acc: 0.8570 - val_loss: 0.6715 - val_acc: 0.8851
Epoch 60/500
52s 103ms/step - loss: 0.7472 - acc: 0.8562 - val_loss: 0.7045 - val_acc: 0.8751
Epoch 61/500
52s 103ms/step - loss: 0.7425 - acc: 0.8596 - val_loss: 0.6793 - val_acc: 0.8836
Epoch 62/500
52s 103ms/step - loss: 0.7433 - acc: 0.8587 - val_loss: 0.6963 - val_acc: 0.8752
Epoch 63/500
52s 103ms/step - loss: 0.7472 - acc: 0.8560 - val_loss: 0.6911 - val_acc: 0.8773
Epoch 64/500
52s 103ms/step - loss: 0.7423 - acc: 0.8577 - val_loss: 0.6808 - val_acc: 0.8809
Epoch 65/500
52s 103ms/step - loss: 0.7404 - acc: 0.8594 - val_loss: 0.7071 - val_acc: 0.8703
Epoch 66/500
52s 103ms/step - loss: 0.7418 - acc: 0.8580 - val_loss: 0.6881 - val_acc: 0.8776
Epoch 67/500
52s 103ms/step - loss: 0.7404 - acc: 0.8584 - val_loss: 0.6973 - val_acc: 0.8770
Epoch 68/500
52s 103ms/step - loss: 0.7458 - acc: 0.8569 - val_loss: 0.6871 - val_acc: 0.8783
Epoch 69/500
52s 103ms/step - loss: 0.7342 - acc: 0.8625 - val_loss: 0.6856 - val_acc: 0.8778
Epoch 70/500
52s 103ms/step - loss: 0.7358 - acc: 0.8603 - val_loss: 0.6999 - val_acc: 0.8742
Epoch 71/500
52s 103ms/step - loss: 0.7419 - acc: 0.8600 - val_loss: 0.6921 - val_acc: 0.8816
Epoch 72/500
52s 103ms/step - loss: 0.7368 - acc: 0.8607 - val_loss: 0.6825 - val_acc: 0.8813
Epoch 73/500
51s 103ms/step - loss: 0.7376 - acc: 0.8581 - val_loss: 0.6771 - val_acc: 0.8846
Epoch 74/500
51s 102ms/step - loss: 0.7371 - acc: 0.8598 - val_loss: 0.6963 - val_acc: 0.8787
Epoch 75/500
51s 102ms/step - loss: 0.7409 - acc: 0.8593 - val_loss: 0.6884 - val_acc: 0.8794
Epoch 76/500
51s 102ms/step - loss: 0.7348 - acc: 0.8633 - val_loss: 0.6655 - val_acc: 0.8904
Epoch 77/500
51s 102ms/step - loss: 0.7358 - acc: 0.8624 - val_loss: 0.6860 - val_acc: 0.8812
Epoch 78/500
51s 102ms/step - loss: 0.7392 - acc: 0.8616 - val_loss: 0.7251 - val_acc: 0.8686
Epoch 79/500
51s 102ms/step - loss: 0.7342 - acc: 0.8645 - val_loss: 0.6891 - val_acc: 0.8804
Epoch 80/500
51s 102ms/step - loss: 0.7341 - acc: 0.8635 - val_loss: 0.6761 - val_acc: 0.8847
Epoch 81/500
51s 102ms/step - loss: 0.7382 - acc: 0.8605 - val_loss: 0.7034 - val_acc: 0.8752ETA: 43s - loss: 0.6977 - acc: 0.8793
Epoch 82/500
51s 102ms/step - loss: 0.7377 - acc: 0.8617 - val_loss: 0.6670 - val_acc: 0.8867
Epoch 83/500
51s 102ms/step - loss: 0.7347 - acc: 0.8643 - val_loss: 0.6862 - val_acc: 0.8815
Epoch 84/500
51s 102ms/step - loss: 0.7301 - acc: 0.8660 - val_loss: 0.6837 - val_acc: 0.8818
Epoch 85/500
51s 102ms/step - loss: 0.7322 - acc: 0.8648 - val_loss: 0.6842 - val_acc: 0.8809
Epoch 86/500
51s 102ms/step - loss: 0.7303 - acc: 0.8640 - val_loss: 0.6906 - val_acc: 0.8823
Epoch 87/500
51s 102ms/step - loss: 0.7285 - acc: 0.8651 - val_loss: 0.6701 - val_acc: 0.8847
Epoch 88/500
51s 102ms/step - loss: 0.7313 - acc: 0.8645 - val_loss: 0.6774 - val_acc: 0.8832
Epoch 89/500
51s 102ms/step - loss: 0.7303 - acc: 0.8625 - val_loss: 0.6827 - val_acc: 0.8851
Epoch 90/500
51s 102ms/step - loss: 0.7283 - acc: 0.8647 - val_loss: 0.6886 - val_acc: 0.8821
Epoch 91/500
51s 103ms/step - loss: 0.7256 - acc: 0.8662 - val_loss: 0.6889 - val_acc: 0.8803
Epoch 92/500
52s 103ms/step - loss: 0.7313 - acc: 0.8634 - val_loss: 0.6747 - val_acc: 0.8865
Epoch 93/500
52s 103ms/step - loss: 0.7257 - acc: 0.8656 - val_loss: 0.6831 - val_acc: 0.8836
Epoch 94/500
52s 103ms/step - loss: 0.7303 - acc: 0.8645 - val_loss: 0.7008 - val_acc: 0.8772
Epoch 95/500
52s 103ms/step - loss: 0.7268 - acc: 0.8657 - val_loss: 0.6998 - val_acc: 0.8783
Epoch 96/500
51s 102ms/step - loss: 0.7245 - acc: 0.8659 - val_loss: 0.6927 - val_acc: 0.8789
Epoch 97/500
51s 102ms/step - loss: 0.7236 - acc: 0.8685 - val_loss: 0.6620 - val_acc: 0.8919
Epoch 98/500
51s 102ms/step - loss: 0.7249 - acc: 0.8669 - val_loss: 0.6741 - val_acc: 0.8857
Epoch 99/500
51s 102ms/step - loss: 0.7252 - acc: 0.8659 - val_loss: 0.6770 - val_acc: 0.8844
Epoch 100/500
51s 102ms/step - loss: 0.7239 - acc: 0.8672 - val_loss: 0.6815 - val_acc: 0.8851
Epoch 101/500
51s 102ms/step - loss: 0.7272 - acc: 0.8667 - val_loss: 0.6752 - val_acc: 0.8867
Epoch 102/500
51s 102ms/step - loss: 0.7268 - acc: 0.8651 - val_loss: 0.6985 - val_acc: 0.8766
Epoch 103/500
51s 102ms/step - loss: 0.7243 - acc: 0.8685 - val_loss: 0.7136 - val_acc: 0.8714
Epoch 104/500
51s 102ms/step - loss: 0.7293 - acc: 0.8646 - val_loss: 0.6930 - val_acc: 0.8813
Epoch 105/500
51s 102ms/step - loss: 0.7237 - acc: 0.8665 - val_loss: 0.6919 - val_acc: 0.8812
Epoch 106/500
51s 102ms/step - loss: 0.7275 - acc: 0.8669 - val_loss: 0.6722 - val_acc: 0.8857
Epoch 107/500
51s 102ms/step - loss: 0.7149 - acc: 0.8703 - val_loss: 0.6730 - val_acc: 0.8827
Epoch 108/500
51s 102ms/step - loss: 0.7214 - acc: 0.8689 - val_loss: 0.6530 - val_acc: 0.8930
Epoch 109/500
51s 102ms/step - loss: 0.7212 - acc: 0.8678 - val_loss: 0.7015 - val_acc: 0.8784
Epoch 110/500
51s 102ms/step - loss: 0.7238 - acc: 0.8676 - val_loss: 0.6730 - val_acc: 0.8833
Epoch 111/500
51s 102ms/step - loss: 0.7260 - acc: 0.8670 - val_loss: 0.6541 - val_acc: 0.8919
Epoch 112/500
51s 102ms/step - loss: 0.7209 - acc: 0.8688 - val_loss: 0.6577 - val_acc: 0.8926
Epoch 113/500
51s 102ms/step - loss: 0.7258 - acc: 0.8654 - val_loss: 0.6864 - val_acc: 0.8832
Epoch 114/500
51s 102ms/step - loss: 0.7211 - acc: 0.8695 - val_loss: 0.6749 - val_acc: 0.8873
Epoch 115/500
51s 102ms/step - loss: 0.7196 - acc: 0.8707 - val_loss: 0.6660 - val_acc: 0.8873
Epoch 116/500
51s 102ms/step - loss: 0.7189 - acc: 0.8684 - val_loss: 0.6945 - val_acc: 0.8809
Epoch 117/500
51s 102ms/step - loss: 0.7246 - acc: 0.8672 - val_loss: 0.7153 - val_acc: 0.8706
Epoch 118/500
51s 102ms/step - loss: 0.7189 - acc: 0.8719 - val_loss: 0.6718 - val_acc: 0.8849
Epoch 119/500
51s 102ms/step - loss: 0.7214 - acc: 0.8695 - val_loss: 0.6883 - val_acc: 0.8802
Epoch 120/500
51s 102ms/step - loss: 0.7154 - acc: 0.8706 - val_loss: 0.6846 - val_acc: 0.8844
Epoch 121/500
51s 102ms/step - loss: 0.7150 - acc: 0.8691 - val_loss: 0.6820 - val_acc: 0.8836
Epoch 122/500
51s 102ms/step - loss: 0.7190 - acc: 0.8696 - val_loss: 0.6737 - val_acc: 0.8857
Epoch 123/500
51s 102ms/step - loss: 0.7191 - acc: 0.8686 - val_loss: 0.6752 - val_acc: 0.8848
Epoch 124/500
51s 102ms/step - loss: 0.7185 - acc: 0.8701 - val_loss: 0.6841 - val_acc: 0.8828
Epoch 125/500
51s 103ms/step - loss: 0.7221 - acc: 0.8689 - val_loss: 0.6739 - val_acc: 0.8837
Epoch 126/500
51s 102ms/step - loss: 0.7202 - acc: 0.8699 - val_loss: 0.6787 - val_acc: 0.8888
Epoch 127/500
51s 102ms/step - loss: 0.7172 - acc: 0.8703 - val_loss: 0.6889 - val_acc: 0.8815
Epoch 128/500
51s 102ms/step - loss: 0.7157 - acc: 0.8719 - val_loss: 0.6832 - val_acc: 0.8852
Epoch 129/500
51s 102ms/step - loss: 0.7149 - acc: 0.8704 - val_loss: 0.6777 - val_acc: 0.8859
Epoch 130/500
51s 102ms/step - loss: 0.7205 - acc: 0.8698 - val_loss: 0.6675 - val_acc: 0.8908
Epoch 131/500
51s 102ms/step - loss: 0.7146 - acc: 0.8721 - val_loss: 0.6741 - val_acc: 0.8916
Epoch 132/500
51s 102ms/step - loss: 0.7140 - acc: 0.8720 - val_loss: 0.6649 - val_acc: 0.8891
Epoch 133/500
52s 103ms/step - loss: 0.7128 - acc: 0.8714 - val_loss: 0.6883 - val_acc: 0.8834
Epoch 134/500
52s 103ms/step - loss: 0.7203 - acc: 0.8699 - val_loss: 0.6899 - val_acc: 0.8854
Epoch 135/500
52s 103ms/step - loss: 0.7131 - acc: 0.8716 - val_loss: 0.6907 - val_acc: 0.8823
Epoch 136/500
52s 103ms/step - loss: 0.7127 - acc: 0.8706 - val_loss: 0.6957 - val_acc: 0.8786
Epoch 137/500
52s 103ms/step - loss: 0.7147 - acc: 0.8726 - val_loss: 0.7036 - val_acc: 0.8764
Epoch 138/500
52s 103ms/step - loss: 0.7125 - acc: 0.8735 - val_loss: 0.6704 - val_acc: 0.8896
Epoch 139/500
52s 103ms/step - loss: 0.7153 - acc: 0.8692 - val_loss: 0.6620 - val_acc: 0.8923
Epoch 140/500
52s 103ms/step - loss: 0.7125 - acc: 0.8707 - val_loss: 0.6862 - val_acc: 0.8839
Epoch 141/500
52s 103ms/step - loss: 0.7149 - acc: 0.8722 - val_loss: 0.6573 - val_acc: 0.8951
Epoch 142/500
51s 103ms/step - loss: 0.7154 - acc: 0.8726 - val_loss: 0.6658 - val_acc: 0.8898
Epoch 143/500
52s 103ms/step - loss: 0.7115 - acc: 0.8728 - val_loss: 0.6868 - val_acc: 0.8848
Epoch 144/500
52s 103ms/step - loss: 0.7116 - acc: 0.8733 - val_loss: 0.6679 - val_acc: 0.8894
Epoch 145/500
52s 103ms/step - loss: 0.7198 - acc: 0.8696 - val_loss: 0.6865 - val_acc: 0.8824
Epoch 146/500
52s 103ms/step - loss: 0.7146 - acc: 0.8726 - val_loss: 0.6906 - val_acc: 0.8826
Epoch 147/500
52s 103ms/step - loss: 0.7165 - acc: 0.8718 - val_loss: 0.6597 - val_acc: 0.8898
Epoch 148/500
52s 103ms/step - loss: 0.7130 - acc: 0.8725 - val_loss: 0.6596 - val_acc: 0.8926
Epoch 149/500
52s 103ms/step - loss: 0.7096 - acc: 0.8728 - val_loss: 0.6855 - val_acc: 0.8816
Epoch 150/500
52s 103ms/step - loss: 0.7131 - acc: 0.8730 - val_loss: 0.6872 - val_acc: 0.8838
Epoch 151/500
lr changed to 0.010000000149011612
52s 103ms/step - loss: 0.5994 - acc: 0.9120 - val_loss: 0.5846 - val_acc: 0.9190
Epoch 152/500
51s 103ms/step - loss: 0.5434 - acc: 0.9286 - val_loss: 0.5670 - val_acc: 0.9214
Epoch 153/500
52s 103ms/step - loss: 0.5249 - acc: 0.9328 - val_loss: 0.5552 - val_acc: 0.9216
Epoch 154/500
51s 103ms/step - loss: 0.5089 - acc: 0.9363 - val_loss: 0.5436 - val_acc: 0.9275
Epoch 155/500
51s 102ms/step - loss: 0.4992 - acc: 0.9393 - val_loss: 0.5400 - val_acc: 0.9263
Epoch 156/500
51s 102ms/step - loss: 0.4838 - acc: 0.9424 - val_loss: 0.5373 - val_acc: 0.9269
Epoch 157/500
51s 102ms/step - loss: 0.4733 - acc: 0.9442 - val_loss: 0.5283 - val_acc: 0.9264ETA: 24s - loss: 0.4741 - acc: 0.9435
Epoch 158/500
51s 103ms/step - loss: 0.4641 - acc: 0.9458 - val_loss: 0.5195 - val_acc: 0.9296
Epoch 159/500
51s 102ms/step - loss: 0.4579 - acc: 0.9462 - val_loss: 0.5159 - val_acc: 0.9316
Epoch 160/500
51s 102ms/step - loss: 0.4515 - acc: 0.9467 - val_loss: 0.5058 - val_acc: 0.9299
Epoch 161/500
51s 102ms/step - loss: 0.4420 - acc: 0.9485 - val_loss: 0.5039 - val_acc: 0.9296
Epoch 162/500
51s 102ms/step - loss: 0.4357 - acc: 0.9485 - val_loss: 0.4972 - val_acc: 0.9325
Epoch 163/500
51s 102ms/step - loss: 0.4238 - acc: 0.9514 - val_loss: 0.4934 - val_acc: 0.9320
Epoch 164/500
51s 102ms/step - loss: 0.4187 - acc: 0.9523 - val_loss: 0.4884 - val_acc: 0.9314
Epoch 165/500
51s 102ms/step - loss: 0.4139 - acc: 0.9523 - val_loss: 0.4825 - val_acc: 0.9311
Epoch 166/500
51s 102ms/step - loss: 0.4071 - acc: 0.9538 - val_loss: 0.4861 - val_acc: 0.9308
Epoch 167/500
51s 102ms/step - loss: 0.4004 - acc: 0.9539 - val_loss: 0.4780 - val_acc: 0.9292
Epoch 168/500
51s 102ms/step - loss: 0.3950 - acc: 0.9550 - val_loss: 0.4754 - val_acc: 0.9311
Epoch 169/500
51s 102ms/step - loss: 0.3948 - acc: 0.9541 - val_loss: 0.4717 - val_acc: 0.9328
Epoch 170/500
51s 102ms/step - loss: 0.3828 - acc: 0.9570 - val_loss: 0.4741 - val_acc: 0.9313
Epoch 171/500
51s 102ms/step - loss: 0.3831 - acc: 0.9554 - val_loss: 0.4666 - val_acc: 0.9302
Epoch 172/500
51s 102ms/step - loss: 0.3781 - acc: 0.9561 - val_loss: 0.4653 - val_acc: 0.9324
Epoch 173/500
51s 102ms/step - loss: 0.3707 - acc: 0.9575 - val_loss: 0.4633 - val_acc: 0.9300
Epoch 174/500
51s 102ms/step - loss: 0.3679 - acc: 0.9573 - val_loss: 0.4688 - val_acc: 0.9262
Epoch 175/500
51s 102ms/step - loss: 0.3621 - acc: 0.9585 - val_loss: 0.4521 - val_acc: 0.9307
Epoch 176/500
51s 102ms/step - loss: 0.3619 - acc: 0.9571 - val_loss: 0.4465 - val_acc: 0.9329ETA: 42s - loss: 0.3637 - acc: 0.9586
Epoch 177/500
51s 102ms/step - loss: 0.3586 - acc: 0.9569 - val_loss: 0.4481 - val_acc: 0.9315
Epoch 178/500
51s 102ms/step - loss: 0.3485 - acc: 0.9598 - val_loss: 0.4531 - val_acc: 0.9305
Epoch 179/500
51s 102ms/step - loss: 0.3469 - acc: 0.9590 - val_loss: 0.4452 - val_acc: 0.9324
Epoch 180/500
51s 102ms/step - loss: 0.3456 - acc: 0.9590 - val_loss: 0.4466 - val_acc: 0.9312
Epoch 181/500
51s 102ms/step - loss: 0.3440 - acc: 0.9584 - val_loss: 0.4395 - val_acc: 0.9324
Epoch 182/500
51s 102ms/step - loss: 0.3393 - acc: 0.9595 - val_loss: 0.4456 - val_acc: 0.9295
Epoch 183/500
51s 102ms/step - loss: 0.3372 - acc: 0.9591 - val_loss: 0.4441 - val_acc: 0.9268
Epoch 184/500
51s 102ms/step - loss: 0.3342 - acc: 0.9592 - val_loss: 0.4347 - val_acc: 0.9283
Epoch 185/500
51s 102ms/step - loss: 0.3313 - acc: 0.9604 - val_loss: 0.4392 - val_acc: 0.9277
Epoch 186/500
51s 102ms/step - loss: 0.3245 - acc: 0.9618 - val_loss: 0.4281 - val_acc: 0.9318
Epoch 187/500
51s 102ms/step - loss: 0.3279 - acc: 0.9596 - val_loss: 0.4449 - val_acc: 0.9266
Epoch 188/500
51s 102ms/step - loss: 0.3244 - acc: 0.9593 - val_loss: 0.4367 - val_acc: 0.9262
Epoch 189/500
51s 102ms/step - loss: 0.3223 - acc: 0.9601 - val_loss: 0.4250 - val_acc: 0.9316
Epoch 190/500
51s 102ms/step - loss: 0.3197 - acc: 0.9597 - val_loss: 0.4245 - val_acc: 0.9273
Epoch 191/500
51s 102ms/step - loss: 0.3179 - acc: 0.9591 - val_loss: 0.4279 - val_acc: 0.9299
Epoch 192/500
51s 102ms/step - loss: 0.3166 - acc: 0.9603 - val_loss: 0.4243 - val_acc: 0.9285
Epoch 193/500
51s 102ms/step - loss: 0.3156 - acc: 0.9600 - val_loss: 0.4209 - val_acc: 0.9299
Epoch 194/500
51s 103ms/step - loss: 0.3102 - acc: 0.9622 - val_loss: 0.4253 - val_acc: 0.9267
Epoch 195/500
51s 102ms/step - loss: 0.3044 - acc: 0.9627 - val_loss: 0.4237 - val_acc: 0.9283
Epoch 196/500
51s 102ms/step - loss: 0.3085 - acc: 0.9609 - val_loss: 0.4174 - val_acc: 0.9297
Epoch 197/500
51s 102ms/step - loss: 0.3044 - acc: 0.9617 - val_loss: 0.4135 - val_acc: 0.9283
Epoch 198/500
51s 102ms/step - loss: 0.2983 - acc: 0.9631 - val_loss: 0.4152 - val_acc: 0.9312
Epoch 199/500
51s 102ms/step - loss: 0.3054 - acc: 0.9601 - val_loss: 0.4156 - val_acc: 0.9296
Epoch 200/500
51s 102ms/step - loss: 0.3007 - acc: 0.9619 - val_loss: 0.4084 - val_acc: 0.9305
Epoch 201/500
51s 102ms/step - loss: 0.3018 - acc: 0.9608 - val_loss: 0.4119 - val_acc: 0.9285
Epoch 202/500
51s 102ms/step - loss: 0.3023 - acc: 0.9596 - val_loss: 0.4075 - val_acc: 0.9312
Epoch 203/500
51s 102ms/step - loss: 0.2984 - acc: 0.9608 - val_loss: 0.4125 - val_acc: 0.9315
Epoch 204/500
51s 102ms/step - loss: 0.2937 - acc: 0.9621 - val_loss: 0.4088 - val_acc: 0.9267
Epoch 205/500
51s 102ms/step - loss: 0.2918 - acc: 0.9620 - val_loss: 0.4138 - val_acc: 0.9258
Epoch 206/500
51s 102ms/step - loss: 0.2922 - acc: 0.9617 - val_loss: 0.4048 - val_acc: 0.9310
Epoch 207/500
51s 102ms/step - loss: 0.2914 - acc: 0.9615 - val_loss: 0.3929 - val_acc: 0.9343
Epoch 208/500
51s 102ms/step - loss: 0.2915 - acc: 0.9615 - val_loss: 0.4042 - val_acc: 0.9291
Epoch 209/500
51s 102ms/step - loss: 0.2880 - acc: 0.9619 - val_loss: 0.4065 - val_acc: 0.9257
Epoch 210/500
51s 102ms/step - loss: 0.2912 - acc: 0.9601 - val_loss: 0.4155 - val_acc: 0.9238
Epoch 211/500
51s 102ms/step - loss: 0.2896 - acc: 0.9616 - val_loss: 0.3948 - val_acc: 0.9304
Epoch 212/500
51s 102ms/step - loss: 0.2890 - acc: 0.9605 - val_loss: 0.4077 - val_acc: 0.9288
Epoch 213/500
51s 102ms/step - loss: 0.2913 - acc: 0.9596 - val_loss: 0.3965 - val_acc: 0.9282
Epoch 214/500
51s 102ms/step - loss: 0.2865 - acc: 0.9599 - val_loss: 0.4121 - val_acc: 0.9240
Epoch 215/500
51s 102ms/step - loss: 0.2841 - acc: 0.9610 - val_loss: 0.4025 - val_acc: 0.9302
Epoch 216/500
51s 102ms/step - loss: 0.2870 - acc: 0.9603 - val_loss: 0.4064 - val_acc: 0.9244
Epoch 217/500
51s 102ms/step - loss: 0.2811 - acc: 0.9610 - val_loss: 0.4086 - val_acc: 0.9276
Epoch 218/500
51s 102ms/step - loss: 0.2837 - acc: 0.9606 - val_loss: 0.3963 - val_acc: 0.9269
Epoch 219/500
51s 102ms/step - loss: 0.2801 - acc: 0.9623 - val_loss: 0.3981 - val_acc: 0.9290
Epoch 220/500
51s 102ms/step - loss: 0.2786 - acc: 0.9631 - val_loss: 0.4038 - val_acc: 0.9261
Epoch 221/500
51s 103ms/step - loss: 0.2792 - acc: 0.9617 - val_loss: 0.3992 - val_acc: 0.9275
Epoch 222/500
52s 104ms/step - loss: 0.2780 - acc: 0.9619 - val_loss: 0.3951 - val_acc: 0.9290
Epoch 223/500
51s 103ms/step - loss: 0.2816 - acc: 0.9602 - val_loss: 0.3909 - val_acc: 0.9290
Epoch 224/500
51s 102ms/step - loss: 0.2766 - acc: 0.9622 - val_loss: 0.4014 - val_acc: 0.9276
Epoch 225/500
51s 102ms/step - loss: 0.2801 - acc: 0.9612 - val_loss: 0.3954 - val_acc: 0.9264
Epoch 226/500
51s 102ms/step - loss: 0.2803 - acc: 0.9595 - val_loss: 0.3920 - val_acc: 0.9309
Epoch 227/500
51s 102ms/step - loss: 0.2759 - acc: 0.9608 - val_loss: 0.3968 - val_acc: 0.9281
Epoch 228/500
51s 102ms/step - loss: 0.2814 - acc: 0.9586 - val_loss: 0.3927 - val_acc: 0.9288
Epoch 229/500
51s 102ms/step - loss: 0.2732 - acc: 0.9636 - val_loss: 0.4051 - val_acc: 0.9249
Epoch 230/500
51s 102ms/step - loss: 0.2770 - acc: 0.9606 - val_loss: 0.3944 - val_acc: 0.9276
Epoch 231/500
51s 102ms/step - loss: 0.2765 - acc: 0.9616 - val_loss: 0.3943 - val_acc: 0.9279
Epoch 232/500
51s 102ms/step - loss: 0.2762 - acc: 0.9603 - val_loss: 0.3855 - val_acc: 0.9271
Epoch 233/500
51s 102ms/step - loss: 0.2747 - acc: 0.9609 - val_loss: 0.3948 - val_acc: 0.9272
Epoch 234/500
51s 102ms/step - loss: 0.2776 - acc: 0.9595 - val_loss: 0.3861 - val_acc: 0.9285
Epoch 235/500
51s 102ms/step - loss: 0.2763 - acc: 0.9606 - val_loss: 0.3802 - val_acc: 0.9301
Epoch 236/500
51s 102ms/step - loss: 0.2747 - acc: 0.9616 - val_loss: 0.3883 - val_acc: 0.9295
Epoch 237/500
51s 102ms/step - loss: 0.2763 - acc: 0.9604 - val_loss: 0.3882 - val_acc: 0.9288
Epoch 238/500
51s 103ms/step - loss: 0.2699 - acc: 0.9616 - val_loss: 0.3973 - val_acc: 0.9260
Epoch 239/500
52s 103ms/step - loss: 0.2720 - acc: 0.9612 - val_loss: 0.3893 - val_acc: 0.9255
Epoch 240/500
51s 102ms/step - loss: 0.2758 - acc: 0.9595 - val_loss: 0.3975 - val_acc: 0.9267
Epoch 241/500
51s 102ms/step - loss: 0.2724 - acc: 0.9619 - val_loss: 0.3955 - val_acc: 0.9242
Epoch 242/500
51s 102ms/step - loss: 0.2704 - acc: 0.9622 - val_loss: 0.3889 - val_acc: 0.9263
Epoch 243/500
51s 102ms/step - loss: 0.2699 - acc: 0.9612 - val_loss: 0.4002 - val_acc: 0.9267
Epoch 244/500
51s 102ms/step - loss: 0.2733 - acc: 0.9605 - val_loss: 0.3858 - val_acc: 0.9290
Epoch 245/500
51s 102ms/step - loss: 0.2733 - acc: 0.9603 - val_loss: 0.3944 - val_acc: 0.9272
Epoch 246/500
51s 102ms/step - loss: 0.2682 - acc: 0.9616 - val_loss: 0.3787 - val_acc: 0.9306
Epoch 247/500
51s 103ms/step - loss: 0.2679 - acc: 0.9623 - val_loss: 0.3862 - val_acc: 0.9281
Epoch 248/500
52s 103ms/step - loss: 0.2688 - acc: 0.9619 - val_loss: 0.3904 - val_acc: 0.9266
Epoch 249/500
52s 103ms/step - loss: 0.2683 - acc: 0.9626 - val_loss: 0.3898 - val_acc: 0.9269
Epoch 250/500
52s 103ms/step - loss: 0.2706 - acc: 0.9600 - val_loss: 0.3884 - val_acc: 0.9294
Epoch 251/500
52s 103ms/step - loss: 0.2681 - acc: 0.9612 - val_loss: 0.3848 - val_acc: 0.9289
Epoch 252/500
52s 103ms/step - loss: 0.2696 - acc: 0.9618 - val_loss: 0.3852 - val_acc: 0.9268
Epoch 253/500
52s 103ms/step - loss: 0.2717 - acc: 0.9603 - val_loss: 0.3837 - val_acc: 0.9270
Epoch 254/500
51s 102ms/step - loss: 0.2718 - acc: 0.9596 - val_loss: 0.3855 - val_acc: 0.9264
Epoch 255/500
51s 103ms/step - loss: 0.2658 - acc: 0.9626 - val_loss: 0.3890 - val_acc: 0.9286
Epoch 256/500
51s 102ms/step - loss: 0.2660 - acc: 0.9623 - val_loss: 0.3866 - val_acc: 0.9314
Epoch 257/500
51s 103ms/step - loss: 0.2713 - acc: 0.9605 - val_loss: 0.3863 - val_acc: 0.9271
Epoch 258/500
52s 103ms/step - loss: 0.2715 - acc: 0.9599 - val_loss: 0.3833 - val_acc: 0.9301
Epoch 259/500
52s 103ms/step - loss: 0.2705 - acc: 0.9596 - val_loss: 0.3897 - val_acc: 0.9257
Epoch 260/500
52s 103ms/step - loss: 0.2682 - acc: 0.9613 - val_loss: 0.3911 - val_acc: 0.9274
Epoch 261/500
52s 103ms/step - loss: 0.2692 - acc: 0.9601 - val_loss: 0.3798 - val_acc: 0.9292
Epoch 262/500
52s 103ms/step - loss: 0.2650 - acc: 0.9622 - val_loss: 0.3910 - val_acc: 0.9270
Epoch 263/500
52s 103ms/step - loss: 0.2639 - acc: 0.9620 - val_loss: 0.3870 - val_acc: 0.9285
Epoch 264/500
52s 103ms/step - loss: 0.2651 - acc: 0.9620 - val_loss: 0.3994 - val_acc: 0.9263
Epoch 265/500
52s 103ms/step - loss: 0.2694 - acc: 0.9601 - val_loss: 0.3807 - val_acc: 0.9314
Epoch 266/500
52s 103ms/step - loss: 0.2691 - acc: 0.9613 - val_loss: 0.3811 - val_acc: 0.9282
Epoch 267/500
52s 103ms/step - loss: 0.2618 - acc: 0.9631 - val_loss: 0.3789 - val_acc: 0.9334
Epoch 268/500
51s 102ms/step - loss: 0.2636 - acc: 0.9631 - val_loss: 0.3905 - val_acc: 0.9254
Epoch 269/500
51s 102ms/step - loss: 0.2695 - acc: 0.9609 - val_loss: 0.4035 - val_acc: 0.9235
Epoch 270/500
51s 102ms/step - loss: 0.2661 - acc: 0.9615 - val_loss: 0.3942 - val_acc: 0.9264
Epoch 271/500
51s 102ms/step - loss: 0.2640 - acc: 0.9623 - val_loss: 0.3878 - val_acc: 0.9294
Epoch 272/500
52s 103ms/step - loss: 0.2658 - acc: 0.9620 - val_loss: 0.3952 - val_acc: 0.9260
Epoch 273/500
52s 103ms/step - loss: 0.2654 - acc: 0.9623 - val_loss: 0.4001 - val_acc: 0.9225
Epoch 274/500
51s 102ms/step - loss: 0.2648 - acc: 0.9621 - val_loss: 0.3885 - val_acc: 0.9285
Epoch 275/500
51s 102ms/step - loss: 0.2618 - acc: 0.9625 - val_loss: 0.3925 - val_acc: 0.9260
Epoch 276/500
51s 103ms/step - loss: 0.2615 - acc: 0.9628 - val_loss: 0.3860 - val_acc: 0.9279
Epoch 277/500
51s 103ms/step - loss: 0.2740 - acc: 0.9584 - val_loss: 0.3831 - val_acc: 0.9279
Epoch 278/500
51s 102ms/step - loss: 0.2653 - acc: 0.9616 - val_loss: 0.3898 - val_acc: 0.9257
Epoch 279/500
51s 102ms/step - loss: 0.2623 - acc: 0.9630 - val_loss: 0.3902 - val_acc: 0.9271
Epoch 280/500
52s 103ms/step - loss: 0.2652 - acc: 0.9624 - val_loss: 0.3899 - val_acc: 0.9274
Epoch 281/500
51s 103ms/step - loss: 0.2674 - acc: 0.9614 - val_loss: 0.3948 - val_acc: 0.9232
Epoch 282/500
51s 102ms/step - loss: 0.2641 - acc: 0.9622 - val_loss: 0.3880 - val_acc: 0.9284
Epoch 283/500
51s 103ms/step - loss: 0.2689 - acc: 0.9600 - val_loss: 0.3856 - val_acc: 0.9281
Epoch 284/500
51s 102ms/step - loss: 0.2616 - acc: 0.9618 - val_loss: 0.3795 - val_acc: 0.9294
Epoch 285/500
51s 102ms/step - loss: 0.2645 - acc: 0.9618 - val_loss: 0.3904 - val_acc: 0.9270
Epoch 286/500
51s 102ms/step - loss: 0.2656 - acc: 0.9612 - val_loss: 0.3848 - val_acc: 0.9264
Epoch 287/500
51s 102ms/step - loss: 0.2645 - acc: 0.9621 - val_loss: 0.3850 - val_acc: 0.9273
Epoch 288/500
51s 103ms/step - loss: 0.2607 - acc: 0.9628 - val_loss: 0.3806 - val_acc: 0.9295
Epoch 289/500
51s 102ms/step - loss: 0.2627 - acc: 0.9619 - val_loss: 0.3875 - val_acc: 0.9279
Epoch 290/500
51s 103ms/step - loss: 0.2641 - acc: 0.9618 - val_loss: 0.3967 - val_acc: 0.9284
Epoch 291/500
52s 103ms/step - loss: 0.2624 - acc: 0.9633 - val_loss: 0.3906 - val_acc: 0.9253
Epoch 292/500
51s 103ms/step - loss: 0.2651 - acc: 0.9623 - val_loss: 0.3843 - val_acc: 0.9273
Epoch 293/500
52s 103ms/step - loss: 0.2604 - acc: 0.9632 - val_loss: 0.3887 - val_acc: 0.9264
Epoch 294/500
51s 103ms/step - loss: 0.2605 - acc: 0.9633 - val_loss: 0.3969 - val_acc: 0.9244
Epoch 295/500
51s 102ms/step - loss: 0.2636 - acc: 0.9615 - val_loss: 0.3853 - val_acc: 0.9295
Epoch 296/500
51s 103ms/step - loss: 0.2620 - acc: 0.9625 - val_loss: 0.3993 - val_acc: 0.9248
Epoch 297/500
52s 103ms/step - loss: 0.2657 - acc: 0.9616 - val_loss: 0.3840 - val_acc: 0.9283
Epoch 298/500
52s 103ms/step - loss: 0.2630 - acc: 0.9627 - val_loss: 0.3899 - val_acc: 0.9257
Epoch 299/500
52s 103ms/step - loss: 0.2623 - acc: 0.9619 - val_loss: 0.3894 - val_acc: 0.9249
Epoch 300/500
52s 103ms/step - loss: 0.2709 - acc: 0.9589 - val_loss: 0.3775 - val_acc: 0.9304
Epoch 301/500
lr changed to 0.0009999999776482583
52s 103ms/step - loss: 0.2329 - acc: 0.9743 - val_loss: 0.3555 - val_acc: 0.9362
Epoch 302/500
52s 103ms/step - loss: 0.2176 - acc: 0.9795 - val_loss: 0.3499 - val_acc: 0.9380
Epoch 303/500
52s 103ms/step - loss: 0.2121 - acc: 0.9810 - val_loss: 0.3486 - val_acc: 0.9394
Epoch 304/500
52s 103ms/step - loss: 0.2088 - acc: 0.9825 - val_loss: 0.3498 - val_acc: 0.9401
Epoch 305/500
52s 103ms/step - loss: 0.2045 - acc: 0.9837 - val_loss: 0.3475 - val_acc: 0.9406
Epoch 306/500
52s 103ms/step - loss: 0.2032 - acc: 0.9846 - val_loss: 0.3479 - val_acc: 0.9404
Epoch 307/500
52s 103ms/step - loss: 0.2013 - acc: 0.9845 - val_loss: 0.3481 - val_acc: 0.9407
Epoch 308/500
51s 103ms/step - loss: 0.2013 - acc: 0.9846 - val_loss: 0.3519 - val_acc: 0.9397
Epoch 309/500
51s 103ms/step - loss: 0.1980 - acc: 0.9865 - val_loss: 0.3485 - val_acc: 0.9411
Epoch 310/500
52s 103ms/step - loss: 0.1949 - acc: 0.9868 - val_loss: 0.3500 - val_acc: 0.9407
Epoch 311/500
51s 102ms/step - loss: 0.1946 - acc: 0.9866 - val_loss: 0.3517 - val_acc: 0.9394
Epoch 312/500
51s 102ms/step - loss: 0.1950 - acc: 0.9864 - val_loss: 0.3522 - val_acc: 0.9395
Epoch 313/500
51s 103ms/step - loss: 0.1925 - acc: 0.9874 - val_loss: 0.3497 - val_acc: 0.9406
Epoch 314/500
52s 103ms/step - loss: 0.1917 - acc: 0.9873 - val_loss: 0.3515 - val_acc: 0.9400
Epoch 315/500
51s 102ms/step - loss: 0.1919 - acc: 0.9876 - val_loss: 0.3528 - val_acc: 0.9391
Epoch 316/500
51s 102ms/step - loss: 0.1914 - acc: 0.9872 - val_loss: 0.3521 - val_acc: 0.9391
Epoch 317/500
51s 103ms/step - loss: 0.1891 - acc: 0.9883 - val_loss: 0.3520 - val_acc: 0.9387
Epoch 318/500
51s 102ms/step - loss: 0.1891 - acc: 0.9883 - val_loss: 0.3529 - val_acc: 0.9389
Epoch 319/500
51s 102ms/step - loss: 0.1885 - acc: 0.9881 - val_loss: 0.3531 - val_acc: 0.9402
Epoch 320/500
51s 102ms/step - loss: 0.1866 - acc: 0.9890 - val_loss: 0.3549 - val_acc: 0.9397
Epoch 321/500
51s 102ms/step - loss: 0.1869 - acc: 0.9885 - val_loss: 0.3530 - val_acc: 0.9401
Epoch 322/500
51s 103ms/step - loss: 0.1871 - acc: 0.9873 - val_loss: 0.3572 - val_acc: 0.9394
Epoch 323/500
52s 103ms/step - loss: 0.1862 - acc: 0.9881 - val_loss: 0.3538 - val_acc: 0.9403
Epoch 324/500
51s 103ms/step - loss: 0.1861 - acc: 0.9883 - val_loss: 0.3545 - val_acc: 0.9394
Epoch 325/500
51s 102ms/step - loss: 0.1832 - acc: 0.9894 - val_loss: 0.3555 - val_acc: 0.9394
Epoch 326/500
51s 102ms/step - loss: 0.1813 - acc: 0.9904 - val_loss: 0.3542 - val_acc: 0.9399
Epoch 327/500
51s 102ms/step - loss: 0.1842 - acc: 0.9887 - val_loss: 0.3552 - val_acc: 0.9382
Epoch 328/500
51s 102ms/step - loss: 0.1825 - acc: 0.9890 - val_loss: 0.3541 - val_acc: 0.9402
Epoch 329/500
51s 102ms/step - loss: 0.1804 - acc: 0.9899 - val_loss: 0.3554 - val_acc: 0.9397
Epoch 330/500
51s 103ms/step - loss: 0.1816 - acc: 0.9896 - val_loss: 0.3549 - val_acc: 0.9403
Epoch 331/500
51s 102ms/step - loss: 0.1804 - acc: 0.9899 - val_loss: 0.3518 - val_acc: 0.9407
Epoch 332/500
51s 102ms/step - loss: 0.1806 - acc: 0.9900 - val_loss: 0.3515 - val_acc: 0.9419
Epoch 333/500
51s 102ms/step - loss: 0.1786 - acc: 0.9903 - val_loss: 0.3517 - val_acc: 0.9401
Epoch 334/500
51s 102ms/step - loss: 0.1812 - acc: 0.9894 - val_loss: 0.3529 - val_acc: 0.9407
Epoch 335/500
51s 102ms/step - loss: 0.1796 - acc: 0.9894 - val_loss: 0.3528 - val_acc: 0.9424
Epoch 336/500
51s 102ms/step - loss: 0.1792 - acc: 0.9896 - val_loss: 0.3534 - val_acc: 0.9400
Epoch 337/500
51s 102ms/step - loss: 0.1777 - acc: 0.9902 - val_loss: 0.3531 - val_acc: 0.9401
Epoch 338/500
51s 102ms/step - loss: 0.1774 - acc: 0.9904 - val_loss: 0.3525 - val_acc: 0.9392
Epoch 339/500
52s 103ms/step - loss: 0.1768 - acc: 0.9905 - val_loss: 0.3517 - val_acc: 0.9414
Epoch 340/500
51s 103ms/step - loss: 0.1781 - acc: 0.9899 - val_loss: 0.3540 - val_acc: 0.9417
Epoch 341/500
51s 102ms/step - loss: 0.1750 - acc: 0.9908 - val_loss: 0.3574 - val_acc: 0.9396
Epoch 342/500
51s 102ms/step - loss: 0.1756 - acc: 0.9906 - val_loss: 0.3560 - val_acc: 0.9407
Epoch 343/500
51s 102ms/step - loss: 0.1758 - acc: 0.9908 - val_loss: 0.3568 - val_acc: 0.9401
Epoch 344/500
51s 102ms/step - loss: 0.1767 - acc: 0.9904 - val_loss: 0.3518 - val_acc: 0.9418
Epoch 345/500
51s 102ms/step - loss: 0.1748 - acc: 0.9911 - val_loss: 0.3542 - val_acc: 0.9399
Epoch 346/500
51s 102ms/step - loss: 0.1737 - acc: 0.9909 - val_loss: 0.3557 - val_acc: 0.9399
Epoch 347/500
51s 102ms/step - loss: 0.1727 - acc: 0.9915 - val_loss: 0.3572 - val_acc: 0.9395
Epoch 348/500
51s 102ms/step - loss: 0.1722 - acc: 0.9914 - val_loss: 0.3557 - val_acc: 0.9395
Epoch 349/500
51s 102ms/step - loss: 0.1743 - acc: 0.9904 - val_loss: 0.3515 - val_acc: 0.9407
Epoch 350/500
51s 102ms/step - loss: 0.1730 - acc: 0.9910 - val_loss: 0.3529 - val_acc: 0.9395
Epoch 351/500
51s 102ms/step - loss: 0.1732 - acc: 0.9906 - val_loss: 0.3518 - val_acc: 0.9390
Epoch 352/500
51s 102ms/step - loss: 0.1724 - acc: 0.9911 - val_loss: 0.3526 - val_acc: 0.9409
Epoch 353/500
51s 102ms/step - loss: 0.1705 - acc: 0.9913 - val_loss: 0.3529 - val_acc: 0.9404
Epoch 354/500
51s 103ms/step - loss: 0.1712 - acc: 0.9912 - val_loss: 0.3517 - val_acc: 0.9399
Epoch 355/500
51s 102ms/step - loss: 0.1697 - acc: 0.9915 - val_loss: 0.3525 - val_acc: 0.9416
Epoch 356/500
51s 102ms/step - loss: 0.1709 - acc: 0.9906 - val_loss: 0.3489 - val_acc: 0.9410
Epoch 357/500
51s 102ms/step - loss: 0.1689 - acc: 0.9921 - val_loss: 0.3503 - val_acc: 0.9412
Epoch 358/500
51s 102ms/step - loss: 0.1692 - acc: 0.9914 - val_loss: 0.3512 - val_acc: 0.9410
Epoch 359/500
51s 102ms/step - loss: 0.1703 - acc: 0.9914 - val_loss: 0.3504 - val_acc: 0.9407
Epoch 360/500
51s 102ms/step - loss: 0.1705 - acc: 0.9910 - val_loss: 0.3535 - val_acc: 0.9412
Epoch 361/500
51s 102ms/step - loss: 0.1684 - acc: 0.9922 - val_loss: 0.3494 - val_acc: 0.9413
Epoch 362/500
51s 102ms/step - loss: 0.1682 - acc: 0.9922 - val_loss: 0.3516 - val_acc: 0.9401
Epoch 363/500
51s 102ms/step - loss: 0.1687 - acc: 0.9910 - val_loss: 0.3511 - val_acc: 0.9403
Epoch 364/500
51s 102ms/step - loss: 0.1679 - acc: 0.9917 - val_loss: 0.3539 - val_acc: 0.9394
Epoch 365/500
51s 102ms/step - loss: 0.1685 - acc: 0.9909 - val_loss: 0.3545 - val_acc: 0.9389
Epoch 366/500
51s 102ms/step - loss: 0.1672 - acc: 0.9914 - val_loss: 0.3546 - val_acc: 0.9385
Epoch 367/500
51s 102ms/step - loss: 0.1663 - acc: 0.9921 - val_loss: 0.3563 - val_acc: 0.9391
Epoch 368/500
51s 102ms/step - loss: 0.1658 - acc: 0.9920 - val_loss: 0.3546 - val_acc: 0.9398
Epoch 369/500
51s 102ms/step - loss: 0.1675 - acc: 0.9908 - val_loss: 0.3553 - val_acc: 0.9393
Epoch 370/500
51s 102ms/step - loss: 0.1659 - acc: 0.9916 - val_loss: 0.3556 - val_acc: 0.9387
Epoch 371/500
51s 102ms/step - loss: 0.1656 - acc: 0.9915 - val_loss: 0.3538 - val_acc: 0.9386
Epoch 372/500
51s 102ms/step - loss: 0.1651 - acc: 0.9918 - val_loss: 0.3547 - val_acc: 0.9400
Epoch 373/500
51s 102ms/step - loss: 0.1656 - acc: 0.9916 - val_loss: 0.3567 - val_acc: 0.9399
Epoch 374/500
51s 102ms/step - loss: 0.1625 - acc: 0.9931 - val_loss: 0.3539 - val_acc: 0.9399
Epoch 375/500
51s 102ms/step - loss: 0.1644 - acc: 0.9920 - val_loss: 0.3547 - val_acc: 0.9399
Epoch 376/500
51s 102ms/step - loss: 0.1632 - acc: 0.9920 - val_loss: 0.3592 - val_acc: 0.9378
Epoch 377/500
51s 102ms/step - loss: 0.1620 - acc: 0.9927 - val_loss: 0.3564 - val_acc: 0.9392
Epoch 378/500
51s 102ms/step - loss: 0.1633 - acc: 0.9926 - val_loss: 0.3542 - val_acc: 0.9378
Epoch 379/500
51s 102ms/step - loss: 0.1629 - acc: 0.9925 - val_loss: 0.3539 - val_acc: 0.9391
Epoch 380/500
51s 102ms/step - loss: 0.1614 - acc: 0.9924 - val_loss: 0.3527 - val_acc: 0.9389
Epoch 381/500
51s 102ms/step - loss: 0.1632 - acc: 0.9917 - val_loss: 0.3532 - val_acc: 0.9400
Epoch 382/500
51s 102ms/step - loss: 0.1627 - acc: 0.9922 - val_loss: 0.3520 - val_acc: 0.9411
Epoch 383/500
51s 102ms/step - loss: 0.1615 - acc: 0.9922 - val_loss: 0.3541 - val_acc: 0.9399
Epoch 384/500
51s 102ms/step - loss: 0.1611 - acc: 0.9928 - val_loss: 0.3528 - val_acc: 0.9395
Epoch 385/500
51s 102ms/step - loss: 0.1613 - acc: 0.9921 - val_loss: 0.3521 - val_acc: 0.9406
Epoch 386/500
51s 102ms/step - loss: 0.1612 - acc: 0.9918 - val_loss: 0.3535 - val_acc: 0.9400
Epoch 387/500
51s 102ms/step - loss: 0.1606 - acc: 0.9927 - val_loss: 0.3505 - val_acc: 0.9412
Epoch 388/500
51s 102ms/step - loss: 0.1607 - acc: 0.9919 - val_loss: 0.3497 - val_acc: 0.9419
Epoch 389/500
51s 102ms/step - loss: 0.1589 - acc: 0.9925 - val_loss: 0.3491 - val_acc: 0.9431
Epoch 390/500
51s 102ms/step - loss: 0.1604 - acc: 0.9923 - val_loss: 0.3523 - val_acc: 0.9429
Epoch 391/500
51s 102ms/step - loss: 0.1588 - acc: 0.9927 - val_loss: 0.3509 - val_acc: 0.9417
Epoch 392/500
51s 102ms/step - loss: 0.1596 - acc: 0.9924 - val_loss: 0.3496 - val_acc: 0.9404
Epoch 393/500
51s 102ms/step - loss: 0.1584 - acc: 0.9928 - val_loss: 0.3477 - val_acc: 0.9425
Epoch 394/500
51s 102ms/step - loss: 0.1588 - acc: 0.9921 - val_loss: 0.3503 - val_acc: 0.9397
Epoch 395/500
51s 103ms/step - loss: 0.1582 - acc: 0.9925 - val_loss: 0.3512 - val_acc: 0.9399
Epoch 396/500
51s 102ms/step - loss: 0.1570 - acc: 0.9931 - val_loss: 0.3508 - val_acc: 0.9406
Epoch 397/500
51s 102ms/step - loss: 0.1573 - acc: 0.9927 - val_loss: 0.3472 - val_acc: 0.9397
Epoch 398/500
51s 102ms/step - loss: 0.1562 - acc: 0.9928 - val_loss: 0.3480 - val_acc: 0.9408ETA: 0s - loss: 0.1562 - acc: 0.9928
Epoch 399/500
51s 102ms/step - loss: 0.1569 - acc: 0.9930 - val_loss: 0.3514 - val_acc: 0.9402
Epoch 400/500
51s 102ms/step - loss: 0.1557 - acc: 0.9932 - val_loss: 0.3516 - val_acc: 0.9390
Epoch 401/500
51s 102ms/step - loss: 0.1567 - acc: 0.9929 - val_loss: 0.3554 - val_acc: 0.9403
Epoch 402/500
51s 102ms/step - loss: 0.1572 - acc: 0.9922 - val_loss: 0.3525 - val_acc: 0.9385
Epoch 403/500
51s 102ms/step - loss: 0.1560 - acc: 0.9929 - val_loss: 0.3521 - val_acc: 0.9395
Epoch 404/500
51s 102ms/step - loss: 0.1564 - acc: 0.9927 - val_loss: 0.3491 - val_acc: 0.9402
Epoch 405/500
51s 103ms/step - loss: 0.1543 - acc: 0.9932 - val_loss: 0.3494 - val_acc: 0.9417
Epoch 406/500
51s 102ms/step - loss: 0.1553 - acc: 0.9926 - val_loss: 0.3483 - val_acc: 0.9408
Epoch 407/500
51s 102ms/step - loss: 0.1548 - acc: 0.9930 - val_loss: 0.3520 - val_acc: 0.9399
Epoch 408/500
51s 102ms/step - loss: 0.1548 - acc: 0.9929 - val_loss: 0.3523 - val_acc: 0.9411
Epoch 409/500
51s 102ms/step - loss: 0.1552 - acc: 0.9926 - val_loss: 0.3506 - val_acc: 0.9399
Epoch 410/500
51s 102ms/step - loss: 0.1530 - acc: 0.9939 - val_loss: 0.3460 - val_acc: 0.9406
Epoch 411/500
51s 102ms/step - loss: 0.1549 - acc: 0.9924 - val_loss: 0.3445 - val_acc: 0.9397
Epoch 412/500
51s 102ms/step - loss: 0.1533 - acc: 0.9931 - val_loss: 0.3445 - val_acc: 0.9410
Epoch 413/500
51s 102ms/step - loss: 0.1534 - acc: 0.9928 - val_loss: 0.3471 - val_acc: 0.9406
Epoch 414/500
51s 102ms/step - loss: 0.1542 - acc: 0.9922 - val_loss: 0.3478 - val_acc: 0.9417
Epoch 415/500
51s 102ms/step - loss: 0.1518 - acc: 0.9934 - val_loss: 0.3527 - val_acc: 0.9414
Epoch 416/500
51s 102ms/step - loss: 0.1529 - acc: 0.9926 - val_loss: 0.3484 - val_acc: 0.9423
Epoch 417/500
51s 102ms/step - loss: 0.1533 - acc: 0.9928 - val_loss: 0.3468 - val_acc: 0.9421
Epoch 418/500
51s 103ms/step - loss: 0.1531 - acc: 0.9927 - val_loss: 0.3497 - val_acc: 0.9418
Epoch 419/500
51s 102ms/step - loss: 0.1519 - acc: 0.9931 - val_loss: 0.3500 - val_acc: 0.9413
Epoch 420/500
51s 102ms/step - loss: 0.1512 - acc: 0.9932 - val_loss: 0.3480 - val_acc: 0.9418
Epoch 421/500
51s 102ms/step - loss: 0.1512 - acc: 0.9931 - val_loss: 0.3491 - val_acc: 0.9414
Epoch 422/500
51s 102ms/step - loss: 0.1518 - acc: 0.9927 - val_loss: 0.3464 - val_acc: 0.9412
Epoch 423/500
51s 102ms/step - loss: 0.1505 - acc: 0.9935 - val_loss: 0.3495 - val_acc: 0.9400
Epoch 424/500
51s 102ms/step - loss: 0.1502 - acc: 0.9930 - val_loss: 0.3467 - val_acc: 0.9418
Epoch 425/500
51s 102ms/step - loss: 0.1513 - acc: 0.9928 - val_loss: 0.3472 - val_acc: 0.9391
Epoch 426/500
51s 102ms/step - loss: 0.1496 - acc: 0.9933 - val_loss: 0.3430 - val_acc: 0.9400
Epoch 427/500
51s 102ms/step - loss: 0.1490 - acc: 0.9935 - val_loss: 0.3452 - val_acc: 0.9398
Epoch 428/500
51s 102ms/step - loss: 0.1499 - acc: 0.9931 - val_loss: 0.3456 - val_acc: 0.9394
Epoch 429/500
51s 102ms/step - loss: 0.1496 - acc: 0.9933 - val_loss: 0.3405 - val_acc: 0.9426
Epoch 430/500
51s 102ms/step - loss: 0.1486 - acc: 0.9933 - val_loss: 0.3467 - val_acc: 0.9404
Epoch 431/500
51s 103ms/step - loss: 0.1481 - acc: 0.9933 - val_loss: 0.3448 - val_acc: 0.9408
Epoch 432/500
51s 102ms/step - loss: 0.1479 - acc: 0.9936 - val_loss: 0.3479 - val_acc: 0.9390
Epoch 433/500
51s 102ms/step - loss: 0.1485 - acc: 0.9934 - val_loss: 0.3484 - val_acc: 0.9380
Epoch 434/500
51s 102ms/step - loss: 0.1479 - acc: 0.9935 - val_loss: 0.3495 - val_acc: 0.9384
Epoch 435/500
51s 102ms/step - loss: 0.1468 - acc: 0.9940 - val_loss: 0.3490 - val_acc: 0.9425
Epoch 436/500
51s 103ms/step - loss: 0.1482 - acc: 0.9929 - val_loss: 0.3538 - val_acc: 0.9386
Epoch 437/500
51s 102ms/step - loss: 0.1478 - acc: 0.9932 - val_loss: 0.3494 - val_acc: 0.9400
Epoch 438/500
51s 102ms/step - loss: 0.1476 - acc: 0.9937 - val_loss: 0.3508 - val_acc: 0.9386
Epoch 439/500
51s 102ms/step - loss: 0.1483 - acc: 0.9932 - val_loss: 0.3483 - val_acc: 0.9386
Epoch 440/500
51s 103ms/step - loss: 0.1475 - acc: 0.9932 - val_loss: 0.3466 - val_acc: 0.9402
Epoch 441/500
51s 102ms/step - loss: 0.1473 - acc: 0.9932 - val_loss: 0.3477 - val_acc: 0.9406
Epoch 442/500
51s 102ms/step - loss: 0.1469 - acc: 0.9931 - val_loss: 0.3461 - val_acc: 0.9402
Epoch 443/500
51s 102ms/step - loss: 0.1460 - acc: 0.9936 - val_loss: 0.3483 - val_acc: 0.9404
Epoch 444/500
51s 102ms/step - loss: 0.1463 - acc: 0.9936 - val_loss: 0.3438 - val_acc: 0.9407
Epoch 445/500
51s 102ms/step - loss: 0.1454 - acc: 0.9936 - val_loss: 0.3488 - val_acc: 0.9401
Epoch 446/500
51s 102ms/step - loss: 0.1452 - acc: 0.9935 - val_loss: 0.3450 - val_acc: 0.9402
Epoch 447/500
51s 102ms/step - loss: 0.1452 - acc: 0.9933 - val_loss: 0.3494 - val_acc: 0.9401
Epoch 448/500
51s 102ms/step - loss: 0.1447 - acc: 0.9936 - val_loss: 0.3448 - val_acc: 0.9418
Epoch 449/500
51s 102ms/step - loss: 0.1446 - acc: 0.9936 - val_loss: 0.3507 - val_acc: 0.9403
Epoch 450/500
51s 103ms/step - loss: 0.1442 - acc: 0.9935 - val_loss: 0.3464 - val_acc: 0.9401
Epoch 451/500
lr changed to 9.999999310821295e-05
51s 102ms/step - loss: 0.1437 - acc: 0.9936 - val_loss: 0.3447 - val_acc: 0.9410
Epoch 452/500
51s 102ms/step - loss: 0.1435 - acc: 0.9938 - val_loss: 0.3450 - val_acc: 0.9408ETA: 19s - loss: 0.1439 - acc: 0.9936
Epoch 453/500
51s 102ms/step - loss: 0.1421 - acc: 0.9943 - val_loss: 0.3444 - val_acc: 0.9402
Epoch 454/500
51s 102ms/step - loss: 0.1419 - acc: 0.9945 - val_loss: 0.3443 - val_acc: 0.9407
Epoch 455/500
51s 102ms/step - loss: 0.1424 - acc: 0.9943 - val_loss: 0.3445 - val_acc: 0.9407
Epoch 456/500
51s 102ms/step - loss: 0.1425 - acc: 0.9942 - val_loss: 0.3442 - val_acc: 0.9406
Epoch 457/500
51s 102ms/step - loss: 0.1420 - acc: 0.9946 - val_loss: 0.3440 - val_acc: 0.9408
Epoch 458/500
51s 102ms/step - loss: 0.1422 - acc: 0.9938 - val_loss: 0.3443 - val_acc: 0.9407
Epoch 459/500
51s 102ms/step - loss: 0.1399 - acc: 0.9953 - val_loss: 0.3442 - val_acc: 0.9411
Epoch 460/500
51s 102ms/step - loss: 0.1417 - acc: 0.9944 - val_loss: 0.3446 - val_acc: 0.9415
Epoch 461/500
51s 102ms/step - loss: 0.1420 - acc: 0.9942 - val_loss: 0.3451 - val_acc: 0.9415
Epoch 462/500
51s 102ms/step - loss: 0.1402 - acc: 0.9949 - val_loss: 0.3443 - val_acc: 0.9413
Epoch 463/500
51s 102ms/step - loss: 0.1414 - acc: 0.9944 - val_loss: 0.3437 - val_acc: 0.9407
Epoch 464/500
51s 102ms/step - loss: 0.1411 - acc: 0.9947 - val_loss: 0.3442 - val_acc: 0.9408
Epoch 465/500
51s 102ms/step - loss: 0.1403 - acc: 0.9949 - val_loss: 0.3440 - val_acc: 0.9414
Epoch 466/500
51s 103ms/step - loss: 0.1406 - acc: 0.9948 - val_loss: 0.3439 - val_acc: 0.9408
Epoch 467/500
51s 102ms/step - loss: 0.1407 - acc: 0.9947 - val_loss: 0.3438 - val_acc: 0.9412
Epoch 468/500
51s 102ms/step - loss: 0.1419 - acc: 0.9942 - val_loss: 0.3438 - val_acc: 0.9413
Epoch 469/500
51s 102ms/step - loss: 0.1417 - acc: 0.9945 - val_loss: 0.3443 - val_acc: 0.9406
Epoch 470/500
51s 102ms/step - loss: 0.1409 - acc: 0.9945 - val_loss: 0.3439 - val_acc: 0.9407
Epoch 471/500
51s 102ms/step - loss: 0.1418 - acc: 0.9939 - val_loss: 0.3439 - val_acc: 0.9410
Epoch 472/500
51s 102ms/step - loss: 0.1407 - acc: 0.9948 - val_loss: 0.3434 - val_acc: 0.9408
Epoch 473/500
51s 102ms/step - loss: 0.1413 - acc: 0.9942 - val_loss: 0.3438 - val_acc: 0.9410
Epoch 474/500
51s 102ms/step - loss: 0.1394 - acc: 0.9950 - val_loss: 0.3439 - val_acc: 0.9413
Epoch 475/500
51s 102ms/step - loss: 0.1405 - acc: 0.9946 - val_loss: 0.3438 - val_acc: 0.9414
Epoch 476/500
51s 102ms/step - loss: 0.1396 - acc: 0.9949 - val_loss: 0.3438 - val_acc: 0.9411
Epoch 477/500
51s 102ms/step - loss: 0.1401 - acc: 0.9950 - val_loss: 0.3439 - val_acc: 0.9411
Epoch 478/500
51s 102ms/step - loss: 0.1406 - acc: 0.9947 - val_loss: 0.3442 - val_acc: 0.9408
Epoch 479/500
51s 102ms/step - loss: 0.1394 - acc: 0.9952 - val_loss: 0.3446 - val_acc: 0.9408
Epoch 480/500
51s 102ms/step - loss: 0.1399 - acc: 0.9950 - val_loss: 0.3447 - val_acc: 0.9402
Epoch 481/500
51s 102ms/step - loss: 0.1409 - acc: 0.9945 - val_loss: 0.3451 - val_acc: 0.9405
Epoch 482/500
51s 102ms/step - loss: 0.1392 - acc: 0.9955 - val_loss: 0.3446 - val_acc: 0.9406
Epoch 483/500
51s 102ms/step - loss: 0.1403 - acc: 0.9948 - val_loss: 0.3450 - val_acc: 0.9402
Epoch 484/500
51s 102ms/step - loss: 0.1396 - acc: 0.9950 - val_loss: 0.3450 - val_acc: 0.9403
Epoch 485/500
51s 102ms/step - loss: 0.1395 - acc: 0.9949 - val_loss: 0.3449 - val_acc: 0.9401
Epoch 486/500
51s 102ms/step - loss: 0.1413 - acc: 0.9944 - val_loss: 0.3450 - val_acc: 0.9402
Epoch 487/500
51s 103ms/step - loss: 0.1406 - acc: 0.9945 - val_loss: 0.3447 - val_acc: 0.9403
Epoch 488/500
51s 102ms/step - loss: 0.1403 - acc: 0.9943 - val_loss: 0.3448 - val_acc: 0.9409
Epoch 489/500
51s 102ms/step - loss: 0.1408 - acc: 0.9948 - val_loss: 0.3442 - val_acc: 0.9413
Epoch 490/500
51s 102ms/step - loss: 0.1398 - acc: 0.9951 - val_loss: 0.3438 - val_acc: 0.9416
Epoch 491/500
51s 102ms/step - loss: 0.1398 - acc: 0.9947 - val_loss: 0.3438 - val_acc: 0.9409
Epoch 492/500
51s 102ms/step - loss: 0.1396 - acc: 0.9951 - val_loss: 0.3437 - val_acc: 0.9414
Epoch 493/500
51s 102ms/step - loss: 0.1402 - acc: 0.9947 - val_loss: 0.3434 - val_acc: 0.9416
Epoch 494/500
51s 102ms/step - loss: 0.1400 - acc: 0.9951 - val_loss: 0.3441 - val_acc: 0.9413
Epoch 495/500
51s 102ms/step - loss: 0.1395 - acc: 0.9952 - val_loss: 0.3442 - val_acc: 0.9409
Epoch 496/500
51s 102ms/step - loss: 0.1395 - acc: 0.9948 - val_loss: 0.3446 - val_acc: 0.9413
Epoch 497/500
51s 102ms/step - loss: 0.1386 - acc: 0.9953 - val_loss: 0.3444 - val_acc: 0.9415
Epoch 498/500
51s 102ms/step - loss: 0.1396 - acc: 0.9951 - val_loss: 0.3440 - val_acc: 0.9415
Epoch 499/500
51s 102ms/step - loss: 0.1396 - acc: 0.9950 - val_loss: 0.3444 - val_acc: 0.9414
Epoch 500/500
51s 102ms/step - loss: 0.1386 - acc: 0.9951 - val_loss: 0.3449 - val_acc: 0.9417
Train loss: 0.128365815192461
Train accuracy: 0.9987200012207031
Test loss: 0.3449158281087875
Test accuracy: 0.941700000166893

相较于调参记录18的94.28%，这次的测试准确率低了一点。

但是，值得指出的是，这次只训练了500个epoch，而调参记录18训练了5000个epoch。通过观察loss可以发现，其实这次的loss下降得更快。

Minghang Zhao, Shisheng Zhong, Xuyun Fu, Baoping Tang, Shaojiang Dong, Michael Pecht, Deep Residual Networks with Adaptively Parametric Rectifier Linear Units for Fault Diagnosis, IEEE Transactions on Industrial Electronics, 2020, DOI: 10.1109/TIE.2020.2972458

https://ieeexplore.ieee.org/document/8998530

dangqing1988
关注
Resnet网络残差连接与relu顺序
u012189988的博客
08-18 3302
class Bottleneck(nn.Module): expansion = 4 def __init__(self, inplanes, planes, stride=1, downsample=None): super(Bottleneck, self).__init__() self.conv1 = nn.Conv2d(inplanes, planes, kernel_s...
深度残差网络+自适应参数化ReLU激活函数（调参记录22）Cifar10~95.25%
dangqing1988的博客
05-18 1518
本文在调参记录21的基础上，将残差模块的个数，从60个增加到120个，测试深度残差网络+自适应参数化ReLU激活函数在Cifar10数据集上的效果。 自适应参数化ReLU激活函数的基本原理如下： Keras程序： 实验结果： Minghang Zhao, Shisheng Zhong, Xuyun Fu, Baoping Tang, Shaojiang Dong, Michael Pecht, Deep Residual Networks with Adaptively Parametric Re
深度残差收缩网络：通过软阈值化与ReLU激活函数的对比，进行探讨
A609006332的博客
04-09 3375
本文介绍了一种新的深度学习方法——深度残差收缩网络(Deep Residual Shrinkage Network)，从软阈值函数在删除冗余特征时的灵活度，进行了探讨。 1.深度残差收缩网络的基础知识 从名字可以看出，深度残差收缩网络是残差网络的一种改进方法。其特色是“收缩”，在这里指的是软阈值化，而软阈值化几乎是现在信号降噪算法的必备步骤。 因此，深度残差收缩网络是一种面向含噪数据的深度学习方法...
relu函数_深度残差网络+自适应参数化ReLU激活函数（调参记录15）
weixin_40009026的博客
12-04 141
在调参记录14里，只有2个残差模块，结果遭遇欠拟合了。这次增加一个残差模块，试一试在Cifar10数据集上的效果。自适应参数化ReLU激活函数的基本原理如下：自适应参数化ReLU激活函数Keras程序如下：#!/usr/bin/env python3# -*- coding: utf-8 -*-"""Created on Tue Apr 14 04:17:45 2020Implemented us...
深度学习——残差神经网络ResNet在分别在Keras和tensorflow框架下的应用案例
loveliuzz的博客
01-21 4万+
一、残差神经网络——ResNet的综述 深度学习网络的深度对最后的分类和识别的效果有着很大的影响，所以正常想法就是能把网络设计的越深越好， 但是事实上却不是这样，常规的网络的堆叠（plain network）在网络很深的时候，效果却越来越差了。其中原因之一 即是网络越深，梯度消失的现象就越来越明显，网络的训练效果也不会很好。 但是现在浅层的网络（shallower network） 又
【模型解读】resnet中的残差连接，你确定真的看懂了？
hacker_long的专栏
10-08 4297
文章首发于微信公众号《与有三学AI》 【模型解读】resnet中的残差连接，你确定真的看懂了？ 这是深度学习模型解读第6篇，本篇我们将介绍深度学习模型中的残差连接。 作者&编辑 | 言有三 1 残差连接 想必做深度学习的都知道skipconnect，也就是残差连接，那什么是skipconnect呢？如下图 上面是来自于resnet【1】的skipblock的...
深度残差网络+自适应参数化ReLU激活函数（调参记录24）Cifar10~95.80%
dangqing1988的博客
05-20 2038
本文在调参记录23的基础上，增加卷积核的个数，最少是64个，最多是256个，继续测试深度残差网络+自适应参数化ReLU激活函数在cifar10数据集上的效果。 自适应参数化ReLU激活函数被放在了残差模块的第二个卷积层之后，它的基本原理如下： Keras程序： #!/usr/bin/env python3 # -*- coding: utf-8 -*- """ Created on Tue Apr 14 04:17:45 2020 Implemented using TensorFlow 1.0.1 an
深度残差网络+自适应参数化ReLU激活函数（调参记录19）Cifar10~93.96%
dangqing1988的博客
05-09 2160
本文修改了数据预处理的方式。在原先的代码中，首先通过除以255将每个样本归一化到0和1之间，然后再减去训练样本集的均值，代码如下： x_train = x_train.astype('float32') / 255. x_test = x_test.astype('float32') / 255. x_test = x_test-np.mean(x_train) x_train = x_train...
深度残差网络+自适应参数化ReLU激活函数（调参记录21）Cifar10~95.12%
dangqing1988的博客
05-17 935
本文在调参记录20的基础上，将残差模块的个数，从27个增加到60个，继续测试深度残差网络ResNet+自适应参数化ReLU激活函数在Cifar10数据集上的表现。 自适应参数化ReLU函数的原理如下 Keras程序如下： #!/usr/bin/env python3 # -*- coding: utf-8 -*- """ Created on Tue Apr 14 04:17:45 2020 Implemented using TensorFlow 1.0.1 and Keras 2.2.1 Mingh
深度残差网络+自适应参数化ReLU激活函数（调参记录4）
dangqing1988的博客
04-19 1909
续上一篇： 深度残差网络+自适应参数化ReLU激活函数（调参记录3） https://blog.csdn.net/dangqing1988/article/details/105601313 本文在深度残差网络中采用了自适应参数化ReLU激活函数，继续测试其在Cifar10图像集上的效果。与上一篇不同的是，这次修改了残差模块里面的结构，原先是两个3×3的卷积层，现在改成了1×1→3×3→1×1的瓶...
残差网络(ResNets)的残差块（Residual block）
csdn_xmj的博客
05-08 5万+
来源：Coursera吴恩达深度学习课程 五一假期结束了，听着梁博的《日落大道》，码字中。非常非常深的神经网络是很难训练的，因为存在梯度消失和梯度爆炸问题。跳跃连接（Skip connection）可以从某一层网络层获取激活，然后迅速反馈给另外一层，甚至是神经网络的更深层。利用跳跃连接构建能够训练深度网络的ResNets，有时深度能够超过100层。ResNets是由残差块（Residual block）构建的，首先看一下什么是残差块。 上图是一个两层神经网络。回顾之前的计算过程： 在残差网络中
深度残差网络+自适应参数化ReLU激活函数（调参记录9）Cifar10~93.71%
dangqing1988的博客
04-23 3124
本文在调参记录6的基础上，继续调整超参数，测试Adaptively Parametric ReLU（APReLU）激活函数在Cifar10图像集上的效果。 深度残差网络+自适应参数化ReLU激活函数（调参记录6） https://blog.csdn.net/dangqing1988/article/details/105628681 在Keras里，Batch Normalization的mom...
Python实现Keras搭建神经网络训练CNN模型
ZYYRWish_97的博客
06-03 2316
# CNN example import numpy as np # for reproducibility np.random.seed(1337) from keras.utils import np_utils from keras.models import Sequential from keras.layers import Dense, Activation, Conv2D, MaxPooling2D, Flatten from keras.optimizers import Adam .
深度学习——激活函数Sigmoid/Tanh/ReLU
热门推荐
zchang81的博客
04-18 8万+
一般情况下，使用ReLU会比较好 1、使用 ReLU，就要注意设置 learning rate，不要让网络训练过程中出现很多 “dead” 神经元； 2、如果“dead”无法解决，可以尝试 Leaky ReLU、PReLU 、RReLU等Relu变体来替代ReLU； 3、不建议使用 sigmoid，如果一定要使用，也可以用 tanh来替代。
LeetCode之Candy
SmartSi
01-25 4996
【题目】 There areNchildren standing in a line. Each child is assigned a rating value. You are giving candies to these children subjected to the following requirements: Each child must have at l...
07登录登出
xiaogeldx的博客
06-01 663
superuser 创建管理员账号：python manage.py createsuperuser，然后填用户名，手机号，密码 前端代码 base.html {% load static %} <!DOCTYPE html> <html> <head> <meta charset="utf-8"> {% block meta %}{%...
Java微信公众平台开发(六)——微信开发中的token获取
myitsite的博客
03-31 1万+
（一）token的介绍 引用：access_token是公众号的全局唯一票据，公众号调用各接口时都需使用access_token。开发者需要进行妥善保存。access_token的存储至少要保留512个字符空间。access_token的有效期目前为2个小时，需定时刷新，重复获取将导致上次获取的access_token失效！ （二）token的获取参考文档 获取的流程我们完全
深度残差网络+自适应参数化ReLU激活函数（调参记录10）
dangqing1988的博客
04-24 1807
本文在调参记录9的基础上，在数据增强部分添加了shear_range = 30，测试Adaptively Parametric ReLU（APReLU）激活函数在Cifar10图像集上的效果。 深度残差网络+自适应参数化ReLU激活函数（调参记录9） https://blog.csdn.net/dangqing1988/article/details/105688144 自适应参数化ReLU激活函...
深度残差网络+自适应参数化ReLU激活函数
最新发布
07-28
深度残差网络（Deep Residual Network）是一种深度学习网络结构，通过引入残差连接来解决深层网络训练过程中的梯度消失...深度残差网络和自适应参数化ReLU激活函数可以结合使用，以提高深度学习模型的性能和训练效果。
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值