Tensorflow2.0 搭建CNN网络进行猫狗分类（一）

在搭建好tensorflow和cuda的配置后，就可以搭建自己的神经网络了，我将使用tensorflow2.0来搭建一个cnn网络，搭建网络的方法有很多，我这边会使用多种方法搭建，通过在处理训练数据上，训练的方法上搭建多个神经网络来比较之间的差别，我将使用的第一种方法如下：

通过cv2手动读取每张图片，转换尺寸大小并划分数据集和测试集，对数据集进行one-hot编码后，搭建三层卷积神经网络模型，建立神经网络(平坦层、隐藏层、输出层)，通过调整上千万的参数，采用多分类损失函数等函数来调参，进行大约20次epoch左右的训练，对损失值和准确率的训练曲线画图，最后再手动选取几张图片，判断模型的大概性能和准确率

所需第三方库：

• numpy
• os
• cv2
• tensorflow2.0
• matplotlib
• keras

我的猫狗图片文件夹路径如下：

一级目录	二级目录
train	dog
train	cat

注意：我这里没有test测试集，我是从train训练集中划分出一部分当测试集和验证集

1. 制作训练集和测试集

制作的方法非常简单，依次读取文件目录下的每一张图片，读取的数量按照比例划分为训练集和测试集，所有的图片尺寸转换为100*100像素并转换为float64类型，然后转换为numpy数组，我的标签字典定义为{ 0 : ‘cat’, 1 : ‘dog’ }，下面是部分代码。

# 制作训练集和测试集
train_imgs = []
train_labels = []
test_imgs = []
test_labels = []

train_normalize_imgs = []
test_normalize_imgs = []

train_num = 10000       # 使用训练的图片数量
train_proportion = 0.9  # 训练集和测试集划分的比例
# 获取训练数据和训练标签
for label, path in enumerate(os.listdir(train_path)):
    images = os.listdir(os.path.join(train_path, path))
    images = images[:train_num]
    for index, img in enumerate(images):
        # 读取图片路径
        img = cv2.imread(os.path.join(train_path, path, img))
        img = cv2.resize(img, (100, 100), interpolation=cv2.INTER_AREA)
        # 判断训练数量，按训练比例分成训练集和测试集
        if index < train_num * train_proportion:
            train_labels.append(label)
            train_imgs.append(img)
            train_normalize_imgs.append(img.astype('float64'))
        else:
            test_labels.append(label)
            test_imgs.append(img)
            test_normalize_imgs.append(img.astype('float64'))

训练集和测试机的形状：

train data: images: (18000, 100, 100, 3)  labels: (18000,)
test  data: images: (2000, 100, 100, 3)  labels: (2000,)

2. 建立三层卷积神经网络模型

model = Sequential()

# 卷积层1与池化层1
model.add(Conv2D(filters=32, kernel_size=(3, 3), input_shape=(100, 100, 3),
                 activation='relu', padding='same'))

model.add(Conv2D(filters=32, kernel_size=(3, 3),
                 activation='relu', padding='same'))
model.add(MaxPooling2D(pool_size=(2, 2)))

# 卷积层2与池化层2
model.add(Conv2D(filters=64, kernel_size=(3, 3),
                 activation='relu', padding='same'))
model.add(Conv2D(filters=64, kernel_size=(3, 3),
                 activation='relu', padding='same'))
model.add(MaxPooling2D(pool_size=(2, 2)))

# 卷积层3与池化层3

model.add(Conv2D(filters=128, kernel_size=(3, 3),
                 activation='relu', padding='same'))
model.add(Conv2D(filters=128, kernel_size=(3, 3),
                 activation='relu', padding='same'))
model.add(MaxPooling2D(pool_size=(2, 2)))

# 建立神经网络(平坦层、隐藏层、输出层)
model.add(Flatten())
model.add(Dropout(0.15))
model.add(Dense(1000, activation='relu'))
model.add(Dropout(0.15))
model.add(Dense(500, activation='relu'))
model.add(Dropout(0.15))
model.add(Dense(2, activation='softmax'))

通过获取模型的参数图大概是这样

Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d (Conv2D)              (None, 100, 100, 32)      896       
_________________________________________________________________
conv2d_1 (Conv2D)            (None, 100, 100, 32)      9248      
_________________________________________________________________
max_pooling2d (MaxPooling2D) (None, 50, 50, 32)        0         
_________________________________________________________________
conv2d_2 (Conv2D)            (None, 50, 50, 64)        18496     
_________________________________________________________________
conv2d_3 (Conv2D)            (None, 50, 50, 64)        36928     
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 25, 25, 64)        0         
_________________________________________________________________
conv2d_4 (Conv2D)            (None, 25, 25, 128)       73856     
_________________________________________________________________
conv2d_5 (Conv2D)            (None, 25, 25, 128)       147584    
_________________________________________________________________
max_pooling2d_2 (MaxPooling2 (None, 12, 12, 128)       0         
_________________________________________________________________
flatten (Flatten)            (None, 18432)             0         
_________________________________________________________________
dropout (Dropout)            (None, 18432)             0         
_________________________________________________________________
dense (Dense)                (None, 1000)              18433000  
_________________________________________________________________
dropout_1 (Dropout)          (None, 1000)              0         
_________________________________________________________________
dense_1 (Dense)              (None, 500)               500500    
_________________________________________________________________
dropout_2 (Dropout)          (None, 500)               0         
_________________________________________________________________
dense_2 (Dense)              (None, 2)                 1002      
=================================================================
Total params: 19,221,510
Trainable params: 19,221,510
Non-trainable params: 0

3. 训练模型

# 训练模型
epochs = 20  # 训练次数
model.compile(loss='categorical_crossentropy', optimizer='adam',
              metrics=['accuracy'])

train_history = model.fit(train_normalize_imgs, train_labels_OneHot,
                          validation_split=0.0005,shuffle=True,
                          epochs=epochs, batch_size=256)

通过训练模型大概的进度条：

Epoch 1/20
71/71 [==============================] - 28s 395ms/step - loss: 12.9789 - accuracy: 0.5620 - val_loss: 0.9277 - val_accuracy: 0.2222
Epoch 2/20
71/71 [==============================] - 27s 381ms/step - loss: 0.6529 - accuracy: 0.6069 - val_loss: 0.7149 - val_accuracy: 0.3333
Epoch 3/20
71/71 [==============================] - 27s 382ms/step - loss: 0.6450 - accuracy: 0.6229 - val_loss: 0.6330 - val_accuracy: 0.7778
Epoch 4/20
71/71 [==============================] - 27s 383ms/step - loss: 0.5849 - accuracy: 0.6858 - val_loss: 0.7419 - val_accuracy: 0.4444
Epoch 5/20
71/71 [==============================] - 27s 384ms/step - loss: 0.5392 - accuracy: 0.7247 - val_loss: 0.5105 - val_accuracy: 0.6667
Epoch 6/20
71/71 [==============================] - 27s 385ms/step - loss: 0.4872 - accuracy: 0.7669 - val_loss: 0.3197 - val_accuracy: 0.7778
Epoch 7/20
71/71 [==============================] - 27s 387ms/step - loss: 0.4455 - accuracy: 0.7916 - val_loss: 0.4260 - val_accuracy: 0.6667
Epoch 8/20
71/71 [==============================] - 27s 387ms/step - loss: 0.3954 - accuracy: 0.8217 - val_loss: 0.4303 - val_accuracy: 0.7778
Epoch 9/20
71/71 [==============================] - 28s 387ms/step - loss: 0.3538 - accuracy: 0.8441 - val_loss: 0.2989 - val_accuracy: 0.7778
Epoch 10/20
71/71 [==============================] - 28s 387ms/step - loss: 0.3126 - accuracy: 0.8617 - val_loss: 0.1530 - val_accuracy: 1.0000
Epoch 11/20
71/71 [==============================] - 28s 388ms/step - loss: 0.2473 - accuracy: 0.8958 - val_loss: 0.3851 - val_accuracy: 0.8889
Epoch 12/20
71/71 [==============================] - 28s 388ms/step - loss: 0.1989 - accuracy: 0.9187 - val_loss: 0.1342 - val_accuracy: 1.0000
Epoch 13/20
71/71 [==============================] - 28s 388ms/step - loss: 0.1620 - accuracy: 0.9361 - val_loss: 0.8813 - val_accuracy: 0.8889
Epoch 14/20
71/71 [==============================] - 28s 388ms/step - loss: 0.1206 - accuracy: 0.9548 - val_loss: 0.6381 - val_accuracy: 0.8889
Epoch 15/20
71/71 [==============================] - 28s 388ms/step - loss: 0.1035 - accuracy: 0.9624 - val_loss: 0.1802 - val_accuracy: 0.8889
Epoch 16/20
71/71 [==============================] - 28s 389ms/step - loss: 0.0796 - accuracy: 0.9708 - val_loss: 0.3545 - val_accuracy: 0.7778
Epoch 17/20
71/71 [==============================] - 28s 389ms/step - loss: 0.0852 - accuracy: 0.9685 - val_loss: 0.0121 - val_accuracy: 1.0000
Epoch 18/20
71/71 [==============================] - 28s 389ms/step - loss: 0.0644 - accuracy: 0.9769 - val_loss: 0.0853 - val_accuracy: 1.0000
Epoch 19/20
71/71 [==============================] - 28s 389ms/step - loss: 0.0681 - accuracy: 0.9762 - val_loss: 0.6664 - val_accuracy: 0.8889
Epoch 20/20
71/71 [==============================] - 28s 388ms/step - loss: 0.0514 - accuracy: 0.9837 - val_loss: 0.0181 - val_accuracy: 1.0000

4. 评估模型的准确率

def show_train_history(train_history, train_acc, test_acc):
    plt.plot(train_history.history[train_acc])
    plt.plot(train_history.history[test_acc])
    plt.title('Train History')
    plt.ylabel('Accuracy')
    plt.xlabel('Epoch')
    plt.legend(['train', 'test'], loc='upper left')
    plt.show()


show_train_history(train_history, 'accuracy', 'val_accuracy')
show_train_history(train_history, 'loss', 'val_loss')

训练集和验证集准确率对比

训练集和验证集损失值对比

5. 进行预测查看概率

# 进行预测
prediction = model.predict_classes(test_normalize_imgs)
from collections import Counter
result = Counter(prediction[:1000])
print('cat 识别数量：' , result)
result = Counter(prediction[1000:])
print('dog 识别数量：' , result)

通过预测获得大概识别出的数量

cat 识别数量： Counter({0: 769, 1: 231})
dog 识别数量： Counter({1: 819, 0: 181})

6. 保存模型权重以及json文件

# 保存模型和json文件
open('Keras_Cifar_CNN_Conv3_architecture.json', 'w').write(model.to_json())
model.save('CNN.h5')
model.save_weights('Keras_Cifar_CNN_Conv3_weights.h5', overwrite=True)

7. 总结

首先是制作训练集的过程当中，采用像cv2+os来循环读取图像的方式，这种制作的方式肯定会比较慢，像我就只能享受这个过程了，后面我会制作tensorflow的dataset来进行训练，这里训练出的效果可能不是最好的，但其实这个，可能是因为我的数据集也没多少，哈哈，我也没有太好的配置来跑模型，大概以我的模型跑出来的准确率在75%-85%之间。另外有什么我不对的地方或者需要修改的地方请通知我，谢谢，我也是慢慢学会的。

写文章不易，点个赞吧

需要猫狗分类的图片的可以自行去百度上找

下面是完整代码和模型权重文件的百度网盘链接

猫狗完整代码和模型权重

提取码：tf20