人工智能与机器学习 --- 深度学习-猫狗数据集

一、anaconda安装TensorFlow，Keras

直接在命令行使用pip安装，通过此方法将安装TensorFlow的最新版。

pip install tensorflow

通过在TensorFlow后加上版本控制，能够安装指定版本的TensorFlow，可以避免最新版的某些问题。

pip install tensorflow==版本号

使用pip在命令行安装Keras

pip install keras

二、狗猫数据集的两阶段分类实验

下载猫狗数据集
1、可以到kaggle官网下载，不过好像需要注册：https://www.kaggle.com/c/dogs-vs-cats/data
2、也可以到百度网盘下载数据集：https://pan.baidu.com/s/13hw4LK8ihR6-6-8mpjLKDA 密码：dmp4
下载好数据集之后，放到相应的文件路径下，需要将压缩文件解压出来。

查看Keras的版本

import keras
keras.__version__

'2.3.1'

使用小型数据集的卷积网络

import os, shutil

# 原目录的路径
# 数据集是未压缩的
original_dataset_dir = 'F:/人工智能与机器学习/kaggle/train'

# 目的路径
# 存储较小的数据集
base_dir = 'F:/人工智能与机器学习/kaggle/cats_and_dogs_small'
os.mkdir(base_dir)

# 训练目录
# 验证和测试拆分
train_dir = os.path.join(base_dir, 'train')
os.mkdir(train_dir)
validation_dir = os.path.join(base_dir, 'validation')
os.mkdir(validation_dir)
test_dir = os.path.join(base_dir, 'test1')
os.mkdir(test_dir)

# 训练猫的图片目录
train_cats_dir = os.path.join(train_dir, 'cats')
os.mkdir(train_cats_dir)

# 训练狗图片目录
train_dogs_dir = os.path.join(train_dir, 'dogs')
os.mkdir(train_dogs_dir)

# 验证猫的图片目录
validation_cats_dir = os.path.join(validation_dir, 'cats')
os.mkdir(validation_cats_dir)

# 验证狗的图片目录
validation_dogs_dir = os.path.join(validation_dir, 'dogs')
os.mkdir(validation_dogs_dir)

# 验证猫的图片目录
test_cats_dir = os.path.join(test_dir, 'cats')
os.mkdir(test_cats_dir)

# 验证狗的图片目录
test_dogs_dir = os.path.join(test_dir, 'dogs')
os.mkdir(test_dogs_dir)

# 将前1000张cat图像复制到train_cats_dir
fnames = ['cat.{}.jpg'.format(i) for i in range(1000)]
for fname in fnames:
    src = os.path.join(original_dataset_dir, fname)
    dst = os.path.join(train_cats_dir, fname)
    shutil.copyfile(src, dst)

# 复制接下来的500张cat图像到validation_cats_dir
fnames = ['cat.{}.jpg'.format(i) for i in range(1000, 1500)]
for fname in fnames:
    src = os.path.join(original_dataset_dir, fname)
    dst = os.path.join(validation_cats_dir, fname)
    shutil.copyfile(src, dst)
    
# 复制接下来的500张cat图像到test_cats_dir
fnames = ['cat.{}.jpg'.format(i) for i in range(1500, 2000)]
for fname in fnames:
    src = os.path.join(original_dataset_dir, fname)
    dst = os.path.join(test_cats_dir, fname)
    shutil.copyfile(src, dst)
    
# 将前1000张狗的图像复制到train_dogs_dir
fnames = ['dog.{}.jpg'.format(i) for i in range(1000)]
for fname in fnames:
    src = os.path.join(original_dataset_dir, fname)
    dst = os.path.join(train_dogs_dir, fname)
    shutil.copyfile(src, dst)
    
# 复制接下来的500张狗图片到validation_dogs_dir
fnames = ['dog.{}.jpg'.format(i) for i in range(1000, 1500)]
for fname in fnames:
    src = os.path.join(original_dataset_dir, fname)
    dst = os.path.join(validation_dogs_dir, fname)
    shutil.copyfile(src, dst)
    
# 复制接下来的500张狗的图片到test_dogs_dir
fnames = ['dog.{}.jpg'.format(i) for i in range(1500, 2000)]
for fname in fnames:
    src = os.path.join(original_dataset_dir, fname)
    dst = os.path.join(test_dogs_dir, fname)
    shutil.copyfile(src, dst)

作为一个完整的检查，让我们计算一下在每个训练分割(训练/验证/测试)中有多少张图片：

print('total training cat images:', len(os.listdir(train_cats_dir)))

total training cat images: 1000

print('total training dog images:', len(os.listdir(train_dogs_dir)))

total training dog images: 1000

print('total validation cat images:', len(os.listdir(validation_cats_dir)))

total validation cat images: 500

print('total validation dog images:', len(os.listdir(validation_dogs_dir)))

total validation dog images: 500

print('total test cat images:', len(os.listdir(test_cats_dir)))

total test cat images: 500

print('total test dog images:', len(os.listdir(test_dogs_dir)))

total test dog images: 500

我们可以确定有2000张训练图像，1000张验证图像和1000张测试图像。在每一次分割中，来自每个类的样本数量都是相同的。这是一个平衡的二元分类问题，这意味着分类的准确性将是衡量成功的一个合适的标准。

创建我们自己的网络：

from keras import layers
from keras import models

model = models.Sequential()
model.add(layers.Conv2D(32, (3, 3), activation='relu',
                        input_shape=(150, 150, 3)))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(64, (3, 3), activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(128, (3, 3), activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(128, (3, 3), activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Flatten())
model.add(layers.Dense(512, activation='relu'))
model.add(layers.Dense(1, activation='sigmoid'))

让我们看看特征图的尺寸是如何随着每一层变化的:

model.summary()

Model: "sequential_4"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_9 (Conv2D)            (None, 148, 148, 32)      896       
_________________________________________________________________
max_pooling2d_9 (MaxPooling2 (None, 74, 74, 32)        0         
_________________________________________________________________
conv2d_10 (Conv2D)           (None, 72, 72, 64)        18496     
_________________________________________________________________
max_pooling2d_10 (MaxPooling (None, 36, 36, 64)        0         
_________________________________________________________________
conv2d_11 (Conv2D)           (None, 34, 34, 128)       73856     
_________________________________________________________________
max_pooling2d_11 (MaxPooling (None, 17, 17, 128)       0         
_________________________________________________________________
conv2d_12 (Conv2D)           (None, 15, 15, 128)       147584    
_________________________________________________________________
max_pooling2d_12 (MaxPooling (None, 7, 7, 128)         0         
_________________________________________________________________
flatten_3 (Flatten)          (None, 6272)              0         
_________________________________________________________________
dense_7 (Dense)              (None, 512)               3211776   
_________________________________________________________________
dense_8 (Dense)              (None, 1)                 513       
=================================================================
Total params: 3,453,121
Trainable params: 3,453,121
Non-trainable params: 0
_________________________________________________________________

from keras import optimizers

model.compile(loss='binary_crossentropy',
              optimizer=optimizers.RMSprop(lr=1e-4),
              metrics=['acc'])

数据预处理：<