图片分类及Paddle派部署实例

项目简介

这是一个demo 项目，用于演示如何在 AI Studio 上训练一个“小”模型，然后把它转化成一个可以部署到Paddle派硬件上的模型。

为了简单起见，在此只训练一个猫猫和狗狗的二分类模型。

进入项目时，已经引用了 AI Studio 的公开数据集"猫狗大战数据集"作为训练数据。数据存储在 data/data62/ 目录下，以压缩包的形式存在。执行下面的代码，进入目录，将训练数据解压

In[1]

!cd /home/aistudio/data/data62 && unzip -q train.zip 
!cd /home/aistudio/data/data62 && unzip -q test.zip

数据预处理

训练集中的数据按照

cat.123.jpg

dog.456.jpg

的命名方式。由于数据中存在一些破损的图片，所以需要先清洗一下数据。同时，为了方便训练，将数据的一行准备成

文件名\t类别

的格式，并输出到和图片同级目录下的label.txt文件中。猫猫的类别是1，狗狗的类别是0。执行以下代码，进行数据的简单清洗

In[2]

#数据清洗
import codecs
import os
from PIL import Image

train_file_list = os.listdir('data/data62/train')
with codecs.open("data/data62/train/label.txt", 'w') as out_file:
    for file in train_file_list:
        try:
            img = Image.open(os.path.join('data/data62/train', file))
            if file.find('cat') != -1:
                out_file.write("{0}\t{1}\n".format(file, 1))
            else:
                out_file.write("{0}\t{1}\n".format(file, 0))
        except Exception as e:
            pass
            # 存在一些文件打不开，此处需要稍作清洗        

参数设置

设置基础训练参数，例如

• 图片尺寸，注意是 chw 格式
• 训练数据路径
• 保存模型的输出路径
• 训练轮数、训练批次大小
• 是否使用GPU
• 学习率变化等

其中类别数量会在读取数据时提前计算，初始为-1，仅用作占位

In[3]

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
import os
import numpy as np
import uuid
import random
import time
import six
import sys
import functools
import math
import paddle
import paddle.fluid as fluid
import paddle.dataset.flowers as flowers
import argparse
import functools
import subprocess
import codecs
import distutils.util
from paddle.fluid import core
from paddle.fluid.initializer import MSRA
from paddle.fluid.param_attr import ParamAttr
from PIL import Image, ImageEnhance
import logging


train_parameters = {
    "input_size": [3, 224, 224],
    "class_dim": -1,
    "data_dir": "data/data62/train",
    "save_model_dir": "./classify-model",
    "mode": "train",
    "num_epochs": 120,
    "image_count": -1,
    "train_batch_size": 50,
    "mean_rgb": [127.5, 127.5, 127.5],
    "use_gpu": True,            # 根据自己的环境，选择适当的设备进行训练
    "image_distort_strategy": {
        "need_distort": True,
        "expand_prob": 0.5,
        "expand_max_ratio": 4,
        "hue_prob": 0.5,
        "hue_delta": 18,
        "contrast_prob": 0.5,
        "contrast_delta": 0.5,
        "saturation_prob": 0.5,
        "saturation_delta": 0.5,
        "brightness_prob": 0.5,
        "brightness_delta": 0.125
    },
    "rsm_strategy": {
        "learning_rate": 0.02,
        "lr_epochs": [20, 40, 60, 80, 100],
        "lr_decay": [1, 0.5, 0.25, 0.1, 0.01, 0.002]
    },
    "momentum_strategy": {
        "learning_rate": 0.005,
        "lr_epochs": [20, 40, 60, 80, 100],
        "lr_decay": [1, 0.5, 0.25, 0.1, 0.01, 0.002]
    }
}

定义网络结构

设定网络结构，此处定义了三个常用的网络结构

• resnet
• mobile-net
• vgg-net

为了训练一个小模型，此处使用mobile-net。如果是其他项目或者其他用途，使用其他网络结构亦可

In[4]

class ResNet():
    def __init__(self, layers=50):
        self.layers = layers
        
    def name(self):
        return 'resnet'

    def net(self, input, class_dim=1000):
        layers = self.layers
        supported_layers = [50, 101, 152]
        assert layers in supported_layers, \
            "supported layers are {} but input layer is {}".format(supported_layers, layers)

        if layers == 50:
            depth = [3, 4, 6, 3]
        elif layers == 101:
            depth = [3, 4, 23, 3]
        elif layers == 152:
            depth = [3, 8, 36, 3]
        num_filters = [64, 128, 256, 512]

        conv = self.conv_bn_layer(
            input=input,
            num_filters=64,
            filter_size=7,
            stride=2,
            act='relu',
            name="conv1")
        conv = fluid.layers.pool2d(
            input=conv,
            pool_size=3,
            pool_stride=2,
            pool_padding=1,
            pool_type='max')

        for block in range(len(depth)):
            for i in range(depth[block]):
                if layers in [101, 152] and block == 2:
                    if i == 0:
                        conv_name = "res" + str(block + 2) + "a"
                    else:
                        conv_name = "res" + str(block + 2) + "b" + str(i)
                else:
                    conv_name = "res" + str(block + 2) + chr(97 + i)
                conv = self.bottleneck_block(
                    input=conv,
                    num_filters=num_filters[block],
                    stride=2 if i == 0 and block != 0 else 1,
                    name=conv_name)

        pool = fluid.layers.pool2d(
            input=conv, pool_size=7, pool_type='avg', global_pooling=True)
        stdv = 1.0 / math.sqrt(pool.shape[1] * 1.0)
        out = fluid.layers.fc(input=pool,
                              size=class_dim,
                              act='softmax', 
                              param_attr=fluid.param_attr.ParamAttr(
                                  initializer=fluid.initializer.Uniform(-stdv,
                                                                        stdv)))
        return out

    def conv_bn_layer(self,
                      input,
                      num_filters,
                      filter_size,
                      stride=1,
                      groups=1,
                      act=None,
                      name=None):
        conv = fluid.layers.conv2d(
            input=input,
            num_filters=num_filters,
            filter_size=filter_size,
            stride=stride,
            padding=(filter_size - 1) // 2,
            groups=groups,
            act=None,
            param_attr=ParamAttr(name=name + "_weights"),
            bias_attr=False,
            name=name + '.conv2d.output.1')
        if name == "conv1":
            bn_name = "bn_" + name
        else:
            bn_name = "bn" + name[3:]
        return fluid.layers.batch_norm(
            input=conv,
            act=act,
            name=bn_name + '.output.1',
            param_attr=ParamAttr(name=bn_name + '_scale'),
            bias_attr=ParamAttr(bn_name + '_offset'),
            moving_mean_name=bn_name + '_mean',
            moving_variance_name=bn_name + '_variance', )

    def shortcut(self, input, ch_out, stride, name):
        ch_in = input.shape[1]
        if ch_in != ch_out or stri