目录
一级目录
二级目录
三级目录
[AI训练营]paddleclas实现图像分类练习
项目背景
1.这是Ai训练营的baseline,经过这段时间的学习,感觉还是有许多地方不懂,于是跟着baseline做一遍
2.该项目完成食物5分类
3.基于baseline的代码,根据自己的理解做了一点注释
# 先导入库
from sklearn.utils import shuffle
import os
import pandas as pd
import numpy as np
from PIL import Image
import paddle
import paddle.nn as nn
import random
# 忽略(垃圾)警告信息
# 在python中运行代码经常会遇到的情况是——代码可以正常运行但是会提示警告,有时特别讨厌。
# 那么如何来控制警告输出呢?其实很简单,python通过调用warnings模块中定义的warn()函数来发出警告。我们可以通过警告过滤器进行控制是否发出警告消息。
import warnings
warnings.filterwarnings("ignore")
解压数据集,查看数据的结构
# 项目挂载的数据集先解压出来,待解压完毕,刷新后可发现左侧文件夹根目录出现五个zip
#代码前面的!意思是在终端执行,直接把代码放到终端执行出来的效果也是一样的
!unzip -oq /home/aistudio/data/data103736/五种图像分类数据集.zip
左侧可以看到如图所示五个zip
# 本项目以食物分类为例进行介绍,因为分类大多数情况下是不存在标签文件的,猫分类已经有了标签,省去了数据处理的操作
# (此处需要你根据自己的选择进行解压对应的文件)
# 解压完毕左侧出现文件夹,即为需要分类的文件
#文件一般建议用英文,用中文有时会出问题
!unzip -oq /home/aistudio/食物5分类.zip
# 查看结构,正为一个类别下有一系列对应的图片
!tree foods/
五类食物图片
- beef_carpaccio
- baby_back_ribs
- beef_tartare
- apple_pie
- baklava
具体结构如下:
foods/
├── apple_pie
│ ├── 1005649.jpg
│ ├── 1011328.jpg
│ ├── 101251.jpg
拿到总的训练数据txt
import os
# -*- coding: utf-8 -*-
# 根据官方paddleclas的提示,我们需要把图像变为两个txt文件
# train_list.txt(训练集)
# val_list.txt(验证集)
# 先把路径搞定 比如:foods/beef_carpaccio/855780.jpg ,读取到并写入txt
# 根据左侧生成的文件夹名字来写根目录
dirpath = "foods"
# 先得到总的txt后续再进行划分,因为要划分出验证集,所以要先打乱,因为原本是有序的
def get_all_txt():
all_list = []
i = 0 # 标记总文件数量
j = 0 # 标记文件类别
for root,dirs,files in os.walk(dirpath): # 分别代表根目录、文件夹、文件
for file in files:
i = i + 1
# 文件中每行格式: 图像相对路径 图像的label_id(数字类别)(注意:中间有空格)。
imgpath = os.path.join(root,file)
all_list.append(imgpath+" "+str(j)+"\n")
j = j + 1
allstr = ''.join(all_list)
f = open('all_list.txt','w',encoding='utf-8')
f.write(allstr)
return all_list , i
all_list,all_lenth = get_all_txt()
print(all_lenth)
5000
数据打乱
# 把数据打乱
all_list = shuffle(all_list)
allstr = ''.join(all_list)
f = open('all_list.txt','w',encoding='utf-8')
f.write(allstr)
print("打乱成功,并重新写入文本")
打乱成功,并重新写入文本
数据划分
# 按照比例划分数据集 食品的数据有5000张图片,不算大数据,一般9:1即可
train_size = int(all_lenth * 0.9)
train_list = all_list[:train_size]
val_list = all_list[train_size:]
print(len(train_list))
print(len(val_list))
4500
500
# 运行cell,生成训练集txt
train_txt = ''.join(train_list)
f_train = open('train_list.txt','w',encoding='utf-8')
f_train.write(train_txt)
f_train.close()
print("train_list.txt 生成成功!")
# 运行cell,生成验证集txt
val_txt = ''.join(val_list)
f_val = open('val_list.txt','w',encoding='utf-8')
f_val.write(val_txt)
f_val.close()
print("val_list.txt 生成成功!")
train_list.txt 生成成功!
val_list.txt 生成成功!
补充
我第一次做的跟这个差不多,txt文件是用以下代码做的
import random
import os
#生成train.txt和val.txt
#train.txt和val.txt是对数据集的划分,看到下面的ratio=0.9,意为按9:1划分为训练集和验证集
random.seed(2020)
xml_dir = '/home/aistudio/work/Annotations'#标签文件地址
#xml后缀文件是标签文件,打开Annotations文件夹,可以看见里面的文件是xml后缀的
img_dir = '/home/aistudio/work/images'#图像文件地址
#该文件地址是存放图片的,跟上面的文件是对应的
path_list = list()
for img in os.listdir(img_dir):
#os.listdir会返回指定路径下的文件
img_path = os.path.join(img_dir,img)
#os.path.join是将两个或多个拼起来
xml_path = os.path.join(xml_dir,img.replace('jpg', 'xml'))
path_list.append((img_path, xml_path))#将标签与图片对应
#可能还是有些同学不是很清楚是什么意思,举个例子可能好理解些
#已知img_dir='/home/aistudio/work/images'路径下都是图片
#假设这些图片是为1~20为名字的jpg
#那么os.listdir(img_dir)返回的是1.jpg,2.jpg……20.jpg
#for img in os.listdir(img_dir)就是上面的每一个jpg
#img_path = os.path.join(img_dir,img)拼接出来的就是这样:
#/home/aistudio/work/images/1.jpg
#/home/aistudio/work/images/2.jpg
#……
#xml_path 得到的就是/home/aistudio/work/images/2.xml
#两者的区别只是将最后面的jpg换成xml
#path_list就是把上面img_path和xml_path连起来
random.shuffle(path_list)#shuffle意为洗牌,意思是将数据集打乱
ratio = 0.9
train_f = open('/home/aistudio/work/train.txt','w') #生成训练文件
#open就是打开,后面的'w'是写入命令,即write
val_f = open('/home/aistudio/work/val.txt' ,'w')#生成验证文件
for i ,content in enumerate(path_list):
#enumerate(path_list)是一个枚举
#i,content分别是索引和值
#可以这样理解:path_list是一个C语言里的数组
#i为数组的序号,意思是第i个;content则为第i个的内容
img, xml = content
text = img + ' ' + xml + '\n'
if i < len(path_list) * ratio:
train_f.write(text)#写入
else:
val_f.write(text)
train_f.close()
val_f.close()
#生成标签文档
label = ['smoke']#设置你想检测的类别
with open('/home/aistudio/work/label_list.txt', 'w') as f:
for text in label:
f.write(text+'\n')
安装paddleclas
数据集核实完搞定成功的前提下,可以准备更改原文档的参数进行实现自己的图片分类了!
这里采用paddleclas的2.2版本,好用!
# 先把paddleclas安装上再说
# 安装paddleclas以及相关三方包(好像studio自带的已经够用了,无需安装了)
!git clone https://gitee.com/paddlepaddle/PaddleClas.git -b release/2.2
# 我这里安装相关包时,花了30几分钟还有错误提示,不管他即可
#!pip install --upgrade -r PaddleClas/requirements.txt -i https://mirror.baidu.com/pypi/simple
Cloning into 'PaddleClas'...
remote: Enumerating objects: 538, done.[K
remote: Counting objects: 100% (538/538), done.[K
remote: Compressing objects: 100% (323/323), done.[K
remote: Total 15290 (delta 347), reused 349 (delta 210), pack-reused 14752[K
Receiving objects: 100% (15290/15290), 113.56 MiB | 13.49 MiB/s, done.
Resolving deltas: 100% (10239/10239), done.
Checking connectivity... done.
#因为后续paddleclas的命令需要在PaddleClas目录下,所以进入PaddleClas根目录,执行此命令
%cd PaddleClas
!ls
/home/aistudio/PaddleClas
dataset hubconf.py MANIFEST.in README_ch.md requirements.txt
deploy __init__.py paddleclas.py README_en.md setup.py
docs LICENSE ppcls README.md tools
# 将图片移动到paddleclas下面的数据集里面
# 至于为什么现在移动,也是我的一点小技巧,防止之前移动的话,生成的txt的路径是全路径,反而需要去掉路径的一部分
!mv ../foods/ dataset/
# 挪动文件到对应目录
!mv ../all_list.txt dataset/foods
!mv ../train_list.txt dataset/foods
!mv ../val_list.txt dataset/foods
修改配置文件
说明
主要是以下几点:分类数、图片总量、训练和验证的路径、图像尺寸、数据预处理、训练和预测的num_workers: 0
路径如下:
PaddleClas/ppcls/configs/quick_start/new_user/ShuffleNetV2_x0_25.yaml
(主要的参数已经进行注释,一定要过一遍)
# global configs
Global:
checkpoints: null
pretrained_model: null
output_dir: ./output/
# 使用GPU训练
device: gpu
# 每几个轮次保存一次
save_interval: 1
eval_during_train: True
# 每几个轮次验证一次
eval_interval: 1
# 训练轮次
epochs: 20
print_batch_step: 1
use_visualdl: True #开启可视化(目前平台不可用)
# used for static mode and model export
# 图像大小
image_shape: [3, 224, 224]
save_inference_dir: ./inference
# training model under @to_static
to_static: False
# model architecture
Arch:
# 采用的网络
name: ResNet50
# 类别数 多了个0类 0-5 0无用
class_num: 6
# loss function config for traing/eval process
Loss:
Train:
- CELoss:
weight: 1.0
Eval:
- CELoss:
weight: 1.0
Optimizer:
name: Momentum
momentum: 0.9
lr:
name: Piecewise
learning_rate: 0.015
decay_epochs: [30, 60, 90]
values: [0.1, 0.01, 0.001, 0.0001]
regularizer:
name: 'L2'
coeff: 0.0005
# data loader for train and eval
DataLoader:
Train:
dataset:
name: ImageNetDataset
# 根路径
image_root: ./dataset/
# 前面自己生产得到的训练集文本路径
cls_label_path: ./dataset/foods/train_list.txt
# 数据预处理
transform_ops:
- DecodeImage:
to_rgb: True
channel_first: False
- ResizeImage:
resize_short: 256
- CropImage:
size: 224
- RandFlipImage:
flip_code: 1
- NormalizeImage:
scale: 1.0/255.0
mean: [0.485, 0.456, 0.406]
std: [0.229, 0.224, 0.225]
order: ''
sampler:
name: DistributedBatchSampler
batch_size: 128
drop_last: False
shuffle: True
loader:
num_workers: 0
use_shared_memory: True
Eval:
dataset:
name: ImageNetDataset
# 根路径
image_root: ./dataset/
# 前面自己生产得到的验证集文本路径
cls_label_path: ./dataset/foods/val_list.txt
# 数据预处理
transform_ops:
- DecodeImage:
to_rgb: True
channel_first: False
- ResizeImage:
resize_short: 256
- CropImage:
size: 224
- NormalizeImage:
scale: 1.0/255.0
mean: [0.485, 0.456, 0.406]
std: [0.229, 0.224, 0.225]
order: ''
sampler:
name: DistributedBatchSampler
batch_size: 128
drop_last: False
shuffle: True
loader:
num_workers: 0
use_shared_memory: True
Infer:
infer_imgs: ./dataset/foods/beef_carpaccio/855780.jpg
batch_size: 10
transforms:
- DecodeImage:
to_rgb: True
channel_first: False
- ResizeImage:
resize_short: 256
- CropImage:
size: 224
- NormalizeImage:
scale: 1.0/255.0
mean: [0.485, 0.456, 0.406]
std: [0.229, 0.224, 0.225]
order: ''
- ToCHWImage:
PostProcess:
name: Topk
# 输出的可能性最高的前topk个
topk: 5
# 标签文件 需要自己新建文件
class_id_map_file: ./dataset/label_list.txt
Metric:
Train:
- TopkAcc:
topk: [1, 5]
Eval:
- TopkAcc:
topk: [1, 5]
标签文件
这个是在预测时生成对照的依据,在上个文件有提到这个
# 标签文件 需要自己新建文件
class_id_map_file: dataset/label_list.txt
按照对应的进行编写:
如食品分类(要对照之前的txt的类别确认无误)
1 beef_carpaccio
2 baby_back_ribs
3 beef_tartare
4 apple_pie
5 baklava
开始训练
# 提示,运行过程中可能存在坏图的情况,但是不用担心,训练过程不受影响。
# 仅供参考,我只跑了五轮,准确率很低
!python3 tools/train.py \
-c ./ppcls/configs/quick_start/new_user/ShuffleNetV2_x0_25.yaml
[2021/08/10 22:11:17] root INFO: Already save model in ./output/ResNet50/latest
预测一张
# 更换为你训练的网络,需要预测的文件,上面训练所得到的的最优模型文件
# 我这里是不严谨的,直接使用训练集的图片进行验证,大家可以去百度搜一些相关的图片传上来,进行预测
!python3 tools/infer.py \
-c ./ppcls/configs/quick_start/new_user/ShuffleNetV2_x0_25.yaml \
-o Infer.infer_imgs=dataset/foods/baby_back_ribs/319516.jpg \
-o Global.pretrained_model=output/ResNet50/best_model
/home/aistudio/PaddleClas/ppcls/arch/backbone/model_zoo/vision_transformer.py:15: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
from collections import Callable
[2021/08/10 22:12:14] root INFO:
===========================================================
== PaddleClas is powered by PaddlePaddle ! ==
===========================================================
== ==
== For more info please go to the following website. ==
== ==
== https://github.com/PaddlePaddle/PaddleClas ==
===========================================================
[2021/08/10 22:12:14] root INFO: Arch :
[2021/08/10 22:12:14] root INFO: class_num : 6
[2021/08/10 22:12:14] root INFO: name : ResNet50
[2021/08/10 22:12:14] root INFO: DataLoader :
[2021/08/10 22:12:14] root INFO: Eval :
[2021/08/10 22:12:14] root INFO: dataset :
[2021/08/10 22:12:14] root INFO: cls_label_path : ./dataset/foods/val_list.txt
[2021/08/10 22:12:14] root INFO: image_root : ./dataset/
[2021/08/10 22:12:14] root INFO: name : ImageNetDataset
[2021/08/10 22:12:14] root INFO: transform_ops :
[2021/08/10 22:12:14] root INFO: DecodeImage :
[2021/08/10 22:12:14] root INFO: channel_first : False
[2021/08/10 22:12:14] root INFO: to_rgb : True
[2021/08/10 22:12:14] root INFO: ResizeImage :
[2021/08/10 22:12:14] root INFO: resize_short : 256
[2021/08/10 22:12:14] root INFO: CropImage :
[2021/08/10 22:12:14] root INFO: size : 224
[2021/08/10 22:12:14] root INFO: NormalizeImage :
[2021/08/10 22:12:14] root INFO: mean : [0.485, 0.456, 0.406]
[2021/08/10 22:12:14] root INFO: order :
[2021/08/10 22:12:14] root INFO: scale : 1.0/255.0
[2021/08/10 22:12:14] root INFO: std : [0.229, 0.224, 0.225]
[2021/08/10 22:12:14] root INFO: loader :
[2021/08/10 22:12:14] root INFO: num_workers : 0
[2021/08/10 22:12:14] root INFO: use_shared_memory : True
[2021/08/10 22:12:14] root INFO: sampler :
[2021/08/10 22:12:14] root INFO: batch_size : 128
[2021/08/10 22:12:14] root INFO: drop_last : False
[2021/08/10 22:12:14] root INFO: name : DistributedBatchSampler
[2021/08/10 22:12:14] root INFO: shuffle : True
[2021/08/10 22:12:14] root INFO: Train :
[2021/08/10 22:12:14] root INFO: dataset :
[2021/08/10 22:12:14] root INFO: cls_label_path : ./dataset/foods/train_list.txt
[2021/08/10 22:12:14] root INFO: image_root : ./dataset/
[2021/08/10 22:12:14] root INFO: name : ImageNetDataset
[2021/08/10 22:12:14] root INFO: transform_ops :
[2021/08/10 22:12:14] root INFO: DecodeImage :
[2021/08/10 22:12:14] root INFO: channel_first : False
[2021/08/10 22:12:14] root INFO: to_rgb : True
[2021/08/10 22:12:14] root INFO: ResizeImage :
[2021/08/10 22:12:14] root INFO: resize_short : 256
[2021/08/10 22:12:14] root INFO: CropImage :
[2021/08/10 22:12:14] root INFO: size : 224
[2021/08/10 22:12:14] root INFO: RandFlipImage :
[2021/08/10 22:12:14] root INFO: flip_code : 1
[2021/08/10 22:12:14] root INFO: NormalizeImage :
[2021/08/10 22:12:14] root INFO: mean : [0.485, 0.456, 0.406]
[2021/08/10 22:12:14] root INFO: order :
[2021/08/10 22:12:14] root INFO: scale : 1.0/255.0
[2021/08/10 22:12:14] root INFO: std : [0.229, 0.224, 0.225]
[2021/08/10 22:12:14] root INFO: loader :
[2021/08/10 22:12:14] root INFO: num_workers : 0
[2021/08/10 22:12:14] root INFO: use_shared_memory : True
[2021/08/10 22:12:14] root INFO: sampler :
[2021/08/10 22:12:14] root INFO: batch_size : 128
[2021/08/10 22:12:14] root INFO: drop_last : False
[2021/08/10 22:12:14] root INFO: name : DistributedBatchSampler
[2021/08/10 22:12:14] root INFO: shuffle : True
[2021/08/10 22:12:14] root INFO: Global :
[2021/08/10 22:12:14] root INFO: checkpoints : None
[2021/08/10 22:12:14] root INFO: device : gpu
[2021/08/10 22:12:14] root INFO: epochs : 20
[2021/08/10 22:12:14] root INFO: eval_during_train : True
[2021/08/10 22:12:14] root INFO: eval_interval : 1
[2021/08/10 22:12:14] root INFO: image_shape : [3, 224, 224]
[2021/08/10 22:12:14] root INFO: output_dir : ./output/
[2021/08/10 22:12:14] root INFO: pretrained_model : output/ResNet50/best_model
[2021/08/10 22:12:14] root INFO: print_batch_step : 1
[2021/08/10 22:12:14] root INFO: save_inference_dir : ./inference
[2021/08/10 22:12:14] root INFO: save_interval : 1
[2021/08/10 22:12:14] root INFO: to_static : False
[2021/08/10 22:12:14] root INFO: use_visualdl : True
[2021/08/10 22:12:14] root INFO: Infer :
[2021/08/10 22:12:14] root INFO: PostProcess :
[2021/08/10 22:12:14] root INFO: class_id_map_file : ./dataset/label_list.txt
[2021/08/10 22:12:14] root INFO: name : Topk
[2021/08/10 22:12:14] root INFO: topk : 5
[2021/08/10 22:12:14] root INFO: batch_size : 10
[2021/08/10 22:12:14] root INFO: infer_imgs : dataset/foods/baby_back_ribs/319516.jpg
[2021/08/10 22:12:14] root INFO: transforms :
[2021/08/10 22:12:14] root INFO: DecodeImage :
[2021/08/10 22:12:14] root INFO: channel_first : False
[2021/08/10 22:12:14] root INFO: to_rgb : True
[2021/08/10 22:12:14] root INFO: ResizeImage :
[2021/08/10 22:12:14] root INFO: resize_short : 256
[2021/08/10 22:12:14] root INFO: CropImage :
[2021/08/10 22:12:14] root INFO: size : 224
[2021/08/10 22:12:14] root INFO: NormalizeImage :
[2021/08/10 22:12:14] root INFO: mean : [0.485, 0.456, 0.406]
[2021/08/10 22:12:14] root INFO: order :
[2021/08/10 22:12:14] root INFO: scale : 1.0/255.0
[2021/08/10 22:12:14] root INFO: std : [0.229, 0.224, 0.225]
[2021/08/10 22:12:14] root INFO: ToCHWImage : None
[2021/08/10 22:12:14] root INFO: Loss :
[2021/08/10 22:12:14] root INFO: Eval :
[2021/08/10 22:12:14] root INFO: CELoss :
[2021/08/10 22:12:14] root INFO: weight : 1.0
[2021/08/10 22:12:14] root INFO: Train :
[2021/08/10 22:12:14] root INFO: CELoss :
[2021/08/10 22:12:14] root INFO: weight : 1.0
[2021/08/10 22:12:14] root INFO: Metric :
[2021/08/10 22:12:14] root INFO: Eval :
[2021/08/10 22:12:14] root INFO: TopkAcc :
[2021/08/10 22:12:14] root INFO: topk : [1, 5]
[2021/08/10 22:12:14] root INFO: Train :
[2021/08/10 22:12:14] root INFO: TopkAcc :
[2021/08/10 22:12:14] root INFO: topk : [1, 5]
[2021/08/10 22:12:14] root INFO: Optimizer :
[2021/08/10 22:12:14] root INFO: lr :
[2021/08/10 22:12:14] root INFO: decay_epochs : [30, 60, 90]
[2021/08/10 22:12:14] root INFO: learning_rate : 0.015
[2021/08/10 22:12:14] root INFO: name : Piecewise
[2021/08/10 22:12:14] root INFO: values : [0.1, 0.01, 0.001, 0.0001]
[2021/08/10 22:12:14] root INFO: momentum : 0.9
[2021/08/10 22:12:14] root INFO: name : Momentum
[2021/08/10 22:12:14] root INFO: regularizer :
[2021/08/10 22:12:14] root INFO: coeff : 0.0005
[2021/08/10 22:12:14] root INFO: name : L2
W0810 22:12:14.715823 1852 device_context.cc:404] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 11.0, Runtime API Version: 10.1
W0810 22:12:14.721333 1852 device_context.cc:422] device: 0, cuDNN Version: 7.6.
[2021/08/10 22:12:18] root INFO: train with paddle 2.1.0 and device CUDAPlace(0)
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/tensor/creation.py:125: DeprecationWarning: `np.object` is a deprecated alias for the builtin `object`. To silence this warning, use `object` by itself. Doing this will not modify any behavior and is safe.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
if data.dtype == np.object:
[{'class_ids': [3, 5, 4, 2, 1], 'scores': [0.82606, 0.10882, 0.04856, 0.01256, 0.0036], 'file_name': 'dataset/foods/baby_back_ribs/319516.jpg', 'label_names': ['beef_tartare', 'baklava', 'apple_pie', 'baby_back_ribs', 'beef_carpaccio']}]
运行结果
修改参数
修改一些参数,看看会不会有更好的效果
验证用的图片保持不变
修改模型
先只修改模型看看
!python3 tools/train.py \
-c ./ppcls/configs/quick_start/new_user/ModifyParameter.yaml
[2021/08/10 22:36:58] root INFO: Already save model in ./output/MobileNetV1/latest
!python3 tools/infer.py \
-c ./ppcls/configs/quick_start/new_user/ModifyParameter.yaml \
-o Infer.infer_imgs=dataset/foods/baby_back_ribs/319516.jpg \
-o Global.pretrained_model=output/MobileNetV1/best_model
/home/aistudio/PaddleClas/ppcls/arch/backbone/model_zoo/vision_transformer.py:15: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
from collections import Callable
[2021/08/10 22:40:03] root INFO:
===========================================================
== PaddleClas is powered by PaddlePaddle ! ==
===========================================================
== ==
== For more info please go to the following website. ==
== ==
== https://github.com/PaddlePaddle/PaddleClas ==
===========================================================
[2021/08/10 22:40:03] root INFO: Arch :
[2021/08/10 22:40:03] root INFO: class_num : 6
[2021/08/10 22:40:03] root INFO: name : MobileNetV1
[2021/08/10 22:40:03] root INFO: DataLoader :
[2021/08/10 22:40:03] root INFO: Eval :
[2021/08/10 22:40:03] root INFO: dataset :
[2021/08/10 22:40:03] root INFO: cls_label_path : ./dataset/foods/val_list.txt
[2021/08/10 22:40:03] root INFO: image_root : ./dataset/
[2021/08/10 22:40:03] root INFO: name : ImageNetDataset
[2021/08/10 22:40:03] root INFO: transform_ops :
[2021/08/10 22:40:03] root INFO: DecodeImage :
[2021/08/10 22:40:03] root INFO: channel_first : False
[2021/08/10 22:40:03] root INFO: to_rgb : True
[2021/08/10 22:40:03] root INFO: ResizeImage :
[2021/08/10 22:40:03] root INFO: resize_short : 256
[2021/08/10 22:40:03] root INFO: CropImage :
[2021/08/10 22:40:03] root INFO: size : 224
[2021/08/10 22:40:03] root INFO: NormalizeImage :
[2021/08/10 22:40:03] root INFO: mean : [0.485, 0.456, 0.406]
[2021/08/10 22:40:03] root INFO: order :
[2021/08/10 22:40:03] root INFO: scale : 1.0/255.0
[2021/08/10 22:40:03] root INFO: std : [0.229, 0.224, 0.225]
[2021/08/10 22:40:03] root INFO: loader :
[2021/08/10 22:40:03] root INFO: num_workers : 0
[2021/08/10 22:40:03] root INFO: use_shared_memory : True
[2021/08/10 22:40:03] root INFO: sampler :
[2021/08/10 22:40:03] root INFO: batch_size : 128
[2021/08/10 22:40:03] root INFO: drop_last : False
[2021/08/10 22:40:03] root INFO: name : DistributedBatchSampler
[2021/08/10 22:40:03] root INFO: shuffle : True
[2021/08/10 22:40:03] root INFO: Train :
[2021/08/10 22:40:03] root INFO: dataset :
[2021/08/10 22:40:03] root INFO: cls_label_path : ./dataset/foods/train_list.txt
[2021/08/10 22:40:03] root INFO: image_root : ./dataset/
[2021/08/10 22:40:03] root INFO: name : ImageNetDataset
[2021/08/10 22:40:03] root INFO: transform_ops :
[2021/08/10 22:40:03] root INFO: DecodeImage :
[2021/08/10 22:40:03] root INFO: channel_first : False
[2021/08/10 22:40:03] root INFO: to_rgb : True
[2021/08/10 22:40:03] root INFO: ResizeImage :
[2021/08/10 22:40:03] root INFO: resize_short : 256
[2021/08/10 22:40:03] root INFO: CropImage :
[2021/08/10 22:40:03] root INFO: size : 224
[2021/08/10 22:40:03] root INFO: RandFlipImage :
[2021/08/10 22:40:03] root INFO: flip_code : 1
[2021/08/10 22:40:03] root INFO: NormalizeImage :
[2021/08/10 22:40:03] root INFO: mean : [0.485, 0.456, 0.406]
[2021/08/10 22:40:03] root INFO: order :
[2021/08/10 22:40:03] root INFO: scale : 1.0/255.0
[2021/08/10 22:40:03] root INFO: std : [0.229, 0.224, 0.225]
[2021/08/10 22:40:03] root INFO: loader :
[2021/08/10 22:40:03] root INFO: num_workers : 0
[2021/08/10 22:40:03] root INFO: use_shared_memory : True
[2021/08/10 22:40:03] root INFO: sampler :
[2021/08/10 22:40:03] root INFO: batch_size : 128
[2021/08/10 22:40:03] root INFO: drop_last : False
[2021/08/10 22:40:03] root INFO: name : DistributedBatchSampler
[2021/08/10 22:40:03] root INFO: shuffle : True
[2021/08/10 22:40:03] root INFO: Global :
[2021/08/10 22:40:03] root INFO: checkpoints : None
[2021/08/10 22:40:03] root INFO: device : gpu
[2021/08/10 22:40:03] root INFO: epochs : 20
[2021/08/10 22:40:03] root INFO: eval_during_train : True
[2021/08/10 22:40:03] root INFO: eval_interval : 1
[2021/08/10 22:40:03] root INFO: image_shape : [3, 224, 224]
[2021/08/10 22:40:03] root INFO: output_dir : ./output/
[2021/08/10 22:40:03] root INFO: pretrained_model : output/MobileNetV1/best_model
[2021/08/10 22:40:03] root INFO: print_batch_step : 1
[2021/08/10 22:40:03] root INFO: save_inference_dir : ./inference
[2021/08/10 22:40:03] root INFO: save_interval : 1
[2021/08/10 22:40:03] root INFO: to_static : False
[2021/08/10 22:40:03] root INFO: use_visualdl : True
[2021/08/10 22:40:03] root INFO: Infer :
[2021/08/10 22:40:03] root INFO: PostProcess :
[2021/08/10 22:40:03] root INFO: class_id_map_file : ./dataset/label_list.txt
[2021/08/10 22:40:03] root INFO: name : Topk
[2021/08/10 22:40:03] root INFO: topk : 5
[2021/08/10 22:40:03] root INFO: batch_size : 10
[2021/08/10 22:40:03] root INFO: infer_imgs : dataset/foods/baby_back_ribs/319516.jpg
[2021/08/10 22:40:03] root INFO: transforms :
[2021/08/10 22:40:03] root INFO: DecodeImage :
[2021/08/10 22:40:03] root INFO: channel_first : False
[2021/08/10 22:40:03] root INFO: to_rgb : True
[2021/08/10 22:40:03] root INFO: ResizeImage :
[2021/08/10 22:40:03] root INFO: resize_short : 256
[2021/08/10 22:40:03] root INFO: CropImage :
[2021/08/10 22:40:03] root INFO: size : 224
[2021/08/10 22:40:03] root INFO: NormalizeImage :
[2021/08/10 22:40:03] root INFO: mean : [0.485, 0.456, 0.406]
[2021/08/10 22:40:03] root INFO: order :
[2021/08/10 22:40:03] root INFO: scale : 1.0/255.0
[2021/08/10 22:40:03] root INFO: std : [0.229, 0.224, 0.225]
[2021/08/10 22:40:03] root INFO: ToCHWImage : None
[2021/08/10 22:40:03] root INFO: Loss :
[2021/08/10 22:40:03] root INFO: Eval :
[2021/08/10 22:40:03] root INFO: CELoss :
[2021/08/10 22:40:03] root INFO: weight : 1.0
[2021/08/10 22:40:03] root INFO: Train :
[2021/08/10 22:40:03] root INFO: CELoss :
[2021/08/10 22:40:03] root INFO: weight : 1.0
[2021/08/10 22:40:03] root INFO: Metric :
[2021/08/10 22:40:03] root INFO: Eval :
[2021/08/10 22:40:03] root INFO: TopkAcc :
[2021/08/10 22:40:03] root INFO: topk : [1, 5]
[2021/08/10 22:40:03] root INFO: Train :
[2021/08/10 22:40:03] root INFO: TopkAcc :
[2021/08/10 22:40:03] root INFO: topk : [1, 5]
[2021/08/10 22:40:03] root INFO: Optimizer :
[2021/08/10 22:40:03] root INFO: lr :
[2021/08/10 22:40:03] root INFO: decay_epochs : [30, 60, 90]
[2021/08/10 22:40:03] root INFO: learning_rate : 0.015
[2021/08/10 22:40:03] root INFO: name : Piecewise
[2021/08/10 22:40:03] root INFO: values : [0.1, 0.01, 0.001, 0.0001]
[2021/08/10 22:40:03] root INFO: momentum : 0.9
[2021/08/10 22:40:03] root INFO: name : Momentum
[2021/08/10 22:40:03] root INFO: regularizer :
[2021/08/10 22:40:03] root INFO: coeff : 0.0005
[2021/08/10 22:40:03] root INFO: name : L2
W0810 22:40:03.151469 3139 device_context.cc:404] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 11.0, Runtime API Version: 10.1
W0810 22:40:03.156962 3139 device_context.cc:422] device: 0, cuDNN Version: 7.6.
[2021/08/10 22:40:06] root INFO: train with paddle 2.1.0 and device CUDAPlace(0)
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/tensor/creation.py:125: DeprecationWarning: `np.object` is a deprecated alias for the builtin `object`. To silence this warning, use `object` by itself. Doing this will not modify any behavior and is safe.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
if data.dtype == np.object:
[{'class_ids': [4, 2, 1, 3, 5], 'scores': [0.49584, 0.16217, 0.13516, 0.11661, 0.09011], 'file_name': 'dataset/foods/baby_back_ribs/319516.jpg', 'label_names': ['apple_pie', 'baby_back_ribs', 'beef_carpaccio', 'beef_tartare', 'baklava']}]
结果
感觉效果一般般
修改更多参数
ModifyMoreParameters.yaml修改了模型、epoch等参数
!python3 tools/train.py \
-c ./ppcls/configs/quick_start/new_user/ModifyMoreParameters.yaml
[2021/08/11 00:03:45] root INFO: Already save model in ./output/DarkNet53/latest
!python3 tools/infer.py \
-c ./ppcls/configs/quick_start/new_user/ModifyMoreParameters.yaml \
-o Infer.infer_imgs=dataset/foods/baby_back_ribs/319516.jpg \
_user/ModifyMoreParameters.yaml \
-o Infer.infer_imgs=dataset/foods/baby_back_ribs/319516.jpg \
-o Global.pretrained_model=output/DarkNet53/best_model
/home/aistudio/PaddleClas/ppcls/arch/backbone/model_zoo/vision_transformer.py:15: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
from collections import Callable
[2021/08/11 00:10:06] root INFO:
===========================================================
== PaddleClas is powered by PaddlePaddle ! ==
===========================================================
== ==
== For more info please go to the following website. ==
== ==
== https://github.com/PaddlePaddle/PaddleClas ==
===========================================================
[2021/08/11 00:10:06] root INFO: Arch :
[2021/08/11 00:10:06] root INFO: class_num : 6
[2021/08/11 00:10:06] root INFO: name : DarkNet53
[2021/08/11 00:10:06] root INFO: DataLoader :
[2021/08/11 00:10:06] root INFO: Eval :
[2021/08/11 00:10:06] root INFO: dataset :
[2021/08/11 00:10:06] root INFO: cls_label_path : ./dataset/foods/val_list.txt
[2021/08/11 00:10:06] root INFO: image_root : ./dataset/
[2021/08/11 00:10:06] root INFO: name : ImageNetDataset
[2021/08/11 00:10:06] root INFO: transform_ops :
[2021/08/11 00:10:06] root INFO: DecodeImage :
[2021/08/11 00:10:06] root INFO: channel_first : False
[2021/08/11 00:10:06] root INFO: to_rgb : True
[2021/08/11 00:10:06] root INFO: ResizeImage :
[2021/08/11 00:10:06] root INFO: resize_short : 256
[2021/08/11 00:10:06] root INFO: CropImage :
[2021/08/11 00:10:06] root INFO: size : 224
[2021/08/11 00:10:06] root INFO: NormalizeImage :
[2021/08/11 00:10:06] root INFO: mean : [0.485, 0.456, 0.406]
[2021/08/11 00:10:06] root INFO: order :
[2021/08/11 00:10:06] root INFO: scale : 1.0/255.0
[2021/08/11 00:10:06] root INFO: std : [0.229, 0.224, 0.225]
[2021/08/11 00:10:06] root INFO: loader :
[2021/08/11 00:10:06] root INFO: num_workers : 0
[2021/08/11 00:10:06] root INFO: use_shared_memory : True
[2021/08/11 00:10:06] root INFO: sampler :
[2021/08/11 00:10:06] root INFO: batch_size : 128
[2021/08/11 00:10:06] root INFO: drop_last : False
[2021/08/11 00:10:06] root INFO: name : DistributedBatchSampler
[2021/08/11 00:10:06] root INFO: shuffle : True
[2021/08/11 00:10:06] root INFO: Train :
[2021/08/11 00:10:06] root INFO: dataset :
[2021/08/11 00:10:06] root INFO: cls_label_path : ./dataset/foods/train_list.txt
[2021/08/11 00:10:06] root INFO: image_root : ./dataset/
[2021/08/11 00:10:06] root INFO: name : ImageNetDataset
[2021/08/11 00:10:06] root INFO: transform_ops :
[2021/08/11 00:10:06] root INFO: DecodeImage :
[2021/08/11 00:10:06] root INFO: channel_first : False
[2021/08/11 00:10:06] root INFO: to_rgb : True
[2021/08/11 00:10:06] root INFO: ResizeImage :
[2021/08/11 00:10:06] root INFO: resize_short : 256
[2021/08/11 00:10:06] root INFO: CropImage :
[2021/08/11 00:10:06] root INFO: size : 224
[2021/08/11 00:10:06] root INFO: RandFlipImage :
[2021/08/11 00:10:06] root INFO: flip_code : 1
[2021/08/11 00:10:06] root INFO: NormalizeImage :
[2021/08/11 00:10:06] root INFO: mean : [0.485, 0.456, 0.406]
[2021/08/11 00:10:06] root INFO: order :
[2021/08/11 00:10:06] root INFO: scale : 1.0/255.0
[2021/08/11 00:10:06] root INFO: std : [0.229, 0.224, 0.225]
[2021/08/11 00:10:06] root INFO: loader :
[2021/08/11 00:10:06] root INFO: num_workers : 0
[2021/08/11 00:10:06] root INFO: use_shared_memory : True
[2021/08/11 00:10:06] root INFO: sampler :
[2021/08/11 00:10:06] root INFO: batch_size : 128
[2021/08/11 00:10:06] root INFO: drop_last : False
[2021/08/11 00:10:06] root INFO: name : DistributedBatchSampler
[2021/08/11 00:10:06] root INFO: shuffle : True
[2021/08/11 00:10:06] root INFO: Global :
[2021/08/11 00:10:06] root INFO: checkpoints : None
[2021/08/11 00:10:06] root INFO: device : gpu
[2021/08/11 00:10:06] root INFO: epochs : 100
[2021/08/11 00:10:06] root INFO: eval_during_train : True
[2021/08/11 00:10:06] root INFO: eval_interval : 1
[2021/08/11 00:10:06] root INFO: image_shape : [3, 224, 224]
[2021/08/11 00:10:06] root INFO: output_dir : ./output/
[2021/08/11 00:10:06] root INFO: pretrained_model : output/DarkNet53/best_model
[2021/08/11 00:10:06] root INFO: print_batch_step : 1
[2021/08/11 00:10:06] root INFO: save_inference_dir : ./inference
[2021/08/11 00:10:06] root INFO: save_interval : 1
[2021/08/11 00:10:06] root INFO: to_static : False
[2021/08/11 00:10:06] root INFO: use_visualdl : True
[2021/08/11 00:10:06] root INFO: Infer :
[2021/08/11 00:10:06] root INFO: PostProcess :
[2021/08/11 00:10:06] root INFO: class_id_map_file : ./dataset/label_list.txt
[2021/08/11 00:10:06] root INFO: name : Topk
[2021/08/11 00:10:06] root INFO: topk : 5
[2021/08/11 00:10:06] root INFO: batch_size : 10
[2021/08/11 00:10:06] root INFO: infer_imgs : dataset/foods/baby_back_ribs/319516.jpg
[2021/08/11 00:10:06] root INFO: transforms :
[2021/08/11 00:10:06] root INFO: DecodeImage :
[2021/08/11 00:10:06] root INFO: channel_first : False
[2021/08/11 00:10:06] root INFO: to_rgb : True
[2021/08/11 00:10:06] root INFO: ResizeImage :
[2021/08/11 00:10:06] root INFO: resize_short : 256
[2021/08/11 00:10:06] root INFO: CropImage :
[2021/08/11 00:10:06] root INFO: size : 224
[2021/08/11 00:10:06] root INFO: NormalizeImage :
[2021/08/11 00:10:06] root INFO: mean : [0.485, 0.456, 0.406]
[2021/08/11 00:10:06] root INFO: order :
[2021/08/11 00:10:06] root INFO: scale : 1.0/255.0
[2021/08/11 00:10:06] root INFO: std : [0.229, 0.224, 0.225]
[2021/08/11 00:10:06] root INFO: ToCHWImage : None
[2021/08/11 00:10:06] root INFO: Loss :
[2021/08/11 00:10:06] root INFO: Eval :
[2021/08/11 00:10:06] root INFO: CELoss :
[2021/08/11 00:10:06] root INFO: weight : 1.0
[2021/08/11 00:10:06] root INFO: Train :
[2021/08/11 00:10:06] root INFO: CELoss :
[2021/08/11 00:10:06] root INFO: weight : 1.0
[2021/08/11 00:10:06] root INFO: Metric :
[2021/08/11 00:10:06] root INFO: Eval :
[2021/08/11 00:10:06] root INFO: TopkAcc :
[2021/08/11 00:10:06] root INFO: topk : [1, 5]
[2021/08/11 00:10:06] root INFO: Train :
[2021/08/11 00:10:06] root INFO: TopkAcc :
[2021/08/11 00:10:06] root INFO: topk : [1, 5]
[2021/08/11 00:10:06] root INFO: Optimizer :
[2021/08/11 00:10:06] root INFO: lr :
[2021/08/11 00:10:06] root INFO: decay_epochs : [30, 60, 90]
[2021/08/11 00:10:06] root INFO: learning_rate : 0.015
[2021/08/11 00:10:06] root INFO: name : Piecewise
[2021/08/11 00:10:06] root INFO: values : [0.1, 0.01, 0.001, 0.0001]
[2021/08/11 00:10:06] root INFO: momentum : 0.9
[2021/08/11 00:10:06] root INFO: name : Momentum
[2021/08/11 00:10:06] root INFO: regularizer :
[2021/08/11 00:10:06] root INFO: coeff : 0.0005
[2021/08/11 00:10:06] root INFO: name : L2
W0811 00:10:06.905934 7037 device_context.cc:404] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 11.0, Runtime API Version: 10.1
W0811 00:10:06.911123 7037 device_context.cc:422] device: 0, cuDNN Version: 7.6.
[2021/08/11 00:10:11] root INFO: train with paddle 2.1.0 and device CUDAPlace(0)
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/tensor/creation.py:125: DeprecationWarning: `np.object` is a deprecated alias for the builtin `object`. To silence this warning, use `object` by itself. Doing this will not modify any behavior and is safe.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
if data.dtype == np.object:
[{'class_ids': [3, 5, 4, 2, 1], 'scores': [0.59343, 0.16844, 0.11845, 0.09278, 0.02689], 'file_name': 'dataset/foods/baby_back_ribs/319516.jpg', 'label_names': ['beef_tartare', 'baklava', 'apple_pie', 'baby_back_ribs', 'beef_carpaccio']}]
结果
最后
通过这次学习,基本了解了paddleclas的使用方法。
想部署到手机上试试