1.准备图片(训练太久就不放那么多图片了)
在caffe根目录下data中新建文件夹6class(意思是6类),在6class文件夹下新建两个文件夹train和val。
train用来存放训练的图片,在train文件夹下新建6个文件夹0-5 。图片有6类,杯子(文件夹0)、书包(文件夹1)、电脑(文件夹2)、猫(3)、狗(4)、鸭子(5),每类10种。
网上下载下来的图片名字都很乱 所以三步大法:
打开图片文件夹终端
c=0;for i in *.jpg;do mv -f $i $((c+=1)).jpg;done #将图片重命名1.jpg-10.jpg
rename 's/\.jpg/.jpeg/' ./* #改图片后缀名
rename 's/^/bag/' * #在图片上加类的名字
然后就这样了:
val 用来放训练过程中用来验证的图片(来计算准确率),val中的图片和train中的不一样,我里面放了6张一样一张。
2. 将图片路径写入txt
在data/6class/中新建train.txt 和val.txt
需要将图片的路径以及标签都写进去,杯子标签为0,包标签为1,等等...
写入路径
find -name *jpeg | grep train | cut -d / -f 3-4 > train.txt
find -name *jpeg | grep val | cut -d / -f 3 > val.txt
写入标签
sed -i "1,10s/.*/& 1/" train.txt # 1~10是杯子,标签为0
sed -i "11,20s/.*/& 0/" train.txt # 11~20是包包,标签为1,后面省略...
![](https://i-blog.csdnimg.cn/blog_migrate/e03c86c6eeb6c306e6ceddc0e875f3d3.png)
转换数据
在caffe/examples目录下新建目录6class。建立空白文档create_(网络名).sh 我这里是alexnet。里面写入:
#!/usr/bin/env sh
# Create the imagenet lmdb inputs
# N.B. set the path to the imagenet train + val data dirs
set -e
EXAMPLE=examples/6class #改成自己的路径
DATA=data/6class #改成自己的路径
TOOLS=build/tools #改成自己的路径
TRAIN_DATA_ROOT=/home/xc/caffe/data/6class/train/ #改成自己的路径
VAL_DATA_ROOT=/home/xc/caffe/data/6class/val/ #改成自己的路径
# Set RESIZE=true to resize the images to 256x256. Leave as false if images have
# already been resized using another tool.
RESIZE=true
if $RESIZE; then
RESIZE_HEIGHT=227
RESIZE_WIDTH=227
else
RESIZE_HEIGHT=0
RESIZE_WIDTH=0
fi
if [ ! -d "$TRAIN_DATA_ROOT" ]; then
echo "Error: TRAIN_DATA_ROOT is not a path to a directory: $TRAIN_DATA_ROOT"
echo "Set the TRAIN_DATA_ROOT variable in create_imagenet.sh to the path" \
"where the ImageNet training data is stored."
exit 1
fi
if [ ! -d "$VAL_DATA_ROOT" ]; then
echo "Error: VAL_DATA_ROOT is not a path to a directory: $VAL_DATA_ROOT"
echo "Set the VAL_DATA_ROOT variable in create_imagenet.sh to the path" \
"where the ImageNet validation data is stored."
exit 1
fi
echo "Creating train lmdb..."
GLOG_logtostderr=1 $TOOLS/convert_imageset \
--resize_height=$RESIZE_HEIGHT \
--resize_width=$RESIZE_WIDTH \
--shuffle \
$TRAIN_DATA_ROOT \
$DATA/train.txt \
$EXAMPLE/6class_train_lmdb #可以改名字
echo "Creating val lmdb..."
GLOG_logtostderr=1 $TOOLS/convert_imageset \
--resize_height=$RESIZE_HEIGHT \
--resize_width=$RESIZE_WIDTH \
--shuffle \
$VAL_DATA_ROOT \
$DATA/val.txt \
$EXAMPLE/6class_val_lmdb #可以改名字
echo "Done."
返回caffe根目录 运行 sh ./examples/6class/create_alexnet.sh
接下来就会生成俩文件
3.训练数据
一般caffe的网络模型有三个文件deploy.prototxt(在模型训练好后用的)、train_val.prototxt(训练数据时用的)、solver.prototxt(训练时的各种参数)我们用Alexnet网络的模型,github可以找到 放在examples/6class下
(1)修改train_val.prototxt
第一步把data/ilsvrc12下的imagenet_mean.binaryproto复制到该文件夹下,data/6class文件夹下myimagenet_mean.binaryproto没有这个文件,并重命名为6class_mean.binaryproto 。
name: "AlexNet"
layer {
name: "data"
type: "Data"
top: "data"
top: "label"
include {
phase: TRAIN
}
transform_param {
mirror: true
crop_size: 227
mean_file: "data/6class/6class_mean.binaryproto" #改路径
}
data_param {
source: "examples/6class/6class_train_lmdb" #改路径
batch_size: 256
backend: LMDB
}
}
layer {
name: "data"
type: "Data"
top: "data"
top: "label"
include {
phase: TEST
}
transform_param {
mirror: false
crop_size: 227
mean_file: "data/6class/6class_mean.binaryproto" #这里也是
}
data_param {
source: "examples/6class/6class_train_lmdb" #这里
batch_size: 50
backend: LMDB
}
}
layer {
name: "conv1"
type: "Convolution"
bottom: "data"
top: "conv1"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 96
kernel_size: 11
stride: 4
weight_filler {
type: "gaussian"
std: 0.01
}
bias_filler {
type: "constant"
value: 0
}
}
}
后面
layer {
name: "fc8"
type: "InnerProduct"
bottom: "fc7"
top: "fc8"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
inner_product_param {
num_output: 6 #有几类就填几
weight_filler {
type: "gaussian"
std: 0.01
}
bias_filler {
type: "constant"
value: 0
}
}
}
layer {
name: "accuracy"
type: "Accuracy"
bottom: "fc8"
bottom: "label"
top: "accuracy"
include {
phase: TEST
}
}
layer {
name: "loss"
type: "SoftmaxWithLoss"
bottom: "fc8"
bottom: "label"
top: "loss"
}
(2)修改solver.prototxt
test_iter: 1000是指测试的批次,我们就10张照片,设置10就可以了。 test_interval: 1000是指每1000次迭代测试一次,我改成了10。 base_lr: 0.01是基础学习率,因为数据量小,0.01就会下降太快了,因此改成0.001 lr_policy: “step”学习率变化 gamma: 0.1学习率变化的比率 stepsize: 100000每100000次迭代减少学习率 display: 20每20层显示一次 max_iter: 1000最大迭代次数, momentum: 0.9学习的参数,不用变 weight_decay: 0.0005学习的参数,不用变 snapshot: 10000每迭代10000次显示状态,这里改为1000次 solver_mode: GPU末尾加一行,代表用GPU进行
net: "/home/xc/caffe/examples/6class/train_val.prototxt"
test_iter: 10
test_interval: 10
base_lr: 0.001
lr_policy: "step"
gamma: 0.1
stepsize: 100000
display: 20
max_iter: 1000
momentum: 0.9
weight_decay: 0.0005
snapshot: 10000
snapshot_prefix: "/home/xc/caffe/examples/6class/caffe_alexnet_train"
solver_mode: GPU
(3)图像均值
减去图像均值会获得更好的效果,所以我们使用tools/compute_image_mean.cpp实现,这个cpp是一个很好的例子去熟悉如何操作多个组建,例如协议的缓冲区,leveldbs,登录等。在examples/6class下创建make_alexnet_mean.sh内容如下
#!/usr/bin/env sh
# Compute the mean image from the imagenet training lmdb
# N.B. this is available in data/ilsvrc12
EXAMPLE=/home/xc/caffe/examples/6class #自己的地址
DATA=/home/xc/caffe/data/6class
TOOLS=/home/xc/caffe/build/tools
$TOOLS/compute_image_mean $EXAMPLE/6class_train_lmdb \ #自己的文件名字
$DATA/alexnet_mean.binaryproto
echo "Done."
(4)运行
创建train_Alexnet.sh文件到example/myself目录下。内容如下:
#!/usr/bin/env sh
set -e
./build/tools/caffe train \
--solver=/home/xc/caffe/examples/6class/solver.prototxt $@
在caffe的主目录下输入命令:./examples/6class/train_Alexnet.sh开始训练网络。
可能遇到CUDAunsuccess这样的问题 把batch_size弄小就ok 原因是显存不足。
5 . 测试数据
找一个你要测试的图片。
修改deploy.prototxt 并编写一个labels.txt
layer {
name: "fc8"
type: "InnerProduct"
bottom: "fc7"
top: "fc8"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
inner_product_param {
num_output: 6 #改成6
}
}
layer {
name: "prob"
type: "Softmax"
bottom: "fc8"
top: "prob"
}
labels.txt的内容如下:
cup
bag
computer
cat
dog
duck
打开data/6class 将6class_mean.binaryproto 转换成 6classmean.npy 写个python小程序:
import caffe
import numpy as np
proto_path='6class_mean.binaryproto'
npy_path='6classmean.npy'
blob=caffe.proto.caffe_pb2.BlobProto()
data=open(proto_path,'rb').read()
blob.ParseFromString(data)
array=np.array(caffe.io.blobproto_to_array(blob))
mean_npy=array[0]
np.save(npy_path,mean_npy)
把生成的6classmean.npy复制到examples/6class下 再用Python写代码对图片进行分类:
import caffe
import sys
import numpy as np
import time
import cv2
caffe_root='/home/xc/caffe/'
sys.path.insert(0,caffe_root+'python')
caffe.set_mode_gpu()
deploy=caffe_root+'examples/6class/deploy.prototxt'
caffe_model=caffe_root+'examples/6class/caffe_alexnet_train_iter_1000.caffemodel'
img=caffe_root+'examples/6class/1.jpeg'
labels_name=caffe_root+'examples/6class/labels.txt'
mean_file=caffe_root+'examples/6class/6classmean.npy'
net=caffe.Net(deploy,caffe_model,caffe.TEST)
transformer=caffe.io.Transformer({'data':net.blobs['data'].data.shape})
transformer.set_transpose('data',(2,0,1))
transformer.set_mean('data',np.load(mean_file).mean(1).mean(1))
transformer.set_raw_scale('data',255)
transformer.set_channel_swap('data',(2,1,0))
image=caffe.io.load_image(img)
net.blobs['data'].data[...]=transformer.preprocess('data',image)
start =time.clock()
out=net.forward()
end=time.clock()
print('classification time: %f s' % (end - start))
labels=np.loadtxt(labels_name,str,delimiter='\t')
prob=net.blobs['prob'].data[0].flatten()
top_k=net.blobs['prob'].data[0].flatten().argsort()[-1:-6:-1]
for i in np.arange(top_k.size):
print top_k[i],labels[top_k[i]],prob[top_k[i]]
'''
class_name=caffe_root+'example/myself/labels.txt'
category = net.blobs['prob'].data[0].argmax()
class_str = labels[int(category)].split(',')
class_str = labels[int(category)].split(',')
cv2.putText(img, class_name, (0, img.shape[0]), cv2.cv.CV_FONT_HERSHEY_SIMPLEX, 1, (55, 255, 155), 2)
'''
这段程序是自己参考多方面自己写的 可能有些不足
结果:还可以~~~