环境:
- Linux 系统
- Python3.5
- Tensorflow-gpu 1.4.0
- TITAN Xp
- tf-faster-rcnn
此篇文章主要记录怎样快速使用tf-faster-rcnn训练自己的数据集,其他的问题可参考其他博客。
根据提示缺少的库,自行安装即可。
一、下载代码
git clone https://github.com/endernewton/tf-faster-rcnn.git
二、根据下图更改下对应的参数
根据自己的显卡 在…/lib/setup.py文件 130行 修改适合的 -arch
三、编译cython模块
在lib路径下,编译Cython模块
make clean
make
cd ..
四、安装COCO API
cd data
git clone https://github.com/pdollar/coco.git
cd coco/PythonAPI
make
cd ../../..
五、Beyond the demo: installation for training and testing models
- Download the training, validation, test data and VOCdevkit
wget http://host.robots.ox.ac.uk/pascal/VOC/voc2007/VOCtrainval_06-Nov-2007.tar
wget http://host.robots.ox.ac.uk/pascal/VOC/voc2007/VOCtest_06-Nov-2007.tar
wget http://host.robots.ox.ac.uk/pascal/VOC/voc2007/VOCdevkit_08-Jun-2007.tar
- Extract all of these tars into one directory named VOCdevkit
tar xvf VOCtrainval_06-Nov-2007.tar
tar xvf VOCtest_06-Nov-2007.tar
tar xvf VOCdevkit_08-Jun-2007.tar
- It should have this basic structure
$VOCdevkit/ # development kit
$VOCdevkit/VOCcode/ # VOC utility code
$VOCdevkit/VOC2007 # image sets, annotations, etc.
# ... and several other directories ...
- 把VOCdevkit文件夹拷贝到tf-fater-rcnn/data路径下,并重命名为VOCdevkit2007
cp -r VOCdevkit /home/ql/env3.6/tf-faster-rcnn/data/VOCdevkit2007
当然,你也可以建立软连接,但是我的有问题,也不愿深究。
六、下载预训练模型
- Download pre-trained model
# Resnet101 for voc pre-trained on 07+12 set
./data/scripts/fetch_faster_rcnn_models.sh
或者网盘下载:Resnet101 for voc pre-trained on 07+12 set 提取码:un64
- Create a folder and a soft link to use the pre-trained model
NET=res101
TRAIN_IMDB=voc_2007_trainval+voc_2012_trainval
mkdir -p output/${NET}/${TRAIN_IMDB}
cd output/${NET}/${TRAIN_IMDB}
ln -s ../../../data/voc_2007_trainval+voc_2012_trainval ./default
cd ../../..
- Demo for testing on custom images
# at repository root
GPU_ID=0
CUDA_VISIBLE_DEVICES=${GPU_ID} ./tools/demo.py
3.Test with pre-trained Resnet101 models
把 tf-faster-rcnn/lib/datasets/voc_eval.py的第121行的
with open(cachefile,‘w’) as f
改成:
with open(cachefile,‘wb’) as f
同时还要把第105行的
cachefile = os.path.join(cachedir, ‘%s_annots.pkl’ % imagesetfile)
改成:
cachefile = os.path.join(cachedir, ‘%s_annots.pkl’ % imagesetfile.split("/")[-1].split(".")[0])
GPU_ID=0
./experiments/scripts/test_faster_rcnn.sh $GPU_ID pascal_voc_0712 res101
在此,表明此模型安装成功且能正常运行。
七、训练自己的数据
注意点:
- 最好使用labelImg对数据进行标注。(保存格式为xml)labelImg
如果你的数据是txt格式,可以使用txtToxml直接转成xml即可 - 图片与xml文件命名格式为六位数字组成。(eg:000001.jpg)data_rename
- 将图片与xml文件分别替换在下面文件夹:
tf-faster-rcnn/data/VOCdevkit2007/VOC2007/JPEGImages
tf-faster-rcnn/data/VOCdevkit2007/VOC2007/Annotations
- 划分训练验证以及测试集
Creat_FRCNN_DataSet
得到test.txt,train.txt,trainval.txt,val.txt 4个文件。将这4个文件放在
tf-faster-rcnn/data/VOCdevkit2007/VOC2007/ImageSets/Layout
tf-faster-rcnn/data/VOCdevkit2007/VOC2007/ImageSets/Main - 下载权重数据
vgg16和res101提取码:1d51 - 修改类别
在tf-faster-rcnn/lib/datasets目录下的pascal_voc.py里第36行更改自己的类别,'background’切记不可删掉,把后面的原来的20个label换成自己的.(类别用小写,别太长。)
- 开始训练
迭代次数设置在
./experiments/scripts/train_faster_rcnn.sh,同时把
./experiments/scripts/test_faster_rcnn.sh的ITERS也相应修改。
#创建文件夹,重命名文件
mkdir -p data/imagenet_weights
mv vgg_16.ckpt vgg16.ckpt
#开始训练 vgg16对应权重数据,0是GPU的ID,pascal_voc是训练使用的数据集
#在开始训练之前,还需要把之前训练产生的模型以及cache删除掉,分别在
#tf-faster-rcnn/output/vgg16/voc_2007_trainval/default
#tf-faster-rcnn/data/cache
./experiments/scripts/train_faster_rcnn.sh 0 pascal_voc vgg16
./experiments/scripts/train_faster_rcnn.sh 0 pascal_voc res101
训练过程:
八、可能会遇到的报错
1 cls = self._class_to_ind[obj.find(‘name’).text.lower().strip()
Set proposal method: gt
Appending horizontally-flipped training examples…
objs : [<Element ‘object’ at 0x7fe2660a85d0>, <Element ‘object’ at 0x7fe2660a8850>, <Element ‘object’ at 0x7fe2660a8d50>]
Traceback (most recent call last):
File “./tools/train_net.py”, line 104, in
imdb, roidb = combined_roidb(args.imdb_name)
File “./tools/train_net.py”, line 69, in combined_roidb
roidbs = [get_roidb(s) for s in imdb_names.split(’+’)]
File “./tools/train_net.py”, line 66, in get_roidb
roidb = get_training_roidb(imdb)
File “/home/sai/py-faster-rcnn/tools/…/lib/fast_rcnn/train.py”, line 118, in get_training_roidb
imdb.append_flipped_images()
File “/home/sai/py-faster-rcnn/tools/…/lib/datasets/imdb.py”, line 106, in append_flipped_images
boxes = self.roidb[i][‘boxes’].copy()
File “/home/sai/py-faster-rcnn/tools/…/lib/datasets/imdb.py”, line 67, in roidb
self._roidb = self.roidb_handler()
File “/home/sai/py-faster-rcnn/tools/…/lib/datasets/pascal_voc.py”, line 111, in gt_roidb
for index in self.image_index]
File “/home/sai/py-faster-rcnn/tools/…/lib/datasets/pascal_voc.py”, line 217, in _load_pascal_annotation
cls = self._class_to_ind[obj.find(‘name’).text.lower().strip()]
KeyError: ‘chair’
解决办法
打开tf-faster-rcnn/lib/datasets/pascal_voc.py
in _load_pascal_annotation method, look for something like
objs = diff_objs (or non_diff_objs)
After that line insert something similar to below code:
cls_objs = [obj for obj in objs if obj.find(‘name’).text in self._classes]
objs = cls_objs
参考博客:
https://blog.csdn.net/char_QwQ/article/details/80980505
https://blog.csdn.net/attitude_yu/article/details/81906193