场景分类总结-caffe版

     参加一个场景分类的比赛。数据集：http://ai.futurelab.tv/dataset/view。解压码：QBRE，数据集不可以用作商业用途。这是一个20类的场景分类，用的方法是https://github.com/CSAILVision/places365。利用这里面的方法进行caffe微调。

一、制作数据集

    参考：caffe学习系列：训练自己的图片集（超详细教程）点击打开链接

1.1原数据集分类

list.csv中存放图片场景标注信息，categories.csv中存放场景分类信息，总共是20类。

 

1、每一类500-1500张图片，首先根据csv文件，将20类图片分到每个文件夹下。数据放在caffe/data下，程序见category_classification.py。

2、将数据统一命名，程序见rename.py

# -*- coding:utf8 -*-
#!/usr/bin/python2.7
import os

class BatchRename():
    '''
    批量重命名文件夹中的图片文件

    '''
    def __init__(self):
        self.path = '/home/ouc/workspace-sjh/caffe/data/testdata/train/3'

    def rename(self):
        filelist = os.listdir(self.path)
        total_num = len(filelist)
        i = 1
	
        for item in filelist:
            if item.endswith('.jpg'):
                src = os.path.join(os.path.abspath(self.path), item)
		str1=str(i)
                dst = os.path.join(os.path.abspath(self.path), str1.zfill(6) + '.jpg')
                try:
                    os.rename(src, dst)
                    print 'converting %s to %s ...' % (src, dst)
                    i = i + 1
                except:
                    continue
        print 'total %d to rename & converted %d jpgs' % (total_num, i)

if __name__ == '__main__':
    demo = BatchRename()
    demo.rename()

3、为了提高分类效果，将数据扩增，使用keras，将每一类的数据扩增到2000，程序见imgAug.py    

#!/usr/bin/env python
#-*- coding: utf-8 -*-
#Author: yuanyong.name

import os
import random
from keras.preprocessing.image import ImageDataGenerator, array_to_img, img_to_array, load_img

num_wanted = 2000

target_path_dir = '/home/ouc/workspace-sjh/caffe/data/testdata/train/19'

datagen = ImageDataGenerator(
        rotation_range = 10,
        width_shift_range = 0.2,
        height_shift_range = 0.2,
        shear_range = 0.2,
        zoom_range = 0.2,
        horizontal_flip = True,
        fill_mode = 'nearest')

sub_dirs = os.walk(target_path_dir).next()
for sub_dir in enumerate(sub_dirs):
    # print 'to be processed: %s (%d/%d), contains images: %d' % (sub_dirs, k+1, len(sub_dirs), sub_dir, len(img_basenames))
    sub_dir_new = sub_dir[1]
    sub_dir_full = os.path.join(target_path_dir, sub_dir_new)
    img_basenames = os.listdir(sub_dir_full)
    num_imgs = len(img_basenames)
    num_perAug = int(float(num_wanted)/float(num_imgs)) - 1
    if num_imgs >= num_wanted:
        continue
    num_total = 0
    for i, img_basename in enumerate(img_basenames):
        num_total = num_imgs + i*num_perAug
        if num_total >= num_wanted:
            break
        img_path = os.path.join(sub_dir_full, img_basename)
        #print "Aug: %s" % img_path
        img = load_img(img_path) # this is a PIL image, please replace to your own file path
        if img == None:
            continue
        try:
            x =