caffe训练输入数据格式介绍(LMDB/imagelist)

目录

引言

LMDB数据格式

imagelist数据

LMDB数据生成

---

引言

这里介绍caffe训练数据的两种输入格式，分别是LMDB，以及普通的imagelist图片输入。主要贴出两种输入数据格式的prototxt作为参考，具体如何工作的可以看caffe源码，参数的介绍参考caffe proto文件：

LMDB数据格式

layer {
  name: "data"
  type: "Data"
  top: "data"
  top: "label"
  include{
    phase:TRAIN
  }
  transform_param {
    mirror: true
    crop_size: 128
  }
  data_param {
    source: "/PATH/lmdb"
    batch_size: 64
    backend: LMDB
  }
}

imagelist数据

layer {  
  name: "data"  
  type: "ImageData"  
  top: "data"  
  top: "label"  
  include {  
    phase: TRAIN  
  }  
  transform_param {
    mirror: true
    crop_size: 128  
    mean_value: 0 
    force_gray: true
  }  
  image_data_param {  
    source: "/PATH/path_train.txt"  
    batch_size: 64  
    is_color: false
    shuffle: true
  }  
} 

LMDB数据生成

#!/usr/bin/env sh
# Create the imagenet lmdb inputs
# N.B. set the path to the imagenet train + val data dirs
set -e

EXAMPLE=examples/imagenet
DATA=data/ilsvrc12
TOOLS=build/tools

TRAIN_DATA_ROOT=/public/xuhaitao/imageNet/ILSVRC2012_img_train/
VAL_DATA_ROOT=/public/xuhaitao/imageNet/ILSVRC2012_img_val/

# Set RESIZE=true to resize the images to 256x256. Leave as false if images have
# already been resized using another tool.
RESIZE=false
if $RESIZE; then
  RESIZE_HEIGHT=256
  RESIZE_WIDTH=256
else
  RESIZE_HEIGHT=0
  RESIZE_WIDTH=0
fi

if [ ! -d "$TRAIN_DATA_ROOT" ]; then
  echo "Error: TRAIN_DATA_ROOT is not a path to a directory: $TRAIN_DATA_ROOT"
  echo "Set the TRAIN_DATA_ROOT variable in create_imagenet.sh to the path" \
       "where the ImageNet training data is stored."
  exit 1
fi

if [ ! -d "$VAL_DATA_ROOT" ]; then
  echo "Error: VAL_DATA_ROOT is not a path to a directory: $VAL_DATA_ROOT"
  echo "Set the VAL_DATA_ROOT variable in create_imagenet.sh to the path" \
       "where the ImageNet validation data is stored."
  exit 1
fi

echo "Creating train lmdb..."

GLOG_logtostderr=1 $TOOLS/convert_imageset \
    --resize_height=$RESIZE_HEIGHT \
    --resize_width=$RESIZE_WIDTH \
    --shuffle \
    $TRAIN_DATA_ROOT \
    $DATA/train.txt \
    $EXAMPLE/ilsvrc12_train_lmdb

echo "Creating val lmdb..."

GLOG_logtostderr=1 $TOOLS/convert_imageset \
    --resize_height=$RESIZE_HEIGHT \
    --resize_width=$RESIZE_WIDTH \
    --shuffle \
    $VAL_DATA_ROOT \
    $DATA/val.txt \
    $EXAMPLE/ilsvrc12_val_lmdb

echo "Done."