ssd-tensorfow-slim生成tfrecord

原

Notes on tensorflow（八）read tfrecords with slim

2017年04月30日 16:21:34 Daniel2333 阅读数：8507

																														</div>
			<div class="operating">
								</div>
		</div>
	</div>
</div>
<article class="baidu_pl">
	<div id="article_content" class="article_content clearfix csdn-tracking-statistics" data-pid="blog" data-mod="popu_307" data-dsm="post">
							<div class="article-copyright">
              					<svg class="icon" title="CSDN认证原创" aria-hidden="true" style="width:53px; height: 18px; vertical-align: -4px;">
						<use xlink:href="#CSDN_Cert"></use>
				</svg>
              					
				版权声明：本文为博主原创文章，转载请注明出处.					https://blog.csdn.net/weixin_35653315/article/details/71023596				</div>
							<link rel="stylesheet" href="https://csdnimg.cn/release/phoenix/template/css/ck_htmledit_views-f57960eb32.css">
							            <div id="content_views" class="markdown_views">
						<!-- flowchart 箭头图标 勿删 -->
						<svg xmlns="http://www.w3.org/2000/svg" style="display: none;"><path stroke-linecap="round" d="M5,0 0,2.5 5,5z" id="raphael-marker-block" style="-webkit-tap-highlight-color: rgba(0, 0, 0, 0);"></path></svg>
						<p><a href="http://blog.csdn.net/weixin_35653315/article/details/71015845" rel="nofollow" target="_blank">http://blog.csdn.net/weixin_35653315/article/details/71015845</a> 中描述了将pascal voc数据集转换tfrecord的过程。本文则通过slim来读取生成的tfrecord。读取和解码操作由<code>tf.TFRecordReader</code>完成</p>

import tensorflow as tf
slim = tf.contrib.slim
file_pattern = './pascal_train_*.tfrecord' #文件名格式

# 适配器1：将example反序列化成存储之前的格式。由tf完成
keys_to_features = {
    'image/encoded': tf.FixedLenFeature((), tf.string, default_value=''),
    'image/format': tf.FixedLenFeature((), tf.string, default_value='jpeg'),
    'image/height': tf.FixedLenFeature([1], tf.int64),
    'image/width': tf.FixedLenFeature([1], tf.int64),
    'image/channels': tf.FixedLenFeature([1], tf.int64),
    'image/shape': tf.FixedLenFeature([3], tf.int64),
    'image/object/bbox/xmin': tf.VarLenFeature(dtype=tf.float32),
    'image/object/bbox/ymin': tf.VarLenFeature(dtype=tf.float32),
    'image/object/bbox/xmax': tf.VarLenFeature(dtype=tf.float32),
    'image/object/bbox/ymax': tf.VarLenFeature(dtype=tf.float32),
    'image/object/bbox/label': tf.VarLenFeature(dtype=tf.int64),
    'image/object/bbox/difficult': tf.VarLenFeature(dtype=tf.int64),
    'image/object/bbox/truncated': tf.VarLenFeature(dtype=tf.int64),
}

#适配器2：将反序列化的数据组装成更高级的格式。由slim完成
items_to_handlers = {
    'image': slim.tfexample_decoder.Image('image/encoded', 'image/format'),
    'shape': slim.tfexample_decoder.Tensor('image/shape'),
    'object/bbox': slim.tfexample_decoder.BoundingBox(
            ['ymin', 'xmin', 'ymax', 'xmax'], 'image/object/bbox/'),
    'object/label': slim.tfexample_decoder.Tensor('image/object/bbox/label'),
    'object/difficult': slim.tfexample_decoder.Tensor('image/object/bbox/difficult'),
    'object/truncated': slim.tfexample_decoder.Tensor('image/object/bbox/truncated'),
}

# 解码器
decoder = slim.tfexample_decoder.TFExampleDecoder(keys_to_features, items_to_handlers)

# dataset对象定义了数据集的文件位置，解码方式等元信息
dataset = slim.dataset.Dataset(
            data_sources=file_pattern,
            reader=tf.TFRecordReader,
            num_samples = 3, # 手动生成了三个文件， 每个文件里只包含一个example
            decoder=decoder,
            items_to_descriptions = {},
            num_classes=21)

#provider对象根据dataset信息读取数据
provider = slim.dataset_data_provider.DatasetDataProvider(
                    dataset,
                    num_readers=3,
                    shuffle=False)

[image, shape, glabels, gbboxes] = provider.get(['image', 'shape',
                                                             'object/label',
                                                             'object/bbox'])
print type(image)
print image.shape
 
 

 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55

<class 'tensorflow.python.framework.ops.Tensor'>
(?, ?, 3)

 
 

 
 
1
2

到目前为止，返回的image是一个tensor，而且是一个3维的，一次只有一张。需要组成一个batch。组成batch之前图片要预处理，一是为了将图片size变成固定大小， 二是数据增长。上面的示例代码来自https://github.com/balancap/SSD-Tensorflow/blob/master/datasets/pascalvoc_common.py#L49
。下面的示例代码来自 https://github.com/balancap/SSD-Tensorflow/blob/master/train_ssd_network.py#L203

# Pre-processing image, labels and bboxes.
image, glabels, gbboxes = \
    image_preprocessing_fn(image, glabels, gbboxes,
                           out_shape=ssd_shape,
                           data_format=DATA_FORMAT)
# Encode groundtruth labels and bboxes.
gclasses, glocalisations, gscores = \
    ssd_net.bboxes_encode(glabels, gbboxes, ssd_anchors)
batch_shape = [1] + [len(ssd_anchors)] * 3

# Training batches and queue.
r = tf.train.batch(
    tf_utils.reshape_list([image, gclasses, glocalisations, gscores]),
    batch_size=FLAGS.batch_size,
    num_threads=FLAGS.num_preprocessing_threads,
    capacity=5 * FLAGS.batch_size)
b_image, b_gclasses, b_glocalisations, b_gscores = \
    tf_utils.reshape_list(r, batch_shape)

# Intermediate queueing: unique batch computation pipeline for all
# GPUs running the training.
batch_queue = slim.prefetch_queue.prefetch_queue(
    tf_utils.reshape_list([b_image, b_gclasses, b_glocalisations, b_gscores]),
    capacity=2 * deploy_config.num_clones)

 
 

 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

					<link href="https://csdnimg.cn/release/phoenix/mdeditor/markdown_views-258a4616f7.css" rel="stylesheet">
                  </div>
</article>