Tensorflow - Dataset 之 repeat(), shuffle(), batch()作用
repeat()
: 该函数让数据集重复的次数, 如没有参数,则数据集可以任意获取
shuffle()
: 打乱数据集的顺序
batch()
: 设置一次操作允许获取的数据个数
import tensorflow as tf
import numpy as np
feature = np.array([1, 2, 3, 4, 5, 6, 7, 8, 9], np.float32)
label = np.array([0, 0, 0, 0, 1, 1, 1, 1, 1])
train_data = tf.data.Dataset.from_tensor_slices((feature, label)) # 定义9个数据的数据集
def print_train_data(data, cnt):
it = data.__iter__()
for i in range(cnt):
x, y = it.next()
print(f"{i}: {x} - {y}")
print_train_data(train_data, 9)
# print_train_data(train_data, 10) # 出错
print("=== after repeat ====")
train_data = train_data.repeat() # 调用该函数后后面可以无限使用该数据集
print_train_data(train_data, 9) # print_train_data(train_data, 10) # 可以无限循环读取: repeat 留空为无限
print("=== after shuffle ====")
train_data = train_data.shuffle(buffer_size=2) # 打乱数据集的顺序( 如果为 1 的话, 打乱顺序则无用)
print_train_data(train_data, 9)
print("=== after batch ====")
dataset_batch = train_data.batch(batch_size=3) # 设置每次回去数据集的数据条数
it = dataset_batch.__iter__()
print_train_data(it, 9)
打印的数据如下:
0: 1.0 - 0
1: 2.0 - 0
2: 3.0 - 0
3: 4.0 - 0
4: 5.0 - 1
5: 6.0 - 1
6: 7.0 - 1
7: 8.0 - 1
8: 9.0 - 1
=== after repeat ====
0: 1.0 - 0
1: 2.0 - 0
2: 3.0 - 0
3: 4.0 - 0
4: 5.0 - 1
5: 6.0 - 1
6: 7.0 - 1
7: 8.0 - 1
8: 9.0 - 1
=== after shuffle ====
0: 2.0 - 0
1: 3.0 - 0
2: 1.0 - 0
3: 5.0 - 1
4: 6.0 - 1
5: 7.0 - 1
6: 4.0 - 0
7: 9.0 - 1
8: 8.0 - 1
=== after batch ====
0: [2. 3. 4.] - [0 0 0]
1: [5. 1. 6.] - [1 0 1]
2: [8. 7. 9.] - [1 1 1]
3: [1. 3. 4.] - [0 0 0]
4: [2. 5. 6.] - [0 1 1]
5: [8. 9. 7.] - [1 1 1]
6: [2. 1. 4.] - [0 0 0]
7: [5. 3. 6.] - [1 0 1]
8: [7. 8. 9.] - [1 1 1]
reference
@online{BibEntry2022May,
title = {{Tensorflow - Dataset 之 repeat(), shuffle(), batch()作用_aaronychen的博客-CSDN博客_train_dataset.shuffle}},
year = {2022},
month = may,
date = {2022-05-13},
urldate = {2022-05-13},
language = {chinese},
hyphenation = {chinese},
note = {[Online; accessed 13. May 2022]},
url = {https://blog.csdn.net/aaronychen/article/details/122879141},
keywords = {train_dataset.shuffle},
abstract = {{该文章简要描述了tensorflow 下 DataSet 一些函数的基本操作}}
}