Tensorflow - Dataset 之 repeat(), shuffle(), batch()作用

Tensorflow - Dataset 之 repeat(), shuffle(), batch()作用

repeat(): 该函数让数据集重复的次数， 如没有参数，则数据集可以任意获取

shuffle(): 打乱数据集的顺序

batch(): 设置一次操作允许获取的数据个数

import tensorflow as tf
import numpy as np

feature = np.array([1, 2, 3, 4, 5, 6, 7, 8, 9], np.float32)

label = np.array([0, 0, 0, 0, 1, 1, 1, 1, 1])

train_data = tf.data.Dataset.from_tensor_slices((feature, label))  # 定义9个数据的数据集


def print_train_data(data, cnt):
    it = data.__iter__()

    for i in range(cnt):
        x, y = it.next()

        print(f"{i}: {x} - {y}")


print_train_data(train_data, 9)
# print_train_data(train_data, 10)  # 出错

print("=== after repeat ====")

train_data = train_data.repeat()  # 调用该函数后后面可以无限使用该数据集

print_train_data(train_data, 9)  # print_train_data(train_data, 10)  # 可以无限循环读取: repeat 留空为无限

print("=== after shuffle ====")

train_data = train_data.shuffle(buffer_size=2)  # 打乱数据集的顺序( 如果为 1 的话, 打乱顺序则无用)

print_train_data(train_data, 9)

print("=== after batch ====")

dataset_batch = train_data.batch(batch_size=3)  # 设置每次回去数据集的数据条数 

it = dataset_batch.__iter__()

print_train_data(it, 9)

打印的数据如下:

0: 1.0 - 0
1: 2.0 - 0
2: 3.0 - 0
3: 4.0 - 0
4: 5.0 - 1
5: 6.0 - 1
6: 7.0 - 1
7: 8.0 - 1
8: 9.0 - 1
=== after repeat ====
0: 1.0 - 0
1: 2.0 - 0
2: 3.0 - 0
3: 4.0 - 0
4: 5.0 - 1
5: 6.0 - 1
6: 7.0 - 1
7: 8.0 - 1
8: 9.0 - 1
=== after shuffle ====
0: 2.0 - 0
1: 3.0 - 0
2: 1.0 - 0
3: 5.0 - 1
4: 6.0 - 1
5: 7.0 - 1
6: 4.0 - 0
7: 9.0 - 1
8: 8.0 - 1
=== after batch ====
0: [2. 3. 4.] - [0 0 0]
1: [5. 1. 6.] - [1 0 1]
2: [8. 7. 9.] - [1 1 1]
3: [1. 3. 4.] - [0 0 0]
4: [2. 5. 6.] - [0 1 1]
5: [8. 9. 7.] - [1 1 1]
6: [2. 1. 4.] - [0 0 0]
7: [5. 3. 6.] - [1 0 1]
8: [7. 8. 9.] - [1 1 1]

---

reference

@online{BibEntry2022May,
title = {{Tensorflow - Dataset 之 repeat(), shuffle(), batch()作用_aaronychen的博客-CSDN博客_train_dataset.shuffle}},
year = {2022},
month = may,
date = {2022-05-13},
urldate = {2022-05-13},
language = {chinese},
hyphenation = {chinese},
note = {[Online; accessed 13. May 2022]},
url = {https://blog.csdn.net/aaronychen/article/details/122879141},
keywords = {train_dataset.shuffle},
abstract = {{该文章简要描述了tensorflow 下 DataSet 一些函数的基本操作}}
}