【Pytorch学习】-- torchvision.datasets&DataLoader使用

最新推荐文章于 2024-03-07 22:35:20 发布

丶Dylan

最新推荐文章于 2024-03-07 22:35:20 发布

阅读量786

点赞数 3

文章标签： pytorch 深度学习机器学习

本文链接：https://blog.csdn.net/weixin_45591452/article/details/122730671

版权

学习视频：https://www.bilibili.com/video/BV1hE411t7RN?p=1，内含环境搭建

torchvision.datasets使用

Pytorch有自带许多数据集可供学习使用，因此，本次挑选一个来进行学习，官方地址

CIFAR10

本次使用的是CIFAR10数据集。

import torchvision
train_set = torchvision.datasets.CIFAR10(root = "./CIFAR10_Dataset",train = True,download = True)
test_set = torchvision.datasets.CIFAR10(root = "./CIFAR10_Dataset",train = False,download = True)
# root：下载的根目录
# train：true为训练集，false为测试集
# download：是否从网上下载，若下载速度慢可以用迅雷下载：
# https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz

查看一下数据集的数据是什么类型的

print(test_set[0])

由输出结果可以看出是一个PIL Image类型，"3"代表类别时三，CIFAR10有10个类别，具体类别参考官方文档

(<PIL.Image.Image image mode=RGB size=32x32 at 0x281591063D0>, 3)

转化为Tensor类型

在调用参数中多加一个自带的transform的参数：

test_data = torchvision.datasets.CIFAR10(root = "./CIFAR10_Dataset",
                                                    train = False,
                                                    transform = torchvision.transforms.ToTensor())
print(test_data[0])

得到的就是tensor数据类型

(tensor([[[0.6196, 0.6235, 0.6471,  ..., 0.5373, 0.4941, 0.4549],
         [0.5961, 0.5922, 0.6235,  ..., 0.5333, 0.4902, 0.4667],
         [0.5922, 0.5922, 0.6196,  ..., 0.5451, 0.5098, 0.4706],
         ...,
         [0.2667, 0.1647, 0.1216,  ..., 0.1490, 0.0510, 0.1569],
         [0.2392, 0.1922, 0.1373,  ..., 0.1020, 0.1137, 0.0784],
         [0.2118, 0.2196, 0.1765,  ..., 0.0941, 0.1333, 0.0824]],

        [[0.4392, 0.4353, 0.4549,  ..., 0.3725, 0.3569, 0.3333],
         [0.4392, 0.4314, 0.4471,  ..., 0.3725, 0.3569, 0.3451],
         [0.4314, 0.4275, 0.4353,  ..., 0.3843, 0.3725, 0.3490],
         ...,
         [0.4863, 0.3922, 0.3451,  ..., 0.3804, 0.2510, 0.3333],
         [0.4549, 0.4000, 0.3333,  ..., 0.3216, 0.3216, 0.2510],
         [0.4196, 0.4118, 0.3490,  ..., 0.3020, 0.3294, 0.2627]],

        [[0.1922, 0.1843, 0.2000,  ..., 0.1412, 0.1412, 0.1294],
         [0.2000, 0.1569, 0.1765,  ..., 0.1216, 0.1255, 0.1333],
         [0.1843, 0.1294, 0.1412,  ..., 0.1333, 0.1333, 0.1294],
         ...,
         [0.6941, 0.5804, 0.5373,  ..., 0.5725, 0.4235, 0.4980],
         [0.6588, 0.5804, 0.5176,  ..., 0.5098, 0.4941, 0.4196],
         [0.6275, 0.5843, 0.5176,  ..., 0.4863, 0.5059, 0.4314]]]), 3)

DataLoader使用

Dataloader就是把数据整理成适合输入到神经网络形式的工具

from torch.utils.data import DataLoader

test_loader = DataLoader(dataset = test_data,
                        batch_size = 4,   	#  how many samples per batch to load 一次取多少
                        shuffle = True,  	# set to True to have the data reshuffled at every epoch 是否打乱
                        num_workers = 0, 	# how many subprocesses to use for data loading 多少个子进程进行读取
                        drop_last = False)  # set to True to drop the last incomplete batch,if the dataset size is not divisible by the batch size. 是否丢弃最后不能形成一个batch的数据

for data in test_loader:
    imgs,targets = data
    print(imgs.shape) # batch_size = 4：四张图，每张图3通道，像素为32*32
    print(targets)    # 四张图的所对应的类别

一个输出结果

torch.Size([4, 3, 32, 32])
tensor([0, 8, 5, 1])

借助TensorBoard更直观地查看：

from torch.utils.tensorboard import SummaryWriter

test_loader = DataLoader(dataset = test_data,
                        batch_size = 64,   
                        shuffle = True,  
                        num_workers = 0, 
                        drop_last = False)

writer = SummaryWriter("dataloader")
step = 0
for data in test_loader:
    imgs,targets = data
    writer.add_images("test_data",imgs,step)
    step = step + 1
    
writer.close()