Pytorch01——数据集介绍
参考:https://pytorch.org/docs/stable/torchvision/datasets.html
本文主要是对Pytorch图像数据集的官方文档翻译,以及梳理和总结。有错误的地方请诸位大佬指正!转载请注明来源!
主要涉及Libraries库中的torchvision.datasets。
目录
整体介绍
用于image classification:
手写字符识别:EMNIST、MNIST、QMNIST、USPS、SVHN、KMNIST、Omniglot
实物分类:Fashion MNIST、CIFAR、LSUN、SLT-10、ImageNet
人脸识别:CelebA
场景分类:LSUN、Places365
用于object detection:SVHN、VOCDetection、COCODetection
用于semantic/instance segmentation:
语义分割:Cityscapes、VOCSegmentation
语义边界:SBD
用于image captioning:Flickr、COCOCaption
用于video classification:HMDB51、Kinetics
用于3D reconstruction:PhotoTour
用于shadow detectors:SBU
EMNIST
torchvision.datasets.EMNIST(root: str, split: str, **kwargs: Any)
Parameters:
root (string) – Root directory of dataset where EMNIST/processed/training.pt and EMNIST/processed/test.pt exist.
split (string) – The dataset has 6 different splits: byclass, bymerge, balanced, letters, digits and mnist. This argument specifies which one to use.
train (bool, optional) – If True, creates dataset from training.pt, otherwise from test.pt.
download (bool, optional) – If true, downloads the dataset from the internet and puts it in root directory. If dataset is already downloaded, it is not downloaded again.
transform (callable, optional) – A function/transform that takes in an PIL image and returns a transformed version. E.g, transforms.RandomCrop
target_transform (callable, optional) – A function/transform that takes in the target and transforms it.
下载地址:
https://www.nist.gov/itl/products-and-services/emnist-dataset
作者:
Gregory Cohen, Saeed Afshar, Jonathan Tapson, Andre van Schaik
The MARCS Institute for Brain, Behaviour and Development, Western Sydney University
引用:
Cohen, G., Afshar, S., Tapson, J., & van Schaik, A. (2017). EMNIST: an extension of MNIST to handwritten letters. Retrieved from http://arxiv.org/abs/1702.05373
简介:
EMNIST来自NIST Special Database 19,包含了数字和大小写字母。大小为1.65GB,分为6部分:
By Class和By Merge的数据分布:
Balanced:
Letters:
Digits 和 MNIST:
MNIST
torchvision.datasets.MNIST(root: str, train: bool = True, transform: Optional[Callable] = None, target_transform: Optional[Callable] = None, download: bool = False)
Parameters:
root (string) – Root directory of dataset where MNIST/processed/training.pt and MNIST/processed/test.pt exist.
train (bool, optional) – If True, creates dataset from training.pt, otherwise from test.pt.
download (bool, optional) – If true, downloads the dataset from the internet and puts it in root directory. If dataset is already downloaded, it is not downloaded again.
transform (callable, optional) – A function/transform that takes in an PIL image and returns a transformed version. E.g, transforms.RandomCrop
target_transform (callable, optional) – A function/transform that takes in the target and transforms it.
下载地址:
http://yann.lecun.com/exdb/mnist/
作者:
Yann LeCun, Courant Institute,,NYU
Corinna Cortes,Google Labs,New York
Christopher J.C. Burges,Microsoft Research,Redmond
简介:
前世:MNIST由NIST的Special Database 3和Special Database 1组成。
- 在NIST中SD-3由Census Bureau的雇员书写,作为训练集,SD-1由高中生书写,作为测试集,SD-3比SD-1更干净和便于识别。SD-1包含了58527张来自500位作者的手写数字图片。SD-3的数据是顺序写入的,同一个人写的10个数字是放在一起的,SD-1的数据是打乱的,但是数据中包含了作者的ID。
- 所以将SD-1的前250个作者写的近3万数字图像放入MNIST的训练集,剩下的由SD-3补全至6万张训练集。将SD-1的后250个作者写的3万数字图像放入MNIST的测试集,剩下的由SD-3补全至6万张测试集。但是,只能下载到6万张的训练集,和1万张的测试集(从6万张测试集选出)。
今生:MNIST训练集6万张图片,分别从SD-3和SD-1中选择3万张,测试集1万张图片,分别从SD-3和SD-1中选择5千张。训练集6万张图片大约来自250位作者,训练集和测试集中的作者不相交。
- 手写数字识别,样本为28*28的二值图,数字尺度统一,数字质心在图片正中。
- 训练集60k,测试集10k,共70k。分为10个数字类别,每类的图片数量相同。
QMNIST
torchvision.datasets.QMNIST(root: str, what: Optional[str] = None, compat: bool = True, train: bool = True, **kwargs: Any)
Parameters:
root (string) – Root directory of dataset whose ``processed’’ subdir contains torch binary files with the datasets.
what (string,optional) – Can be ‘train’, ‘test’, ‘test10k’, ‘test50k’, or ‘nist’ for respectively the mnist compatible training set, the 60k qmnist testing set, the 10k qmnist examples that match the mnist testing set, the 50k remaining qmnist testing examples, or all the nist digits. The default is to select ‘train’ or ‘test’ according to the compatibility argument ‘train’.
compat (bool,optional) – A boolean that says whether the target for each example is class number (for compatibility with the MNIST dataloader) or a torch vector containing the full qmnist information. Default=True.
download (bool, optional) – If true, downloads the dataset from the internet and puts it in root directory. If dataset is already downloaded, it is not downloaded again.
transform (callable, optional) – A function/transform that takes in an PIL image and returns a transformed version. E