解决：pytorch同时让两个dataloader打乱的顺序是相同

最新推荐文章于 2024-04-07 14:08:18 发布

碧寒

最新推荐文章于 2024-04-07 14:08:18 发布

阅读量1.1k

点赞数 3

文章标签： pytorch 深度学习 python

本文链接：https://blog.csdn.net/qq_39324954/article/details/128102254

版权

class SamplerDef(object):

    def __init__(self, data_source, indices):
        self.data_source = data_source
        self.indices = indices

    def __iter__(self):
        return iter(self.indices)

    def __len__(self):
        return len(self.data_source)

n = len(dataset_train1)
indices = torch.randperm(n)
mySampler = SamplerDef(data_source=dataset_train1, indices=indices)

ITS_train_loader1 = torch.utils.data.DataLoader(dataset_train1, batch_size=BATCH_SIZE, shuffle=False,pin_memory=True, sampler=mySampler)
ITS_train_loader2 = torch.utils.data.DataLoader(dataset_train2, batch_size=BATCH_SIZE, shuffle=False,pin_memory=True, sampler=mySampler)

解决了可以同步的问题，但是每次获取的样本都一样，不满足要求，想将两个数据集的打乱读，而且还要同步，每次还不一样。

解决：

class MyDataset(Dataset):
    def __init__(self, datasetA, datasetB):
        self.datasetA = datasetA
        self.datasetB = datasetB
        
    def __getitem__(self, index):
        xA = self.datasetA[index]
        xB = self.datasetB[index]
        return xA, xB
    
    def __len__(self):
        return len(self.datasetA)
    
datasetA = ...
datasetB = ...
dataset = MyDataset(datasetA, datasetB)
loader = DataLoader(dataset, batch_size=10, shuffle=True)