AttributeError: Can't pickle local object 'BaseDataset.img_transformer.<locals>.<lambda>'

目录：

1. 问题：

2. 最终解决方法：

3. 解决问题的过程： 

      1）另外定义一个函数放在transforms.Lambda()括号里面，没有用。

      2）import dill as pickle，没有用。

      3）在命令后添加 --workers 0，没有用。

      4）有人说是windows下pickling、multiprocessing的问题。

      5）找出项目里含有workers的文件，把workers的值改为0，解决问题。

---

1. 问题：

File "E:\ganimation_replicate-master\data\data_loader.py", line 46, in __iter__
    for i, data in enumerate(self.dataloader):

File "E:\Anaconda3\envs\pytorch36\lib\multiprocessing\reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object 'BaseDataset.img_transformer.<locals>.<lambda>'

  File "E:\Anaconda3\envs\pytorch36\lib\multiprocessing\spawn.py", line 115, in _main
    self = reduction.pickle.load(from_parent)
EOFError: Ran out of input

Traceback (most recent call last):
  File "main.py", line 16, in <module>
    solver.run_solver()
  File "E:\ganimation_replicate-master\solvers.py", line 38, in run_solver
    self.test_networks(self.opt)
  File "E:\ganimation_replicate-master\solvers.py", line 106, in test_networks
    self.test_ops()
  File "E:\ganimation_replicate-master\solvers.py", line 113, in test_ops
    for batch_idx, batch in enumerate(self.test_dataset):
  File "E:\ganimation_replicate-master\data\data_loader.py", line 46, in __iter__
    for i, data in enumerate(self.dataloader):
  File "E:\Anaconda3\envs\pytorch36\lib\site-packages\torch\utils\data\dataloader.py", line 451, in __iter__
    return _DataLoaderIter(self)
  File "E:\Anaconda3\envs\pytorch36\lib\site-packages\torch\utils\data\dataloader.py", line 239, in __init__
    w.start()
  File "E:\Anaconda3\envs\pytorch36\lib\multiprocessing\process.py", line 105, in start
    self._popen = self._Popen(self)
  File "E:\Anaconda3\envs\pytorch36\lib\multiprocessing\context.py", line 223, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "E:\Anaconda3\envs\pytorch36\lib\multiprocessing\context.py", line 322, in _Popen
    return Popen(process_obj)
  File "E:\Anaconda3\envs\pytorch36\lib\multiprocessing\popen_spawn_win32.py", line 65, in __init__
    reduction.dump(process_obj, to_child)
  File "E:\Anaconda3\envs\pytorch36\lib\multiprocessing\reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object 'BaseDataset.img_transformer.<locals>.<lambda>'

(pytorch36) E:\ganimation_replicate-master>Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "E:\Anaconda3\envs\pytorch36\lib\multiprocessing\spawn.py", line 105, in spawn_main
    exitcode = _main(fd)
  File "E:\Anaconda3\envs\pytorch36\lib\multiprocessing\spawn.py", line 115, in _main
    self = reduction.pickle.load(from_parent)

2. 最终解决方法：

https://github.com/matterport/Mask_RCNN/issues/93   直到我看到了这篇博客。我就开始搜索我项目里的所有.py代码文件，一个文件一个文件的Ctrl+F搜索，看代码里面是否含有workers、mutilprocessing这两个关键字。最终，我在我项目的data_loader.py文件里搜索到了workers，把workers的值改为0，如下图所示，然后，我的错误就解决了！！！！！！

 

3. 解决问题的过程： 

1）另外定义一个函数放在transforms.Lambda()括号里面，没有用。

https://github.com/donydchen/ganimation_replicate/issues/4 与我有一模一样的问题，作者提到了一个相似问题的解决链接https://discuss.pytorch.org/t/cant-pickle-local-object-dataloader-init-locals-lambda/31857

 

链接里提到一个解决方法，如下图所示：

我的原代码如下所示，也含有与Lambda相关的代码transforms.Lambda(lambda image: image)。

def img_transformer(self):
        transform_list = []
        if self.opt.resize_or_crop == 'resize_and_crop':
            transform_list.append(transforms.Resize([self.opt.load_size, self.opt.load_size], Image.BICUBIC))
            transform_list.append(transforms.RandomCrop(self.opt.final_size))
        elif self.opt.resize_or_crop == 'crop':
            transform_list.append(transforms.RandomCrop(self.opt.final_size))
        elif self.opt.resize_or_crop == 'none':
            transform_list.append(transforms.Lambda(lambda image: image))
        else:
            raise ValueError("--resize_or_crop %s is not a valid option." % self.opt.resize_or_crop)

所以，我改了自己的代码，改成transforms.Lambda(tmp_func)，里面是一个自己定义的函数，如下所示：

def img_transformer(self):
        transform_list = []
        def tmp_func(image):
            return image
        if self.opt.resize_or_crop == 'resize_and_crop':
            transform_list.append(transforms.Resize([self.opt.load_size, self.opt.load_size], Image.BICUBIC))
            transform_list.append(transforms.RandomCrop(self.opt.final_size))
        elif self.opt.resize_or_crop == 'crop':
            transform_list.append(transforms.RandomCrop(self.opt.final_size))
        elif self.opt.resize_or_crop == 'none':
            transform_list.append(transforms.Lambda(tmp_func))
        else:
            raise ValueError("--resize_or_crop %s is not a valid option." % self.opt.resize_or_crop)

结果，并没有用。依旧报错。

 

2）import dill as pickle，没有用。

https://github.com/chenchongthu/DeepCoNN/issues/3  AttributeError: Can't pickle local object 'numerize.<locals>.<lambda>' #3，   里面提到使用dill解决import dill as pickle。    给了一个链接https://stackoverflow.com/questions/25348532/can-python-pickle-lambda-functions，里面提到用dill解决的方法，和另外定义一个函数放在transforms.Lambda()括号里面，对我都没有用。

 

3）在命令后添加 --workers 0，没有用。

https://github.com/fizyr/keras-retinanet/issues/967 里提到添加 --workers 0

https://github.com/fizyr/keras-retinanet/issues/857  里提到 solve this by turning off multiprocessing. You can do this by providing the "--workers=0" parameter or by changing it directly in the train.py file in the line "parser.add_argument('--workers',
help='Number of multiprocessing workers. To disable multiprocessing, set workers to 0',type=int, default=0)"

https://blog.csdn.net/qq_20373723/article/details/85258535 里提到torch.utils.data.DataLoader函数里面有个参数num_workers表示进程个数，在windows下改为0就可以了

https://discuss.pytorch.org/t/pytorch-windows-eoferror-ran-out-of-input-when-num-workers-0/25918

 

4）有人说是windows下pickling、multiprocessing的问题。

https://github.com/thstkdgus35/EDSR-PyTorch/issues/105 

https://github.com/yahoo/TensorFlowOnSpark/issues/198

https://github.com/yahoo/TensorFlowOnSpark/issues/36

 

5）找出项目里含有workers的文件，把workers的值改为0，解决问题。

https://github.com/matterport/Mask_RCNN/issues/93   直到我看到了这篇博客。我就开始搜索我项目里的所有.py代码文件，一个文件一个文件的Ctrl+F搜索，看代码里面是否含有workers、mutilprocessing这两个关键字。最终，我在我项目的data_loader.py文件里搜索到了workers，把workers的值改为0，然后，我的错误就解决了！！！！！！