multiprocessing.pool.MaybeEncodingError

multiprocessing.pool.MaybeEncodingError: Error sending result: '<multiprocessing.pool.ExceptionWithTraceback object at 0x000000000B5D6EC8>'. Reason: 'PicklingError("Can't pickle <class 'MemoryError'>: it's not the same object as builtins.MemoryError")'

出现上图问题，个人感觉应该是进程数开的太多，崩溃了；

可用以下方法解决：

 

    if not os.path.exists(finished_flag):
        annos = np.array(pandas.read_csv(luna_label))
        pool = Pool(4)     #进程数修改为具体的数值，可以改小一点儿
        if not os.path.exists(savepath):
            os.mkdir(savepath)
        for setidx in range(10):
            print('process subset', setidx)
            filelist = [f.split('.mhd')[0] for f in os.listdir(luna_data + 'subset' + str(setidx)) if
                        f.endswith('.mhd')]
            if not os.path.exists(savepath + 'subset' + str(setidx)):
                os.mkdir(savepath + 'subset' + str(setidx))
            partial_savenpy_luna = partial(savenpy_luna, annos=annos, filelist=filelist,
                                           luna_segment=luna_segment,
                                           luna_data=luna_data + 'subset' + str(setidx) + '/',
                                           savepath=savepath + 'subset' + str(setidx) + '/')
            N = len(filelist)
            #pool2 = pool(20)
            # savenpy(1)
            _ = pool.map(partial_savenpy_luna, range(N))
        pool.close()
        pool.join()