matlab parfor如何传递large共享变量

cvpr2019在超分辨率重建方面，旷视觉科技有一篇论文：Meta-SR，实现单一模型任意缩放因子超分，非常感谢作者的付出和贡献．

作者已经开放了源码，下面是我的分支，由于自己改的比较随意，所以今后的一些修改将在这个分支上继续：

https://github.com/idealboy/Meta-SR-Pytorch.git

在prepare_dataset的时候，源码里没有采用并行parfor，所以处理会比较慢，关键代码中的循环如下：

%%　DIV2K_HR中存储所有图片，是一个站内存很大的数组
%%  如果将这个大数组传入parfor的每个worker中，效率会相当慢，启动不了parfor循环
%%  一个每个worker都会拷贝一份这个变量

for idx_im = 1:nb_im
    fprintf('Read HR :%d\n', idx_im);
    ImHR = imread(fullfile(path_src, filepaths(idx_im).name));
    DIV2K_HR{idx_im} = ImHR;
end
 
%% generate and save LR via imresize() with Bicubic

%% parfor只支持integer做循环索引，所以这里改变一下循环的形式

scales = 1.0:0.1:4.0;
scale_num = length(scales);

parfor i = 2:1:scale_num　　　%% 这里，DIV2K_HR会传入到parfor的每个worker中，效率很差

    scale = scales(i)

    FolderLR = fullfile(path_save,'LR_bicubic', sprintf('X%.2f',scale));
    
    if ~exist(FolderLR，＇dir＇)  %%这里，需要'dir'这个参数，否则在parfor中会报错
        mkdir(FolderLR)
    end
    for IdxIm = 1:nb_im
        fprintf('IdxIm=%d\n', IdxIm);
        ImHR = DIV2K_HR{IdxIm};
        [h,w,n]=size(ImHR);
        if mod(h,scale) ~= 0
            h = h - 4;
        end
        if mod(w,scale) ~= 0
            w = w - 4;
        end
        ImHR =ImHR(1:h,1:w,:);
        ImLR = imresize(ImHR, 1/scale, 'bicubic');
        % name image
        fileName = filepaths(IdxIm).name
        NameLR = fullfile(FolderLR, fileName);
        % save image
        imwrite(ImLR, NameLR, 'png');
    end
end

　　　　原始代码中，parfor循环中每个worker会传入大的数组变量DIV2K_HR，并拷贝，造成负载(load)很大，所以效率很慢，那么，怎么办？

　　　　调整内外for循环，使得传入parfor workers中的变量体积变小即可．

　　　　调整的代码详见如下（多线程parfor版本）：

　　　　https://github.com/idealboy/Meta-SR-Pytorch/blob/master/prepare_dataset/generate_LR_metasr_X1_X4_mt.m

　　　　

for idx_im = 1:nb_im
    fprintf('Read HR :%d\n', idx_im);
    ImHR = imread(fullfile(path_src, filepaths(idx_im).name));
    DIV2K_HR{idx_im} = ImHR;
end

FolderLR_bicubic = fullfile(path_save,'LR_bicubic')

if ~exist(FolderLR_bicubic)
    mkdir(FolderLR_bicubic)
end
    
%% generate and save LR via imresize() with Bicubic

scales = 1.0:0.1:4.0;

scale_num = length(scales);

%% look at the original source code, when applying parfor on outter loop, the large variable 'DIV2K_HR' wiil be
%% passed to each workers, and it will be very very slow just for start up the parfor.

%% we just exchange the original outter-loop and inner-loop,
%% so that, when applying parfor on current inner-loop('parfor i = 2:1:scale_num'),
%% it will not cause a heavy load to each workers when passing the shared variable,(image = ImHR;)
%% and, now, for each image, all the resized images (with diffrent scale) will be processed parallelly(speed up)

%% 内外循环切换，每张图片，生成所有的scale图片后，再处理下一张图片
for IdxIm = 1:nb_im
    fprintf('IdxIm=%d\n', IdxIm);
    ImHR = DIV2K_HR{IdxIm};
    
    %% 每张图像的ImHR将在每个parfor woker中拷贝，ｌｏａｄ开销会很小
    parfor i = 2:1:scale_num
        image = ImHR;
        scale = scales(i);
        FolderLR = fullfile(path_save,'LR_bicubic', sprintf('X%.2f',scale));
        
        if ~exist(FolderLR, 'dir')
            mkdir(FolderLR)
        end

        [h,w,n]=size(image);
        if mod(h,scale) ~= 0
            h = h - 4;
        end
        if mod(w,scale) ~= 0
            w = w - 4;
        end
        image =image(1:h,1:w,:);

        image= imresize(image, 1/scale, 'bicubic');
        % name image
        fileName = filepaths(IdxIm).name
        NameLR = fullfile(FolderLR, fileName);
        % save image
        imwrite(image, NameLR, 'png');
    end
end