的核数和进程_MATLAB并行计算之多进程连续滤波

为了提高MATALB的滤波效率,采用了spmd并行计算来提高滤波速度。针对大数据分块后滤波的不连续问题,每次分段滤波时,设置每段overlap值等于滤波器中寄存器的个数。

下面直接上代码。

function multiProcessFilter(objFilter,rawFile,processFile,M)
% 实现本地文件数据读取、多进程连续滤波、滤波后信号写入

% objFilter      1×1 dffir    FIR滤波器对象
% rawFile        1×n char     待滤波的二进制文件(int8)名称
% processFile    1×m char     滤波后的二进制文件(int8)名称
% M              1×1 double   进程数目,小于等于CPU核数

% FIR滤波器中寄存器个数
overlap  = length(objFilter.States);
% 打开要滤波的文件
fidRead  = fopen(rawFile,'rb');
fidWrite = fopen(processFile,'wb');
% 预处理一小段数据
preData   = fread(fidRead,overlap,'int8=>int8');
preFilterData = filter(objFilter,preData);
fwrite(fidWrite,preFilterData,'int8');
while(~feof(fidRead))
    % 文件读取不能多进程
    fseek(fidRead,- overlap,'cof');
    [data,count]  = fread(fidRead,M*1e8 + overlap,'int8=>int8');
    workLoad = round((count - overlap)/M);
    % 多进程滤波
    spmd(M)
        id = labindex;
        if isequal(id,1)
            fdata = filter(objFilter,data(1:workLoad+overlap));
            fdata = int8(fdata(overlap+1:end));
        elseif isequal(id,M)
            fdata = filter(objFilter,data((M-1)*workLoad+1 : end));
            fdata = int8(fdata(overlap+1:end));
        else
            fdata = filter(objFilter,data((id-1)*workLoad+1: id*workLoad+overlap));
            fdata = int8(fdata(overlap+1:end));
        end
    end
    % 文件写入不能多进程操作
    for ii = 1:M
        fwrite(fidWrite,fdata{ii},'int8');
    end
end
% 关闭文件
fclose('all');
end

作为对比,下面是单进程滤波的代码

function singleProcessFilter(objFilter,rawFile,processFile)
% PersistentMemory设为1,保持连续滤波
objFilter.PersistentMemory = 1;
fidRead  = fopen(rawFile,'rb');
fidWrite = fopen(processFile,'wb');

while(~feof(fidRead))
    data  = fread(fidRead,1e8,'int8=>int8');
    fdata = filter(objFilter,data);
    fwrite(fidWrite,int8(fdata),'int8');
end
fclose('all');
end

测试,比较单进程与多进程滤波的运行时间,并二进制比较滤波后的输出文件是否相同。

function main()
% 68阶等纹波滤波器
objFilter = lowpassFilter;

% 单进程滤波
tic;singleProcessFilter(objFilter,'raw.bin','single.bin');toc
% 时间已过 138.555783 秒。

% parpool(6)
% 6进程滤波
tic;multiProcessFilter(objFilter,'raw.bin','multi.bin',6);toc
% 时间已过 46.245316 秒。

% 比较滤波结果
visdiff('single.bin','multi.bin','binary')
% 这些文件相同
end

function Hd = lowpassFilter
Fpass = 0.2;              % Passband Frequency
Fstop = 0.25;             % Stopband Frequency
Dpass = 0.057501127785;   % Passband Ripple
Dstop = 0.0031622776602;  % Stopband Attenuation
dens  = 20;               % Density Factor
[N, Fo, Ao, W] = firpmord([Fpass, Fstop], [1 0], [Dpass, Dstop]);
b  = firpm(N, Fo, Ao, W, {dens});
Hd = dfilt.dffir(b);
% set(Hd, 'Arithmetic', 'single');
end

测试结果表明,使用68阶FIR滤波器对1.2G的int8文件进行滤波,单进程需要138秒,6进程需要46秒,花费的时间约为1/3。(之所以不是1/6,是因为文件读取与写入是单进程的。)

上述代码的测试环境为MATLAB R2019a,Intel i7 8700 4.3GHz, 6核6线程(关闭了超线程),内存32G。

易夕:MATLAB Tricks 专栏目录​zhuanlan.zhihu.com
表情包
插入表情
评论将由博主筛选后显示,对所有人可见 | 还能输入1000个字符
相关推荐
<p> 课程演示环境:Windows10  </p> <p> 需要学习<span>Ubuntus</span>系统<span>YOLOv4-tiny</span>同学请前往《<span>YOLOv4-tiny</span>目标检测实战:训练自己数据集》 <span></span> </p> <p> <span> </span> </p> <p> <span style="color:#E53333;">YOLOv4-tiny</span><span style="color:#E53333;">来了!速度大幅提升!</span><span></span> </p> <p> <span> </span> </p> <p> <span>YOLOv4-tiny</span>在<span>COCO</span>上性能可达到:<span>40.2% AP50, 371 FPS (GTX 1080 Ti)</span>。相较于<span>YOLOv3-tiny</span>,<span>AP</span><span>FPS</span>性能有巨大提升。并且,<span>YOLOv4-tiny</span>权重文件只有<span>23MB</span>,适合在移动端、嵌入式设备、边缘计算设备上部署。<span></span> </p> <p> <span> </span> </p> <p> 本课程将手把手地教大家使用<span>labelImg</span>标注使用<span>YOLOv4-tiny</span>训练自己数据集。课程实战分为两个项目:单目标检测(足球目标检测)多目标检测(足球梅西同时检测)。<span></span> </p> <p> <span> </span> </p> <p> 本课程<span>YOLOv4-tiny</span>使用<span>AlexAB/darknet</span>,在<span>Windows10</span>系统上做项目演示。包括:<span>YOLOv4-tiny</span>网络结构、安装<span>YOLOv4-tiny</span>、标注自己数据集、整理自己数据集、修改配置文件、训练自己数据集、测试训练出网络模型、性能统计<span>(mAP</span>计算<span>)</span>先验框聚类分析。 <span> </span> </p> <p> <span> </span> </p> <p> 除本课程《<span>Windows</span>版<span>YOLOv4-tiny</span>目标检测实战:训练自己数据集》外,本人推出了有关<span>YOLOv4</span>目标检测系列课程。请持续关注该系列其它视频课程,包括:<span></span> </p> <p> 《<span>Windows</span>版<span>YOLOv4</span>目标检测实战:训练自己数据集》<span></span> </p> <p> 《<span>Windows</span>版<span>YOLOv4</span>目标检测实战:人脸口罩佩戴识别》<span></span> </p> <p> 《<span>Windows</span>版<span>YOLOv4</span>目标检测实战:中国交通标志识别》<span></span> </p> <p> 《<span>Windows</span>版<span>YOLOv4</span>目标检测:原理与源码解析》<span></span> </p> <p> <span> <img alt="" src="https://img-bss.csdnimg.cn/202007061503586145.jpg" /></span> </p> <p> <span><img alt="" src="https://img-bss.csdnimg.cn/202007061504169339.jpg" /><br /> </span> </p>
©️2020 CSDN 皮肤主题: 游动-白 设计师:白松林 返回首页