介绍
本篇文章将简要总结一下python和MATLAB中并行计算的实现。需要强调的是,并行计算只可以用于计算步骤之间独立没有递进关系的情况。
I
MATLAB
在MATLAB中,实现起来较为容易,只需要将“for”改为“parfor”即可,还可以手动指定需要的核心数,命令为kernum=parpool(poolnum)
,但是需要注意的是,记得在运行完后关掉delete(kernum)
poolnum=10; % numbers of kernels
kernum=parpool(poolnum)
parfor l=1:numloop
%do something here
end
delete(kernum)
%the delete step is very important because in cluster we may use more than a dozen cores.If we stop the program directly without closing the parallel pool, the cluster may crash or take a long time to become normal.
当然,parfor也有其他的许多限制,比如对于矩阵的切片A(:,l)=B
这种赋值方式可能是行不通的。我这只是一个简单的介绍,而且足以应对大部分场景了。
Python
python虽然一直被诟病就是慢,但是有时候有些函数直接调用c语言时,还是可以非常快的,关于python的并行计算的介绍可以常见下面的文章,主要用到的包是 “multiprocessing”
https://docs.python.org/3.4/library/multiprocessing.html?highlight=process
python的并行计算的方法很多,用法多样,我这里展示我自己用到过的一段代码,作为记录
import multiprocessing as mp
import numpy as np
%use a function to define what we want to do
def fun=(a,b,c,d):
#do something here
return fun
# number of process you are going to use
processnum=10
if __name__ == '__main__':#this is needed in windows
pool = mp.Pool(processes=processnum)
#results1 = [pool.apply(fun, args=(l,loop_mat)) for l in range(numloop)]
#get the ordered values directly and store the results in results1
results2 = [pool.apply_async(fun, args=(l,loop_mat)) for l in range(numloop)]
#get the values arbitrarily
value_1= [p.get() for p in results2]# use this get function to obtain the calculated value
pool.terminate() #shut down
注意上面的if __name__ == '__main__':#this is needed in windows
是一段保险的代码,在windows上是需要的。