python中for循环加速_如何加快python函数中的for循环?

您没有发布指向image1.tif文件的链接, 因此下面的示例代码使用中的pysheds/data/dem.tifhttps://github.com/mdbartos/pysheds。基本思想是拆分输入参数, 根据您的情况, 将xs和ys分成子集, 然后为每个CPU提供不同的子集进行处理。

main()计算解决方案两次, 顺序一次, 并行一次, 然后比较每个解决方案。并行解决方案效率低下, 因为每个CPU都会读取图像文件, 因此还有改进的空间(即, 在并行部分之外读取图像文件, 然后将生成的grid对象提供给每个实例)。

import numpy as np

from pysheds.grid import Grid

from dask.distributed import Client

from dask import delayed, compute

xs = 10, 20, 30, 40, 50, 60, 70, 80, 90, 100

ys = 25, 35, 45, 55, 65, 75, 85, 95, 105, 115, 125

def var(image_file, x_in, y_in):

grid = Grid.from_raster(image_file, data_name='map')

variable_avg = []

for (x,y) in zip(x_in,y_in):

grid.catchment(data='map', x=x, y=y, out_name='catch')

variable = grid.view('catch', nodata=np.nan)

variable_avg.append( np.array(variable).mean() )

return(variable_avg)

def var_parallel(n_cpu, image_file, x_in, y_in):

tasks = []

for cpu in range(n_cpu):

x_in = xs[cpu::n_cpu] # eg, cpu = 0: x_in = (10, 40, 70, 100)

y_in = ys[cpu::n_cpu] #

tasks.append( delayed(var)(image_file, x_in, y_in) )

ans = compute(tasks)

# reassemble solution in the right order

par_avg = [None]*len(xs)

for cpu in range(n_cpu):

par_avg[cpu::n_cpu] = ans[0][cpu]

print('AVG (parallel) =',par_avg)

return par_avg

def main():

image_file = 'pysheds/data/dem.tif'

# sequential solution:

seq_avg = var(image_file, xs, ys)

print('AVG (sequential)=',seq_avg)

# parallel solution:

n_cpu = 3

dask_client = Client(n_workers=n_cpu)

par_avg = var_parallel(n_cpu, image_file, xs, ys)

dask_client.shutdown()

print('max error=',

max([ abs(seq_avg[i]-par_avg[i]) for i in range(len(seq_avg))]))

if __name__ == '__main__': main()