python输出顺序_使用Python以真实顺序打印Parellel函数输出

Looking to print everything in order, for a Python parallelized script. Note the c3 is printed prior to the b2 -- out of order. Any way to make the below function with a wait feature? If you rerun, sometimes the print order is correct for shorter batches. However, looking for a reproducible solution to this issue.

from joblib import Parallel, delayed, parallel_backend

import multiprocessing

testFrame = [['a',1], ['b', 2], ['c', 3]]

def testPrint(letr, numbr):

print(letr + str(numbr))

return letr + str(numbr)

with parallel_backend('multiprocessing'):

num_cores = multiprocessing.cpu_count()

results = Parallel(n_jobs = num_cores)(delayed(testPrint)(letr = testFrame[i][0],

numbr = testFrame[i][1]) for i in range(len(testFrame)))

print('##########')

for test in results:

print(test)

Output:

b2

c3

a1

##########

a1

b2

c3

Seeking:

a1

b2

c3

##########

a1

b2

c3

解决方案

Once you launch tasks in separate processes you no longer control the order of execution so you cannot expect the actions of those tasks to execute in any predictable order - especially if the tasks can take varying lengths of time.

If you are parallelizing(?) a task/function with a sequence of arguments and you want to reorder the results to match the order of the original sequence you can pass sequence information to the task/function that will be returned by the task and can be used to reconstruct the original order.

If the original function looks like this:

def f(arg):

l,n = arg

#do stuff

time.sleep(random.uniform(.1,10.))

result = f'{l}{n}'

return result

Refactor the function to accept the sequence information and pass it through with the return value.

def f(arg):

indx, (l,n) = arg

time.sleep(random.uniform(.1,10.))

result = (indx,f'{l}{n}')

return result

enumerate could be used to add the sequence information to the sequence of data:

originaldata = list(zip('abcdefghijklmnopqrstuvwxyz', range(26)))

dataplus = enumerate(originaldata)

Now the arguments have the form (index,originalarg) ... (0, ('a',0'), (1, ('b',1)).

And the returned values from the multi-processes look like this (if collected in a list) -

[(14, 'o14'), (23, 'x23'), (1, 'b1'), (4, 'e4'), (13, 'n13'),...]

Which is easily sorted on the first item of each result, key=lambda item: item[0], and the values you really want obtained by picking out the second items after sorting results = [item[1] for item in results].