原因是实例方法不能被pickle。上面的代码并不是执行没有错误,只是错误没有被报出来。错误只有在调用结果时,才会报错。如下:
import os
import time
from multiprocessing import Pool
import random
class MyThread(object):
def long_time_task(self,i):
print 'Run task %s (%s)...' % (i, os.getpid())
time.sleep(random.random() * 3)
print i
def parse_thread(self):
print 'Parent process %s.' % os.getpid()
p = Pool()
results = []
for i in range(10):
results.append(p.apply_async(self.long_time_task,args=(i,)))
# If trying to get the result, error will be reported here
for res in results:
print res.get()
print 'Waiting for all subprocesses done...'
p.close()
p.join()
print 'All subprocesses done.'
def main():
print "start"
tt=MyThread()
tt.parse_thread()
if __name__=="__main__"
main()
运行结果会在print res.get() 那一行报错,错误如下:
PicklingError: Can't pickle : attribute lookup __builtin__.instancemethod failed
解决方法:用一个可被实例化的普通函数包装类方法,将实例对象作为参数传递给函数即可。代码如下:
class MyThread(object):
def __init__(self, func):
self.func = func
def long_time_task(self,i):
print 'Run task %s (%s)...' % (i, os.getpid())
time.sleep(random.random() * 3)
print i
return (i, os.getpid())
def parse_thread(self):
print 'Parent process %s.' % os.getpid()
p = Pool()
results = []
for i in range(10):
results.append(p.apply_async(long_time_task_wrapper,args=(self,i,)))
# Now can get the result
for res in results:
print res.get()
print 'Waiting for all subprocesses done...'
p.close()
p.join()
print 'All subprocesses done.'
def long_time_task_wrapper(cls_instance, i):
return cls_instance.long_time_task(i)
def main():
print "start"
tt=MyThread(long_time_task_wrapper)
tt.parse_thread()
if __name__=="__main__":
main()
运行结果如下:
start
Parent process 3604.
(0, 6412)
(1, 3904)
(2, 2528)
(3, 5492)
(4, 5492)
(5, 5492)
(6, 5492)
(7, 6412)
(8, 5492)
(9, 6412)
Waiting for all subprocesses done...
Run task 2 (2528)...
2
Run task 1 (3904)...
1
Run task 3 (5492)...
3
Run task 4 (5492)...
4
Run task 5 (5492)...
5
Run task 6 (5492)...
6
Run task 8 (5492)...
8
Run task 0 (6412)...
0
Run task 7 (6412)...
7
Run task 9 (6412)...
9
All subprocesses done.
希望对你有帮助。