这是一张list of what can be pickled。特别是,函数只有在模块的顶层定义时才是可选择的。
这段代码:import multiprocessing as mp
class Foo():
@staticmethod
def work(self):
pass
if __name__ == '__main__':
pool = mp.Pool()
foo = Foo()
pool.apply_async(foo.work)
pool.close()
pool.join()
产生的错误与您发布的错误几乎相同:Exception in thread Thread-2:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 552, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 505, in run
self.__target(*self.__args, **self.__kwargs)
File "/usr/lib/python2.7/multiprocessing/pool.py", line 315, in _handle_tasks
put(task)
PicklingError: Can't pickle : attribute lookup __builtin__.function failed
问题是pool方法都使用mp.SimpleQueue将任务传递给工作进程。通过mp.SimpleQueue的所有内容都必须是可选取的,并且foo.work不可选取,因为它不是在模块的顶层定义的。
它可以通过在顶层定义一个函数来修复,该函数调用foo.work():def work(foo):
foo.work()
pool.apply_async(work,args=(foo,))
注意foo是可选取的,因为Foo是在顶层定义的,而foo.__dict__是可选取的。