python multiprocess.Queue - RuntimeError: Queue objects should only be shared between processes thro...

相关代码如下：

queue = multiprocessing.Queue()

db = HandleSQL(conf_db, data_db, queue)

错误信息如下：

Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python2.6/threading.py", line 532, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.6/threading.py", line 484, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/usr/lib/python2.6/multiprocessing/pool.py", line 225, in _handle_tasks
    put(task)
  File "/usr/lib/python2.6/multiprocessing/queues.py", line 51, in __getstate__
    assert_spawning(self)
  File "/usr/lib/python2.6/multiprocessing/forking.py", line 25, in assert_spawning
    ' through inheritance' % type(self).__name__
RuntimeError: Queue objects should only be shared between processes through inheritance

 

修改如下：

manager = multiprocessing.Manager()
queue = manager.Queue()

 

Queue对象只能使用继承（inheritance）的方式共享。这是因为Queue本身基于unix的Pipe对象实现，而Pipe对象的共享需要通过继承。

因此，在一个典型的应用实现模型当中，应该是父进程创建Queue，然后创建子进程共享该Queue，由父进程和子进程分别读写。另一种实现方式是父进程创建Queue，创建多个子进程，有的子进程读Queue，有的子进程写Queue。

详见参考中的链接。

 

参考：

http://stackoverflow.com/questions/3217002/how-do-you-pass-a-queue-reference-to-a-function-managed-by-pool-map-async

http://blog.ftofficer.com/2009/11/using-python-multiprocessing-1-process-model/

http://blog.ftofficer.com/2009/12/python-multiprocessing-2-object-sharing-across-process/

http://blog.ftofficer.com/2009/12/python-multiprocessing-3-about-queue/

转载于:https://www.cnblogs.com/congbo/archive/2012/08/22/2642697.html