python 回溯_Python：从多处理进程中获取回溯

I am trying to get hold of a traceback object from a multiprocessing.Process.

Unfortunately passing the exception info through a pipe does not work because traceback objects can not be pickled:

def foo(pipe_to_parent):

try:

raise Exception('xxx')

except:

pipe_to_parent.send(sys.exc_info())

to_child, to_self = multiprocessing.Pipe()

process = multiprocessing.Process(target = foo, args = (to_self,))

process.start()

exc_info = to_child.recv()

process.join()

print traceback.format_exception(*exc_info)

to_child.close()

to_self.close()

Traceback:

Traceback (most recent call last):

File "/usr/lib/python2.6/multiprocessing/process.py", line 231, in _bootstrap

self.run()

File "/usr/lib/python2.6/multiprocessing/process.py", line 88, in run

self._target(*self._args, **self._kwargs)

File "foo", line 7, in foo

to_parent.send(sys.exc_info())

PicklingError: Can't pickle : attribute lookup __builtin__.traceback failed

Is there another way to access the exception info? I'd like to avoid passing the formatted string.

解决方案

Using tblib you can pass wrapped exceptions and reraise them later:

import tblib.pickling_support

tblib.pickling_support.install()

from multiprocessing import Pool

import sys

class ExceptionWrapper(object):

def __init__(self, ee):

self.ee = ee

__, __, self.tb = sys.exc_info()

def re_raise(self):

raise self.ee.with_traceback(self.tb)

# for Python 2 replace the previous line by:

# raise self.ee, None, self.tb

# example how to use ExceptionWrapper

def inverse(i):

"""will fail for i == 0"""

try:

return 1.0 / i

except Exception as e:

return ExceptionWrapper(e)

def main():

p = Pool(1)

results = p.map(inverse, [0, 1, 2, 3])

for result in results:

if isinstance(result, ExceptionWrapper):

result.re_raise()

if __name__ == "__main__":

main()

So, if you catch an exception in your remote process, wrap it with ExceptionWrapper and then pass it back. Calling re_reraise in the main process will do the work.