import multiprocessing
import time
class Test:
def __init__(self):
self.pool = multiprocessing.Pool()
# self.queue = multiprocessing.Queue()
m = multiprocessing.Manager()
self.queue = m.Queue()
def subprocess(self):
for i in range(10):
print("Running")
time.sleep(1)
print("Subprocess Completed")
def start(self):
self.pool.apply_async(func=self.subprocess)
print("Subprocess has been started")
self.pool.close()
self.pool.join()
def __getstate__(self):
self_dict = self.__dict__.copy()
del self_dict['pool']
return self_dict
def __setstate__(self, state):
self.__dict__.update(state)
if __name__ == '__main__':
test = Test()
test.start()
主要是这两行代码:
m = multiprocessing.Manager()
self.queue = m.Queue()
A manager object returned by Manager() controls a server process which holds Python objects and allows other processes to manipulate them using proxies.
A manager returned by Manager() will support types list, dict, Namespace, Lock, RLock, Semaphore, BoundedSemaphore, Condition, Event, Barrier, Queue, Value and Array.
共享数据主要通过在内存中共享某个数据类型实现的,共享数据类型比较多:列表,字典,同步锁,递归锁,信号量,时间,还有队列等,这个进程池用的就是共享数据的队列类型数据
话说__getstate__和__setstate__是什么鬼?
这个是Python中有些无法被pickle序列化的对象,调用这两个方法后就可以进行pickle序列化和反序列化处理处理。示例:
class Foo(object):
def __init__(self, val=2):
self.val = val
def __getstate__(self):
# print("I'm being pickled")
self.val *= 2
print(self.__dict__)
return self.__dict__
print('=--------->')
def __setstate__(self, d):
# print("I'm being unpickled with these values:", d)
self.__dict__ = d
# self.val *= 3
# pass
print('============>')
import pickle
f = Foo()
f_string = pickle.dumps(f)
f_new = pickle.loads(f_string)
print(f_new.val)
上面的示例可以很好的解释了__getstate__和__setstate__用法,Python 将只 pickle 当它调用该实例的__getstate__() 方法时返回给它的值,在 unpickle 时,Python 将提供经过 pickle 的值作为参数传递给实例的__setstate__() 方法。在 __setstate__() 方法内,可以根据经过 pickle 的名称和位置信息来重建对象,并将该对象分配给这个实例的 属性。
这个博客写的超级赞:
pickle序列化全接触