python 共享内存 对象_在Python多处理中将Pool.map与共享内存数组结合

千万里不及你

当我刚看到赏金的时候再试一次;)基本上，我认为错误消息的含义是它的意思-多处理共享内存数组不能作为参数传递(通过酸洗)。序列化数据没有意义-关键是数据是共享内存。因此，您必须使共享数组成为全局数组。我认为像我的第一个答案一样，将其作为模块的属性比较整洁，但在示例中将其保留为全局变量也可以很好地工作。考虑到您不想在fork之前设置数据的观点，这是一个修改后的示例。如果您希望拥有多个共享数组(这就是为什么要将toShare作为参数传递的原因)，则可以类似地创建共享数组的全局列表，然后将索引传递给count_it(将变为for c in toShare[i]:)。from sys import stdinfrom multiprocessing import Pool, Array, Processdef count_it( key ):  count = 0  for c in toShare:    if c == key:      count += 1  return countif __name__ == '__main__':  # allocate shared array - want lock=False in this case since we   # aren't writing to it and want to allow multiple processes to access  # at the same time - I think with lock=True there would be little or   # no speedup  maxLength = 50  toShare = Array('c', maxLength, lock=False)  # fork  pool = Pool()  # can set data after fork  testData = "abcabcs bsdfsdf gdfg dffdgdfg sdfsdfsd sdfdsfsdf"  if len(testData) > maxLength:      raise ValueError, "Shared array too small to hold data"  toShare[:len(testData)] = testData  print pool.map( count_it, ["a", "b", "s", "d"] )[编辑：以上内容由于未使用fork而无法在Windows上运行。但是，下面的方法在Windows上仍然可以使用Pool，但仍然可以使用，因此我认为这与您想要的最接近：from sys import stdinfrom multiprocessing import Pool, Array, Processimport mymoduledef count_it( key ):  count = 0  for c in mymodule.toShare:    if c == key:      count += 1  return countdef initProcess(share):  mymodule.toShare = shareif __name__ == '__main__':  # allocate shared array - want lock=False in this case since we   # aren't writing to it and want to allow multiple processes to access  # at the same time - I think with lock=True there would be little or   # no speedup  maxLength = 50  toShare = Array('c', maxLength, lock=False)  # fork  pool = Pool(initializer=initProcess,initargs=(toShare,))  # can set data after fork  testData = "abcabcs bsdfsdf gdfg dffdgdfg sdfsdfsd sdfdsfsdf"  if len(testData) > maxLength:      raise ValueError, "Shared array too small to hold data"  toShare[:len(testData)] = testData  print pool.map( count_it, ["a", "b", "s", "d"] )不知道为什么map不会腌制数组，而Process和Pool会腌制-我想也许它已经在Windows上的子进程初始化时转移了。请注意，尽管在派生之后仍然设置了数据。