我有一个用python编写的程序,它读取4个输入文本文件并将它们全部写入一个名为ListOutput的列表中,这是我程序中使用的4个进程之间的共享内存(我使用了4个进程,所以我的程序运行得更快!)在
我还有一个名为processedFiles的共享内存变量,该变量存储任何进程已经读取的输入文件的名称,因此当前进程不会再次读取它们(我使用了lock,因此进程不会同时检查processedFiles中是否存在文件)。在
当我只使用一个进程时,我的程序运行得更快(7毫秒)-我的计算机有8个核心。为什么会这样?在import glob
from multiprocessing import Process, Manager,Lock
import timeit
import os
os.chdir("files")
# Define a function for the Processes
def print_content(ProcessName,processedFiles,ListOutput,lock):
for file in glob.glob("*.txt"):
newfile=0
lock.acquire()
print "\n Current Process:",ProcessName
if file not in processedFiles:
print "\n", file, " not in ", processedFiles," for ",ProcessName
processedFiles.append(file)
newfile=1 #it is a new file
lock.release()
#if it is a new file
if newfile==1:
f = open(file,"r")
lines = f.readlines()
ListOutput.append(lines)
f.close()
#print "%s: %s" % ( ProcessName, time.ctime(time.time()) )
# Create processes as follows
try:
manager = Manager()
processedFiles = manager.list()
ListOutput = manager.list()
start = timeit.default_timer()
lock=Lock()
p1 = Process(target=print_content, args=("Procees-1",processedFiles,ListOutput,lock))
p2 = Process(target=print_content, args=("Process-2",processedFiles,ListOutput,lock))
p3 = Process(target=print_content, args=("Process-3",processedFiles,ListOutput,lock))
p4 = Process(target=print_content, args=("Process-4",processedFiles,ListOutput,lock))
p1.start()
p2.start()
p3.start()
p4.start()
p1.join()
p2.join()
p3.join()
p4.join()
print "ListOutput",ListOutput
stop = timeit.default_timer()
print stop - start
except:
print "Error: unable to start process"