subprocess模块
subprocess模块是python中子进程模块,可以用来在python程序之中调用其他程序,或者执行系统命令。官方建议用subprocess模块来替代一些原有的函数,比如os.system()
subprocess.Popen
- Popen()
Popen启动新的进程与父进程并行执行,默认父进程不等待新进程结束。
def TestPopen():
import subprocess
p = subprocess.Popen("calc",shell=True)
for i in range(10) :
print (i)
- Popen.wait()
Popen.wait()函数使得父进程等待新创建的进程运行结束,然后再继续父进程的其他任务。且此时可以在Popen.returncode中得到新进程的返回值。
def TestWait():
import subprocess
import datetime
print (datetime.datetime.now())
p = subprocess.Popen("calc",shell=True)
p.wait()
print (p.returncode)
print (datetime.datetime.now())
- Popen.poll()
Popen.poll()函数可以用来检测新创建的进程是否结束。
import subprocess
import datetime
import time
print (datetime.datetime.now())
p=subprocess.Popen("calc",shell=True)
t = 1
while(t <= 5):
time.sleep(1)
if p.poll() == None:
print "process not finished"
else:
print "process finished"
print (p.returncode)
t+=1
print (datetime.datetime.now())
- Popen.kill()&Popen.terminate()
Popen.kill()&Popen.terminate()用来结束创建的新进程,在windows系统上相当于调用TerminateProcess(),在posix系统上相当于发送信号SIGTERM和SIGKILL。
def TestKillAndTerminate():
p=subprocess.Popen("notepad.exe")
t = 1
while(t <= 5):
time.sleep(1)
t +=1
p.kill()
#p.terminate()
print ("new process was killed")
- Popen.communicate()
Popen.communicate()可以与新进程交互,但是必须要在popen构造时候将管道重定向。
def TestCommunicate():
import subprocess
cmd = "dir"
p=subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
(stdoutdata, stderrdata) = p.communicate()
if p.returncode != 0:
print (cmd + "error !")
#defaultly the return stdoutdata is bytes, need convert to str and utf8
for r in str(stdoutdata,encoding='utf8' ).split("\n"):
print (r)
print (p.returncode)
def TestCommunicate2():
import subprocess
cmd = "dir"
#universal_newlines=True, it means by text way to open stdout and stderr
p = subprocess.Popen(cmd, shell=True, universal_newlines=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
curline = p.stdout.readline()
while(curline != ""):
print (curline)
curline = p.stdout.readline()
p.wait()
print (p.returncode)
subprocess.call()
call函数可以认为是对popen和wait的分装,直接对call函数传入要执行的命令行,将命令行的退出code返回。
def TestCall():
retcode = subprocess.call("c:\\test.bat")
print (retcode)
subprocess.getoutput&subprocess.getstatusoutput
subprocess.getoutput&subprocess.getstatusoutput,基本上等价于subprocess.call函数,但是可以返回output,或者同时返回退出code和output。
但是可惜的是好像不能在windows平台使用,在windows上有如下错误:’{’ is not recognized as an internal or external command, operable program or batch file.
def TestGetOutput():
outp = subprocess.getoutput("ls -la")
print (outp)
def TestGetStatusOutput():
(status, outp) = subprocess.getstatusoutput('ls -la')
print (status)
print (outp)
总结
Popen的参数,
第一个为字符串(或者也可以为多个非命名的参数),表示你要执行的命令和命令的参数;后面的均为命名参数;
shell=True,表示你前面的传入的命令将在shell下执行,如果你的命令是个可执行文件或bat,不需要指定此参数;
stdout=subprocess.PIPE用来将新进程的输出重定向,stderr=subprocess.STDOUT将新进程的错误输出重定向到stdout,stdin=subprocess.PIPE用来将新进程的输入重定向;
universal_newlines=True表示以text的方式打开stdout和stderr。
其他的不推荐使用的模块:
os.system
os.spawn
os.popen
popen2.
commands.
范例
def process(workPath, resultPath, fileName,timeout = 200):
sha1 = fileName.split('.')[0]
filePath = os.path.join(workPath,fileName)
resultPath = os.path.join(resultPath,sha1 + ".txt")
if os.path.exists(resultPath):
os.remove(resultPath)
os.mknod(resultPath)
cmd = parseCmd(filePath, resultPath)
start = datetime.datetime.now()
process = subprocess.Popen(cmd, shell=True)
while process.poll() is None:
time.sleep(0.5)
now = datetime.datetime.now()
if (now - start).seconds > timeout:
istimeout = True
try:
process.kill()
time.sleep(0.1)
except Exception, e:
print 'kill timeout process [%s] error, [%s]' % (pid, e)
break
这个程序有一个很大的问题,因为是使用子进程是使用shell的形式启动的,之后shell中又会创建一个子进程来执行java命令,这样当超时的时候使用process.kill()的时候将shell进程kill掉,并不能kill java进程!
所以不应该用子进程的执行不应当使用shell的形式。但是如果不使用shell形式而直接传递cmd命令的话是会报错的,因为是命令,这种形式执行的是可执行程序。(报错原因就是找不到java.jar)
这个时候可以采用数组传递参数的形式来执行,跳过了shell:
def process(workPath, resultPath, fileName,timeout = 200):
sha1 = fileName.split('.')[0]
filePath = os.path.join(workPath,fileName)
resultPath = os.path.join(resultPath,sha1 + ".txt")
if os.path.exists(resultPath):
os.remove(resultPath)
os.mknod(resultPath)
cmd = parseCmd(filePath, resultPath)
pipeio = open('result', "w")
cmd = cmd.split(' ')
start = datetime.datetime.now()
process = subprocess.Popen(cmd, universal_newlines=True,
stdout=pipeio, stderr=pipeio)
while process.poll() is None:
time.sleep(0.5)
now = datetime.datetime.now()
if (now - start).seconds > timeout:
istimeout = True
try:
process.kill()
time.sleep(0.1)
except Exception, e:
print 'kill timeout process [%s] error, [%s]' % (pid, e)
break
使用subprocess模块替换其他函数
以下都是官方推荐的做法
- os.system()
status = os.system("mycmd" + " myarg")
=>
status = subprocess.call("mycmd" + " myarg", shell=True)
- os.spawn家族
P_NOWAIT example:
pid = os.spawnlp(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg")
==>
pid = Popen(["/bin/mycmd", "myarg"]).pid
P_WAIT example:
retcode = os.spawnlp(os.P_WAIT, "/bin/mycmd", "mycmd", "myarg")
==>
retcode = call(["/bin/mycmd", "myarg"])
Vector example:
os.spawnvp(os.P_NOWAIT, path, args)
==>
Popen([path] + args[1:])
Environment example:
os.spawnlpe(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg", env)
==>
Popen(["/bin/mycmd", "myarg"], env={"PATH": "/usr/bin"})
- os.popen(), os.popen2(), os.popen3()
pipe = os.popen("cmd", 'r', bufsize)
==>
pipe = Popen("cmd", shell=True, bufsize=bufsize, stdout=PIPE).stdout
pipe = os.popen("cmd", 'w', bufsize)
==>
pipe = Popen("cmd", shell=True, bufsize=bufsize, stdin=PIPE).stdin
(child_stdin, child_stdout) = os.popen2("cmd", mode, bufsize)
==>
p = Popen("cmd", shell=True, bufsize=bufsize,
stdin=PIPE, stdout=PIPE, close_fds=True)
(child_stdin, child_stdout) = (p.stdin, p.stdout)
(child_stdin,child_stdout,child_stderr) = os.popen3("cmd", mode, bufsize)
==>
p = Popen("cmd", shell=True, bufsize=bufsize,
stdin=PIPE, stdout=PIPE, stderr=PIPE, close_fds=True)
(child_stdin,child_stdout,child_stderr) = (p.stdin, p.stdout, p.stderr)
(child_stdin, child_stdout_and_stderr) = os.popen4("cmd", mode,
bufsize)
==>
p = Popen("cmd", shell=True, bufsize=bufsize,
stdin=PIPE, stdout=PIPE, stderr=STDOUT, close_fds=True)
(child_stdin, child_stdout_and_stderr) = (p.stdin, p.stdout)