subprocess

subprocess模块

subprocess模块是python中子进程模块，可以用来在python程序之中调用其他程序，或者执行系统命令。官方建议用subprocess模块来替代一些原有的函数，比如os.system()

subprocess.Popen

1. Popen()
   Popen启动新的进程与父进程并行执行，默认父进程不等待新进程结束。

def TestPopen():
    import subprocess
    p = subprocess.Popen("calc",shell=True)
    for i in range(10) :
        print (i)

1. Popen.wait()
   Popen.wait()函数使得父进程等待新创建的进程运行结束，然后再继续父进程的其他任务。且此时可以在Popen.returncode中得到新进程的返回值。

def TestWait():
    import subprocess
    import datetime
    print (datetime.datetime.now())
    p = subprocess.Popen("calc",shell=True)
    p.wait()
    print (p.returncode)
    print (datetime.datetime.now())

1. Popen.poll()
   Popen.poll()函数可以用来检测新创建的进程是否结束。

import subprocess
import datetime
import time
print (datetime.datetime.now())
p=subprocess.Popen("calc",shell=True)
t = 1
while(t <= 5):
    time.sleep(1)
    if p.poll() == None:
        print "process not finished"
    else:
        print "process finished"
    print (p.returncode)
    t+=1
print (datetime.datetime.now())

1. Popen.kill()&Popen.terminate()
   Popen.kill()&Popen.terminate()用来结束创建的新进程，在windows系统上相当于调用TerminateProcess()，在posix系统上相当于发送信号SIGTERM和SIGKILL。

def TestKillAndTerminate():
    p=subprocess.Popen("notepad.exe")
    t = 1
    while(t <= 5):
      time.sleep(1)
      t +=1
    p.kill()
    #p.terminate()
    print ("new process was killed")

1. Popen.communicate()
   Popen.communicate()可以与新进程交互，但是必须要在popen构造时候将管道重定向。

def TestCommunicate():
    import subprocess
    cmd = "dir"
    p=subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
    (stdoutdata, stderrdata) = p.communicate()

    if p.returncode != 0:
          print (cmd + "error !")
    #defaultly the return stdoutdata is bytes, need convert to str and utf8
    for r in str(stdoutdata,encoding='utf8' ).split("\n"):
      print (r)
    print (p.returncode)


def TestCommunicate2():
  import subprocess
  cmd = "dir"
  #universal_newlines=True, it means by text way to open stdout and stderr
  p = subprocess.Popen(cmd, shell=True, universal_newlines=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
  curline = p.stdout.readline()

  while(curline != ""):
        print (curline)
        curline = p.stdout.readline()
  p.wait()
  print (p.returncode)

subprocess.call()

call函数可以认为是对popen和wait的分装，直接对call函数传入要执行的命令行，将命令行的退出code返回。

def TestCall():
    retcode = subprocess.call("c:\\test.bat")
    print (retcode)

subprocess.getoutput&subprocess.getstatusoutput

subprocess.getoutput&subprocess.getstatusoutput，基本上等价于subprocess.call函数，但是可以返回output，或者同时返回退出code和output。

但是可惜的是好像不能在windows平台使用，在windows上有如下错误：’{’ is not recognized as an internal or external command, operable program or batch file.

def TestGetOutput():
    outp = subprocess.getoutput("ls -la")
    print (outp)

def TestGetStatusOutput():
    (status, outp) = subprocess.getstatusoutput('ls -la')
    print (status)
    print (outp)

总结

Popen的参数，
第一个为字符串（或者也可以为多个非命名的参数），表示你要执行的命令和命令的参数；后面的均为命名参数；

shell=True,表示你前面的传入的命令将在shell下执行，如果你的命令是个可执行文件或bat，不需要指定此参数；

stdout=subprocess.PIPE用来将新进程的输出重定向，stderr=subprocess.STDOUT将新进程的错误输出重定向到stdout，stdin=subprocess.PIPE用来将新进程的输入重定向；

universal_newlines=True表示以text的方式打开stdout和stderr。

其他的不推荐使用的模块：
os.system
os.spawn
os.popen
popen2.
commands.

范例

def process(workPath, resultPath, fileName,timeout = 200):
    sha1 = fileName.split('.')[0]
    filePath = os.path.join(workPath,fileName)
    resultPath = os.path.join(resultPath,sha1 + ".txt")
    if os.path.exists(resultPath):
        os.remove(resultPath)
    os.mknod(resultPath)
    cmd = parseCmd(filePath, resultPath)
    start = datetime.datetime.now()
    process = subprocess.Popen(cmd, shell=True)
    while process.poll() is None:
        time.sleep(0.5)
        now = datetime.datetime.now()
        if (now - start).seconds > timeout:
            istimeout = True
            try:
                process.kill()
                time.sleep(0.1)                
            except Exception, e:
                print 'kill timeout process [%s] error, [%s]' % (pid, e)
            break

这个程序有一个很大的问题，因为是使用子进程是使用shell的形式启动的，之后shell中又会创建一个子进程来执行java命令，这样当超时的时候使用process.kill()的时候将shell进程kill掉，并不能kill java进程！

所以不应该用子进程的执行不应当使用shell的形式。但是如果不使用shell形式而直接传递cmd命令的话是会报错的，因为是命令，这种形式执行的是可执行程序。（报错原因就是找不到java.jar）

这个时候可以采用数组传递参数的形式来执行，跳过了shell：

def process(workPath, resultPath, fileName,timeout = 200):
    sha1 = fileName.split('.')[0]
    filePath = os.path.join(workPath,fileName)
    resultPath = os.path.join(resultPath,sha1 + ".txt")
    if os.path.exists(resultPath):
        os.remove(resultPath)
    os.mknod(resultPath)
    cmd = parseCmd(filePath, resultPath)
    pipeio = open('result', "w")
    cmd = cmd.split(' ')
    start = datetime.datetime.now()
    process = subprocess.Popen(cmd, universal_newlines=True, 
                            stdout=pipeio, stderr=pipeio)
    while process.poll() is None:
        time.sleep(0.5)
        now = datetime.datetime.now()
        if (now - start).seconds > timeout:
            istimeout = True
            try:
                process.kill()
                time.sleep(0.1)                
            except Exception, e:
                print 'kill timeout process [%s] error, [%s]' % (pid, e)
            break

使用subprocess模块替换其他函数

以下都是官方推荐的做法

1. os.system()

status = os.system("mycmd" + " myarg")
=>
status = subprocess.call("mycmd" + " myarg", shell=True)

1. os.spawn家族

P_NOWAIT example:
pid = os.spawnlp(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg")
==>
pid = Popen(["/bin/mycmd", "myarg"]).pid

P_WAIT example:
retcode = os.spawnlp(os.P_WAIT, "/bin/mycmd", "mycmd", "myarg")
==>
retcode = call(["/bin/mycmd", "myarg"])

Vector example:
os.spawnvp(os.P_NOWAIT, path, args)
==>
Popen([path] + args[1:])

Environment example:
os.spawnlpe(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg", env)
==>
Popen(["/bin/mycmd", "myarg"], env={"PATH": "/usr/bin"})

1. os.popen(), os.popen2(), os.popen3()

pipe = os.popen("cmd", 'r', bufsize)
==>
pipe = Popen("cmd", shell=True, bufsize=bufsize, stdout=PIPE).stdout

pipe = os.popen("cmd", 'w', bufsize)
==>
pipe = Popen("cmd", shell=True, bufsize=bufsize, stdin=PIPE).stdin

(child_stdin, child_stdout) = os.popen2("cmd", mode, bufsize)
==>
p = Popen("cmd", shell=True, bufsize=bufsize,
          stdin=PIPE, stdout=PIPE, close_fds=True)
(child_stdin, child_stdout) = (p.stdin, p.stdout)

(child_stdin,child_stdout,child_stderr) = os.popen3("cmd", mode, bufsize)
==>
p = Popen("cmd", shell=True, bufsize=bufsize,
          stdin=PIPE, stdout=PIPE, stderr=PIPE, close_fds=True)
(child_stdin,child_stdout,child_stderr) = (p.stdin, p.stdout, p.stderr)

(child_stdin, child_stdout_and_stderr) = os.popen4("cmd", mode,
                                                   bufsize)
==>
p = Popen("cmd", shell=True, bufsize=bufsize,
          stdin=PIPE, stdout=PIPE, stderr=STDOUT, close_fds=True)
(child_stdin, child_stdout_and_stderr) = (p.stdin, p.stdout)