Python主进程hang住的两个原因

最近使用Python遇到两个非常不好定位的问题，表现都是Python主进程hang住。最终定位出一个是subprocess模块的问题，一个是threading.Timer线程的问题。

---

subprocess模块不当使用的问题

Python的subprocess比较强大，基本上能替换os.system、os.popen、commands.getstatusoutput的功能，但是在使用的过程中需要注意参数stdin/stdout/stderr使用subprocess.PIPE的情况，因为管道通常会有默认的buffer size(Linux x86_64下实测是64K，这里有个疑问io.DEFAULT_BUFFER_SIZE是8K，而ulimit -a的pipe size为512 * 8 = 4K?)，父进程如果不使用communicate消耗掉子进程write pipe(stdout/stderr)中的数据，直接进入wait，此时子进程可能阻塞在了pipe的写上，从而导致父子进程都hang住。下面是测试代码。

# main.py
#!/usr/bin/env python
# encoding: utf-8

import subprocess
import os
import tempfile
import sys
import traceback
import commands


# both parent and child process will hang 
# if run.py stdout/stderr exceed 64K, since
# parent process is waiting child process exit
# but child process is blocked by writing pipe
def testSubprocessCallPipe():
    # call: just Popen().wait()
    p = subprocess.Popen(["python", "run.py"], 
        stdin=subprocess.PIPE, 
        stdout=subprocess.PIPE, 
        stderr=subprocess.PIPE)
    ret = p.wait()
    print ret


# will not hang since the parent process which
# call communicate will poll or thread to comsume
# the pipe buffer, so the child process can write
# all it's data to stdout or stderr pipe and it will
# not be blocked.
def testSubprocessCommunicate():
    p = subprocess.Popen(["python", "run.py"], 
        stdin=subprocess.PIPE, 
        stdout=subprocess.PIPE, 
        stderr=subprocess.PIPE)
    print p.communicate()[0]


# will not hang since sys.stdout and sys.stderr 
# don't have 64K default buffer limita