HTTP POST输出流与内存优化

 

起因

有个功能，比如上传文件，想在HTTPPOST时候想打开一个OutputStream，然后将File Content通过FileInputStream边读边上传。以节省内存。

之前一直理解不到位，以前是这样实现的：

URL url = new URL("http://ddaitp.sinaapp.com/test.php");
       HttpURLConnection conn =(HttpURLConnection) url.openConnection();
       conn.setRequestMethod("POST");
       conn.setDoOutput(true);
       conn.setDoInput(true);
       conn.connect();
    OutputStreamos = conn.getOutputStream();

后来发现这样没有解决问题，当文件比较大的时候依然会OutOfMemory。

原因

但是仔细读代码发现这样不能解决问题，还是将要上传的内容都读到内存，然后在flush() 时上传。

因为打印conn发现实际为libcore.net.http.ChunkedOutputStream。conn.getOutputStream()得到的OutputStream为RetryableOutputStream。

如果要实现目的，需要setFixedLengthStreamingMode()或者setChunkedStreamingMode(int)。

RetryableOutputStream中关键实现如下：

private final ByteArrayOutputStream content;
public RetryableOutputStream() {
        this.limit = -1;
        this.content = new ByteArrayOutputStream();
    }
public synchronized void write(byte[] buffer, int offset, int count)
            throws IOException {
        checkNotClosed();
        Arrays.checkOffsetAndCount(buffer.length, offset, count);
        if (limit != -1 && content.size() > limit - count) {
            throw new IOException("exceeded content-length limit of " + limit + " bytes");
        }
        content.write(buffer, offset, count);
    }

所有的内容都写到了ByteArrayOutputStream中，存在于在内存。

 关键代码如下：

libcore.net.http.HttpURLConnectionImpl

https://android.googlesource.com/platform/libcore/+/4f81a06/luni/src/main/java/libcore/net/http/HttpURLConnectionImpl.java

 

public final OutputStream getOutputStream() throws IOException {
        connect();
 
        OutputStreamresult = httpEngine.getRequestBody();
        if (result == null) {
            throw new ProtocolException("method does not support a request body:" + method);
        } else if (httpEngine.hasResponse()) {
            throw new ProtocolException("cannot write request body afterresponse has been read");
        }
 
        return result;
     }

libcore.net.http.HttpEngine

https://android.googlesource.com/platform/libcore/+/4f81a06/luni/src/main/java/libcore/net/http/HttpEngine.java

protected void initRequestBodyOut() throws IOException {
        int chunkLength = policy.getChunkLength();
        if (chunkLength > 0 || requestHeaders.isChunked()) {
            sendChunked = true;
            if (chunkLength == -1) {
                chunkLength = DEFAULT_CHUNK_LENGTH;
            }
        }
 
        if (socketOut == null) {
            throw new IllegalStateException("No socket to write to; was a POSTcached?");
        }
 
        if (httpMinorVersion == 0) {
            sendChunked = false;
        }
 
        intfixedContentLength = policy.getFixedContentLength();
        if (requestBodyOut != null) {
            // request body was already initialized bythe predecessor HTTP engine
        } else if (fixedContentLength != -1) {
            writeRequestHeaders(fixedContentLength);
            requestBodyOut = new FixedLengthOutputStream(requestOut, fixedContentLength);
        } else if (sendChunked) {
            writeRequestHeaders(-1);
            requestBodyOut = new ChunkedOutputStream(requestOut, chunkLength);
        } else if (requestHeaders.getContentLength() != -1) {
            writeRequestHeaders(requestHeaders.getContentLength());
            requestBodyOut = new RetryableOutputStream(requestHeaders.getContentLength());
        } else {
            requestBodyOut = new RetryableOutputStream();
        }
    }
}

修改

结论，修改办法也简单。有两种办法

Chunked Streaming
添加conn.setChunkedStreamingMode(0);

修改后如下：

     HttpURLConnection conn;
           try {
              conn = (HttpURLConnection) new URL(
                     "http://10.2.167.14/").openConnection();
              conn.setRequestMethod("POST");
              conn.setRequestProperty("User-Agent", "ddai-agent");
              conn.setChunkedStreamingMode(0);
              conn.setConnectTimeout(0);
              conn.setReadTimeout(0);
              conn.setDoOutput(true);
              conn.setDoInput(true);
              conn.connect();
              BufferedOutputStream os = new BufferedOutputStream(
                     conn.getOutputStream());
              upload(os, conn);
              os.flush();
              os.close();
              conn.disconnect();
           } catch (MalformedURLException e) {
              e.printStackTrace();
           } catch (IOException e) {
              e.printStackTrace();
           }
 

  

Fixed Length

设置POST内容的总长度：

conn.setFixedLengthStreamingMode();

由于我写出只是一个File，所以长度不包括HTTP标准的其他内容。

HttpURLConnection conn;
           try {
              conn = (HttpURLConnection) new URL(
                     "http://ddaitp.sinaapp.com/test.php").openConnection();
              conn.setRequestMethod("POST");
              conn.setDoOutput(true);
              conn.setDoInput(true);
              conn.setFixedLengthStreamingMode((int) uploadFile.length());
              conn.connect();
              OutputStream os =conn.getOutputStream();
              upload(os, conn);
              conn.disconnect();
           } catch (MalformedURLException e) {
              e.printStackTrace();
           } catch (IOException e) {
              e.printStackTrace();
           }

 

测试

上传同一个文件，SDCARD/aa.rar 20MB左右

未上传时候：

用以前的错误方式上传，上传过程中内存占用一直上升，然后OutOfMemory。崩溃时内存占用：

然后是Chunked方式：
上传过程中内存占用稳定的保持很低。

Fiexd Length 方式：

和Chunked方式差不多一直很低。