HTTP客户端警告：Going to buffer response body of large or unknown size

HTTP客户端警告：Going to buffer response body of large or unknown size

   点关注不迷路，欢迎再访！	

精简博客内容，尽量已行业术语来分享。
努力做到对每一位认可自己的读者负责。
帮助别人的同时更是丰富自己的良机。

原编码问题

HttpClient发现抛出Going to buffer response body of large or unknown size. Using getResponseBodyAsStream instead is recommended，查看原有代码中的逻辑如下：

HttpClient httpclient = new HttpClient();
GetMethod getMethod = new GetMethod(url);
int statusCode = httpclient.executeMethod(getMethod);
String respContent = getMethod.getResponseBodyAsString();

源码分析 getResponseBodyAsString()源码

	//getResponseBodyAsString()方法源码
    public String getResponseBodyAsString() throws IOException {
        byte[] rawdata = null;
        if (this.responseAvailable()) {
        	//调用了getResponseBody()，容易消耗内存
            rawdata = this.getResponseBody();
        }

        return rawdata != null ? EncodingUtil.getString(rawdata, this.getResponseCharSet()) : null;
    }
    //responseAvailable()方法源码
    private boolean responseAvailable() {
        return this.responseBody != null || this.responseStream != null;
    }
    
    //getResponseBody()方法源码
    public byte[] getResponseBody() throws IOException {
        if (this.responseBody == null) {
            InputStream instream = this.getResponseBodyAsStream();
            if (instream != null) {
                long contentLength = this.getResponseContentLength();
                if (contentLength > 2147483647L) {
                    throw new IOException("Content too large to be buffered: " + contentLength + " bytes");
                }

                int limit = this.getParams().getIntParameter("http.method.response.buffer.warnlimit", 1048576);
                if (contentLength == -1L || contentLength > (long)limit) {
                //这里是warn的原文
                    LOG.warn("Going to buffer response body of large or unknown size. Using getResponseBodyAsStream instead is recommended.");
                }

                LOG.debug("Buffering response body");
                ByteArrayOutputStream outstream = new ByteArrayOutputStream(contentLength > 0L ? (int)contentLength : 4096);
                byte[] buffer = new byte[4096];

                int len;
                while((len = instream.read(buffer)) > 0) {
                    outstream.write(buffer, 0, len);
                }

                outstream.close();
                this.setResponseStream((InputStream)null);
                this.responseBody = outstream.toByteArray();
            }
        }

        return this.responseBody;
    }

从源码中可以看出，warn的条件是(contentLength == -1L || contentLength > (long)limit)，如果http头没有指定contentLength或大于上限值（默认1M），就会抛异常。其实，如果返回的结果比较确定，对程序没有太大影响。而对于返回结果不确定时，源码也建议我们使用下面的getResponseBodyAsStream()方法。

getResponseBodyAsStream()源码

public InputStream getResponseBodyAsStream() throws IOException {
        if (this.responseStream != null) {
            return this.responseStream;
        } else if (this.responseBody != null) {
            InputStream byteResponseStream = new ByteArrayInputStream(this.responseBody);
            LOG.debug("re-creating response stream from byte array");
            return byteResponseStream;
        } else {
            return null;
        }
    }

从源码中可以看出，getResponseBodyAsStream()内部没有使用getResponseBody()方法，避免了内存耗尽问题，而是使用了InputStream流方式处理。

优化原编码

HttpClient httpclient = new HttpClient();
GetMethod getMethod = new GetMethod(url);
int statusCode = httpclient.executeMethod(getMethod);
//String respContent = getMethod.getResponseBodyAsString();
//使用getResponseBodyAsStream()
InputStream inputStream = getMethod.getResponseBodyAsStream();
BufferedReader br = new BufferedReader(new InputStreamReader(inputStream));
StringBuffer stringBuffer = new StringBuffer();
String str = "";
while ((str = br.readLine()) != null) {
       stringBuffer.append(str);
}
LOGGER.info("respContent: {}", stringBuffer.toString());