HttpURLConnection keep-alive 问题

问题

private static void download(String downloadURL) {
        File localReviseFile = new File("/home/admin/1.log");
        HttpURLConnection connection = null;
        InputStream inputStream = null;
        try {
            URL url = new URL(downloadURL);
            connection = (HttpURLConnection) url.openConnection();
            connection.setConnectTimeout(10000);
            connection.setReadTimeout(60000);
            connection.setDoOutput(true);
            connection.setRequestMethod("GET");
            connection.setIfModifiedSince(999999999);
            connection.connect();
            inputStream = connection.getInputStream();
            if (inputStream == null) {
                System.out.println("failed to download " + downloadURL + ", input stream is null");
            } else {
                int fileLength = connection.getContentLength();
                System.out.println(downloadURL + " content length : "+ fileLength + " , response code : "+connection.getResponseCode() + " , msg "+connection.getResponseMessage());
            }
        } catch (IOException e) {
            System.out.println("failed to download checkResult.log from remote url:" + downloadURL);
        } finally {
            connection.disconnect();
            inputStream.close();
        }
    }

        String url1 = "http://10.81.52.159:10183/getLoggerFile?filePath=/home/admin/logs/jwlog/ECS-HZ_T_MD_191534/Result.log";
        String url2 = "http://10.81.52.159:10183/getLoggerFile?filePath=/home/admin/logs/jwlog/ECS-HZ_T_MD_191536/Result.log";

        download(url1);
        download(url2);

发现两次返回 connection.getContentLength() 均为 0。
实际上 ECS-HZ_T_MD_191534/Result.log 为 0，ECS-HZ_T_MD_191536/Result.log 的不为 0。

那么很明显，这是由缓存引起的问题。

解决1，不行

使用了这些配置，问题未解决：
connection.setRequestProperty(“Connection”, “close”);
connection.setDefaultUseCaches(false);
connection.setUseCaches(false);
在 finally 中添加
connection.disconnect()
仍不行

解决2，可行

添加：System.setProperty(“http.keepAlive”, “false”);
将整个 APP 的 http 长连接支持关掉。
问题解决。

解决3，可行

采用 okhttp

<dependency>
    <groupId>com.squareup.okhttp3</groupId>
    <artifactId>okhttp</artifactId>
    <version>4.2.2</version>
</dependency>

OkHttpClient client = new OkHttpClient();
        Request request = new Request.Builder().url(url).build();
        Response response = client.newCall(request).execute();
        System.out.println("url: " + url);
        System.out.println("response: " + response.body().string());
response.close()

原因

参考

• https://github.com/bingoohuang/blog/issues/66

从相关源代码（HttpClient, KeepAliveCache, HttpURLConnection）来看，disconnect 只会释放空余的连接，而keep-alive的连接，会放到 KeepAliveCache的缓存中，并且默认有5秒的缓存失效时间。因此，为了保持一致性，建议还是调用disconnect，这样如果keep-alive不生效时，能及时释放链接 ，keep-alive生效时，也不影响连接 的重用。

参考

https://docs.oracle.com/javase/1.5.0/docs/guide/net/http-keepalive.html

参考

• https://stackoverflow.com/questions/4767553/safe-use-of-httpurlconnection
  According to http://docs.oracle.com/javase/6/docs/technotes/guides/net/http-keepalive.html and OpenJDK source code.

(When keepAlive == true)

If client called HttpURLConnection.getInputSteam().close(), the later call to HttpURLConnection.disconnect() will NOT close the Socket. i.e. The Socket is reused (cached)

If client does not call close(), call disconnect() will close the InputStream and close the Socket.

So in order to reuse the Socket, just call InputStream.close(). Do not call HttpURLConnection.disconnect().

参考

• https://stackoverflow.com/questions/272910/in-java-when-does-a-url-connection-close
  If you cast to an HttpURLConnection, there is a disconnect() method. If the connection is idle, it will probably disconnect immediately. No guarantees.

参考

• https://www.cnblogs.com/whoislcj/p/5520384.html
  HttpUrlConnection 是Android SDK的标准实现，而 HttpClient 是apache的开源实现；
  HttpUrlConnection 直接支持GZIP压缩；HttpClient 也支持，但要自己写代码处理；
  HttpUrlConnection 直接支持系统级连接池，即打开的连接不会直接关闭，在一段时间内所有程序可共用；HttpClient 当然也能做到，但毕竟不如官方直接系统底层支持好；
  HttpUrlConnection 直接在系统层面做了缓存策略处理，加快重复请求的速度。

参考

• https://hongjiang.info/httpurlconnection-socket-reuse/
• https://mttkay.github.io/blog/2013/03/02/herding-http-requests-or-why-your-keep-alive-connection-may-be-dead/
• https://stackoverflow.com/questions/4767553/safe-use-of-httpurlconnection

总结

• 简单说，就是 HttpURLConnection 在系统层做了缓存，即便是每次都是新建的 HttpURLConnection 对象，但在 TCP 层是复用的。
• System.setProperty(“http.keepAlive”, “false”); 关闭了整个应用的 HTTP 连接的复用，自然能解决这个问题。
• 如果只是 inputStream.close(); 那么 TCP 层一定会重用。
• 如果是 connection.disconnect(); 那么 TCP 层也不能保证不会重用。disconnect 只会释放空余的连接，而 keep-alive 的连接，会放到 KeepAliveCache 的缓存中，并且默认有5秒的缓存失效时间