线上事故:Unexpected end of file from server

2019独角兽企业重金招聘Python工程师标准>>>

事故说明

短信服务单独抽出来,提供http接口访问.通过HAProxy 提供负载均衡.

日志中偶现:

java.net.SocketException: Unexpected end of file from server
	at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:792)
	at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:647)
	at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:789)
	at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:647)
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1569)
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474)

当时没有太过在意,直到请求短信服务不可用,才来排查问题所在.

由于http请求代码没有设置超时,导致HAProxy 挂满请求,从而整个服务不可用

异常代码

 try {
            u = new URL(url);
            con = (HttpURLConnection) u.openConnection();
            con.setRequestMethod("POST");
            con.setDoOutput(true);
            con.setDoInput(true);
            con.setUseCaches(false);
            con.setRequestProperty("Content-Type", "application/json");
            OutputStreamWriter osw = new OutputStreamWriter(con.getOutputStream(), "UTF-8");
            osw.write(msg);
            osw.flush();
            osw.close();
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            if (con != null) {
                con.disconnect();
            }

        }

修复方案

** 添加超时,所有的io操作都应该添加超时 **

try {
            u = new URL(url);
            con = (HttpURLConnection) u.openConnection();
            con.setRequestMethod("POST");
            con.setDoOutput(true);
            con.setDoInput(true);
            con.setUseCaches(false);
            con.setConnectTimeout(3000);
            con.setReadTimeout(3000);
            con.setRequestProperty("Content-Type", "application/json");
            OutputStreamWriter osw = new OutputStreamWriter(con.getOutputStream(), "UTF-8");
            osw.write(msg);
            osw.flush();
            osw.close();
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            if (con != null) {
                con.disconnect();
            }

        }

第二种,由于以前老代码直接使用的net包下的,个人觉得不是很好用,直接升级到了httpclient fluent

try {
            return Request.Post(url)
                    .useExpectContinue()
                    .version(HttpVersion.HTTP_1_1)
                    .bodyString(param, ContentType.APPLICATION_JSON)
                    .connectTimeout(3000)
                    .socketTimeout(3000)
                    .execute()
                    .returnContent()
                    .asString();
        } catch (IOException e) {
            LOGGER.error("请求地址:{},参数:{}",url,param);
            LOGGER.error("post请求出错",e);
        }

pom依赖

            <dependency>
                <groupId>org.apache.httpcomponents</groupId>
                <artifactId>fluent-hc</artifactId>
                <version>4.5.3</version>
            </dependency>

转载于:https://my.oschina.net/superwen/blog/893028