Java原生API实现http请求单路复用

    一、http请求

    一个完整的http请求事务粗略分为4个步骤：

1.   建立网络连接
2.   发送请求包
3.   接收响应数据
4.   关闭连接

    其中第1步是消耗的时间是最多的。显然当对同域服务器发起N次请求就会造成N-1次建立网络连接的时间浪费，

所以现在主流程的web服务都已经支持单路复（甚至多路复用）从而避免这种资源浪费。

 

    二、发起单路复用请求

    加上请求头Connection: keep-alive，就可以保持连接并且可以重复使用。

   GET / HTTP/1.1
   Content-Type: text/xml
   User-Agent: httpclient/1.0
   Connection: keep-alive
   Accept-Encoding: *;q=0
   Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2
   Host: xxxxxxx:xx
   Content-Length: 0

   如果目标服务器不支持在影响头里会带上Connection: close，这要注意的点。

 

   三、用Java实现单路复用

    HttpURLConnection已经提供很多实用的方法，几行代码就可以实现一次http请求:

URL url = new URL(raw);
conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
//conn.setRequestProperty("Connection", "keep-alive");
conn.setDoInput(true);
conn.setDoOutput(true);
conn.setConnectTimeout(3000);
conn.connect();
int code = conn.getResponseCode();
if (code == 200) {
    byte[] bytes = new byte[conn.getInputStream().available()];
    int size = conn.getInputStream().read(bytes);
    System.out.println(new String(bytes, 0, size));
}
conn.disconnect();

   但HttpURLConnection不支持单路复用，否则会抛出IO异常。下面使用socket实现一个具有单路复用的简单的RestClient。

    RestClient对外开放3个接口：

    1、connect建立网络连接

public boolean connect(String ip, int port){
    try{
        this.disconnect();

        host = ip + ":" + port;
        socket = new Socket();
        socket.setSoTimeout(5000);
        SocketAddress addr = new InetSocketAddress(ip, port);
        socket.connect(addr, 5000);
        isConnected = true;
    }catch(Exception e) {
        logger.error("打开" + ip + ":" + port + "接口失败：", e);
    }

    return isConnected;
}

   2、send发送请求并接收响应

/**
 * 发送请求并读取响应数据
 */
public Response send(String req) throws IOException{
    Response result = null;

    if(this.isConnected) {
        logger.debug(host + "请求：\n" + req);
        OutputStream out = socket.getOutputStream();
        //发送请求
        out.write(req.getBytes());
        out.flush();

        InputStream in = socket.getInputStream();
        //读取响应
        byte[] bytes = this.readResp(in);
        if(bytes.length > 0) {
            String resp = new String(bytes).trim();
            logger.debug(host + "响应：\n" + resp);
            result = new Response(resp);
        }
    }

    return result;
}

/**
 * 读取响应数据
 */
private byte[] readResp(InputStream in) throws IOException {
    byte[] bytes = new byte[1024 << 2];
    int i = 0;
    int endFlagCounter = 0;

    //读取响应头
    do {
        int val = in.read();
        byte b = (byte)val;
        if(b == '\r') {
            continue;
        }

        bytes[i++] = b;

        if(i >= bytes.length) {
            break;
        }

        if(b == '\n') {
            endFlagCounter++;
        }else{
            endFlagCounter = 0;
        }

        if(endFlagCounter == 2) {
            break;
        }
    }while(true);

    bytes = ArrayUtils.subarray(bytes, 0, i);
    //读取响应体
    String respBody = this.readRespBody(bytes, in);
    if(StringUtils.isNotBlank(respBody)) {
        bytes = ArrayUtils.addAll(bytes, respBody.getBytes());
    }

    return bytes;
}

/**
 * 提取content-length的数值
 */
private int getContentLength(String[] headers){
    int len = -1;

    for(String s : headers) {
        String headerLine = s.trim().toLowerCase();
        if(headerLine.startsWith("content-length")) {
            String[] kv = s.split(":");
            len = Integer.parseInt(kv[1].trim());
        }
    }

    return len;
}

/**
 * 识别服务返回responseBody的方式
 */
private int selectReadingMode(String[] headers) {
    int result = OTHER;

    for(String s : headers) {
        String headerLine = s.trim().toLowerCase();
        if(StringUtils.equals(headerLine, "transfer-encoding: chunked")) {
            result = TRANSFER_ENCODING;
            break;
        }else if(headerLine.startsWith("content-length")){
            result = CONTENT_LENGHT;
            break;
        }
    }

    return result;
}

/**
 * 读取响应体数据
 */
private String readRespBody(byte[] bytes, InputStream in) throws IOException {
    String result = null;
    byte[] respBody = null;
    String header = new String(bytes);
    String[] headers = header.split("\n");
    int readingMode = this.selectReadingMode(headers);
    int blankLineNum = 0;

    if (readingMode == TRANSFER_ENCODING) {
        do {
            byte[] lineBytes = this.readLine(in);
            String line = new String(lineBytes).trim();
            if(StringUtils.isBlank(line)) {
                blankLineNum++;
                if(blankLineNum >= 2) { //连续两个换行符结束读取
                    break;
                }else {
                    continue;
                }
            }

            blankLineNum = 0;
            if(false == line.matches("^[\\dabcdefABCDEF]+$")){
                continue;
            }

            int size = Integer.parseInt(line, 16);
            if(size > 0) {
                byte[] block = this.read(in, size);

                if(respBody == null) {
                    respBody = block;
                }else{
                    respBody = ArrayUtils.addAll(respBody, block);
                }
            }else{
                break;
            }
        }while(true);
    }else if(readingMode == CONTENT_LENGHT){
        int len = getContentLength(headers);
        if(len > 0) {
            respBody = this.read(in, len);
        }
    }

    int remain = in.available();
    if(remain > 0) {
        //提取余下的数据不作处理，为了不响应下次读取
        this.read(in, remain);
    }

    if(respBody != null) {
        result = new String(respBody);
    }

    return result;
}

/**
 * 读取指定大小的数据块
 */
private byte[] read(InputStream in, int size) throws IOException {
    byte[] bytes = new  byte[size];
    int offset = 0;
    int len = size;
    int num = -1;
    do{
        num = in.read(bytes, offset, len);
        if(num == -1) {
            break;
        }
        offset += num;
        len = size - offset;
    }while(offset < size);

    if(offset < size) {
        return Arrays.copyOfRange(bytes, 0, offset);
    }else{
        return bytes;
    }
}

/**
 * 读取一行数据
 */
private byte[] readLine(InputStream in) throws IOException {
    ByteArrayOutputStream byteBuf = new ByteArrayOutputStream();

    do{
        byte c = (byte)in.read();
        if(c == '\r') {
            continue;
        }

        byteBuf.write(c);
        if(c == '\n') {
            break;
        }
    }while(true);

    return byteBuf.toByteArray();
}

  3、disconnect关闭连接

public void disconnect(){
    try {
        if(this.socket != null && false == this.socket.isClosed()) {
            this.socket.close();
        }
    } catch (IOException e) {
        logger.error(host + "关闭连接失败：", e);
    }

    this.isConnected = false;
}

 4、Response对象

public class Response {

    private Map<String, String> headerMap;

    private int statusCode;

    private String body;

    private String respString;

    public Response(String resp){
        this.headerMap = new HashMap<>();
        this.respString = resp;

        if(StringUtils.isNotBlank(resp.trim())) {
            String[] lines = resp.trim().split("\n");

            if (lines[0].length() > 9) {
                String line = lines[0].substring(9);
                this.statusCode = Integer.parseInt(line.substring(0, 3));
            }

            int i = 1;
            for(; i < lines.length; i++) {
                if(StringUtils.isBlank(lines[i])) {
                    break;
                }

                String[] keyValue = lines[i].split(":", 2);
                headerMap.put(keyValue[0], keyValue[1].trim());
            }

            StringBuffer buf = new StringBuffer();
            for(i+=1; i  < lines.length; i++) {
                buf.append(lines[i]);
                buf.append("\n");
            }
            this.body = buf.toString();
        }
    }

    public Map<String, String> getHeaderMap() {
        return headerMap;
    }

    public void setHeaderMap(Map<String, String> headerMap) {
        this.headerMap = headerMap;
    }

    public String getBody() {
        return body;
    }

    public void setBody(String body) {
        this.body = body;
    }

    @Override
    public String toString(){
        return this.respString;
    }

    public int getStatusCode() {
        return statusCode;
    }

    public void setStatusCode(int statusCode) {
        this.statusCode = statusCode;
    }
}

     到此一个简单的RestClient就完成了，其中比较复杂的代码都是针对http协议的解析，有兴趣可以自行百度在这就不做过多的说明。

     四、100次访问测试

     分别使用RestClient和HttpURLConnection访问100次http://www.w3school.com.cn/

•      RestClient花费：4235毫秒
•      HttpURLConnection花费：7485毫秒

    RestClient的没有处理压缩编码Accept-Encoding，所以在使用的时候要在请求头加上：Accept-Encoding: *;q=0

    测试代码：

String raw = "http://www.w3school.com.cn:80";
RestClient client = new RestClient();

try {
    URL url = new URL(raw);
    client.connect(url.getHost(), url.getPort());

    StringBuffer buf = new StringBuffer();
    buf.append("GET / HTTP/1.1");
    buf.append("\r\n");
    buf.append("Content-Type: text/plain");
    buf.append("\r\n");
    buf.append("User-Agent: restclient/1.0");
    buf.append("\r\n");
    buf.append("Connection: keep-alive"); //保持连接，单路复用
    buf.append("\r\n");
    buf.append("Accept-Encoding: *;q=0"); //不支持任何压缩编码
    buf.append("\r\n");
    buf.append("Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2");
    buf.append("\r\n");
    buf.append("Host: " + url.getHost() + ":" + url.getPort());
    buf.append("\r\n");
    buf.append("Content-Length: 0");
    buf.append("\r\n");
    buf.append("\r\n");

    for(int i = 0; i < 100; i++) {
        Response resp = client.send(buf.toString());
        if (resp.getStatusCode() == 200) {
           System.out.println(resp.getBody());
        }
    }
}catch(Exception e) {
   logger.error(e);
}finally {
    client.disconnect();
}

      对于请求数据没有任何封装， 有兴趣可以自行实现。