OkHttp原理解析之连接拦截器

目标:

  1. DNS解析
  2. HTTPS:SSL握手与加密
  3. HTTP代理:普通代理与隧道代理
  4. SOCKS代理

前置知识

分析OkHttp的连接拦截器ConnectInterceptor之前,先来学习预备知识,包括代理和DNS。

代理

  • 不使用代理的情况(普通http请求)
  • 使用HTTP代理的情况(使用HTTP代理时又分为发送Http请求,发送Https请求的情况)
  • 使用SOCKS代理的情况

DNS

  • DNS简介
  • DNS原理
  • DNS特点

下面对代理和DNS进行详细讲解。

代理

普通http请求(即不使用代理)

http请求报文:
GET /v3/weather/weatherInfo?city=长沙&key=13cb58f5884f9749287abbead9c658f2 HTTP/1.1
Host: restapi.amap.com

    /**
     * 普通http请求,没有使用代理
     *
     * @throws IOException
     */
    public void testHttpNoProxy() throws IOException {

        Socket socket = new Socket();
        socket.connect(new InetSocketAddress("restapi.amap.com", 808));//restapi.amap.com是需要请求的目标主机
        InputStream is = socket.getInputStream();
        OutputStream os = socket.getOutputStream();

        StringBuilder sb = new StringBuilder();
        sb.append("GET /v3/weather/weatherInfo?city=长沙&key" +
                "=13cb58f5884f9749287abbead9c658f2 HTTP/1.1\r\n");
        sb.append("Host: restapi.amap.com\r\n\r\n");
        os.write(sb.toString().getBytes());
        os.flush();

        BufferedReader reader = new BufferedReader(new InputStreamReader(is));
        String msg;
        while ((msg = reader.readLine()) != null) {
            System.out.println(msg);
        }
    }

使用HTTP代理

发送Http请求

使用http代理,发送Http请求时,发送的请求行中需要加上域名。
http请求报文:
GET http://restapi.amap.com/v3/weather/weatherInfo?city=长沙&key=13cb58f5884f9749287abbead9c658f2 HTTP/1.1
Host: restapi.amap.com

  /**
     * 使用http代理,发送Http请求
     *
     * @throws IOException
     */
    public void testHttpProxy() throws IOException {
        //okhttp的用法,还可以
        // new Socket(new Proxy(Type.HTTP,new InetSocketAddress("114.239.145.90", 808)))
        // connect(new InetSocketAddress("restapi.amap.com", 80))
        //然后直接 发送准确的http数据就可以了即: GET /v3/weather/weatherInfo... HTTP/1.1
        Socket socket = new Socket();
        socket.connect(new InetSocketAddress("114.239.145.90", 808));//114.239.145.90是代理服务器,代理服务器就是转发的作用
        InputStream is = socket.getInputStream();
        OutputStream os = socket.getOutputStream();

        /**注意使用 http普通代理 ,发送的请求行中需要加上域名:
         *    sb.append("GET http://restapi.amap.com/v3/weather/weatherInfo?city=长沙&key" +
         *                 "=13cb58f5884f9749287abbead9c658f2 HTTP/1.1\r\n");
         *    sb.append("Host: restapi.amap.com\r\n\r\n");
         *
         * 而如果没有使用代理,发送的http请求报文是这样的:
         *    sb.append("GET /v3/weather/weatherInfo?city=长沙&key" +
         *                 "=13cb58f5884f9749287abbead9c658f2 HTTP/1.1\r\n");
         *    sb.append("Host: restapi.amap.com\r\n\r\n");
         *    请求行中没有域名,域名是加在Host请求头中的。
         *
         */
        StringBuilder sb = new StringBuilder();
        sb.append("GET http://restapi.amap.com/v3/weather/weatherInfo?city=长沙&key" +
                "=13cb58f5884f9749287abbead9c658f2 HTTP/1.1\r\n");
        sb.append("Host: restapi.amap.com\r\n\r\n");
        os.write(sb.toString().getBytes());
        os.flush();

        BufferedReader reader = new BufferedReader(new InputStreamReader(is));
        String msg;
        while ((msg = reader.readLine()) != null) {
            System.out.println(msg);
        }
    }

当然这些工作okhttp会帮我们完成:

//RequestLine.java

  /**
   * Returns the request status line, like "GET / HTTP/1.1". This is exposed to the application by
   * {@link HttpURLConnection#getHeaderFields}, so it needs to be set even if the transport is
   * HTTP/2.
   */
  public static String get(Request request, Proxy.Type proxyType) {
    StringBuilder result = new StringBuilder();
    result.append(request.method());
    result.append(' ');

    if (includeAuthorityInRequestLine(request, proxyType)) {
      result.append(request.url());//http代理请求的请求行需要保留域名,即完整的url即可
    } else {
      result.append(requestPath(request.url()));//从url中解析path,普通http请求的请求行只需要路径,不需要域名
    }

    result.append(" HTTP/1.1");
    return result.toString();
  }

  /**
   * Returns true if the request line should contain the full URL with host and port (like "GET
   * http://android.com/foo HTTP/1.1") or only the path (like "GET /foo HTTP/1.1").
   */
  private static boolean includeAuthorityInRequestLine(Request request, Proxy.Type proxyType) {
    return !request.isHttps() && proxyType == Proxy.Type.HTTP;
  }
发送Https请求

需要先给代理服务器发送CONNECT请求(而不是直接发送GET请求),代理服务器返回成功后再使用ssl包装与代理服务器的socket,即生成sslSocket,然后利用这个sslSocket发送GET请求。

先发送 CONNECT 请求:
CONNECT restapi.amap.com HTTP/1.1
Host: restapi.amap.com
服务器返回200的响应后,再发送 GET 请求:
GET /v3/weather/weatherInfo?city=长沙&key=13cb58f5884f9749287abbead9c658f2 HTTP/1.1
Host: restapi.amap.com

 /**
     * 使用http代理,发送Https请求
     *
     * @throws IOException
     */
    public void testHttpsProxy() throws IOException {
        Socket socket = new Socket();
        socket.connect(new InetSocketAddress("114.239.145.90", 808));
        InputStream is = socket.getInputStream();
        OutputStream os = socket.getOutputStream();

        //1.先发送CONNECT请求,与代理服务器完成代理协议连接
        StringBuilder sb = new StringBuilder();
        sb.append("CONNECT restapi.amap.com " +
                "HTTP/1.1\r\n");
        sb.append("Host: restapi.amap.com\r\n\r\n");

        os.write(sb.toString().getBytes());
        os.flush();

        //读取代理服务器返回的结果
        BufferedReader reader = new BufferedReader(new InputStreamReader(is));
        String msg;
        while ((msg = reader.readLine()) != null) {
            if (msg.isEmpty()) {
                break;
            }
            System.out.println(msg);//代理服务器返回的结果
        }

        //2.成功后再使用ssl包装与代理服务的socket,然后发送GET请求
        SSLSocketFactory sslSocketFactory = (SSLSocketFactory) SSLSocketFactory.getDefault();
        SSLSocket sslSocket = (SSLSocket) sslSocketFactory.createSocket(
                socket, "restapi.amap.com", 443, true);

        OutputStream outputStream = sslSocket.getOutputStream();
        InputStream inputStream = sslSocket.getInputStream();

        //这个请求会被代理转发给connect协商的目标服务器
        StringBuilder request = new StringBuilder();
        request.append("GET /v3/weather/weatherInfo?city=长沙&key=13cb58f5884f9749287abbead9c658f2 " +
                "HTTP/1.1\r\n");
        request.append("Host: restapi.amap.com\r\n");
        request.append("\r\n");

        outputStream.write(request.toString().getBytes());
        outputStream.flush();

        BufferedReader br = new BufferedReader(new InputStreamReader(inputStream));
        while ((msg = br.readLine()) != null) {
            System.out.println(msg);
        }
    }

有了上面这个基础知识,接下来分析okhttp中的实现:

//Route.java

  /**
   * 所谓的隧道就是指:在HTTP代理中发送HTTPS请求
   * Returns true if this route tunnels HTTPS through an HTTP proxy. See <a
   * href="http://www.ietf.org/rfc/rfc2817.txt">RFC 2817, Section 5.2</a>.
   */
  public boolean requiresTunnel() {
    //如果使用了HTTP代理,同时本次请求是HTTPS请求
    return address.sslSocketFactory != null && proxy.type() == Proxy.Type.HTTP;
  }
//RealConnection.java

public void connect(int connectTimeout, int readTimeout, int writeTimeout,
                    int pingIntervalMillis, boolean connectionRetryEnabled, Call call,
                    EventListener eventListener) {

			...
	
            if (route.requiresTunnel()) {
                //todo http隧道代理
                connectTunnel(connectTimeout, readTimeout, writeTimeout, call, eventListener);
                if (rawSocket == null) {
                    // We were unable to connect the tunnel but properly closed down our
                    // resources.
                    break;
                }
            } else {
                connectSocket(connectTimeout, readTimeout, call, eventListener);
            }

			...


}

connect的逻辑很清晰,就是判断是否要建立隧道(即是否在HTTP代理中发送HTTPS请求),如果需要,则调用connectTunnel方法建立隧道,否则调用connectSocket建立Socket连接。而connectTunnel方法建立隧道做的工作就是先调用connectSocket建立Socket连接,然后再调用createTunnel方法发送一个CONNECT请求。

使用SOCKS代理

先补充一下SOCKS相关的知识点:

采用SOCKS协议的代理服务器就是SOCKS代理服务器,是一种通用的代理服务器。 SOCKS协议是一组由Internal工程工作小组(IETF)所开发出来的开放标准,工作在OSI模型中的第五层(会话层)。由于SOCKS工作在会话层上,因此它是一个提供会话层到会话层间安全服务的方案,不受高层应用程序变更的影响。

SOCKS协议是一种网络代理协议。该协议所描述的是一种内部主机(使用私有ip地址)通过SOCKS 服务器获得完全的Internet访问的方法。具体说来是这样一个环境:用一台运行SOCKS的服务器(双宿主主机)连接内部网和Internet,内部网主机使用的都是私有的ip地址,内部网主机请求访问Internet时,首先和SOCKS 服务器建立一个SOCKS通道,然后再将请求通过这个通道发送给SOCKS服务器,SOCKS服务器在收到客户请求后,向客户请求的Internet主机发出请求,得到响应后,SOCKS服务器再通过原先建立的SOCKS通道将数据返回给客户。当然在建立SOCKS通道的过程中可能有一个用户认证的过程。

SOCKS代理服务器和一般的应用层代理服务器完全不同。一般的应用层代理服务器工作在应用层,并且针对不用的网络应用提供不同的处理方法,比如HTTP、FTP、SMTP等,这样,一旦有新的网络应用出现时,应用层代理服务器就不能提供对该应用的代理,因此应用层代理服务器的可扩展性并不好;与应用层代理服务器不同的是,SOCKS代理服务器旨在提供一种广义的代理服务,它与具体的应用无关,不管再出现什么新的应用都能提供代理服务,因为SOCKS代理工作在会话层(即应用层和传输层之间),这和单纯工作在网络层或传输层的ip欺骗(或者叫做网络地址转换NAT)又有所不同,因为SOCKS不能提供网络层网关服务,比如ICMP包转发等。这三种技术的比较如下表所示:

类别ip欺骗(NAT)SOCKS v5应用层代理
工作区域网络层或传输层会话层应用层
用户认证
应用可扩展性
网络服务

目前的SOCKS版本是第五版,第五版同第四版的区别主要在于第五版提供多种不同的用户认证方法和UDP代理。

那SOCKS4和SOCKS5又有什么不同?SOCKS4和SOCKS5都属于SOCKS协议,只是由于所支持的协议不同而存在差异,SOCKS4只能支持TCP协议,而SOCKS5支持TCP和UDP协议。因此,SOCKS4代理只支持TCP应用,而SOCKS5代理则可以支持TCP和UDP应用。比如QQ使用的是UDP协议,所以它不能使用SOCKS4代理,而像国外的ICQ使用的是TCP协议(TCP协议比UDP协议安全),所以就可以使用SOCKS4代理。

那SOCKS代理和HTTP代理有什么不同?从上文我们知道SOCKS工作在会话层上,而HTTP工作在应用层上,SOCKS代理只是简单地传递数据包,而不必关心是何种应用协议(比如FTP、HTTP和NNTP请求),所以SOCKS代理服务器比HTTP代理服务器等应用层代理服务器要快得多。

测试使用SOCKS代理

    /**
     * 使用SOCKS代理
     * @throws IOException
     */
    public void testSocksProxy() throws IOException {
        //先启动本地主机的SOCKS代理服务器
        new Thread() {
            @Override
            public void run() {
                try {
                    SocksProxy.start();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
        }.start();

        // 这是JDK中对网络请求使用SOCKS代理的入口方法,要实现SOCKS代理,就需要传递进去一个Proxy对象给Socket
        // 这里是把本地主机作为SOCKS代理服务器
        Socket socket = new Socket(new Proxy(Proxy.Type.SOCKS, new InetSocketAddress("localhost", 808)));
        //让SOCKS代理服务器解析域名:设置了SOCKS代理就传递不解析的域名,让SOCKS代理服务器解析。(okhttp也是这么做的)
        //InetSocketAddress.createUnresolved("restapi.amap.com", 80)这行代码非常关键,创建了一个未解析(Unresolved)的SocketAddress,
        //在SOCKS协议握手阶段,InetSocketAddress信息会原封不动的发送到代理服务器,由代理服务器解析出具体的IP地址。
        socket.connect(InetSocketAddress.createUnresolved("restapi.amap.com", 80));


        InputStream is = socket.getInputStream();
        OutputStream os = socket.getOutputStream();

        StringBuilder sb = new StringBuilder();
        sb.append("GET /v3/weather/weatherInfo?city=长沙&key" +
                "=13cb58f5884f9749287abbead9c658f2 HTTP/1.1\r\n");
        sb.append("Host: restapi.amap.com\r\n");
        sb.append("\r\n");
        os.write(sb.toString().getBytes());
        os.flush();

        BufferedReader reader = new BufferedReader(new InputStreamReader(is));
        String msg;
        while ((msg = reader.readLine()) != null) {
            System.out.println(msg);
        }
    }

让SOCKS代理服务器解析域名
场景:运行HttpClient的进程所在主机可能并不能上公网,大部分时候,也无法进行DNS解析,这时通常会出现域名无法解析的IO异常,下面介绍怎么避免在客户端解析域名。
上面有一行代码非常关键:

InetSocketAddress.createUnresolved("restapi.amap.com", 80)

restapi.amap.com80是你发起http请求的目标主机和端口信息,这里创建了一个未解析(Unresolved)的SocketAddress,在SOCKS协议握手阶段,InetSocketAddress信息会原封不动的发送到代理服务器,由代理服务器解析出具体的IP地址。

Socks的协议描述中有个片段:

The SOCKS request is formed as follows:
 
     +----+-----+-------+------+----------+----------+
     |VER | CMD |  RSV  | ATYP | DST.ADDR | DST.PORT |
     +----+-----+-------+------+----------+----------+
     | 1  |  1  | X'00' |  1   | Variable |    2     |
     +----+-----+-------+------+----------+----------+
 
  Where:
 
       o  VER    protocol version: X'05'
       o  CMD
          o  CONNECT X'01'
          o  BIND X'02'
          o  UDP ASSOCIATE X'03'
       o  RSV    RESERVED
       o  ATYP   address type of following address
          o  IP V4 address: X'01'
          o  DOMAINNAME: X'03'
          o  IP V6 address: X'04'

代码按上面方法写,协议握手发送的是ATYP=X’03’,即采用域名的地址类型。否则,HttpClient会尝试在客户端解析,然后发送ATYP=X’01’进行协商。当然,大多数时候HttpClient在解析域名的时候就挂了。

DNS

DNS负责把域名解析为IP地址。

  1. DNS程序运行DNS协议,工作在应用层。
  2. DNS运行模式:客户端-服务器模式,即DNS客户端程序向DNS服务器发起查询报文,接收响应。
  3. 用户的电脑上的DNS服务器地址必须配置固定的IP地址。

更多dns相关内容参考:DNS原理

连接拦截器

ConnectInterceptor,打开与目标服务器的连接,并执行下一个拦截器。

它比较简短,代码可以直接完整贴在这里:

//ConnectInterceptor.java

/**
 * Opens a connection to the target server and proceeds to the next interceptor.
 */
public final class ConnectInterceptor implements Interceptor {
    public final OkHttpClient client;

    public ConnectInterceptor(OkHttpClient client) {
        this.client = client;
    }

    @Override
    public Response intercept(Chain chain) throws IOException {
        RealInterceptorChain realChain = (RealInterceptorChain) chain;
        Request request = realChain.request();
        StreamAllocation streamAllocation = realChain.streamAllocation();

        // We need the network to satisfy this request. Possibly for validating a conditional GET.
        boolean doExtensiveHealthChecks = !request.method().equals("GET");
        HttpCodec httpCodec = streamAllocation.newStream(client, chain, doExtensiveHealthChecks);
        RealConnection connection = streamAllocation.connection();

        return realChain.proceed(request, streamAllocation, httpCodec, connection);
    }
}

虽然代码量很少,实际上大部分功能都封装到其它类去了,这里只是调用而已。

首先我们看到的StreamAllocation这个对象是在第一个拦截器(重试及重定向拦截器)里创建的,但是真正使用的地方却在这里。

“当一个请求发出,需要建立连接,连接建立后需要使用流来读写数据”;而这个StreamAllocation就是协调请求连接数据流三者之间的关系,它负责为一次请求寻找连接,然后获得流来实现网络通信。

这里使用StreamAllocationnewStream方法实际上就是去查找或者新建一个与请求主机有效的连接,然后返回连接的HttpCodecHttpCodec中包含了输入输出流,并且封装了对HTTP请求报文的编码与解码,直接使用它就能够与请求主机完成HTTP通信。

StreamAllocation中简单来说就是维护连接RealConnection(封装了Socket与连接池ConnectionPool)。

//okhttp3.internal.connection.StreamAllocation.java

    public HttpCodec newStream(OkHttpClient client, Interceptor.Chain chain, boolean doExtensiveHealthChecks) {
        int connectTimeout = chain.connectTimeoutMillis();
        int readTimeout = chain.readTimeoutMillis();
        int writeTimeout = chain.writeTimeoutMillis();
        int pingIntervalMillis = client.pingIntervalMillis();
        boolean connectionRetryEnabled = client.retryOnConnectionFailure();

        try {
            //todo  找到一个健康的连接,即创建或复用已有的连接
            RealConnection resultConnection = findHealthyConnection(connectTimeout, readTimeout,
                    writeTimeout, pingIntervalMillis, connectionRetryEnabled,
                    doExtensiveHealthChecks);
            //todo 利用连接实例化流HttpCodec对象,如果是HTTP/2返回Http2Codec,否则返回Http1Codec
            HttpCodec resultCodec = resultConnection.newCodec(client, chain, this);

            synchronized (connectionPool) {
                codec = resultCodec;
                return resultCodec;
            }
        } catch (IOException e) {
            throw new RouteException(e);
        }
    }

    /**
     * Finds a connection and returns it if it is healthy. If it is unhealthy the process is
     * repeated until a healthy connection is found.
     * <p>
     * 寻找并返回一个健康的连接,如果没有则一直重复,直到找到这个连接
     */
    private RealConnection findHealthyConnection(int connectTimeout, int readTimeout,
                                                 int writeTimeout, int pingIntervalMillis,
                                                 boolean connectionRetryEnabled,
                                                 boolean doExtensiveHealthChecks) throws IOException {
        while (true) {
            //todo 找到一个连接
            // findConnection()方法是核心代码,真正创建或者复用链接的地方
            RealConnection candidate = findConnection(connectTimeout, readTimeout, writeTimeout,
                    pingIntervalMillis, connectionRetryEnabled);

            //todo 如果这个连接是新建立的,那肯定是健康的,直接返回
            //If this is a brand new connection, we can skip the extensive health checks.
            synchronized (connectionPool) {
                if (candidate.successCount == 0) {
                    return candidate;
                }
            }

            //todo 如果不是新创建的,需要检查是否健康
            //Do a (potentially slow) check to confirm that the pooled connection is still good.
            // If it
            // isn't, take it out of the pool and start again.
            if (!candidate.isHealthy(doExtensiveHealthChecks)) {
                //todo 不健康 关闭连接,释放Socket,从连接池移除
                // 继续下次寻找连接操作
                noNewStreams();
                continue;
            }

            return candidate;
        }
    }


    /**
     * Returns a connection to host a new stream. This prefers the existing connection if it exists,
     * then the pool, finally building a new connection.
     */
    private RealConnection findConnection(int connectTimeout, int readTimeout, int writeTimeout,
                                          int pingIntervalMillis, boolean connectionRetryEnabled) throws IOException {
        boolean foundPooledConnection = false;
        RealConnection result = null;
        Route selectedRoute = null;
        Connection releasedConnection;
        Socket toClose;
        synchronized (connectionPool) {
            if (released) throw new IllegalStateException("released");
            if (codec != null) throw new IllegalStateException("codec != null");
            if (canceled) throw new IOException("Canceled");

            // Attempt to use an already-allocated connection. We need to be careful here because
            // our already-allocated connection may have been restricted from creating new streams.
            releasedConnection = this.connection;
            toClose = releaseIfNoNewStreams();
            if (this.connection != null) {
                // We had an already-allocated connection and it's good.
                result = this.connection; //todo 步骤1.如果需要的连接是当前连接,则记录result为当前连接
                releasedConnection = null;
            }
            if (!reportedAcquired) {
                // If the connection was never reported acquired, don't report it as released!
                releasedConnection = null;
            }

            if (result == null) {
                //todo 步骤2.尝试从连接池获取连接,如果有可复用的连接,会给第三个参数 this的connection赋值
                //Attempt to get a connection from the pool.
                Internal.instance.get(connectionPool, address, this, null);
                if (connection != null) {
                    foundPooledConnection = true;
                    result = connection;
                } else {
                    selectedRoute = route;
                }
            }
        }
        closeQuietly(toClose);

        if (releasedConnection != null) {
            eventListener.connectionReleased(call, releasedConnection);
        }
        if (foundPooledConnection) {
            eventListener.connectionAcquired(call, result);
        }
        if (result != null) {//说明从当前连接(this.connection)或者连接池中(connectionPool)找到可复用的连接
            // If we found an already-allocated or pooled connection, we're done.
            return result;
        }

        // If we need a route selection, make one. This is a blocking operation.
        //todo 创建一个路由 (dns解析的所有ip与代理的组合)
        boolean newRouteSelection = false;
        if (selectedRoute == null && (routeSelection == null || !routeSelection.hasNext())) {
            newRouteSelection = true;
            routeSelection = routeSelector.next();
        }

        //todo   步骤3.配置路由后,再次尝试去连接池获取连接
        synchronized (connectionPool) {
            if (canceled) throw new IOException("Canceled");

            if (newRouteSelection) {
                // Now that we have a set of IP addresses, make another attempt at getting a
                // connection from the pool. This could match due to connection coalescing.
                //todo 根据代理和不同的ip从连接池中找可复用的连接
                List<Route> routes = routeSelection.getAll();
                for (int i = 0, size = routes.size(); i < size; i++) {
                    Route route = routes.get(i);
                    Internal.instance.get(connectionPool, address, this, route);
                    if (connection != null) {//找到可复用的连接
                        foundPooledConnection = true;
                        result = connection;
                        this.route = route;
                        break;
                    }
                }
            }
            //todo 步骤4.还是没找到,必须新建一个RealConnection连接了
            if (!foundPooledConnection) {
                if (selectedRoute == null) {
                    selectedRoute = routeSelection.next();
                }

                // Create a connection and assign it to this allocation immediately. This makes
                // it possible
                // for an asynchronous cancel() to interrupt the handshake we're about to do.
                route = selectedRoute;
                refusedStreamCount = 0;
                result = new RealConnection(connectionPool, selectedRoute);
                acquire(result, false);
            }
        }

        // If we found a pooled connection on the 2nd time around, we're done.
        if (foundPooledConnection) {//第2轮从连接池中寻找到了可复用的连接,返回该连接
            eventListener.connectionAcquired(call, result);
            return result;
        }


        // Do TCP + TLS handshakes. This is a blocking operation.
        //todo 走到这里说明第2轮从连接池中寻找可复用连接时没有找到
        // 则对于步骤4新建的RealConnection连接执行connect方法
        // connect方法实际上就是创建socket连接,但是要注意的是如果存在http代理的情况
        result.connect(connectTimeout, readTimeout, writeTimeout, pingIntervalMillis,
                connectionRetryEnabled, call, eventListener);
        routeDatabase().connected(result.route());

        Socket socket = null;
        synchronized (connectionPool) {
            reportedAcquired = true;

            // Pool the connection.
            //todo 将新创建的连接放到连接池中
            Internal.instance.put(connectionPool, result);

            // If another multiplexed connection to the same address was created concurrently, then
            // release this connection and acquire that one.
            if (result.isMultiplexed()) {//对于HTTP/2连接,一条连接(Http2Connection)可以同时执行多个HTTP请求
                socket = Internal.instance.deduplicate(connectionPool, address, this);
                result = connection;
            }
        }
        closeQuietly(socket);

        eventListener.connectionAcquired(call, result);
        return result;
    }

总结一下寻找可复用连接的思路:
1.如果当前streamAllocation中的之前已经分配的连接就是需要的连接,则直接使用该连接
2.如果不是,则从连接池中查找(遍历)是否有可复用的连接,有则直接使用该连接
3.如果连接池中没有,则配置路由,配置后再次从连接池中查找是否有可复用连接,有则直接使用该连接
4.如果连接池中还是没找到可复用的连接,则新建一个连接,使用该连接,并将其放入连接池中
5.对于HTTP/2协议,做些特殊判断,这里先不具体讨论。

获取可复用连接的整体流程图:
在这里插入图片描述

连接池中如何获取可复用的连接?

ConnectionPool的get方法会循环遍历连接池中的所有连接进行判断连接是否可用:

//okhttp3.ConnectionPool.java


    /**
     * todo 最多保存 5个处于空闲状态的连接(idle connections),空闲连接的默认保活时间为 5分钟,5分钟内如果一直没有活动则被移出连接池
     * Create a new connection pool with tuning parameters appropriate for a single-user application.
     * The tuning parameters in this pool are subject to change in future OkHttp releases. Currently
     * this pool holds up to 5 idle connections which will be evicted after 5 minutes of inactivity.
     */
    public ConnectionPool() {
        this(5, 5, TimeUnit.MINUTES);
    }

    /**
     * todo 获取可复用的连接
     * Returns a recycled connection to {@code address}, or null if no such connection exists. The
     * route is null if the address has not yet been routed.
     */
    @Nullable
    RealConnection get(Address address, StreamAllocation streamAllocation, Route route) {
        assert (Thread.holdsLock(this));
        for (RealConnection connection : connections) {
            //todo 要拿到的连接与连接池中的连接的配置(dns/代理/域名等等)全都一致,就可以复用
            if (connection.isEligible(address, route)) {
                streamAllocation.acquire(connection, true); // 在使用了,所以 acquire 会创建弱引用放入集合记录
                return connection;
            }
        }
        return null;
    }

    /**
     * todo 对http2而言,多路复用去重(所有同一地址的请求都应该共享同一个TCP连接) 先不管
     * Replaces the connection held by {@code streamAllocation} with a shared connection if possible.
     * This recovers when multiple multiplexed connections are created concurrently.
     */
    @Nullable
    Socket deduplicate(Address address, StreamAllocation streamAllocation) {
        assert (Thread.holdsLock(this));
        for (RealConnection connection : connections) {
            if (connection.isEligible(address, null)
                    && connection.isMultiplexed()
                    && connection != streamAllocation.connection()) {
                return streamAllocation.releaseAndAcquire(connection);
            }
        }
        return null;
    }

    /**
     * todo 保存连接以复用。
     * 本方法没上锁,只加了断言: 当前线程拥有this(pool)对象的锁。
     * 表示使用这个方法必须要上锁,而且是上pool的对象锁。
     * okhttp中使用到这个函数的地方确实也是这么做的
     */
    void put(RealConnection connection) {
        assert (Thread.holdsLock(this));
        //todo 如果清理任务未执行就启动它,再把新连接加入队列
        if (!cleanupRunning) {
            cleanupRunning = true;
            executor.execute(cleanupRunnable);
        }
        connections.add(connection);
    }

    /**
     * todo 连接用完了,重新变为闲置
     * Notify this pool that {@code connection} has become idle. Returns true if the connection has
     * been removed from the pool and should be closed.
     */
    boolean connectionBecameIdle(RealConnection connection) {
        assert (Thread.holdsLock(this));
        //todo 比如 服务器返回 Connection: close ,那就会把这个连接关掉 (noNewStreams 设置为true)
        if (connection.noNewStreams || maxIdleConnections == 0) {
            connections.remove(connection);
            return true;
        } else {
            //todo 唤醒wait的清理任务 开始工作
            notifyAll(); // Awake the cleanup thread: we may have exceeded the idle connection limit.
            return false;
        }
    }

会根据RealConnection的isEligible()方法判断该连接是否可复用:

//okhttp3.internal.connection.RealConnection.java

public boolean isEligible(Address address, @Nullable Route route) {
    // If this connection is not accepting new streams, we're done.
    if (allocations.size() >= allocationLimit || noNewStreams) return false;

    // If the non-host fields of the address don't overlap, we're done.
    if (!Internal.instance.equalsNonHost(this.route.address(), address)) return false;

    // If the host exactly matches, we're done: this connection can carry the address.
    if (address.url().host().equals(this.route().address().url().host())) {
      return true; // This connection is a perfect match.
    }

    // At this point we don't have a hostname match. But we still be able to carry the request if
    // our connection coalescing requirements are met. See also:
    // https://hpbn.co/optimizing-application-delivery/#eliminate-domain-sharding
    // https://daniel.haxx.se/blog/2016/08/18/http2-connection-coalescing/

    // 1. This connection must be HTTP/2.
    if (http2Connection == null) return false;

    // 2. The routes must share an IP address. This requires us to have a DNS address for both
    // hosts, which only happens after route planning. We can't coalesce connections that use a
    // proxy, since proxies don't tell us the origin server's IP address.
    if (route == null) return false;
    if (route.proxy().type() != Proxy.Type.DIRECT) return false;
    if (this.route.proxy().type() != Proxy.Type.DIRECT) return false;
    if (!this.route.socketAddress().equals(route.socketAddress())) return false;

    // 3. This connection's server certificate's must cover the new host.
    if (route.address().hostnameVerifier() != OkHostnameVerifier.INSTANCE) return false;
    if (!supportsUrl(address.url())) return false;

    // 4. Certificate pinning must match the host.
    try {
      address.certificatePinner().check(address.url().host(), handshake().peerCertificates());
    } catch (SSLPeerUnverifiedException e) {
      return false;
    }

    return true; // The caller's address can be carried by this connection.
  }

需要满足的条件:
1、if (allocations.size() >= allocationLimit || noNewStreams) return false;

​ 连接到达最大并发流或者连接不允许建立新的流;如http1.x正在使用的连接不能给其他人用(最大并发流为:1)或者连接被关闭;那就不允许复用;

2、

if (!Internal.instance.equalsNonHost(this.route.address(), address)) return false;
if (address.url().host().equals(this.route().address().url().host())) {
      return true; // This connection is a perfect match.
}

DNS、代理、SSL证书、服务器域名、端口完全相同则可复用;

如果上述条件都不满足,在HTTP/2的某些场景下可能仍可以复用(http2先不管)。

所以综上,如果在连接池中找到一个连接参数一致并且未被关闭、没被占用的连接,则该连接可以复用。

连接池清理任务

在这里插入图片描述

//okhttp3.ConnectionPool.java

    //清理空闲连接的清理任务,cleanupRunnable什么时候开始执行?执行put方法新放入连接的时候
    private final Runnable cleanupRunnable = new Runnable() {
        @Override
        public void run() {
            while (true) {
                //todo waitNanos表示等待多久后需要再次清理
                long waitNanos = cleanup(System.nanoTime());
                if (waitNanos == -1) return;
                if (waitNanos > 0) {
                    //todo 因为等待是纳秒级,wait方法可以接收纳秒级控制,但是要把毫秒与纳秒分开
                    long waitMillis = waitNanos / 1000000L;
                    waitNanos -= (waitMillis * 1000000L);
                    synchronized (ConnectionPool.this) {
                        try {
                            //todo 参数多传递一个纳秒参数waitNanos,控制更加精准
                            ConnectionPool.this.wait(waitMillis, (int) waitNanos);
                        } catch (InterruptedException ignored) {
                        }
                    }
                }
            }
        }
    };


 	/**
     * Performs maintenance on this pool, evicting the connection that has been idle the longest if
     * either it has exceeded the keep alive limit or the idle connections limit.
     *
     * <p>Returns the duration in nanos to sleep until the next scheduled call to this method. Returns
     * -1 if no further cleanups are required.
     */
    long cleanup(long now) {
        int inUseConnectionCount = 0;
        int idleConnectionCount = 0;
        RealConnection longestIdleConnection = null; //连接池中的闲置了最久的连接
        long longestIdleDurationNs = Long.MIN_VALUE; //连接池中的闲置连接的最长闲置时间

        // Find either a connection to evict, or the time that the next eviction is due.
        synchronized (this) {
            //遍历连接池
            for (Iterator<RealConnection> i = connections.iterator(); i.hasNext(); ) {
                RealConnection connection = i.next();

                // If the connection is in use, keep searching.
              // todo 检查连接是否正在被使用
                if (pruneAndGetAllocationCount(connection, now) > 0) {
                    inUseConnectionCount++;
                    continue;
                }

                //todo 否则记录闲置连接数
                idleConnectionCount++;

                // If the connection is ready to be evicted, we're done.
                //TODO 获得这个连接已经闲置多久
                // 执行完遍历,获得闲置了最久的连接以及最长闲置时间
                long idleDurationNs = now - connection.idleAtNanos;
                if (idleDurationNs > longestIdleDurationNs) {
                    longestIdleDurationNs = idleDurationNs;
                    longestIdleConnection = connection;
                }
            }

            //todo 最长闲置时间超过了保活时间(5分钟) 或者池内的连接数量超过了最大闲置连接数量(5个)
            // 马上移除这个闲置了最久的连接,然后返回0,0表示不等待,马上再次执行清理任务
            if (longestIdleDurationNs >= this.keepAliveDurationNs
                    || idleConnectionCount > this.maxIdleConnections) {
                // We've found a connection to evict. Remove it from the list, then close it below (outside
                // of the synchronized block).
                connections.remove(longestIdleConnection);
            } else if (idleConnectionCount > 0) {
                // A connection will be ready to evict soon.
                //todo 池内存在闲置连接,那就等待 保活时间(5分钟)-最长闲置时间 =还能闲置多久 后再次检查
                return keepAliveDurationNs - longestIdleDurationNs;
            } else if (inUseConnectionCount > 0) {
                // All connections are in use. It'll be at least the keep alive duration 'til we run again.
                //todo 池内所有的连接都在使用中,就等 keepAliveDurationNs(5分钟) 后再次检查
                return keepAliveDurationNs;
            } else {
                // No connections, idle or in use.
                //todo 都不满足,即池内没有任何连接,直接停止清理任务(put后会再次启动清理任务)
                cleanupRunning = false;
                return -1;
            }
        }

        closeQuietly(longestIdleConnection.socket());//移除闲置连接的同时关闭这个闲置连接的socket

        // Cleanup again immediately.
        return 0;
    }

    /**
     * Prunes any leaked allocations and then returns the number of remaining live allocations on
     * {@code connection}. Allocations are leaked if the connection is tracking them but the
     * application code has abandoned them. Leak detection is imprecise and relies on garbage
     * collection.
     */
    private int pruneAndGetAllocationCount(RealConnection connection, long now) {
        //todo 这个连接被使用就会创建一个弱引用放入集合,这个集合不为空就表示这个连接正在被使用
        // 实际上 http1.x 上也只能有一个正在使用的。
        List<Reference<StreamAllocation>> references = connection.allocations;
        for (int i = 0; i < references.size(); ) {
            Reference<StreamAllocation> reference = references.get(i);

            if (reference.get() != null) {
                i++;
                continue;
            }

            // We've discovered a leaked allocation. This is an application bug.
            StreamAllocation.StreamAllocationReference streamAllocRef =
                    (StreamAllocation.StreamAllocationReference) reference;
            String message = "A connection to " + connection.route().address().url()
                    + " was leaked. Did you forget to close a response body?";
            Platform.get().logCloseableLeak(message, streamAllocRef.callStackTrace);

            references.remove(i);
            connection.noNewStreams = true;

            // If this was the last allocation, the connection is eligible for immediate eviction.
            if (references.isEmpty()) {
                connection.idleAtNanos = now - keepAliveDurationNs;
                return 0;
            }
        }

        return references.size();
    }

代理与DNS

无代理、HTTP代理、SOCKS代理三种情况下发送的请求报文(请求报文是http协议规定的特定格式)是有区别的:
在这里插入图片描述

在使用OkHttp时,如果用户在创建OkHttpClient时,配置了proxy或者proxySelector,则会使用配置的代理,并且proxy优先级高于proxySelector。而如果未配置,则会获取机器配置的代理并使用。

    /**
     * 测试JDK的ProxySelector的使用
     * @throws IOException
     */
    public void testProxySelector() throws IOException {
        try {
            URI uri = new URI("http://restapi.amap.com");
            List<Proxy> proxyList = ProxySelector.getDefault().select(uri);
            System.out.println("proxy.address=" + proxyList.get(0).address());//proxy.address=null
            System.out.println("proxy.type=" + proxyList.get(0).type());//proxy.type=DIRECT
        } catch (URISyntaxException e) {
            e.printStackTrace();
        }
    }

因此,如果我们不需要自己的App中的请求走代理,则可以配置一个proxy(Proxy.NO_PROXY),这样也可以避免被抓包。NO_PROXY的定义如下:

//Proxy.java

public class Proxy {

	...
	
    /**
     * A proxy setting that represents a {@code DIRECT} connection,
     * basically telling the protocol handler not to use any proxying.
     * Used, for instance, to create sockets bypassing any other global
     * proxy settings (like SOCKS):
     * <P>
     * {@code Socket s = new Socket(Proxy.NO_PROXY);}
     *
     */
    public final static Proxy NO_PROXY = new Proxy();

    // Creates the proxy that represents a {@code DIRECT} connection.
    private Proxy() {
        type = Type.DIRECT;
        sa = null;
    }
	
	...

}

代理在Java中对应的抽象类有三种类型:

//Proxy.java


/**
 * This class represents a proxy setting, typically a type (http, socks) and
 * a socket address.
 * A {@code Proxy} is an immutable object.
 *
 * @see     java.net.ProxySelector
 * @author Yingxian Wang
 * @author Jean-Christophe Collet
 * @since   1.5
 */
public class Proxy {

    /**
     * Represents the proxy type.
     *
     * @since 1.5
     */
    public enum Type {
        /**
         * Represents a direct connection, or the absence of a proxy.
         */
        DIRECT,
        /**
         * Represents proxy for high level protocols such as HTTP or FTP.
         */
        HTTP,
        /**
         * Represents a SOCKS (V4 or V5) proxy.
         */
        SOCKS
    };

	...

}

DIRECT:无代理,HTTP:HTTP代理,SOCKS:SOCKS代理。

第一种DIRECT自然不用多说,就是普通的http请求,而HTTP代理与SOCKS代理有什么区别?

对于SOCKS代理,在HTTP的场景下,代理服务器完成TCP数据包的转发工作;
而对于HTTP代理服务器,在转发数据之外,还会解析HTTP的请求及响应,并根据请求及响应的内容做一些处理。

RealConnectionconnectSocket方法:

//okhttp3.internal.connection.RealConnection.java

    /**
     * Does all the work necessary to build a full HTTP or HTTPS connection on a raw socket.
     */
    private void connectSocket(int connectTimeout, int readTimeout, Call call,
                               EventListener eventListener) throws IOException {
    	...                          
	
		//如果是Socks代理则 new Socket(proxy); 否则相当于直接:new Socket()
		rawSocket = proxy.type() == Proxy.Type.DIRECT || proxy.type() == Proxy.Type.HTTP
		                ? address.socketFactory().createSocket()
		                : new Socket(proxy);
	
		...
	
	 	Platform.get().connectSocket(rawSocket, route.socketAddress(), connectTimeout);
		
		...
		
	}

//okhttp3.internal.platform.Platform.java

    public void connectSocket(Socket socket, InetSocketAddress address,
                              int connectTimeout) throws IOException {
        socket.connect(address, connectTimeout);//connect方法
    }

设置了SOCKS代理的情况下,创建Socket时,为其传入proxy,与SOCKS代理服务器建立连接;如果设置的是HTTP代理,创建Socket时是与HTTP代理服务器建立连接。

new Socket(proxy)与new Socket()的区别? new Socket(proxy)做了哪些工作,传递的参数proxy如何使用?
答:

connect方法中传递的address来自于下面的集合:RouteSelector中的inetSocketAddressesRouteSelectorresetNextInetSocketAddress方法用于生成inetSocketAddresses

//okhttp3.internal.connection.RouteSelector.java

    /**
     * Prepares the socket addresses to attempt for the current proxy or host.
     */
    private void resetNextInetSocketAddress(Proxy proxy) throws IOException {
        // Clear the addresses. Necessary if getAllByName() below throws!
        inetSocketAddresses = new ArrayList<>();

        String socketHost;
        int socketPort;
        if (proxy.type() == Proxy.Type.DIRECT || proxy.type() == Proxy.Type.SOCKS) {
            //若无代理或者使用SOCKS代理,则使用http服务器的域名与端口
            socketHost = address.url().host();
            socketPort = address.url().port();
        } else {
            //若使用HTTP代理,则使用HTTP代理服务器的域名和端口
            SocketAddress proxyAddress = proxy.address();
            if (!(proxyAddress instanceof InetSocketAddress)) {
                throw new IllegalArgumentException(
                        "Proxy.address() is not an " + "InetSocketAddress: " + proxyAddress.getClass());
            }
            InetSocketAddress proxySocketAddress = (InetSocketAddress) proxyAddress;
            socketHost = getHostString(proxySocketAddress);
            socketPort = proxySocketAddress.getPort();
        }

        if (socketPort < 1 || socketPort > 65535) {
            throw new SocketException("No route to " + socketHost + ":" + socketPort
                    + "; port is out of range");
        }

        if (proxy.type() == Proxy.Type.SOCKS) {//若使用SOCKS代理
            //若使用SOCKS代理,dns没用到,由SOCKS代理服务器解析域名
            //注意InetSocketAddress.createUnresolved()方法:根据主机名和端口号创建未解析的套接字地址。不会尝试将主机名解析为 InetAddress。
            inetSocketAddresses.add(InetSocketAddress.createUnresolved(socketHost, socketPort));
        } else {//若无代理或者使用HTTP代理
            eventListener.dnsStart(call, socketHost);

            //若无代理,则使用dns解析http服务器
            //若使用HTTP代理,则使用dns解析HTTP代理服务器
            // Try each address for best behavior in mixed IPv4/IPv6 environments.
            List<InetAddress> addresses = address.dns().lookup(socketHost);
            if (addresses.isEmpty()) {
                throw new UnknownHostException(address.dns() + " returned no addresses for " + socketHost);
            }

            eventListener.dnsEnd(call, socketHost, addresses);

            for (int i = 0, size = addresses.size(); i < size; i++) {
                InetAddress inetAddress = addresses.get(i);
                inetSocketAddresses.add(new InetSocketAddress(inetAddress, socketPort));
            }
        }
    }

从代码中可以总结:

  1. 如果设置了SOCKS代理时,不使用配置的dns,Http服务器的域名解析会被交给SOCKS代理服务器执行。
  2. 但是如果是设置了HTTP代理,使用OkhttpClient配置的dns解析HTTP代理服务器的域名,Http服务器的域名解析被交给HTTP代理服务器解析。
  3. 如果无代理,使用OkhttpClient配置的dns解析Http服务器的域名。

当然,如果没有配置dns,则okhttp默认使用系统的DNS解析方式:

//okhttp3.OkHttpClient.java

      dns = Dns.SYSTEM;
      
//okhttp3.Dns.java

public interface Dns {
    /**
     * A DNS that uses {@link InetAddress#getAllByName} to ask the underlying operating system to
     * lookup IP addresses. Most custom {@link Dns} implementations should delegate to this instance.
     */
    Dns SYSTEM = new Dns() {
        @Override
        public List<InetAddress> lookup(String hostname) throws UnknownHostException {
            if (hostname == null) throw new UnknownHostException("hostname == null");
            try {
                return Arrays.asList(InetAddress.getAllByName(hostname));
            } catch (NullPointerException e) {
                UnknownHostException unknownHostException =
                        new UnknownHostException("Broken system behaviour for dns lookup of " + hostname);
                unknownHostException.initCause(e);
                throw unknownHostException;
            }
        }
    };

    /**
     * Returns the IP addresses of {@code hostname}, in the order they will be attempted by OkHttp. If
     * a connection to an address fails, OkHttp will retry the connection with the next address until
     * either a connection is made, the set of IP addresses is exhausted, or a limit is exceeded.
     */
    List<InetAddress> lookup(String hostname) throws UnknownHostException;
}

可以看到,Dns.SYSTEM使用的就是JDK中的域名解析的方法:

//java.net.InetAddress.java

public class InetAddress implements java.io.Serializable {

	...
	//getByName与getAllByName的区别:getByName返回地址列表的第一个地址;
    public static InetAddress getByName(String host)
        throws UnknownHostException {
        // Android-changed: Rewritten on the top of Libcore.os.
        return impl.lookupAllHostAddr(host, NETID_UNSET)[0];
    }

	public static InetAddress[] getAllByName(String host)
        throws UnknownHostException {
        // Android-changed: Resolves a hostname using Libcore.os.
        // Also, returns both the Inet4 and Inet6 loopback for null/empty host
        return impl.lookupAllHostAddr(host, NETID_UNSET).clone();
    }

	...


}

impl.lookupAllHostAddr()方法走的就是传统的DNS解析方式(即通过DNS协议发送给本地域名服务器,再发送给互联网运营商的DNS服务器)。

上述代码就是代理与DNS在OkHttp中的使用,但是还有一点需要注意,Http代理也分成两种类型:普通代理与隧道代理。

其中普通代理不需要额外的操作,扮演「中间人」的角色,在两端之间来回传递报文。这个“中间人”在收到客户端发送的请求报文时,需要正确的处理请求和连接状态,同时向服务器发送新的请求,在收到响应后,将响应结果包装成一个响应体返回给客户端。在普通代理的流程中,代理两端都是有可能察觉不到"中间人“的存在。

但是隧道代理不再作为中间人,无法改写客户端的请求,而仅仅是在建立连接后,将客户端的请求,通过建立好的隧道,无脑的转发给终端服务器。隧道代理需要发起Http CONNECT请求,这种请求方式没有请求体,仅供代理服务器使用,并不会传递给终端服务器。请求头部分一旦结束,后面的所有数据,都被视为应该转发给终端服务器的数据,代理需要把他们无脑的直接转发,直到从客户端的 TCP 读通道关闭。CONNECT 的响应报文,在代理服务器和终端服务器建立连接后,可以向客户端返回一个 200 Connect established 的状态码,以此表示和终端服务器的连接,建立成功。

RealConnection的connect方法:

//RealConnection.java

    public void connect(int connectTimeout, int readTimeout, int writeTimeout,
                        int pingIntervalMillis, boolean connectionRetryEnabled, Call call,
                        EventListener eventListener) {

					...
					
                if (route.requiresTunnel()) {
                    //todo http隧道代理
                    connectTunnel(connectTimeout, readTimeout, writeTimeout, call, eventListener);
                    if (rawSocket == null) {
                        // We were unable to connect the tunnel but properly closed down our
                        // resources.
                        break;
                    }
                } else {
                    connectSocket(connectTimeout, readTimeout, call, eventListener);
                }

				...



	}

requiresTunnel()方法的判定逻辑为:当前请求为HTTPS请求并且设置了HTTP代理

//Route.java

  /**
   * 所谓的隧道就是指:在HTTP代理中发送HTTPS请求
   * Returns true if this route tunnels HTTPS through an HTTP proxy. See <a
   * href="http://www.ietf.org/rfc/rfc2817.txt">RFC 2817, Section 5.2</a>.
   */
  public boolean requiresTunnel() {
    //如果使用了HTTP代理,同时本次请求是HTTPS请求
    return address.sslSocketFactory != null && proxy.type() == Proxy.Type.HTTP;
  }

这时候connectTunnel()中会发起:

CONNECT xxxx HTTP/1.1
Host: xxxx
Proxy-Connection: Keep-Alive
User-Agent: okhttp/${version}

的CONNECT请求,连接成功后代理服务器会返回200;如果代理服务器返回407表示代理服务器需要鉴权(如:付费代理),这时需要在请求头中加入Proxy-Authorization

 Authenticator authenticator = new Authenticator() {
        @Nullable
        @Override
        public Request authenticate(Route route, Response response) throws IOException {
          if(response.code == 407){
            //代理鉴权
            String credential = Credentials.basic("代理服务用户名", "代理服务密码");
            return response.request().newBuilder()
                    .header("Proxy-Authorization", credential)
                    .build();
          }
          return null;
        }
      };
new OkHttpClient.Builder().proxyAuthenticator(authenticator);

连接拦截器总结

这个拦截器中的所有实现都是为了获得一个与目标服务器的连接,在这个连接上进行HTTP数据的收发。

OkHttp接入HttpDNS

为什么要使用HttpDNS

一个DNS查询,会先从本地DNS缓存查找,如果没有缓存或者缓存已经过期,就从DNS服务器查询,如果客户端没有主动设置DNS服务器,一般是从网络服务运营商(ISP)的DNS服务器上查找。这就出现了不可控。因为如果使用了ISP的LocalDNS域名服务器,那么基本都会或多或少地无法避免地遭遇到各种域名被缓存、用户跨网访问缓慢、域名劫持等问题。

使用普通域名服务会有什么问题:
1. 域名劫持
一些小服务商以及小地方的服务商非常喜欢干这个事情。根据腾讯给出的数据,DNS劫持率7%,恶意劫持率2%。网速给的劫持率是10-15%。

  • 把你的域名解析到竞争对手那里,然后哭死都不知道,为什么流量下降了。
  • 在你的代码当中,插入广告或者追踪代码。这就是为什么在淘宝或者百度搜索一下东西,很快就有人联系你。
  • 下载APK文件的时候,替换你的文件,下载一个其他应用或者山寨应用。
  • 打开一个页面,先跳转到广告联盟,然后跳转到这个页面。无缘无故多花广告钱,以及对运营的误导。

2. 智能DNS策略失效
智能DNS,就是为了调度用户访问策略。但是这些因素会导致智能DNS策略失效:

  • 小运营商,没有DNS服务器,直接调用别的服务商,导致服务商识别错误,直接跨网传输,速度大大下降。
  • 服务商多长NAT,实际IP,获得不了,结果没有就近访问。
  • 一些运营商将IP设置到开卡地,即使漫游到其他地方,结果也是没有就近访问。

目前国内大多数企业对于域名解析这块问题没有进行特殊处理,这导致了上述说的那些问题,其中域名劫持的问题相当普遍。那么有没有一种方法能够避免上述的情况呢?有,当然有。那就是使用HttpDNS。

什么是HttpDNS

HttpDNS其实是对DNS解析的另一种实现方式,只是将域名解析的协议由DNS协议换成了Http协议,并不复杂。使用HTTP协议向DNS服务器的80端口进行请求,代替传统的DNS协议向DNS服务器的53端口进行请求,绕开了运营商的Local DNS,从而避免了使用网络运营商的Local DNS造成的域名劫持和跨网访问等问题。

接入HttpDNS也是很简单的,使用普通DNS时,客户端发送网络请求时,就直接发送出去了,由底层网络框架进行域名解析。当接入HttpDNS时,就需要自己发送域名解析的HTTP请求,当客户端拿到域名对应的IP之后,就直接往此IP发送连接请求。

这样,就再也不用考虑传统DNS解析会带来的那些问题了,因为是使用专门的HttpDNS服务器,所以不用担心网络运营商的Local DNS的域名劫持问题了;而且,如果选择好的DNS服务器提供商,还保证将用户引导到访问速度最快的IDC节点上。

接入HttpDNS之前

在接入时需要考虑一个问题:HttpDNS服务器用哪家的呢?

选择HttpDNS服务商

目前,比较出名的HttpDNS服务提供商有两家(腾讯和阿里):

阿里云 HTTPDNS
腾讯云 移动解析HTTPDNS

DNSPOD | D+

选择接入SDK

一般来说服务提供商会向所有客户端提供相应的SDK以方便使用。

如果没提供的话可以选择开源的第三方SDK:
新浪-安卓版:https://github.com/CNSRE/HTTPDNSLib

接入HttpDNS

参考HttpDNS服务商提供的SDK或者开源的第三方SDK。

项目中做http相关需求时遇到的问题

1、HTTP请求有时会失败,是概率性的。

现象:请求图片,读取响应时报异常。所以获取的是缓存中原来保存的图片。
原因:这属于修改框架代码时引起的bug。
项目中当时用的是HttpResponseCache这个HTTP缓存框架,对其中的一些代码进行了定制,改代码时,对于有些输入流提早close掉了,框架的作者也做了注释,要等响应完成后在调用端进行手动close,提早关闭输入流导致读取响应时失败,修改回框架原来的流程即可。

2、设置了HTTP代理后不能发送https请求,可以发送http请求

现象:设置了HTTP代理了后,不能发送https请求,抓包抓不到https请求,http请求正常。
原因:对于https请求,正确的做法是:新建一个socket发送connect请求,服务器返回成功后利用ssl包裹原来的这个socket从而生成一个新的sslSocket,然后利用这个sslSocket发送后续的GET请求。
框架的做法是:新建一个socket发送connect请求,服务器返回成功后又新建了一个socket(而不是利用原来的那个socket),然后用ssl包裹这个socket生成一个新的sslSocket,利用这个sslSocket发送后续的GET请求,这会导致GET请求发送失败。

3、客户端无法更新图片

现象:后台更新了图片后客户端没有更新图片
原因:请求图片时,后台返回的响应的Cache-Control: max-age=[秒]是一年(一般图片的max-age都设置的很长,比如设置为一年,三年),结果几天后后台更新了新的图片,但是给客户端的图片url没有更改,所以客户端没反应,发送了请求后显示的还是缓存中的旧图片。
更改图片的同时更新给客户端的图片url即可解决。

参考:
https://docs.oracle.com/javase/8/docs/technotes/guides/net/proxies.html
SOCKS Protocol Version 5: https://www.ietf.org/rfc/rfc1928.txt

给HttpClient添加Socks代理
什么是HTTP隧道,怎么理解HTTP隧道呢?

Android 网络优化,使用 HTTPDNS 优化 DNS,从原理到 OkHttp 集成
OkHttp接入HttpDNS,最佳实践
Android OkHttp实现HttpDns的最佳实践(非拦截器)
HttpDNS介绍,HttpDNS原理详解
新浪:反 DNS 劫持高效实战 & HttpDNS Lib 库解析

DNS智能策略解析

  • 3
    点赞
  • 1
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值