2、HttpClient源码解析之连接的获取

从1996年开始,很多HTTP/1.0浏览器与服务器都对协议进行了扩展,那就是“keep-alive”扩展协议。 注意,这个扩展协议是作为1.0的补充的“实验型持久连接”出现的。keep-alive已经不再使用了,最新的HTTP/1.1规范中也没有对它进行说明,只是很多应用延续了下来。 使用HTTP/1.0的客户端在首部中加上"Connection:Keep-Alive",请求服务端将一条连接保持在打开状态。服务端如果愿意将这条连接保持在打开状态,就会在响应中包含同样的首部。如果响应中没有包含"Connection:Keep-Alive"首部,则客户端会认为服务端不支持keep-alive,会在发送完响应报文之后关闭掉当前连接。

通过keep-alive补充协议,客户端与服务器之间完成了持久连接,然而仍然存在着一些问题:

  • 在HTTP/1.0中keep-alive不是标准协议,客户端必须发送Connection:Keep-Alive来激活keep-alive连接。
  • 代理服务器可能无法支持keep-alive,因为一些代理是"盲中继",无法理解首部的含义,只是将首部逐跳转发。所以可能造成客户端与服务端都保持了连接,但是代理不接受该连接上的数据。

HTTP/1.1采取持久连接的方式替代了Keep-Alive。

HTTP/1.1的连接默认情况下都是持久连接。如果要显式关闭,需要在报文中加上Connection:Close首部。即在HTTP/1.1中,所有的连接都进行了复用。 然而如同Keep-Alive一样,空闲的持久连接也可以随时被客户端与服务端关闭。不发送Connection:Close不意味着服务器承诺连接永远保持打开。

请求执行处理器「handler」为ConnectExec。

1、ConnectExec

  1. 通过InternalExecRuntime获取 InternalConnectionEndpoint,赋予属性 endpointRef。
  2. InternalConnectionEndpoint持有 PoolEntry 实例。
  3. PoolEntry 实例属性 connRef 持有 DefaultManagedHttpClientConnection 实例。【#3.2】
  4. 创建Socket实例,并且与 DefaultManagedHttpClientConnection连接绑定【#6.2 socketHolderRef】。
public ClassicHttpResponse execute(ClassicHttpRequest request,ExecChain.Scope scope,ExecChain chain){
   	...
   	// ExecRuntime为 InternalExecRuntime#acquireEndpoint #2
     execRuntime.acquireEndpoint(exchangeId, route, userToken, context);
    do {
        switch (step) {
        	// 从 Endpoint 获取PoolEntry,从PoolEntry中通过HttpRoute获取到服务端信息.
			// 将Socket 与 DefaultManagedHttpClientConnection连接绑定
            case HttpRouteDirector.CONNECT_TARGET:
            	// 绑定SocketHolder、Socket。
            	//Socket 与 DefaultManagedHttpClientConnection
                execRuntime.connectEndpoint(context);
                tracker.connectTarget(route.isSecure());
                break;
            ...
        }

    } while (step > HttpRouteDirector.COMPLETE);
    //获取服务端响应
    return chain.proceed(request, scope);
}

2、InternalExecRuntime

class InternalExecRuntime implements ExecRuntime, Cancellable {

	private final HttpClientConnectionManager manager;
	private final AtomicReference<ConnectionEndpoint> endpointRef;
	
	@Override
	public void acquireEndpoint(String id, HttpRoute route,Object object, HttpClientContext context) {
	    ...
       // #3.2
       final LeaseRequest connRequest = manager.lease(id, route, connectionRequestTimeout, object);
       ...
       // 创建的Endpoint为 PoolingHttpClientConnectionManager的内部类InternalConnectionEndpoint
       final ConnectionEndpoint connectionEndpoint = connRequest.get(connectionRequestTimeout);
       endpointRef.set(connectionEndpoint);
       ...
	}

	@Override
    public ClassicHttpResponse execute(String id,ClassicHttpRequest request,HttpClientContext context){
        final ConnectionEndpoint endpoint = ensureValid();
        if (!endpoint.isConnected()) {
            connectEndpoint(endpoint, context);
        }
        final RequestConfig requestConfig = context.getRequestConfig();
        final Timeout responseTimeout = requestConfig.getResponseTimeout();
        if (responseTimeout != null) {
            endpoint.setSocketTimeout(responseTimeout);
        }
		// PoolingHttpClientConnectionManager内部类 InternalConnectionEndpoint
		// 参考 https://blog.csdn.net/qq_36851469/article/details/125575198 执行流程
        return endpoint.execute(id, request, requestExecutor, context);
    }
	
	private void connectEndpoint(ConnectionEndpoint endpoint,HttpClientContext context) {
        final RequestConfig requestConfig = context.getRequestConfig();
        final Timeout connectTimeout = requestConfig.getConnectTimeout();
        //manager:PoolingHttpClientConnectionManager
        manager.connect(endpoint, connectTimeout, context);
    }
}

3、PoolingHttpClientConnectionManager

  1. 连接池管理器的作用是创建连接池类型,默认为StrictConnPool
  2. 通过StrictConnPool租借连接。

3.1、连接池管理器的初始化

public class PoolingHttpClientConnectionManager implements HttpClientConnectionManager, ConnPoolControl<HttpRoute> {
    
    private final ManagedConnPool<HttpRoute, ManagedHttpClientConnection> pool;
    private final HttpClientConnectionOperator connectionOperator;
	
	// 默认的连接池最大并发连接数
	public static final int DEFAULT_MAX_TOTAL_CONNECTIONS = 25;
	// 默认每个域名分配的最大并发连接数
    public static final int DEFAULT_MAX_CONNECTIONS_PER_ROUTE = 5;
  
	protected PoolingHttpClientConnectionManager(HttpClientConnectionOperator 
		httpClientConnectionOperator,PoolConcurrencyPolicy poolConcurrencyPolicy,PoolReusePolicy poolReusePolicy,
            final TimeValue timeToLive,HttpConnectionFactory<ManagedHttpClientConnection> connFactory) {
            
        switch (poolConcurrencyPolicy != null ? poolConcurrencyPolicy : PoolConcurrencyPolicy.STRICT) {
            case STRICT://默认方式
                this.pool = new StrictConnPool<>(DEFAULT_MAX_CONNECTIONS_PER_ROUTE,// 5
                        DEFAULT_MAX_TOTAL_CONNECTIONS,timeToLive,poolReusePolicy,null);
                break;
            case LAX:
                this.pool = new LaxConnPool<>(DEFAULT_MAX_CONNECTIONS_PER_ROUTE,timeToLive,poolReusePolicy,null);
                break;
            default:
            ...
        }
        this.connFactory = connFactory != null ? connFactory : ManagedHttpClientConnectionFactory.INSTANCE;
        this.closed = new AtomicBoolean(false);
    }
} 

3.2、开始租借连接

  1. 首次租借连接实体PoolEntry,此时尚未初始化Socket连接ManagedHttpClientConnection。
  2. 初始化endpoint之InternalConnectionEndpoint
public LeaseRequest lease(String id,HttpRoute route,Timeout requestTimeout, Object state) {
 	//StrictConnPool#lease
  final Future<PoolEntry> leaseFuture = this.pool.lease(route, state, requestTimeout, null);
  return new LeaseRequest() {
      private volatile ConnectionEndpoint endpoint;
      
      @Override
      public synchronized ConnectionEndpoint get(Timeout timeout){
            PoolEntry poolEntry = leaseFuture.get(timeout.getDuration(), timeout.getTimeUnit());
            ...
	        final ManagedHttpClientConnection conn = poolEntry.getConnection();
	        if (conn != null) {
	            conn.activate();
	        } else {
	        	// #6.1 connFactory: ManagedHttpClientConnectionFactory
	        	// #4 PoolEntry#assignConnection
	            poolEntry.assignConnection(connFactory.createConnection(null));
	        }
	        if (leaseFuture.isCancelled()) {
	            pool.release(poolEntry, false);
	        } else {
	        	  // 创建默认的Endpoint 之 InternalConnectionEndpoint。
	            this.endpoint = new InternalConnectionEndpoint(poolEntry);
	        }
	        return this.endpoint;
		}
      ...
  };
}

3.2、初始化Socket 套接字

public void connect(final ConnectionEndpoint endpoint, final TimeValue connectTimeout, final HttpContext context){
  	
    final InternalConnectionEndpoint internalEndpoint = cast(endpoint);
    final PoolEntry<HttpRoute, ManagedHttpClientConnection> poolEntry = internalEndpoint.getPoolEntry();
    final HttpRoute route = poolEntry.getRoute();
   	...
   	//DefaultManagedHttpClientConnection
    final ManagedHttpClientConnection conn = poolEntry.getConnection();
    this.connectionOperator.connect(//DefaultHttpClientConnectionOperator
            conn,
            host,
            route.getLocalSocketAddress(),
            connectTimeout,
            defaultSocketConfig != null ? this.defaultSocketConfig : SocketConfig.DEFAULT,
            context);
}

通过DefaultHttpClientConnectionOperator创建Socket实例,并且与 DefaultManagedHttpClientConnection 绑定。【#6.2】

4、StrictConnPool

  • 该类型的连接池缓存【属性routeToPool】HttpRoute vs 对应的连接池 的对应关系。
  • 静态内部类 PerRoutePool 表示某个路由HttpRoute的对应连接池。
  • PerRoutePool中每个连接对象为 PoolEntry实例。

PerRoutePool中属性之Set类型的 leased 集合:缓存租借出的连接 PoolEntry 实例。
PerRoutePool中属性之LinkedList类型的 available 集合:缓存连接池中可用的连接 PoolEntry实例。

  • 每次根据 HttpRoute 获取连接实例 PoolEntry时,都会 available -1 & leased + 1。
public class StrictConnPool<T, C extends ModalCloseable> implements ManagedConnPool<T, C> {
	private final Map<T, PerRoutePool<T, C>> routeToPool;//缓存不同路由HttpRoute对应的连接池
    private final LinkedList<LeaseRequest<T, C>> leasingRequests;
    private final Set<PoolEntry<T, C>> leased;
    private final LinkedList<PoolEntry<T, C>> available;
	private final Map<T, Integer> maxPerRoute;
	private volatile int defaultMaxPerRoute;
    private volatile int maxTotal;
	
	public Future lease(T route,Object state,Timeout requestTimeout,FutureCallback callback) {
		...
		BasicFuture<PoolEntry<T, C>> future = new BasicFuture<>(callback);
		LeaseRequest<T, C> request = new LeaseRequest<>(route, state, requestTimeout, future);
		//true表示存在可重用的连接
		final boolean completed = processPendingRequest(request);
	    if (!request.isDone() && !completed) {
	        this.leasingRequests.add(request);
	    }
	    if (request.isDone()) {
	    	// 表示存在可重用连接
	        this.completedRequests.add(request);
	    }
	    // 首先从 completedRequests中获取 LeaseRequest,其次从LeaseRequest获取BasicFuture,最后将 LeaseRequest中PoolEntry赋值给BasicFuture
	    fireCallbacks();
	    return future;
	}
	
	private boolean processPendingRequest(final LeaseRequest<T, C> request) {
		// 获取当前域名下的路由
	    final T route = request.getRoute();
	    final Object state = request.getState();
	    final Deadline deadline = request.getDeadline();
		...
		//获取当前 route 下的 pool之PerRoutePool
	    final PerRoutePool<T, C> pool = getPool(route);
	    PoolEntry<T, C> entry;
	    for (;;) {
	    	// 从当前PerRoutePool中尝试获取可用连接
	        entry = pool.getFree(state);
	        if (entry == null) {
	            break;
	        }
	        // 如果从 PerRoutePool 获取的连接过期
	        if (entry.getExpiryDeadline().isExpired()) {
	        	// 释放连接 #4
	            entry.discardConnection(CloseMode.GRACEFUL);
	            // 将连接PoolEntry从StrictConnPool池中移除
	            this.available.remove(entry);
	            // 同时从 PerRoutePool 中移除
	            pool.free(entry, false);
	        } else {
	            break;
	        }
	    }
	    if (entry != null) {
	    	// 将获取到的可重用连接从 StrictConnPool中available中移除,并且有leased持有统计控制
	        this.available.remove(entry);
	        this.leased.add(entry);
	        // 将PoolEntry赋值给 LeaseRequest
	        request.completed(entry);
	        if (this.connPoolListener != null) {
	            this.connPoolListener.onLease(entry.getRoute(), this);
	        }
	        // 返回可重用的连接
	        return true;
	    }
	
	    // 以下执行说明,没有可重用的连接,需要创建新的连接
	    final int maxPerRoute = getMax(route);
	    final int excess = Math.max(0, pool.getAllocatedCount() + 1 - maxPerRoute);
	    // excess表示超过maxPerRoute的连接数
	    if (excess > 0) {
	        for (int i = 0; i < excess; i++) {
	            final PoolEntry<T, C> lastUsed = pool.getLastUsed();
	            if (lastUsed == null) {
	                break;
	            }
	            lastUsed.discardConnection(CloseMode.GRACEFUL);
	            this.available.remove(lastUsed);
	            pool.remove(lastUsed);
	        }
	    }
	
	    if (pool.getAllocatedCount() < maxPerRoute) {//this.available.size() + this.leased.size();
	        // 判断已经创建的连接数是否超过最大连接数
	        final int freeCapacity = Math.max(this.maxTotal - this.leased.size(), 0);
	        if (freeCapacity == 0) {
	            return false;
	        }
	        final int totalAvailable = this.available.size();
	        if (totalAvailable > freeCapacity - 1) {
	            if (!this.available.isEmpty()) {
	                final PoolEntry<T, C> lastUsed = this.available.removeLast();
	                lastUsed.discardConnection(CloseMode.GRACEFUL);
	                final PerRoutePool<T, C> otherpool = getPool(lastUsed.getRoute());
	                otherpool.remove(lastUsed);
	            }
	        }
			// 开始新建连接
	        entry = pool.createEntry(this.timeToLive);
	        // 由leased持有当前新建的连接 PoolEntry
	        this.leased.add(entry);
	        // 将PoolEntry赋值给 LeaseRequest
	        request.completed(entry);
	        if (this.connPoolListener != null) {
	            this.connPoolListener.onLease(entry.getRoute(), this);
	        }
	        return true;
	    }
	    return false;
	}
	
	private PerRoutePool<T, C> getPool(final T route) {
        PerRoutePool<T, C> pool = this.routeToPool.get(route);
        if (pool == null) {
            pool = new PerRoutePool<>(route, this.disposalCallback);
            this.routeToPool.put(route, pool);
        }
        return pool;
    }
	
	static class LeaseRequest<T, C extends ModalCloseable> {

        private final T route;
        private final Object state;
        private final Deadline deadline;
        private final BasicFuture<PoolEntry<T, C>> future;
        private final AtomicBoolean completed;
        private volatile PoolEntry<T, C> result;
        private volatile Exception ex;

        public LeaseRequest(
                final T route,
                final Object state,
                final Timeout requestTimeout,
                final BasicFuture<PoolEntry<T, C>> future) {
            super();
            this.route = route;
            this.state = state;
            this.deadline = Deadline.calculate(requestTimeout);
            this.future = future;
            this.completed = new AtomicBoolean(false);
        }
        ...
    }

    static class PerRoutePool<T, C extends ModalCloseable> {

        private final T route;
        private final Set<PoolEntry<T, C>> leased;
        private final LinkedList<PoolEntry<T, C>> available;//表示当前HttpRoute对应的连接池
        private final DisposalCallback<C> disposalCallback;
		...
        public PoolEntry<T, C> getFree(final Object state) {
            if (!this.available.isEmpty()) {
                if (state != null) {
                    final Iterator<PoolEntry<T, C>> it = this.available.iterator();
                    while (it.hasNext()) {
                        final PoolEntry<T, C> entry = it.next();
                        if (state.equals(entry.getState())) {
                            it.remove();
                            this.leased.add(entry);
                            return entry;
                        }
                    }
                }
                final Iterator<PoolEntry<T, C>> it = this.available.iterator();
                while (it.hasNext()) {
                    final PoolEntry<T, C> entry = it.next();
                    if (entry.getState() == null) {
                        it.remove();
                        this.leased.add(entry);
                        return entry;
                    }
                }
            }
            return null;
        }
		// reusable = true,则将 重用连接 放回到连接池中
		// reusable = false,说明连接不可重用,将连接从leased中移除,但不会将连接返回到池中
        public void free(final PoolEntry<T, C> entry, final boolean reusable) {
            final boolean found = this.leased.remove(entry);
            if (reusable) {
                this.available.addFirst(entry);
            }
        }

        public PoolEntry<T, C> createEntry(final TimeValue timeToLive) {
            final PoolEntry<T, C> entry = new PoolEntry<>(this.route, timeToLive, disposalCallback);
            this.leased.add(entry);
            return entry;
        }
}

5、连接实体 PoolEntry

public final class PoolEntry<T, C extends ModalCloseable> {

	private final AtomicReference<C> connRef;

	public void discardConnection(final CloseMode closeMode) {
		// 获取 DefaultManagerHttpClientConnection 连接
	    final C connection = this.connRef.getAndSet(null);
	    ...
	    // closeMode = GRACEFUL 优雅关闭
	    connection.close(closeMode);
	}
	
	public void assignConnection(final C conn) {
        if (this.connRef.compareAndSet(null, conn)) {
            this.created = getCurrentTime();
            this.updated = this.created;
            this.validityDeadline = Deadline.calculate(this.created, this.timeToLive);
            this.expiryDeadline = this.validityDeadline;
            this.state = null;
        } else {
            throw new IllegalStateException("Connection already assigned");
        }
    }
}

6、连接工厂类ManagedHttpClientConnectionFactory

通过ManagedHttpClientConnectionFactory创建连接对象DefaultManagedHttpClientConnection

6.1、ManagedHttpClientConnectionFactory

参考 # 3.2 首次租借连接时实例化。

public ManagedHttpClientConnection createConnection(final Socket socket){
   ...
   DefaultManagedHttpClientConnection conn = new DefaultManagedHttpClientConnection(
            id,
            charDecoder,
            charEncoder,
            h1Config,
            incomingContentStrategy,
            outgoingContentStrategy,
            requestWriterFactory,
            responseParserFactory);
    if (socket != null) {//首次创建socket为null
        conn.bind(socket);
    }
    return conn;
}
  • 生成 DefaultManagedHttpClientConnection 由PoolEntry属性connRef持有。
public void close(final CloseMode closeMode) {
	......
	//BHttpConnectionBase#close
    super.close(closeMode);
}
public void close(final CloseMode closeMode) {
	
    final SocketHolder socketHolder = this.socketHolderRef.getAndSet(null);
    final Socket socket = socketHolder.getSocket();
    // 关闭 Socket 连接
    Closer.closeQuietly(socket);
}

6.2、DefaultManagedHttpClientConnection

final class DefaultManagedHttpClientConnection extends DefaultBHttpClientConnection{

	public void bind(final Socket socket){
        super.bind(this.wireLog.isDebugEnabled() ? new LoggingSocketHolder(socket, this.id, this.wireLog) : new 	
        		SocketHolder(socket));
        socketTimeout = Timeout.ofMilliseconds(socket.getSoTimeout());
    }
}
public class DefaultBHttpClientConnection extends BHttpConnectionBase{

}
class BHttpConnectionBase implements BHttpConnection {

	AtomicReference<SocketHolder> socketHolderRef;
	
	protected void bind(final Socket socket) throws IOException {
        bind(new SocketHolder(socket));
    }
	
	protected void bind(final SocketHolder socketHolder) throws IOException {
        this.socketHolderRef.set(socketHolder);
        this.endpointDetails = null;
    }
}
  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值