CacheInterceptor中如果没能成功使用缓存,接下来就要准备向服务器发起请求。所以接下来的拦截器就是ConnectInterceptor。ConnectInterceptor的主要职责是负责与服务器建立起连接。
ConnectInterceptor的intercept方法很精简,直接上源码:
@Override public Response intercept(Chain chain) throws IOException {
RealInterceptorChain realChain = (RealInterceptorChain) chain;
Request request = realChain.request();
StreamAllocation streamAllocation = realChain.streamAllocation();
// We need the network to satisfy this request. Possibly for validating a conditional GET.
boolean doExtensiveHealthChecks = !request.method().equals("GET");
HttpCodec httpCodec = streamAllocation.newStream(client, chain, doExtensiveHealthChecks);
RealConnection connection = streamAllocation.connection();
return realChain.proceed(request, streamAllocation, httpCodec, connection);
}
代码看上去很简单,只是因为它把绝大部分工作都交给在RetryAndFollowUpInterceptor中创建的streamAllocation完成了。其中的关键在于streamAllocation.newStream(…)方法里,在这里面完成了所有的连接建立工作。要理解连接的建立过程,先来看看OkHttp中连接池(ConnectionPool)的建立。
ConnectionPool
我们知道HTTP连接中需要进行三次握手、四次挥手操作,如果每进行一次请求都建立连接然后释放连接,将会消耗大量的时间(使用HTTPS的情况下更严重)。这种情况下HTTP请求头里提供了Keep-Alive字段,它使得在传输数据之后还能保持连接,下次再进行请求时不需要重新握手。于是OkHttp为了便于管理所有的连接(Connection)的复用设计了ConnectionPool。
先看看ConnectionPool的介绍
Manages reuse of HTTP and HTTP/2 connections for reduced network latency. HTTP requests that share the same {@link Address} may share a {@link Connection}. This class implements the policy of which connections to keep open for future use.
ConnectionPool的主要功能就是为了降低由于频繁建立连接导致的网络延迟。它实现了复用连接的策略。我们可以在创建OkHttpClient的时候自定义一个ConnectionPool,否则系统会为我们创建一个默认的、最大空闲连接数为5、保活时间为5分钟的ConnectionPool。
对ConnectionPool的调用主要在StreamAllocation类里通过Internal.instance进行,我们回到streamAllocation.newStream方法中。这个方法代码量不大:
public HttpCodec newStream(OkHttpClient client, Interceptor.Chain chain, boolean doExtensiveHealthChecks) {
int connectTimeout = chain.connectTimeoutMillis();
int readTimeout = chain.readTimeoutMillis();
int writeTimeout = chain.writeTimeoutMillis();
boolean connectionRetryEnabled = client.retryOnConnectionFailure();
try {
RealConnection resultConnection = findHealthyConnection(connectTimeout, readTimeout,
writeTimeout, connectionRetryEnabled, doExtensiveHealthChecks);
HttpCodec resultCodec = resultConnection.newCodec(client, chain, this);
synchronized (connectionPool) {
codec = resultCodec;
return resultCodec;
}
} catch (IOException e) {
throw new RouteException(e);
}
}
主要的流程有两个,如下:
1. 复用或新建一个Connection对象
2. 新建流,即是创建HttpCodec对象并返回给调用处
1.复用或新建Connection对象
RealConnection resultConnection = findHealthyConnection(connectTimeout, readTimeout,
这行代码中调用了findHealthyConnectino方法来得到一个Connection对象,进入里面看看:
/**
* Finds a connection and returns it if it is healthy. If it is unhealthy the process is repeated
* until a healthy connection is found.
*/
private RealConnection findHealthyConnection(int connectTimeout, int readTimeout,
int writeTimeout, boolean connectionRetryEnabled, boolean doExtensiveHealthChecks)
throws IOException {
while (true) {
RealConnection candidate = findConnection(connectTimeout, readTimeout, writeTimeout,
connectionRetryEnabled);
// If this is a brand new connection, we can skip the extensive health checks.
synchronized (connectionPool) {
if (candidate.successCount == 0) {
return candidate;
}
}
// Do a (potentially slow) check to confirm that the pooled connection is still good. If it
// isn't, take it out of the pool and start again.
if (!candidate.isHealthy(doExtensiveHealthChecks)) {
noNewStreams();
continue;
}
return candidate;
}
}
在上面的findHealthyConnection方法中,有个while(true){}循环,循环不断地从findConnection方法中获取一个候选的Connection对象,然后进行判断是否符合要求,符合则返回,否则继续执行循环。继续前进,看看findConnection里做了些什么:
/**
* Returns a connection to host a new stream. This prefers the existing connection if it exists,
* then the pool, finally building a new connection.
*/
private RealConnection findConnection(int connectTimeout, int readTimeout, int writeTimeout,
boolean connectionRetryEnabled) throws IOException {
boolean foundPooledConnection = false;
RealConnection result = null;
Route selectedRoute = null;
Connection releasedConnection;
Socket toClose;
synchronized (connectionPool) {
if (released) throw new IllegalStateException("released");
if (codec != null) throw new IllegalStateException("codec != null");
if (canceled) throw new IOException("Canceled");
// Attempt to use an already-allocated connection. We need to be careful here because our
// already-allocated connection may have been restricted from creating new streams.
releasedConnection = this.connection;
toClose = releaseIfNoNewStreams();
if (this.connection != null) {
// We had an already-allocated connection and it's good.
result = this.connection;
releasedConnection = null;
}
if (!reportedAcquired) {
// If the connection was never reported acquired, don't report it as released!
releasedConnection = null;
}
if (result == null) {
// Attempt to get a connection from the pool.
Internal.instance.get(connectionPool, address, this, null);
if (connection != null) {
foundPooledConnection = true;
result = connection;
} else {
selectedRoute = route;
}
}
}
closeQuietly(toClose);
if (releasedConnection != null) {
eventListener.connectionReleased(call, releasedConnection);
}
if (foundPooledConnection) {
eventListener.connectionAcquired(call, result);
}
if (result != null) {
// If we found an already-allocated or pooled connection, we're done.
return result;
}
// If we need a route selection, make one. This is a blocking operation.
boolean newRouteSelection = false;
if (selectedRoute == null && (routeSelection == null || !routeSelection.hasNext())) {
newRouteSelection = true;
routeSelection = routeSelector.next();
}
synchronized (connectionPool) {
if (canceled) throw new IOException("Canceled");
if (newRouteSelection) {
// Now that we have a set of IP addresses, make another attempt at getting a connection from
// the pool. This could match due to connection coalescing.
List<Route> routes = routeSelection.getAll();
for (int i = 0, size = routes.size(); i < size; i++) {
Route route = routes.get(i);
Internal.instance.get(connectionPool, address, this, route);
if (connection != null) {
foundPooledConnection = true;
result = connection;
this.route = route;
break;
}
}
}
if (!foundPooledConnection) {
if (selectedRoute == null) {
selectedRoute = routeSelection.next();
}
// Create a connection and assign it to this allocation immediately. This makes it possible
// for an asynchronous cancel() to interrupt the handshake we're about to do.
route = selectedRoute;
refusedStreamCount = 0;
result = new RealConnection(connectionPool, selectedRoute);
acquire(result, false);
}
}
// If we found a pooled connection on the 2nd time around, we're done.
if (foundPooledConnection) {
eventListener.connectionAcquired(call, result);
return result;
}
// Do TCP + TLS handshakes. This is a blocking operation.
result.connect(
connectTimeout, readTimeout, writeTimeout, connectionRetryEnabled, call, eventListener);
routeDatabase().connected(result.route());
Socket socket = null;
synchronized (connectionPool) {
reportedAcquired = true;
// Pool the connection.
Internal.instance.put(connectionPool, result);
// If another multiplexed connection to the same address was created concurrently, then
// release this connection and acquire that one.
if (result.isMultiplexed()) {
socket = Internal.instance.deduplicate(connectionPool, address, this);
result = connection;
}
}
closeQuietly(socket);
eventListener.connectionAcquired(call, result);
return result;
}
findConnectino方法很长,大致流程如下:
1. 首先判断当前StreamAllocation对象是否已经有一个Connection对象了(这种情况会在请求重定向,且重定向的Request的host、port、scheme与之前一致时出现)
2. 如果1不满足,则尝试从ConnectionPool中获取一个
3. 如果2中没有获取到,则遍历所有路由路径,尝试从再次从ConnectionPool中寻找可复用的连接,找到则返回
4. 如果3中没有找到可复用的连接,则尝试新建一个,进行三次握手/TLS握手(如果需要)
5. 把新建的连接放入ConnectionPool中
6. 返回结果
以上步骤的关键在于如何寻找可复用的连接,这又回到了我们上面提到的ConnectionPool。从ConnectionPool中获取一个可复用的Connection是通过ConnectionPool的get方法实现的,其内部代码量不大:
@Nullable RealConnection get(Address address, StreamAllocation streamAllocation, Route route) {
assert (Thread.holdsLock(this));
for (RealConnection connection : connections) {
if (connection.isEligible(address, route)) {
streamAllocation.acquire(connection, true);
return connection;
}
}
return null;
}
大致的逻辑就是循环判断connections队列中的每一个对象,如果有符合的话,则调用streamAllocation.acquire方法绑定(因为对connection的回收管理中用到了引用计数法,后面再介绍),然后返回。判断是否符合的方法是connection.isEligible,代码如下:
public boolean isEligible(Address address, @Nullable Route route) {
// If this connection is not accepting new streams, we're done.
if (allocations.size() >= allocationLimit || noNewStreams) return false;
// If the non-host fields of the address don't overlap, we're done.
if (!Internal.instance.equalsNonHost(this.route.address(), address)) return false;
// If the host exactly matches, we're done: this connection can carry the address.
if (address.url().host().equals(this.route().address().url().host())) {
return true; // This connection is a perfect match.
}
// At this point we don't have a hostname match. But we still be able to carry the request if
// our connection coalescing requirements are met. See also:
// https://hpbn.co/optimizing-application-delivery/#eliminate-domain-sharding
// https://daniel.haxx.se/blog/2016/08/18/http2-connection-coalescing/
// 1. This connection must be HTTP/2.
if (http2Connection == null) return false;
// 2. The routes must share an IP address. This requires us to have a DNS address for both
// hosts, which only happens after route planning. We can't coalesce connections that use a
// proxy, since proxies don't tell us the origin server's IP address.
if (route == null) return false;
if (route.proxy().type() != Proxy.Type.DIRECT) return false;
if (this.route.proxy().type() != Proxy.Type.DIRECT) return false;
if (!this.route.socketAddress().equals(route.socketAddress())) return false;
// 3. This connection's server certificate's must cover the new host.
if (route.address().hostnameVerifier() != OkHostnameVerifier.INSTANCE) return false;
if (!supportsUrl(address.url())) return false;
// 4. Certificate pinning must match the host.
try {
address.certificatePinner().check(address.url().host(), handshake().peerCertificates());
} catch (SSLPeerUnverifiedException e) {
return false;
}
return true; // The caller's address can be carried by this connection.
}
大致就是,如果host匹配了,说明是这条连接是可以复用的。针对HTTP2做了另外一些条件判断,具体的以后有时间再详细研究。
说完get方法,说put方法,put方法是往ConnectionPool中放入一个新的Connection对象,代码如下:
void put(RealConnection connection) {
assert (Thread.holdsLock(this));
if (!cleanupRunning) {
cleanupRunning = true;
executor.execute(cleanupRunnable);
}
connections.add(connection);
}
put方法代码也很精简。在往connections队列中添加对象前,会先判断当前的cleanupRunnable是否有在执行。cleanupRunnable是一个专门用于回收连接池中的无效连接的。代码如下:
private final Runnable cleanupRunnable = new Runnable() {
@Override public void run() {
while (true) {
long waitNanos = cleanup(System.nanoTime());
if (waitNanos == -1) return;
if (waitNanos > 0) {
long waitMillis = waitNanos / 1000000L;
waitNanos -= (waitMillis * 1000000L);
synchronized (ConnectionPool.this) {
try {
ConnectionPool.this.wait(waitMillis, (int) waitNanos);
} catch (InterruptedException ignored) {
}
}
}
}
}
};
在run方法中会循环调用cleanup方法检测是否有无效连接需要清除,核心在cleanup方法内部:
long cleanup(long now) {
int inUseConnectionCount = 0;
int idleConnectionCount = 0;
RealConnection longestIdleConnection = null;
long longestIdleDurationNs = Long.MIN_VALUE;
// Find either a connection to evict, or the time that the next eviction is due.
synchronized (this) {
for (Iterator<RealConnection> i = connections.iterator(); i.hasNext(); ) {
RealConnection connection = i.next();
// If the connection is in use, keep searching.
if (pruneAndGetAllocationCount(connection, now) > 0) {
inUseConnectionCount++;
continue;
}
idleConnectionCount++;
// If the connection is ready to be evicted, we're done.
long idleDurationNs = now - connection.idleAtNanos;
if (idleDurationNs > longestIdleDurationNs) {
longestIdleDurationNs = idleDurationNs;
longestIdleConnection = connection;
}
}
if (longestIdleDurationNs >= this.keepAliveDurationNs
|| idleConnectionCount > this.maxIdleConnections) {
// We've found a connection to evict. Remove it from the list, then close it below (outside
// of the synchronized block).
connections.remove(longestIdleConnection);
} else if (idleConnectionCount > 0) {
// A connection will be ready to evict soon.
return keepAliveDurationNs - longestIdleDurationNs;
} else if (inUseConnectionCount > 0) {
// All connections are in use. It'll be at least the keep alive duration 'til we run again.
return keepAliveDurationNs;
} else {
// No connections, idle or in use.
cleanupRunning = false;
return -1;
}
}
closeQuietly(longestIdleConnection.socket());
// Cleanup again immediately.
return 0;
}
在cleanup方法内使用了类似GC算法中的标记-擦除算法,标记处最不活跃的连接进行清除。步骤如下:
1. 遍历connections队列,如果当前的connection对象正在被使用,continue,inUseConnectionCount+1
2. 否则idleConnectionCount+1
,同时判断当前的connection的空闲时间是否比已知的长,是的话把它记录下来
3. 如果空闲时间超过了保活时间,或者当前空闲连接数超过了最大空闲连接数,说明这个connection对象需要被回收,将它从connections队列中移除,在代码最后两行进行关闭操作,同时返回0通知cleanupRunnable
的线程马上继续执行cleanup方法
4. 如果3不满足,判断idleConnectionCount
是否大于0,是的话返回保活时间与空闲时间差keepAliveDurationNs - longestIdleDurationNs
,cleanupRunnable
的线程等待时间keepAliveDurationNs - longestIdleDurationNs
后继续执行cleanup方法
5. 如果4不满足,说明没有空闲连接,继续判断有没有正在使用的连接,有的话返回保活时间keepAliveDurationNs
,提醒cleanupRunnable
的线程至少等待时间keepAliveDurationNs
后才需要继续执行cleanup方法
6. 如果5也不满足,当前说明连接池中没有连接,返回-1,告诉cleanupRunnable
的线程不需要再执行了。等待下次调用put方法时再执行。
在步骤1中,判断当前connection是否正在被使用,调用的方法是pruneAndGetAllocationCount
,代码如下:
private int pruneAndGetAllocationCount(RealConnection connection, long now) {
List<Reference<StreamAllocation>> references = connection.allocations;
for (int i = 0; i < references.size(); ) {
Reference<StreamAllocation> reference = references.get(i);
if (reference.get() != null) {
i++;
continue;
}
// We've discovered a leaked allocation. This is an application bug.
StreamAllocation.StreamAllocationReference streamAllocRef =
(StreamAllocation.StreamAllocationReference) reference;
String message = "A connection to " + connection.route().address().url()
+ " was leaked. Did you forget to close a response body?";
Platform.get().logCloseableLeak(message, streamAllocRef.callStackTrace);
references.remove(i);
connection.noNewStreams = true;
// If this was the last allocation, the connection is eligible for immediate eviction.
if (references.isEmpty()) {
connection.idleAtNanos = now - keepAliveDurationNs;
return 0;
}
}
return references.size();
}
pruneAndGetAllocationCount方法进行判断时,使用的是类似GC算法中的引用计数法,针对每一个connection对象,里面都维护了一个引用这个connection的StreamAllocation的弱引用(StreamAllocationReference extends WeakReference<StreamAllocation>
)列表。每次有StreamAllocation用到这个connection对象时,都把这个StreamAllocation对象的弱引用添加到connection的allocations集合里。在pruneAndGetAllocationCount方法里只需要遍历这个集合,判断是否每一个元素都为null,就知道这个connection对象当前能否被释放。
以上部分大致说明了StreamAllocation如何从ConnectionPool中复用连接、ConnectionPool如何建立、维护一个连接池的。代码量比较大, 代码整体逻辑也比较清晰。还有一些细节没有研究,以后有空补上(估计没有以后)。
2.新建流(HttpCodec)
在streamAllocation.newStream方法中,获取到Connection对象之后,要进行的就是新建流,即创建HttpCodec对象。HttpCodec对象是封装了底层IO的可以直接用来收发数据的组件(依赖okio库),提供了这些操作:
* 为发送请求而提供的
* 写入请求头
* 创建请求体
* 结束请求发送
* 为获得响应而提供的
* 读取相应头部
* 打开响应体,以用于获取响应体数据
* 取消请求执行
在ConnectInterceptor中对于HttpCodec只是进行了创建工作,具体的调用在下一个拦截器CallServerInterceptor中进行,具体分析也在下一篇笔记中进行。
后话
这篇笔记花了三天的时间(当然不是一直都在写)。看了好多别人的博客,刚开始看别人的博客都一脸懵逼,更别说看源码,到后来慢慢有了点脉络,然后慢慢自己也能跟着源码的思路走,真的是不容易。总结到的经验就是,看不下去想放弃的时候,关上电脑学其它东西吧,死撑着只会越看越烦躁,说不定睡一觉明天醒来就能看懂了(真)。