彻底掌握网络通信(一)Http协议基础知识
彻底掌握网络通信(二)Apache的HttpClient基础知识
彻底掌握网络通信(三)Android源码中HttpClient的在不同版本的使用
彻底掌握网络通信(四)Android源码中HttpClient的发送框架解析
彻底掌握网络通信(五)DefaultRequestDirector解析
彻底掌握网络通信(六)HttpRequestRetryHandler解析
彻底掌握网络通信(七)ConnectionReuseStrategy,ConnectionKeepAliveStrategy解析
彻底掌握网络通信(八)AsyncHttpClient源码解读
彻底掌握网络通信(九)AsyncHttpClient为什么无法用Fiddler来抓包
彻底掌握网络通信(十)AsyncHttpClient如何发送JSON解析JSON,以及一些其他用法
彻底掌握网络通信(十一)HttpURLConnection进行网络请求的知识准备
彻底掌握网络通信(十二)HttpURLConnection进行网络请求概览
彻底掌握网络通信(十三)HttpURLConnection进行网络请求深度分析
彻底掌握网络通信(十四)HttpURLConnection进行网络请求深度分析二:缓存
这篇我们主要分两个部分来介绍下HttpURLConnection中的重要概念
- 发送Http请求
- 接收Http请求
HttpURLConnection中请求发送和接收都是在HttpURLConnectionImpl.getInputStream()方法中完成的(针对http请求)
@Override public final InputStream getInputStream() throws IOException {
//doInput默认为true
if (!doInput) {
throw new ProtocolException("This protocol does not support input");
}
//通过getResponse方法获得HttpEngine实例
HttpEngine response = getResponse();
/*省略部分代码*/
//通过HttpEngine获取响应body
return response.getResponse().body().byteStream();
}
这个方法将Http的发送和接收完美的进行了封装,我们主要调用该方法,便可以读取响应了;
重点看下getResponse方法
/*
* 尝试发送Http请求并获得响应,因为一个请求有可能是重定向,顾使用了while(true)的方式不断去尝试
*/
private HttpEngine getResponse() throws IOException {
/*initHttpEngine方法作用有两个
* 1:构建具体的http请求类Request.java,并封装默认的http请求头
* 2:构建上面构建的http请求类创建HttpEngine.java,HttpEngine主要用来socket的连接,发送Http请求和接收Http响应
*/
initHttpEngine();
if (httpEngine.hasResponse()) {
return httpEngine;
}
while (true) {
/*
* execute方法使用HttpEngine来完成请求的发送和接收
*/
if (!execute(true)) {
continue;
}
Response response = httpEngine.getResponse();
Request followUp = httpEngine.followUpRequest();
if (followUp == null) {
httpEngine.releaseStreamAllocation();//Allocation分配
return httpEngine;
}
if (++followUpCount > HttpEngine.MAX_FOLLOW_UPS) {
throw new ProtocolException("Too many follow-up requests: " + followUpCount);
}
// The first request was insufficient. Prepare for another...
url = followUp.url();
requestHeaders = followUp.headers().newBuilder();
// Although RFC 2616 10.3.2 specifies that a HTTP_MOVED_PERM redirect
// should keep the same method, Chrome, Firefox and the RI all issue GETs
// when following any redirect.
Sink requestBody = httpEngine.getRequestBody();
if (!followUp.method().equals(method)) {
requestBody = null;
}
if (requestBody != null && !(requestBody instanceof RetryableSink)) {
throw new HttpRetryException("Cannot retry streamed HTTP body", responseCode);
}
StreamAllocation streamAllocation = httpEngine.close();
if (!httpEngine.sameConnection(followUp.httpUrl())) {
streamAllocation.release();
streamAllocation = null;
}
httpEngine = newHttpEngine(followUp.method(), streamAllocation, (RetryableSink) requestBody,
response);
}
}
- 疑问:既然第19行已经完成了Http消息的发送和接收,那为什么第15行为什么还用一个while循环?这个循环什么时候跳出去?似乎23行~57行都是无效代码
为了不破坏主线,这个疑问我们将在最后做出解释
我们看下execute方法是怎么完成Http消息的发送和接收的
private boolean execute(boolean readResponse) throws IOException {
boolean releaseConnection = true;
/*省略部分代码*/
try {
//根据本地是否有缓存,来决定Http消息的发送与否
httpEngine.sendRequest();
/*省略部分代码*/
if (readResponse) {
//读取http消息,并写入缓存中
httpEngine.readResponse();
}
releaseConnection = false;
return true;
}
/*省略部分代码*/
}
- sendRequest完成消息的发送
- readResponse完成消息接收
这两个方法都是在HttpEngine.java实现的
public void sendRequest() throws RequestException, RouteException, IOException {
if (cacheStrategy != null) return; // Already sent.
if (httpStream != null) throw new IllegalStateException();
//networkRequest将为请求添加一些属性,如Connection", "Keep-Alive";返回对象代表一个Http请求
Request request = networkRequest(userRequest);
//Internal.java为抽象类,其具体的实现在OkHttpClient中,主要作用是读取用户设置的缓存
InternalCache responseCache = Internal.instance.internalCache(client);
//通过request来获取缓存映射,每一次请求有一个map结构保存了url和对应response之间的关系,这里主要是读取相同请求中的缓存
Response cacheCandidate = responseCache != null
? responseCache.get(request)
: null;
long now = System.currentTimeMillis();
//缓存策略类定义(其中判断了缓存是否过期等等)
cacheStrategy = new CacheStrategy.Factory(now, request, cacheCandidate).get();
networkRequest = cacheStrategy.networkRequest;
cacheResponse = cacheStrategy.cacheResponse;
if (responseCache != null) {
responseCache.trackResponse(cacheStrategy);
}
if (cacheCandidate != null && cacheResponse == null) {
closeQuietly(cacheCandidate.body()); // The cache candidate wasn't applicable. Close it.
}
//在上一步骤中,networkRequest不为null的时候,则通过调用connect的方法完成socket的绑定,http消息的发送
if (networkRequest != null) {
httpStream = connect();
httpStream.setHttpEngine(this);
/*省略部分代码*/
}
//networkRequest为null有可能是缓存可用
else {
//有缓存内容
if (cacheResponse != null) {
// We have a valid cached response. Promote it to the user response immediately.
this.userResponse = cacheResponse.newBuilder()
.request(userRequest)
.priorResponse(stripBody(priorResponse))
.cacheResponse(stripBody(cacheResponse))
.build();
}
//networkRequest为null时的其他情况,构建504:Gateway Time-out的response
else {
// We're forbidden from using the network, and the cache is insufficient.
this.userResponse = new Response.Builder()
.request(userRequest)
.priorResponse(stripBody(priorResponse))
.protocol(Protocol.HTTP_1_1)
.code(504)
.message("Unsatisfiable Request (only-if-cached)")
.body(EMPTY_BODY)
.build();
}
//将userResponse进行压缩并返回userResponse
userResponse = unzip(userResponse);
}
}
sendRequest的总结
当网络可用,并且无本地缓存的情况下,完成socket的绑定和Http消息的发送
当缓存可用,直接返回缓存
其他情况,如网络被禁止,则直接返回504的相应
到目前为止,Http请求已经发送,那是如何读取这些响应的?看下readResponse方法
/*
* 刷新请求头和body,并解析HTTP响应头和消息体
*/
public void readResponse() throws IOException {
//当缓存存在的时候,我们不需要做任何动作,直接使用缓存生成响应即可
if (userResponse != null) {
return; // Already ready.
}
//请求为空,并且无缓存,如网络被禁止的情况
if (networkRequest == null && cacheResponse == null) {
throw new IllegalStateException("call sendRequest() first!");
}
if (networkRequest == null) {
return; // No network response to read.
}
//Response.java代表一个HTTP消息的响应
Response networkResponse;
if (forWebSocket) {
resultStream = new Http2xStream(this, resultConnection.framedConnection);
} else {
//这里的httpStream的实现者在之前调用connect方法的时候完成赋值,其实现者有Http2xStream和Http1xStream;大多数情况下为Http1xStream
//将networkRequest请求中的一些头信息写入到请求中
httpStream.writeRequestHeaders(networkRequest);
//读取响应
networkResponse = readNetworkResponse();
}
//callerWritesRequestBody是在构造HttpEngine时传入的,这个参数为false的时候表示在响应返回之前,需要做拦截处理
else if (!callerWritesRequestBody) {
networkResponse = new NetworkInterceptorChain(0, networkRequest).proceed(networkRequest);
} else {
// Emit the request body's buffer so that everything is in requestBodyOut.
if (bufferedRequestBody != null && bufferedRequestBody.buffer().size() > 0) {
bufferedRequestBody.emit();
}
// Emit the request headers if we haven't yet. We might have just learned the Content-Length.
if (sentRequestMillis == -1) {
if (OkHeaders.contentLength(networkRequest) == -1
&& requestBodyOut instanceof RetryableSink) {
long contentLength = ((RetryableSink) requestBodyOut).contentLength();
networkRequest = networkRequest.newBuilder()
.header("Content-Length", Long.toString(contentLength))
.build();
}
httpStream.writeRequestHeaders(networkRequest);
}
// Write the request body to the socket.
if (requestBodyOut != null) {
if (bufferedRequestBody != null) {
// This also closes the wrapped requestBodyOut.
bufferedRequestBody.close();
} else {
requestBodyOut.close();
}
if (requestBodyOut instanceof RetryableSink) {
//将消息体写入到socket中
httpStream.writeRequestBody((RetryableSink) requestBodyOut);
}
}
networkResponse = readNetworkResponse();
}
receiveHeaders(networkResponse.headers());
// If we have a cache response too, then we're doing a conditional get.
//这里是将networkResponse和cacheResponse做对比
if (cacheResponse != null) {
//validate方法返回true,表示cacheResponse可用,返回false,表示networkResponse可用
if (validate(cacheResponse, networkResponse)) {
//使用cacheResponse来构建响应
userResponse = cacheResponse.newBuilder()
.request(userRequest)
.priorResponse(stripBody(priorResponse))
.headers(combine(cacheResponse.headers(), networkResponse.headers()))
.cacheResponse(stripBody(cacheResponse))
.networkResponse(stripBody(networkResponse))
.build();
networkResponse.body().close();
releaseStreamAllocation();
// Update the cache after combining headers but before stripping the
// Content-Encoding header (as performed by initContentStream()).
//更新本地缓存
InternalCache responseCache = Internal.instance.internalCache(client);
responseCache.trackConditionalCacheHit();
responseCache.update(cacheResponse, stripBody(userResponse));
userResponse = unzip(userResponse);
return;
} else {
closeQuietly(cacheResponse.body());
}
}
//如果没有cacheResponse,则使用networkResponse来构建响应
userResponse = networkResponse.newBuilder()
.request(userRequest)
.priorResponse(stripBody(priorResponse))
.cacheResponse(stripBody(cacheResponse))
.networkResponse(stripBody(networkResponse))
.build();
if (hasBody(userResponse)) {
maybeCache();
userResponse = unzip(cacheWritingResponse(storeRequest, userResponse));
}
}
这个方法主要作用有两个
将请求消息写入到socket中
根据本地缓存和响应来获取真正的缓存,其保存的变量为Response userResponse;
这样就可以通过如下代码获取响应的流信息了
HttpEngine response = getResponse();
response.getResponse().body().byteStream()
- 以上我们分析了一个Http消息的发送和Http消息的相应,但是我们并没有看到socket是如何创建和绑定的,这个动作是在哪处理的?
其实在sendRequest方法中调用了connect(),正是这个connect()方法完成了socket的创建和绑定
private HttpStream connect() throws RouteException, RequestException, IOException {
boolean doExtensiveHealthChecks = !networkRequest.method().equals("GET");
return streamAllocation.newStream(client.getConnectTimeout(),
client.getReadTimeout(), client.getWriteTimeout(),
client.getRetryOnConnectionFailure(), doExtensiveHealthChecks);
}
查看下newStream方法
//该方法通过findHealthyConnection完成socket的创建和绑定的过程
public HttpStream newStream(int connectTimeout, int readTimeout, int writeTimeout,
boolean connectionRetryEnabled, boolean doExtensiveHealthChecks)
throws RouteException, IOException {
try {
RealConnection resultConnection = findHealthyConnection(connectTimeout, readTimeout,
writeTimeout, connectionRetryEnabled, doExtensiveHealthChecks);
//Http2xStream和Http1xStream主要是根据协议版本号等其他条件进行具体返回,这两个类的作用是会将一些头信息或者请求体通过writeRequestHeaders方法写入到socket当中
//这样一个http请求才真正的发送出去
HttpStream resultStream;
if (resultConnection.framedConnection != null) {
resultStream = new Http2xStream(this, resultConnection.framedConnection);
} else {
resultConnection.getSocket().setSoTimeout(readTimeout);
resultConnection.source.timeout().timeout(readTimeout, MILLISECONDS);
resultConnection.sink.timeout().timeout(writeTimeout, MILLISECONDS);
resultStream = new Http1xStream(this, resultConnection.source, resultConnection.sink);
}
synchronized (connectionPool) {
resultConnection.streamCount++;
stream = resultStream;
return resultStream;
}
} catch (IOException e) {
throw new RouteException(e);
}
}
查看下findHealthyConnection方法
private RealConnection findHealthyConnection(int connectTimeout, int readTimeout,
int writeTimeout, boolean connectionRetryEnabled, boolean doExtensiveHealthChecks)
throws IOException, RouteException {
while (true) {
//通过findConnection方法返回一个连接,如果这个连接可以对流进行处理,则说明这个连接是健康的
//一个连接是否健康,是通过对socket的方法来判定的,如调用socket的isClosed来判断是否关闭等,如果正常则返回给调用者,否则连接创建失败,调用connectionFailed方法
RealConnection candidate = findConnection(connectTimeout, readTimeout, writeTimeout,
connectionRetryEnabled);
// If this is a brand new connection, we can skip the extensive health checks.
synchronized (connectionPool) {
if (candidate.streamCount == 0) {
return candidate;
}
}
// Otherwise do a potentially-slow check to confirm that the pooled connection is still good.
if (candidate.isHealthy(doExtensiveHealthChecks)) {
return candidate;
}
connectionFailed();
}
}
通过对findConnection代码的分析可以知道,内部new了一个RealConnection实例,并通过调用RealConnection.java的connect()方法完成socket的创建,绑定过程
最后我们来看下我们遗留的那个问题
疑问:既然第19行已经完成了Http消息的发送和接收,那为什么第15行为什么还用一个while循环?这个循环什么时候跳出去?似乎23行~57行都是无效代码
我们知道一个http请求有可能需要重定向,顾这里是采用循环的方式来发送请求,即第15行的 while (true);当一个请求需要重试的时候,即execute返回为false,他会再次循环,直到这个请求发送成功
重点看下第24行代码
Request followUp = httpEngine.followUpRequest();
这里的followup表示一个冲定下的请求,当followup为null的时候,说明这个请求是被正确执行的,则程序直接返回httpEngine;否则根据这个followUp,重新构建具有重定向含义的httpEngine,再起发起请求,这就是第37~57行代码的含义
- 那Request followUp什么时候为null?即什么时候一个请求正确执行,这里就需要看下followUpRequest方法了
/*
* 一个请求如果需要重定下,或者需要添加认证,则需要重新构建一个Request,再次进行http请求
* 反之返回为null,常见的请求即不重定下,不需要添加认证,会返回null
*/
public Request followUpRequest() throws IOException {
if (userResponse == null) throw new IllegalStateException();
Connection connection = streamAllocation.connection();
Route route = connection != null
? connection.getRoute()
: null;
Proxy selectedProxy = route != null
? route.getProxy()
: client.getProxy();
int responseCode = userResponse.code();
final String method = userRequest.method();
//根据具体的相应状态码来做处理
switch (responseCode) {
//返回407,抛出ProtocolException
case HTTP_PROXY_AUTH:
if (selectedProxy.type() != Proxy.Type.HTTP) {
throw new ProtocolException("Received HTTP_PROXY_AUTH (407) code while not using proxy");
}
// fall-through
//401:当前请求需要用户验证。该响应必须包含一个适用于被请求资源的 WWW-Authenticate 信息头用以询问用户信息。
case HTTP_UNAUTHORIZED:
return OkHeaders.processAuthHeader(client.getAuthenticator(), userResponse, selectedProxy);
case HTTP_PERM_REDIRECT:
case HTTP_TEMP_REDIRECT:
// "If the 307 or 308 status code is received in response to a request other than GET
// or HEAD, the user agent MUST NOT automatically redirect the request"
if (!method.equals("GET") && !method.equals("HEAD")) {
return null;
}
// fall-through
case HTTP_MULT_CHOICE:
case HTTP_MOVED_PERM:
case HTTP_MOVED_TEMP:
case HTTP_SEE_OTHER:
// Does the client allow redirects?
if (!client.getFollowRedirects()) return null;
String location = userResponse.header("Location");
if (location == null) return null;
HttpUrl url = userRequest.httpUrl().resolve(location);
// Don't follow redirects to unsupported protocols.
if (url == null) return null;
// If configured, don't follow redirects between SSL and non-SSL.
boolean sameScheme = url.scheme().equals(userRequest.httpUrl().scheme());
if (!sameScheme && !client.getFollowSslRedirects()) return null;
// Redirects don't include a request body.
Request.Builder requestBuilder = userRequest.newBuilder();
if (HttpMethod.permitsRequestBody(method)) {
if (HttpMethod.redirectsToGet(method)) {
requestBuilder.method("GET", null);
} else {
requestBuilder.method(method, null);
}
requestBuilder.removeHeader("Transfer-Encoding");
requestBuilder.removeHeader("Content-Length");
requestBuilder.removeHeader("Content-Type");
}
// When redirecting across hosts, drop all authentication headers. This
// is potentially annoying to the application layer since they have no
// way to retain them.
if (!sameConnection(url)) {
requestBuilder.removeHeader("Authorization");
}
return requestBuilder.url(url).build();
default:
return null;
}
}
最后:
现在使用HttpURLConnection来进行消息发送的各个主要类都已经相继出现
主要有
HttpURLConnectionImpl
HttpEngine
HttpStream
RealConnection
CacheStrategy
Request
Response
他们之间的关系简单的来说如下
当需要发送一个Http协议的时候,HttpURLConnectionImpl就站出来说让我来,然后HttpURLConnectionImpl代就指派HttpEngine去做这件事情
当HttpEngine接到这个命令的时候,他先完成自己的初始化,然后让CacheStrategy查看有没有之前使用过的http请求或者看看系统有没有之前保存的http内容来决定是不是要重新发起一次http请求
当需要重新发起一次http请求的时候,HttpEngine会构建一个健康的RealConnection来去执行socket的绑定于发送,并将HttpStream附带的一些参数写入到socket里面进行发送
当Request正常发送之后,HttpEngine就可以通过Response来获取响应中的流信息
- 疑问:需要显示调用URLConnection的connect方法吗
不需要,查看URLConnection的connect方法可知(URLConnection的实现者为HttpURLConnectionImpl)
@Override public final void connect() throws IOException {
initHttpEngine();
boolean success;
do {
success = execute(false);
} while (!success);
}
其执行的动作在URLConnection的getInputStream方法中已经被执行,顾不需要显示的调用connect方法,直接调用getInputStream方法即可完成消息的发送和接收