前言
众所周知,浏览器与服务器交互时可对响应的数据进行缓存,这极大的节省了用户的流量和提高了网站的响应性。对于app而言,理应做响应数据缓存。
实现方式
就我目前所知,主要是两种方式:以文件方式缓存和以存数据库方式缓存。
以文件方式缓存,笔记认为有研究意义的当属volley和开源中国源码。以数据库方式笔者目前没碰到,在这里就不说了。这篇主要介绍volley的缓存响应数据机制。
知识准备
了解http协议,javaweb。不会的没关系,我会做简单的说明。
拿tomcat服务器来说,tomcat缓存策略如下图:
客户端发起一次请求,tomcat服务器进行响应。响应头携带Last-Modified,其表示最后的修改时间,客户端发起重复请求时,请求头携带If-Modified-Since,其值就是上次响应中的Last-Modified。服务器收到该请求后,将文件最后修改时间与If-Modified-Since对应的时间进行比对,如果一样表示服务器没有修改过该文件,则304响应,客户端就去读取缓存,如果不一样就响应200,将最新的Last-Modified放到响应头里,客户端读取服务器数据。
If-None-Match与Etag是一对,跟If-Modified-Since与Last-Modified一样。
Volley缓存网络数据详解
从解析响应数据说起,
在NetworkDispatcher类的run()方法中,找到
<pre name="code" class="java">// Parse the response here on the worker thread.
Response<?> response = request.parseNetworkResponse(networkResponse);
request.addMarker("network-parse-complete");
// Write to cache if applicable.写入缓存
// TODO: Only update cache metadata instead of entire record for 304s.
if (request.shouldCache() && response.cacheEntry != null) {
mCache.put(request.getCacheKey(), response.cacheEntry);
request.addMarker("network-cache-written");
}
其第二行处理网络响应,由于Request是抽象类,我们具体拿StringRequest来说,
@Override
protected Response<String> parseNetworkResponse(NetworkResponse response) {
String parsed;
try {
parsed = new String(response.data, HttpHeaderParser.parseCharset(response.headers));
} catch (UnsupportedEncodingException e) {
parsed = new String(response.data);
}
return Response.success(parsed, HttpHeaderParser.parseCacheHeaders(response));
}
关键就是看HttpHeaderParser.parseCacheHeaders(response)。
/**
* Extracts a {@link Cache.Entry} from a {@link NetworkResponse}.
*
* @param response The network response to parse headers from
* @return a cache entry for the given response, or null if the response is not cacheable.
*/
public static Cache.Entry parseCacheHeaders(NetworkResponse response) {
long now = System.currentTimeMillis();
Map<String, String> headers = response.headers;
long serverDate = 0;
long lastModified = 0;
long serverExpires = 0;
long softExpire = 0;
long finalExpire = 0;
long maxAge = 0;
long staleWhileRevalidate = 0;
boolean hasCacheControl = false;
boolean mustRevalidate = false;
String serverEtag = null;
String headerValue;
headerValue = headers.get("Date");
if (headerValue != null) {
serverDate = parseDateAsEpoch(headerValue);
}
headerValue = headers.get("Cache-Control");
if (headerValue != null) {
hasCacheControl = true;
String[] tokens = headerValue.split(",");
for (int i = 0; i < tokens.length; i++) {
String token = tokens[i].trim();
if (token.equals("no-cache") || token.equals("no-store")) {
return null;
} else if (token.startsWith("max-age=")) {
try {
maxAge = Long.parseLong(token.substring(8));
} catch (Exception e) {
}
} else if (token.startsWith("stale-while-revalidate=")) {
try {
staleWhileRevalidate = Long.parseLong(token.substring(23));
} catch (Exception e) {
}
} else if (token.equals("must-revalidate") || token.equals("proxy-revalidate")) {
mustRevalidate = true;
}
}
}
headerValue = headers.get("Expires");
if (headerValue != null) {
serverExpires = parseDateAsEpoch(headerValue);
}
headerValue = headers.get("Last-Modified");
if (headerValue != null) {
lastModified = parseDateAsEpoch(headerValue);
}
serverEtag = headers.get("ETag");
// Cache-Control takes precedence over an Expires header, even if both exist and Expires
// is more restrictive.
if (hasCacheControl) {
softExpire = now + maxAge * 1000;
finalExpire = mustRevalidate
? softExpire
: softExpire + staleWhileRevalidate * 1000;
} else if (serverDate > 0 && serverExpires >= serverDate) {
// Default semantic for Expire header in HTTP specification is softExpire.
softExpire = now + (serverExpires - serverDate);
finalExpire = softExpire;
}
Cache.Entry entry = new Cache.Entry();
entry.data = response.data;
entry.etag = serverEtag;
entry.softTtl = softExpire;
entry.ttl = finalExpire;
entry.serverDate = serverDate;
entry.lastModified = lastModified;
entry.responseHeaders = headers;
return entry;
}
其具体是对http响应头进行分析并一个缓存实体Cache.Entry,将其返回。上面的softTtl是该响应数据对应缓存的软生存时间,ttl是实际生存时间。
从请求头这方来看
在NetworkDispatcher类的run()方法中,找到// Perform the network request.
NetworkResponse networkResponse = mNetwork.performRequest(request);
mNetwork是在Volley的newRequestQueue(Context context, HttpStack stack, int maxDiskCacheBytes)方法中创建的。也就是BasicNetwork,我们看它的performRequest()方法,其关键代码:
<pre name="code" class="java">try {
// Gather headers.
Map<String, String> headers = new HashMap<String, String>();
addCacheHeaders(headers, request.getCacheEntry());<span style="white-space:pre"> </span>//这里从缓存数据中解析出请求头数据
httpResponse = mHttpStack.performRequest(request, headers);
StatusLine statusLine = httpResponse.getStatusLine();
int statusCode = statusLine.getStatusCode();
往addCacheHeaders()看:
private void addCacheHeaders(Map<String, String> headers, Cache.Entry entry) {
// If there's no cache entry, we're done.
if (entry == null) {
return;
}
if (entry.etag != null) {
headers.put("If-None-Match", entry.etag);
}
if (entry.lastModified > 0) {
Date refTime = new Date(entry.lastModified);
headers.put("If-Modified-Since", DateUtils.formatDate(refTime));
}
}
也就是将缓存中etag和lastModified分别放置在请求头里的If-None-Match和If-Modified-Since。
开发中缓存数据的方式
一、由服务器去控制
也就是服务端设置etag、lastModified和Cache-Control等。
二、由安卓开发人员控制
也就在Request的parseNetworkResponse方法中使用自己的HttpHeaderParser。具体例子:
自定义的HttpHeaderParser。
/**
* 自定义的HeaderParser,跟默认的比,可以强制缓存,忽略服务器的设置
*/
public class CustomHttpHeaderParser extends HttpHeaderParser {
/**
* Extracts a {@link com.android.volley.Cache.Entry} from a {@link com.android.volley.NetworkResponse}.
*
* @param response The network response to parse headers from
* @param cacheTime 缓存时间,毫秒
* @return a cache entry for the given response, or null if the response is not cacheable.
*/
public static Cache.Entry parseCacheHeaders(NetworkResponse response,long cacheTime) {
Cache.Entry entry=parseCacheHeaders(response);
if(entry == null){
entry = new Cache.Entry();
}
long now = System.currentTimeMillis();
long softExpire=now+cacheTime;
entry.softTtl = softExpire;
entry.ttl = entry.softTtl;
entry.data = response.data;
return entry;
}
}
使用CustomHttpHeaderParser。
public class StringCacheRequest extends StringRequest {
public StringCacheRequest(int method, String url, Response.Listener<String> listener, Response.ErrorListener errorListener) {
super(method, url, listener, errorListener);
}
public StringCacheRequest(String url, Response.Listener<String> listener, Response.ErrorListener errorListener) {
super(url, listener, errorListener);
}
@Override
protected Response<String> parseNetworkResponse(NetworkResponse response) {
String parsed;
try {
parsed = new String(response.data, HttpHeaderParser.parseCharset(response.headers));
} catch (UnsupportedEncodingException e) {
parsed = new String(response.data);
}
return Response.success(parsed, CustomHttpHeaderParser.parseCacheHeaders(response,10000));
}
}
请求一次网络查看log信息,
看到已经写入缓存了,再调一次
11-15 22:24:46.771 21714-21714/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (16 ms) [ ] http://www.baidu.com 0x101be397 NORMAL 1
11-15 22:24:46.772 21714-21714/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+0 ) [ 1] baidu-look
11-15 22:24:46.775 21714-21714/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+0 ) [ 1] add-to-queue
11-15 22:24:46.777 21714-21714/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+7 ) [4069] cache-queue-take
11-15 22:24:46.779 21714-21714/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+3 ) [4069] cache-hit
11-15 22:24:46.783 21714-21714/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+1 ) [4069] cache-hit-parsed
11-15 22:24:46.787 21714-21714/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+0 ) [4069] post-response
11-15 22:24:46.788 21714-21714/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+5 ) [ 1] done
11-15 22:24:46.772 21714-21714/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+0 ) [ 1] baidu-look
11-15 22:24:46.775 21714-21714/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+0 ) [ 1] add-to-queue
11-15 22:24:46.777 21714-21714/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+7 ) [4069] cache-queue-take
11-15 22:24:46.779 21714-21714/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+3 ) [4069] cache-hit
11-15 22:24:46.783 21714-21714/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+1 ) [4069] cache-hit-parsed
11-15 22:24:46.787 21714-21714/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+0 ) [4069] post-response
11-15 22:24:46.788 21714-21714/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+5 ) [ 1] done
上面就直接用缓存中的数据了。过了cacheTime又会重新从网络获取数据。
现在有这样一个场景,有缓存数据,并且缓存数据没过期,我想从网络上获取数据该咋办。我们一般想到的就是使缓存数据失效。
使缓存数据失效
在Volley的newRequestQueue(Context context, HttpStack stack, int maxDiskCacheBytes)中找到
if (maxDiskCacheBytes <= -1)
{
// No maximum size specified
queue = new RequestQueue(new DiskBasedCache(cacheDir), network);
}
else
{
// Disk cache size specified
queue = new RequestQueue(new DiskBasedCache(cacheDir, maxDiskCacheBytes), network);
}
这里可看到缓存的实现就是DiskBasedCache。在这个类中,有着这样一个方法:
/**
* Invalidates an entry in the cache.
* @param key Cache key
* @param fullExpire True to fully expire the entry, false to soft expire
*/
@Override
public synchronized void invalidate(String key, boolean fullExpire) {
Entry entry = get(key);
if (entry != null) {
entry.softTtl = 0;
if (fullExpire) {
entry.ttl = 0;
}
put(key, entry);
}
}
这个方法其实就是使url对应的缓存失效。这个方法的参数key就是Request.getCacheKey()返回的值。第二个参数为false时表示软过期,一次网络请求会先读取缓存中的数据显示,然后读取网络上的数据显示。log示例:
11-15 22:45:15.328 23488-23488/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (625 ms) [ ] http://www.baidu.com 0x101be397 NORMAL 1
11-15 22:45:15.329 23488-23488/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+0 ) [ 1] baidu-look
11-15 22:45:15.330 23488-23488/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+0 ) [ 1] add-to-queue
11-15 22:45:15.331 23488-23488/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+0 ) [4139] cache-queue-take
11-15 22:45:15.332 23488-23488/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+17 ) [4139] cache-hit
11-15 22:45:15.333 23488-23488/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+0 ) [4139] cache-hit-parsed
11-15 22:45:15.334 23488-23488/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+0 ) [4139] cache-hit-refresh-needed
11-15 22:45:15.336 23488-23488/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+0 ) [4139] post-response
11-15 22:45:15.338 23488-23488/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+282 ) [ 1] intermediate-response
11-15 22:45:15.339 23488-23488/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+0 ) [4141] network-queue-take
11-15 22:45:15.341 23488-23488/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+117 ) [4141] network-http-complete
11-15 22:45:15.343 23488-23488/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+3 ) [4141] network-parse-complete
11-15 22:45:15.344 23488-23488/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+1 ) [4141] network-cache-written
11-15 22:45:15.345 23488-23488/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+0 ) [4141] post-response
11-15 22:45:15.346 23488-23488/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+205 ) [ 1] done
从这里面可以看到有一次intermediate-response中间响应。
11-15 22:45:15.329 23488-23488/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+0 ) [ 1] baidu-look
11-15 22:45:15.330 23488-23488/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+0 ) [ 1] add-to-queue
11-15 22:45:15.331 23488-23488/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+0 ) [4139] cache-queue-take
11-15 22:45:15.332 23488-23488/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+17 ) [4139] cache-hit
11-15 22:45:15.333 23488-23488/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+0 ) [4139] cache-hit-parsed
11-15 22:45:15.334 23488-23488/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+0 ) [4139] cache-hit-refresh-needed
11-15 22:45:15.336 23488-23488/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+0 ) [4139] post-response
11-15 22:45:15.338 23488-23488/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+282 ) [ 1] intermediate-response
11-15 22:45:15.339 23488-23488/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+0 ) [4141] network-queue-take
11-15 22:45:15.341 23488-23488/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+117 ) [4141] network-http-complete
11-15 22:45:15.343 23488-23488/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+3 ) [4141] network-parse-complete
11-15 22:45:15.344 23488-23488/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+1 ) [4141] network-cache-written
11-15 22:45:15.345 23488-23488/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+0 ) [4141] post-response
11-15 22:45:15.346 23488-23488/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+205 ) [ 1] done
从这里面可以看到有一次intermediate-response中间响应。
第二个参数为true时表示硬过期,没有中间响应,log如下:
11-15 22:48:43.735 23876-23876/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (364 ms) [ ] http://www.baidu.com 0x101be397 NORMAL 1
11-15 22:48:43.736 23876-23876/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+0 ) [ 1] baidu-look
11-15 22:48:43.737 23876-23876/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+0 ) [ 1] add-to-queue
11-15 22:48:43.737 23876-23876/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+2 ) [4153] cache-queue-take
11-15 22:48:43.738 23876-23876/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+13 ) [4153] cache-hit-expired
11-15 22:48:43.739 23876-23876/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+2 ) [4154] network-queue-take
11-15 22:48:43.739 23876-23876/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+120 ) [4154] network-http-complete
11-15 22:48:43.741 23876-23876/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+3 ) [4154] network-parse-complete
11-15 22:48:43.742 23876-23876/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+1 ) [4154] network-cache-written
11-15 22:48:43.742 23876-23876/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+0 ) [4154] post-response
11-15 22:48:43.744 23876-23876/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+223 ) [ 1] done
11-15 22:48:43.736 23876-23876/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+0 ) [ 1] baidu-look
11-15 22:48:43.737 23876-23876/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+0 ) [ 1] add-to-queue
11-15 22:48:43.737 23876-23876/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+2 ) [4153] cache-queue-take
11-15 22:48:43.738 23876-23876/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+13 ) [4153] cache-hit-expired
11-15 22:48:43.739 23876-23876/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+2 ) [4154] network-queue-take
11-15 22:48:43.739 23876-23876/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+120 ) [4154] network-http-complete
11-15 22:48:43.741 23876-23876/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+3 ) [4154] network-parse-complete
11-15 22:48:43.742 23876-23876/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+1 ) [4154] network-cache-written
11-15 22:48:43.742 23876-23876/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+0 ) [4154] post-response
11-15 22:48:43.744 23876-23876/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+223 ) [ 1] done
DiskBasedCache中还有另一个方法clear(),从字面意思看都知道是清空缓存了。
源码:http://yunpan.cn/cL6d4SKdCHZ9q (提取码:ab40)