探秘volley缓存网络数据

前言

众所周知，浏览器与服务器交互时可对响应的数据进行缓存，这极大的节省了用户的流量和提高了网站的响应性。对于app而言，理应做响应数据缓存。

实现方式

就我目前所知，主要是两种方式：以文件方式缓存和以存数据库方式缓存。

以文件方式缓存，笔记认为有研究意义的当属volley和开源中国源码。以数据库方式笔者目前没碰到，在这里就不说了。这篇主要介绍volley的缓存响应数据机制。

知识准备

了解http协议，javaweb。不会的没关系，我会做简单的说明。

拿tomcat服务器来说，tomcat缓存策略如下图：

客户端发起一次请求，tomcat服务器进行响应。响应头携带Last-Modified，其表示最后的修改时间，客户端发起重复请求时，请求头携带If-Modified-Since，其值就是上次响应中的Last-Modified。服务器收到该请求后，将文件最后修改时间与If-Modified-Since对应的时间进行比对，如果一样表示服务器没有修改过该文件，则304响应，客户端就去读取缓存，如果不一样就响应200，将最新的Last-Modified放到响应头里，客户端读取服务器数据。

If-None-Match与Etag是一对，跟If-Modified-Since与Last-Modified一样。

Volley缓存网络数据详解

从解析响应数据说起，

在NetworkDispatcher类的run（）方法中，找到

 <pre name="code" class="java">// Parse the response here on the worker thread.
                Response<?> response = request.parseNetworkResponse(networkResponse);
                request.addMarker("network-parse-complete");

                // Write to cache if applicable.写入缓存
                // TODO: Only update cache metadata instead of entire record for 304s.
                if (request.shouldCache() && response.cacheEntry != null) {
                    mCache.put(request.getCacheKey(), response.cacheEntry);
                    request.addMarker("network-cache-written");
                }

其第二行处理网络响应，由于Request是抽象类，我们具体拿StringRequest来说,

@Override
    protected Response<String> parseNetworkResponse(NetworkResponse response) {
        String parsed;
        try {
            parsed = new String(response.data, HttpHeaderParser.parseCharset(response.headers));
        } catch (UnsupportedEncodingException e) {
            parsed = new String(response.data);
        }
        return Response.success(parsed, HttpHeaderParser.parseCacheHeaders(response));
    }

关键就是看HttpHeaderParser.parseCacheHeaders(response)。

/**
     * Extracts a {@link Cache.Entry} from a {@link NetworkResponse}.
     *
     * @param response The network response to parse headers from
     * @return a cache entry for the given response, or null if the response is not cacheable.
     */
    public static Cache.Entry parseCacheHeaders(NetworkResponse response) {
        long now = System.currentTimeMillis();

        Map<String, String> headers = response.headers;

        long serverDate = 0;
        long lastModified = 0;
        long serverExpires = 0;
        long softExpire = 0;
        long finalExpire = 0;
        long maxAge = 0;
        long staleWhileRevalidate = 0;
        boolean hasCacheControl = false;
        boolean mustRevalidate = false;

        String serverEtag = null;
        String headerValue;

        headerValue = headers.get("Date");
        if (headerValue != null) {
            serverDate = parseDateAsEpoch(headerValue);
        }

        headerValue = headers.get("Cache-Control");
        if (headerValue != null) {
            hasCacheControl = true;
            String[] tokens = headerValue.split(",");
            for (int i = 0; i < tokens.length; i++) {
                String token = tokens[i].trim();
                if (token.equals("no-cache") || token.equals("no-store")) {
                    return null;
                } else if (token.startsWith("max-age=")) {
                    try {
                        maxAge = Long.parseLong(token.substring(8));
                    } catch (Exception e) {
                    }
                } else if (token.startsWith("stale-while-revalidate=")) {
                    try {
                        staleWhileRevalidate = Long.parseLong(token.substring(23));
                    } catch (Exception e) {
                    }
                } else if (token.equals("must-revalidate") || token.equals("proxy-revalidate")) {
                    mustRevalidate = true;
                }
            }
        }

        headerValue = headers.get("Expires");
        if (headerValue != null) {
            serverExpires = parseDateAsEpoch(headerValue);
        }

        headerValue = headers.get("Last-Modified");
        if (headerValue != null) {
            lastModified = parseDateAsEpoch(headerValue);
        }

        serverEtag = headers.get("ETag");

        // Cache-Control takes precedence over an Expires header, even if both exist and Expires
        // is more restrictive.
        if (hasCacheControl) {
            softExpire = now + maxAge * 1000;
            finalExpire = mustRevalidate
                    ? softExpire
                    : softExpire + staleWhileRevalidate * 1000;
        } else if (serverDate > 0 && serverExpires >= serverDate) {
            // Default semantic for Expire header in HTTP specification is softExpire.
            softExpire = now + (serverExpires - serverDate);
            finalExpire = softExpire;
        }

        Cache.Entry entry = new Cache.Entry();
        entry.data = response.data;
        entry.etag = serverEtag;
        entry.softTtl = softExpire;
        entry.ttl = finalExpire;
        entry.serverDate = serverDate;
        entry.lastModified = lastModified;
        entry.responseHeaders = headers;

        return entry;
    }

其具体是对http响应头进行分析并一个缓存实体Cache.Entry,将其返回。上面的softTtl是该响应数据对应缓存的软生存时间，ttl是实际生存时间。

从请求头这方来看

在NetworkDispatcher类的run（）方法中，找到

// Perform the network request.
                NetworkResponse networkResponse = mNetwork.performRequest(request);

mNetwork是在Volley的newRequestQueue(Context context, HttpStack stack, int maxDiskCacheBytes)方法中创建的。也就是BasicNetwork，我们看它的performRequest（）方法，其关键代码：

<pre name="code" class="java">try {
                // Gather headers.
                Map<String, String> headers = new HashMap<String, String>();
                addCacheHeaders(headers, request.getCacheEntry());<span style="white-space:pre">	</span>//这里从缓存数据中解析出请求头数据
                httpResponse = mHttpStack.performRequest(request, headers);
                StatusLine statusLine = httpResponse.getStatusLine();
                int statusCode = statusLine.getStatusCode();

往addCacheHeaders()看：

private void addCacheHeaders(Map<String, String> headers, Cache.Entry entry) {
        // If there's no cache entry, we're done.
        if (entry == null) {
            return;
        }

        if (entry.etag != null) {
            headers.put("If-None-Match", entry.etag);
        }

        if (entry.lastModified > 0) {
            Date refTime = new Date(entry.lastModified);
            headers.put("If-Modified-Since", DateUtils.formatDate(refTime));
        }
    }

也就是将缓存中etag和lastModified分别放置在请求头里的If-None-Match和If-Modified-Since。

开发中缓存数据的方式

一、由服务器去控制

也就是服务端设置etag、lastModified和Cache-Control等。

二、由安卓开发人员控制

也就在Request的parseNetworkResponse方法中使用自己的HttpHeaderParser。具体例子：

自定义的HttpHeaderParser。

/**
 * 自定义的HeaderParser,跟默认的比，可以强制缓存，忽略服务器的设置
 */
public class CustomHttpHeaderParser extends HttpHeaderParser {
    /**
     * Extracts a {@link com.android.volley.Cache.Entry} from a {@link com.android.volley.NetworkResponse}.
     *
     * @param response The network response to parse headers from
     * @param cacheTime 缓存时间，毫秒
     * @return a cache entry for the given response, or null if the response is not cacheable.
     */
    public static Cache.Entry parseCacheHeaders(NetworkResponse response,long cacheTime) {
        Cache.Entry entry=parseCacheHeaders(response);
        if(entry == null){
            entry = new Cache.Entry();
        }
        long now = System.currentTimeMillis();
        long softExpire=now+cacheTime;
        entry.softTtl = softExpire;
        entry.ttl = entry.softTtl;
        entry.data = response.data;
        return entry;
    }
}

使用CustomHttpHeaderParser。

public class StringCacheRequest extends StringRequest {


    public StringCacheRequest(int method, String url, Response.Listener<String> listener, Response.ErrorListener errorListener) {
        super(method, url, listener, errorListener);
    }

    public StringCacheRequest(String url, Response.Listener<String> listener, Response.ErrorListener errorListener) {
        super(url, listener, errorListener);
    }

    @Override
    protected Response<String> parseNetworkResponse(NetworkResponse response) {
        String parsed;
        try {
            parsed = new String(response.data, HttpHeaderParser.parseCharset(response.headers));
        } catch (UnsupportedEncodingException e) {
            parsed = new String(response.data);
        }
        return Response.success(parsed, CustomHttpHeaderParser.parseCacheHeaders(response,10000));
    }
}

请求一次网络查看log信息，

看到已经写入缓存了，再调一次

11-15 22:24:46.771  21714-21714/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (16   ms) [ ] http://www.baidu.com 0x101be397 NORMAL 1
11-15 22:24:46.772  21714-21714/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+0   ) [ 1] baidu-look
11-15 22:24:46.775  21714-21714/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+0   ) [ 1] add-to-queue
11-15 22:24:46.777  21714-21714/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+7   ) [4069] cache-queue-take
11-15 22:24:46.779  21714-21714/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+3   ) [4069] cache-hit
11-15 22:24:46.783  21714-21714/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+1   ) [4069] cache-hit-parsed
11-15 22:24:46.787  21714-21714/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+0   ) [4069] post-response
11-15 22:24:46.788  21714-21714/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+5   ) [ 1] done

上面就直接用缓存中的数据了。过了cacheTime又会重新从网络获取数据。

现在有这样一个场景，有缓存数据，并且缓存数据没过期，我想从网络上获取数据该咋办。我们一般想到的就是使缓存数据失效。

使缓存数据失效

在Volley的newRequestQueue(Context context, HttpStack stack, int maxDiskCacheBytes)中找到

if (maxDiskCacheBytes <= -1)
        {
        	// No maximum size specified
        	queue = new RequestQueue(new DiskBasedCache(cacheDir), network);
        }
        else
        {
        	// Disk cache size specified
        	queue = new RequestQueue(new DiskBasedCache(cacheDir, maxDiskCacheBytes), network);
        }

这里可看到缓存的实现就是DiskBasedCache。在这个类中，有着这样一个方法：

/**
     * Invalidates an entry in the cache.
     * @param key Cache key
     * @param fullExpire True to fully expire the entry, false to soft expire
     */
    @Override
    public synchronized void invalidate(String key, boolean fullExpire) {
        Entry entry = get(key);
        if (entry != null) {
            entry.softTtl = 0;
            if (fullExpire) {
                entry.ttl = 0;
            }
            put(key, entry);
        }

    }

这个方法其实就是使url对应的缓存失效。这个方法的参数key就是Request.getCacheKey()返回的值。第二个参数为false时表示软过期，一次网络请求会先读取缓存中的数据显示，然后读取网络上的数据显示。log示例：

11-15 22:45:15.328  23488-23488/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (625  ms) [ ] http://www.baidu.com 0x101be397 NORMAL 1
11-15 22:45:15.329  23488-23488/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+0   ) [ 1] baidu-look
11-15 22:45:15.330  23488-23488/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+0   ) [ 1] add-to-queue
11-15 22:45:15.331  23488-23488/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+0   ) [4139] cache-queue-take
11-15 22:45:15.332  23488-23488/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+17  ) [4139] cache-hit
11-15 22:45:15.333  23488-23488/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+0   ) [4139] cache-hit-parsed
11-15 22:45:15.334  23488-23488/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+0   ) [4139] cache-hit-refresh-needed
11-15 22:45:15.336  23488-23488/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+0   ) [4139] post-response
11-15 22:45:15.338  23488-23488/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+282 ) [ 1] intermediate-response
11-15 22:45:15.339  23488-23488/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+0   ) [4141] network-queue-take
11-15 22:45:15.341  23488-23488/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+117 ) [4141] network-http-complete
11-15 22:45:15.343  23488-23488/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+3   ) [4141] network-parse-complete
11-15 22:45:15.344  23488-23488/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+1   ) [4141] network-cache-written
11-15 22:45:15.345  23488-23488/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+0   ) [4141] post-response
11-15 22:45:15.346  23488-23488/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+205 ) [ 1] done
从这里面可以看到有一次intermediate-response中间响应。

第二个参数为true时表示硬过期，没有中间响应，log如下：

11-15 22:48:43.735  23876-23876/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (364  ms) [ ] http://www.baidu.com 0x101be397 NORMAL 1
11-15 22:48:43.736  23876-23876/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+0   ) [ 1] baidu-look
11-15 22:48:43.737  23876-23876/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+0   ) [ 1] add-to-queue
11-15 22:48:43.737  23876-23876/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+2   ) [4153] cache-queue-take
11-15 22:48:43.738  23876-23876/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+13  ) [4153] cache-hit-expired
11-15 22:48:43.739  23876-23876/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+2   ) [4154] network-queue-take
11-15 22:48:43.739  23876-23876/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+120 ) [4154] network-http-complete
11-15 22:48:43.741  23876-23876/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+3   ) [4154] network-parse-complete
11-15 22:48:43.742  23876-23876/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+1   ) [4154] network-cache-written
11-15 22:48:43.742  23876-23876/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+0   ) [4154] post-response
11-15 22:48:43.744  23876-23876/com.example.volleyhttpcache D/Volley﹕ [1] MarkerLog.finish: (+223 ) [ 1] done

DiskBasedCache中还有另一个方法clear（），从字面意思看都知道是清空缓存了。

源码：http://yunpan.cn/cL6d4SKdCHZ9q （提取码：ab40）