深入Volley(一)volley缓存文件结构

最新推荐文章于 2021-05-27 13:12:33 发布

三岁sansui

最新推荐文章于 2021-05-27 13:12:33 发布

阅读量1.7w

点赞数

文章标签： android java volley

本文链接：https://blog.csdn.net/u013178436/article/details/38406103

版权

首先要感谢@非子墨大牛的系列文章,附上地址,如果没有看过volley源码的朋友可以先去看他的系列文章 : <非子墨>的volley文章系列

还是要再感谢下其于我之讨论,希望本篇文章能够如他一般帮助到别人;话不说直接上正文吧;

缓存流程分析:

正所谓工欲善其事必先利其器,要了解volley整个框架的设计以及各个缓存的细节,就一定要读源代码了,那么我们先来简单分析下他的缓存流程:

首先:

我们知道在之前的volley的newRequestQueue()方法中是有启动一个队列,然后队列start就初始化了两个核心的内容:mCacheDispatcher和NetworkDispatcher的数组,当一个request子类请求add进来的时候,他会根据request.shouldCache()来判断是加入mCacheQueue还是mNetworkQueue这两个阻塞队列;而这两个阻塞队列对应的就是mCacheDispatcher,NetworkDispatcher这两个核心类的阻塞式操作,这里的细节我就不多说啦,非子墨都已经说的很详细了;add源码:

public <T> Request<T> add(Request<T> request) {
        // Tag the request as belonging to this queue and add it to the set of current requests.
        request.setRequestQueue(this);
        synchronized (mCurrentRequests) {
            mCurrentRequests.add(request);
        }

        // Process requests in the order they are added.
        request.setSequence(getSequenceNumber());
        request.addMarker("add-to-queue");

        // If the request is uncacheable, skip the cache queue and go straight to the network.
        if (!request.shouldCache()) {
            mNetworkQueue.add(request);
            return request;
        }

        // Insert request into stage if there's already a request with the same cache key in flight.
        synchronized (mWaitingRequests) {
            String cacheKey = request.getCacheKey();
            if (mWaitingRequests.containsKey(cacheKey)) {
                // There is already a request in flight. Queue up.
                Queue<Request<?>> stagedRequests = mWaitingRequests.get(cacheKey);
                if (stagedRequests == null) {
                    stagedRequests = new LinkedList<Request<?>>();
                }
                stagedRequests.add(request);
                mWaitingRequests.put(cacheKey, stagedRequests);
                if (VolleyLog.DEBUG) {
                    VolleyLog.v("Request for cacheKey=%s is in flight, putting on hold.", cacheKey);
                }
            } else {
                // Insert 'null' queue for this cacheKey, indicating there is now a request in
                // flight.
                mWaitingRequests.put(cacheKey, null);
                mCacheQueue.add(request);
            }
            return request;
        }
    }

但可能还是要补充一点的就是 : 在队列中的两个数据结构--mCurrentRequests和mWaitingRequests用法

可以看到当队列刚初始化的时候他就已经被new好了,他们主要职能是用来标记当前队列中的具体请求,举个实际例子:如果我请求发到一半还在队列中没有被消费掉的话,那么我突然想要取消掉他的话该怎么办呢？那只能通过mCurrentRequests和mWaitingRequests了

在add()方法中进来一个request,其会根据shouldCache字段判断是否加入mWaitingRequests,而在finish()方法中也会根据这个字段来判断是否从mWaitingRequests中移除,我们查看下finish()方法: 从api可以看到他调用的时间是当request调用finish的时候,也就是request被消耗的时候

    /**
     * Called from {@link Request#finish(String)}, indicating that processing of the given request
     * has finished.
     *
     * <p>Releases waiting requests for <code>request.getCacheKey()</code> if
     *      <code>request.shouldCache()</code>.</p>
     */
    void finish(Request<?> request) {
        // Remove from the set of requests currently being processed.
        synchronized (mCurrentRequests) {
            mCurrentRequests.remove(request);
        }

        if (request.shouldCache()) {
            synchronized (mWaitingRequests) {
                String cacheKey = request.getCacheKey();
                Queue<Request<?>> waitingRequests = mWaitingRequests.remove(cacheKey);
                if (waitingRequests != null) {
                    if (VolleyLog.DEBUG) {
                        VolleyLog.v("Releasing %d waiting requests for cacheKey=%s.",
                                waitingRequests.size(), cacheKey);
                    }
                    // Process all queued up requests. They won't be considered as in flight, but
                    // that's not a problem as the cache has been primed by 'request'.
                    mCacheQueue.addAll(waitingRequests);
                }
            }
        }
    }

那么在中途还在队列没有被消耗的时候如何取消掉request呢

    /**
     * A simple predicate or filter interface for Requests, for use by
     * {@link RequestQueue#cancelAll(RequestFilter)}.
     */
    public interface RequestFilter {
        public boolean apply(Request<?> request);
    }

    /**
     * Cancels all requests in this queue for which the given filter applies.
     * @param filter The filtering function to use
     */
    public void cancelAll(RequestFilter filter) {
        synchronized (mCurrentRequests) {
            for (Request<?> request : mCurrentRequests) {
                if (filter.apply(request)) {
                    request.cancel();
                }
            }
        }
    }

    /**
     * Cancels all requests in this queue with the given tag. Tag must be non-null
     * and equality is by identity.
     */
    public void cancelAll(final Object tag) {
        if (tag == null) {
            throw new IllegalArgumentException("Cannot cancelAll with a null tag");
        }
        cancelAll(new RequestFilter() {
            @Override
            public boolean apply(Request<?> request) {
                return request.getTag() == tag;
            }
        });
    }

这里直接调用重载的cancelAll方法就好了,反正他也是调用cancel方法就不需要考虑是否需要缓存的问题了;

还有一点大家要注意的是:善用锁在整个队列里面是十分重要的,可以看到当操作两个容器类的时候volley都加上了一把锁,这是因为Java里面没有锁用这些容器类是十分容易出错的,所以当大家在封装自己的框架的时候一定要善用java锁机制,锁用好了不但可以使得容器类操作保证了原子性和线程可见性并且还可以对线程做很棒的处理,具体的建议大家可以看看wait notify ...等方法和参照《java并发实战这本书》;但如果读者是在不了解他那么直接在方法上加入一个synchronized修饰符也是不错的选择;

其次：

上面我们说到了其根据shouldCache字段是否进行缓存,既然本章主要讲缓存文件结构(上面讲的有点多,只是想在细节上补充下非子墨大牛的文章,让大家更好理解),那么我们直接看mNetworkQueue如何对其做缓存的吧,注意这里只有在shouldCache字段是true的条件下mCacheQueue才会去从本地看看其是否有缓存以及mNetworkQueue 在请求之后对其进行缓存,话不多说,看下代码

    @Override
    public void run() {
        Process.setThreadPriority(Process.THREAD_PRIORITY_BACKGROUND);
        Request<?> request;
        while (true) {
            try {
                // Take a request from the queue.
                request = mQueue.take();
            } catch (InterruptedException e) {
                // We may have been interrupted because it was time to quit.
                if (mQuit) {
                    return;
                }
                continue;
            }

            try {
                request.addMarker("network-queue-take");

                // If the request was cancelled already, do not perform the
                // network request.
                if (request.isCanceled()) {
                    request.finish("network-discard-cancelled");
                    continue;
                }

                addTrafficStatsTag(request);

                // Perform the network request.
                NetworkResponse networkResponse = mNetwork.performRequest(request);
                request.addMarker("network-http-complete");

                // If the server returned 304 AND we delivered a response already,
                // we're done -- don't deliver a second identical response.
                if (networkResponse.notModified && request.hasHadResponseDelivered()) {
                    request.finish("not-modified");
                    continue;
                }

                // Parse the response here on the worker thread.
                Response<?> response = request.parseNetworkResponse(networkResponse);
                request.addMarker("network-parse-complete");

                // Write to cache if applicable.
                // TODO: Only update cache metadata instead of entire record for 304s.
                if (request.shouldCache() && response.cacheEntry != null) {
                    mCache.put(request.getCacheKey(), response.cacheEntry);
                    request.addMarker("network-cache-written");
                }

                // Post the response back.
                request.markDelivered();
                mDelivery.postResponse(request, response);
            } catch (VolleyError volleyError) {
                parseAndDeliverNetworkError(request, volleyError);
            } catch (Exception e) {
                VolleyLog.e(e, "Unhandled exception %s", e.toString());
                mDelivery.postError(request, new VolleyError(e));
            }
        }

从这里看到我们的mNetworkQueue将request先调用performRequest方法将他转化成netWorkResponce,之后再调用

mCache.put(request.getCacheKey(),response.cacheEntry);将其缓存

最后：

mCache.put进行缓存,实则为其子类DishBaseCache进行操作:传入的操作一个是request.getCacheKey()这个就直接是请求的url,另一个是Entiy这个是什么呢,看代码:

    /**
     * Data and metadata for an entry returned by the cache.
     */
    public static class Entry {
        /** The data returned from cache. */
        public byte[] data;

        /** ETag for cache coherency. */
        public String etag;

        /** Date of this response as reported by the server. */
        public long serverDate;

        /** TTL for this record. */
        public long ttl;

        /** Soft TTL for this record. */
        public long softTtl;

        /** Immutable response headers as received from server; must be non-null. */
        public Map<String, String> responseHeaders = Collections.emptyMap();

        /** True if the entry is expired. */
        public boolean isExpired() {
            return this.ttl < System.currentTimeMillis();
        }

        /** True if a refresh is needed from the original data source. */
        public boolean refreshNeeded() {
            return this.softTtl < System.currentTimeMillis();
        }
    }

堆堆数据类型,但是不要管他,因为在put缓存过程中已经将url和ectity进行合并了

合并调用的是CacheHeader e = new CacheHeader(key, entry);这个东西

代码:

/** The size of the data identified by this CacheHeader. (This is not
         * serialized to disk. */
        public long size;

        /** The key that identifies the cache entry. */
        public String key;

        /** ETag for cache coherence. */
        public String etag;

        /** Date of this response as reported by the server. */
        public long serverDate;

        /** TTL for this record. */
        public long ttl;

        /** Soft TTL for this record. */
        public long softTtl;

        /** Headers from the response resulting in this cache entry. */
        public Map<String, String> responseHeaders;

CacheHeader 类的其他方法先不管我们先看数据类型:

key ----就是request url

etag ---和缓存有关的response数据类型

ttl和softttl ---和缓存有关的response数据类型

responseHeaders -- 一些response头

serverDate -- 服务器时间

size -- response数据的长度

缓存文件分析:

上面我们分析了大致缓存流程,已经一些启动细节,现在正片开始,进行了实际缓存流程:

缓存FileName产生:

可以直观看出从这段代码产生 File file = getFileForKey(key); 而getFileForKey里面调用了getFileNameForKey(String url)得到fileName

getFileForKey代码:

    private String getFilenameForKey(String key) {
        int firstHalfLength = key.length() / 2;
        String localFilename = String.valueOf(key.substring(0, firstHalfLength).hashCode());
        localFilename += String.valueOf(key.substring(firstHalfLength).hashCode());
        return localFilename;
    }

其实这个过程很简单就是通过URL转化成一串字符串；没啥好说的,看官们只需要自己复制这个方法然后调用试试就好

内容开始写入代码:

<span style="white-space:pre">	</span>/**
         * Writes the contents of this CacheHeader to the specified OutputStream.
         */
        public boolean writeHeader(OutputStream os) {
            try {
                writeInt(os, CACHE_MAGIC);
                writeString(os, key);
                writeString(os, etag == null ? "" : etag);
                writeLong(os, serverDate);
                writeLong(os, ttl);
                writeLong(os, softTtl);
                writeStringStringMap(responseHeaders, os);
                os.flush();
                return true;
            } catch (IOException e) {
                VolleyLog.d("%s", e.toString());
                return false;
            }
        }

魔数Magic:

讲到魔数就不得不提一下jvm的魔数了,略微了解class文件的朋友都知道魔数是一个很有趣的东西（不清楚的朋友执行谷歌一下）,而jvm的魔数是cafebabe,那么volley的

魔数是 20120504这个数,

 /** Magic number for current version of cache file format. */
    private static final int CACHE_MAGIC = 0x20120504;

但是真的有那么简单吗？

我们用winhex打开一个缓存文件查看他的魔数

这个魔数为什么是倒着的呢？理由很简单,其写魔数是调用了WriterInt方法：

    static void writeInt(OutputStream os, int n) throws IOException {
        os.write((n >> 0) & 0xff);
        os.write((n >> 8) & 0xff);
        os.write((n >> 16) & 0xff);
        os.write((n >> 24) & 0xff);
    }

request Key的写入(Url)

该内容的写入调用的是writerString方法,代码：

    static void writeString(OutputStream os, String s) throws IOException {
        byte[] b = s.getBytes("UTF-8");
        writeLong(os, b.length);
        os.write(b, 0, b.length);
    }

他是先有调用一个writeLong用来表示字符串长度,这个方法和writeInt大致是一样的,都是让其颠倒存入;

蓝色的前八个直接是LONG类型数据用来表示字符串长度;后面是具体的字符串,在这里他是url

Etag、serverDate、ttl、softTtl字段:

这些字段就是和具体怎么缓存有关了,本例先只分析缓存文件结构,再之后的系列文章我会具体的将这几个字段单独成文,从服务器端和客户端来解析他们的用途;

本文例子是设置了服务器端无缓存类型也就是这些都是一些无关数据,但是不影响我们分析文件

看代码可以看到我们写入1个string,3个long则至少需要32字节空间,而且这里的确是32字节因为我设置服务器无缓存则使得string为空

这里和前面分析其实差不多,但只要注意一点:在第二个八位字节表示的serverData它是Unix时间戳,也和java里面的时间有关;

responseHead字段

写入该字段是通过writeStringStringMap写入的

代码:

    static void writeStringStringMap(Map<String, String> map, OutputStream os) throws IOException {
        if (map != null) {
            writeInt(os, map.size());
            for (Map.Entry<String, String> entry : map.entrySet()) {
                writeString(os, entry.getKey());
                writeString(os, entry.getValue());
            }
        } else {
            writeInt(os, 0);
        }
    }

他先写了一个Int进来用来标识相应头的个数,然后就是一堆具体内容进去了