1. 原理
DisKLruCache是一个用于硬盘存储的cache的类,它不是Android API,其GitHub地址为【https://github.com/JakeWharton/DiskLruCache】,也有些博客对其进行了介绍【Android DiskLruCache完全解析,硬盘缓存的最佳方案:http://blog.csdn.net/guolin_blog/article/details/28863651】。与LruCache类似,实现依赖于LinkedHashMap<String,Entry> lruEntries。lruEntries保存了键值对key-value,其中key是String类型,value是字节序列,字节序列最大为Integer.MAX_VALUE。
DisKLruCache本质上是将每个value保存到对应的key生成的文件名的文件中,方便本地索引。一旦总文件超过设定的缓存大小,就删除队头文件。
DisKLruCache本身不是线程安全的,需要额外的线程安全处理。
2. 源码分析
2.1初始化
先来看一下成员变量。
private final File directory; //路径
private final File journalFile; //日志文件
private final File journalFileTmp; //临时日志文件
private final int appVersion; //app的版本
private final long maxSize; //缓存的最大容量
private final int valueCount; //1个key对应多少个缓存文件
private long size = 0;
private Writer journalWriter; //日志文件写入
private final LinkedHashMap<String, Entry> lruEntries
= new LinkedHashMap<String, Entry>(0, 0.75f, true); //缓存队列
private int redundantOpCount;
/**
* To differentiate between old and current snapshots, each entry is given
* a sequence number each time an edit is committed. A snapshot is stale if
* its sequence number is not equal to its entry's sequence number.
*/
private long nextSequenceNumber = 0
DisKLruCache采用静态工厂的模式进行初始化。如果日志文件存在,就从文件中读取相关信息,进一步完善DisKLruCache对象;否则就直接初始化DisKLruCache对象,并且重建日志文件。
private DiskLruCache(File directory, int appVersion, int valueCount, long maxSize) {
this.directory = directory;
this.appVersion = appVersion;
this.journalFile = new File(directory, JOURNAL_FILE);
this.journalFileTmp = new File(directory, JOURNAL_FILE_TMP);
this.valueCount = valueCount;
this.maxSize = maxSize;
}
/**
* Opens the cache in {@code directory}, creating a cache if none exists
* there.
*
* @param directory a writable directory
* @param appVersion
* @param valueCount the number of values per cache entry. Must be positive.
* @param maxSize the maximum number of bytes this cache should use to store
* @throws java.io.IOException if reading or writing the cache directory fails
*/
public static DiskLruCache open(File directory, int appVersion, int valueCount, long maxSize)
throws IOException {
if (maxSize <= 0) {
throw new IllegalArgumentException("maxSize <= 0");
}
if (valueCount <= 0) {
throw new IllegalArgumentException("valueCount <= 0");
}
// prefer to pick up where we left off
DiskLruCache cache = new DiskLruCache(directory, appVersion, valueCount, maxSize);
if (cache.journalFile.exists()) {//日志文件存在
try {
cache.readJournal(); //读取日志文件
cache.processJournal();// 处理日志文件
cache.journalWriter = new BufferedWriter(new FileWriter(cache.journalFile, true),
IO_BUFFER_SIZE);
return cache;
} catch (IOException journalIsCorrupt) {
// System.logW("DiskLruCache " + directory + " is corrupt: "
// + journalIsCorrupt.getMessage() + ", removing");
cache.delete();
}
}
// create a new empty cache
directory.mkdirs();
cache = new DiskLruCache(directory, appVersion, valueCount, maxSize);
cache.rebuildJournal(); //日志文件不存在,就重建日志文件
return cache;
}
第一次使用需要新建日志文件,日志只写入了固定的前5行,稍后会解释该日志文件。
/**
* Creates a new journal that omits redundant information. This replaces the
* current journal if it exists.
*/
private synchronized void rebuildJournal() throws IOException {
if (journalWriter != null) {
journalWriter.close();
}
Writer writer = new BufferedWriter(new FileWriter(journalFileTmp), IO_BUFFER_SIZE);
writer.write(MAGIC);
writer.write("\n");
writer.write(VERSION_1);
writer.write("\n");
writer.write(Integer.toString(appVersion));
writer.write("\n");
writer.write(Integer.toString(valueCount));
writer.write("\n");
writer.write("\n");
for (Entry entry : lruEntries.values()) {
if (entry.currentEditor != null) {
writer.write(DIRTY + ' ' + entry.key + '\n');
} else {
writer.write(CLEAN + ' ' + entry.key + entry.getLengths() + '\n');
}
}
writer.close();
journalFileTmp.renameTo(journalFile);
journalWriter = new BufferedWriter(new FileWriter(journalFile, true), IO_BUFFER_SIZE);
}
2.2 Key-Value入队
Value被保存到本地文件中,留在内存中充当Value角色的是该文件的文件名。这里将value的文件名以静态内部类的形式封装成Entry。Entry接收key为成员变量,并提供了File getCleanFile(int i)为文件的返回值,而File getDirtyFile(inti)返回了临时文件。
private final class Entry {
private final String key;
/** Lengths of this entry's files. */
private final long[] lengths;
/** True if this entry has ever been published */
private boolean readable;
/** The ongoing edit or null if this entry is not being edited. */
private Editor currentEditor;
/** The sequence number of the most recently committed edit to this entry. */
private long sequenceNumber;
private Entry(String key) {
this.key = key;
this.lengths = new long[valueCount];
}
public String getLengths() throws IOException {
StringBuilder result = new StringBuilder();
for (long size : lengths) {
result.append(' ').append(size);
}
return result.toString();
}
/**
* Set lengths using decimal numbers like "10123".
*/
private void setLengths(String[] strings) throws IOException {
if (strings.length != valueCount) {
throw invalidLengths(strings);
}
try {
for (int i = 0; i < strings.length; i++) {
lengths[i] = Long.parseLong(strings[i]);
}
} catch (NumberFormatException e) {
throw invalidLengths(strings);
}
}
private IOException invalidLengths(String[] strings) throws IOException {
throw new IOException("unexpected journal line: " + Arrays.toString(strings));
}
public File getCleanFile(int i) {
return new File(directory, key + "." + i);
}
public File getDirtyFile(int i) {
return new File(directory, key + "." + i + ".tmp");
}
}
然后,另一个静态内部类Editor封装了Entry,提供队Entry的编辑功能,并向外提供了Entry对应文件的输入输出流对象,入队时,调用了Editor的commit()函数。需要说明的一点,Editor中又有另一个静态内部类FaultHidingOutputStream,用来封装输出流,这里返回的临时文件的输出流。
/**
* Edits the values for an entry.
*/
public final class Editor {
private final Entry entry;
private boolean hasErrors;
private Editor(Entry entry) {
this.entry = entry;
}
/**
* Returns an unbuffered input stream to read the last committed value,
* or null if no value has been committed.
*/
public InputStream newInputStream(int index) throws IOException {
synchronized (DiskLruCache.this) {
if (entry.currentEditor != this) {
throw new IllegalStateException();
}
if (!entry.readable) {
return null;
}
return new FileInputStream(entry.getCleanFile(index));
}
}
/**
* Returns the last committed value as a string, or null if no value
* has been committed.
*/
public String getString(int index) throws IOException {
InputStream in = newInputStream(index);
return in != null ? inputStreamToString(in) : null;
}
/**
* Returns a new unbuffered output stream to write the value at
* {@code index}. If the underlying output stream encounters errors
* when writing to the filesystem, this edit will be aborted when
* {@link #commit} is called. The returned output stream does not throw
* IOExceptions.
*/
public OutputStream newOutputStream(int index) throws IOException {
synchronized (DiskLruCache.this) {
if (entry.currentEditor != this) {
throw new IllegalStateException();
}
return new FaultHidingOutputStream(new FileOutputStream(entry.getDirtyFile(index)));
}
}
/**
* Sets the value at {@code index} to {@code value}.
*/
public void set(int index, String value) throws IOException {
Writer writer = null;
try {
writer = new OutputStreamWriter(newOutputStream(index), UTF_8);
writer.write(value);
} finally {
closeQuietly(writer);
}
}
/**
* Commits this edit so it is visible to readers. This releases the
* edit lock so another edit may be started on the same key.
*/
public void commit() throws IOException {
if (hasErrors) {
completeEdit(this, false);
remove(entry.key); // the previous entry is stale
} else {
completeEdit(this, true);
}
}
/**
* Aborts this edit. This releases the edit lock so another edit may be
* started on the same key.
*/
public void abort() throws IOException {
completeEdit(this, false);
}
private class FaultHidingOutputStream extends FilterOutputStream {
private FaultHidingOutputStream(OutputStream out) {
super(out);
}
@Override public void write(int oneByte) {
try {
out.write(oneByte);
} catch (IOException e) {
hasErrors = true;
}
}
@Override public void write(byte[] buffer, int offset, int length) {
try {
out.write(buffer, offset, length);
} catch (IOException e) {
hasErrors = true;
}
}
@Override public void close() {
try {
out.close();
} catch (IOException e) {
hasErrors = true;
}
}
@Override public void flush() {
try {
out.flush();
} catch (IOException e) {
hasErrors = true;
}
}
}
}
入队时,commit()函数,调用了completeEdit(Editoreditor, boolean success)函数。该函数将临时文件更名为正式文件,并调整缓冲区大小。
private synchronized void completeEdit(Editor editor, boolean success) throws IOException {
Entry entry = editor.entry;
if (entry.currentEditor != editor) {
throw new IllegalStateException();
}
// if this edit is creating the entry for the first time, every index must have a value
if (success && !entry.readable) {
for (int i = 0; i < valueCount; i++) {
if (!entry.getDirtyFile(i).exists()) {
editor.abort();
throw new IllegalStateException("edit didn't create file " + i);
}
}
}
for (int i = 0; i < valueCount; i++) {
File dirty = entry.getDirtyFile(i);
if (success) {
if (dirty.exists()) {
File clean = entry.getCleanFile(i);
dirty.renameTo(clean);
long oldLength = entry.lengths[i];
long newLength = clean.length();
entry.lengths[i] = newLength;
size = size - oldLength + newLength;
}
} else {
deleteIfExists(dirty);
}
}
redundantOpCount++;
entry.currentEditor = null;
if (entry.readable | success) {
entry.readable = true;
journalWriter.write(CLEAN + ' ' + entry.key + entry.getLengths() + '\n');
if (success) {
entry.sequenceNumber = nextSequenceNumber++;
}
} else {
lruEntries.remove(entry.key);
journalWriter.write(REMOVE + ' ' + entry.key + '\n');
}
if (size > maxSize || journalRebuildRequired()) {
executorService.submit(cleanupCallable);
}
}
调整缓冲区大小是通过仔细线程池中一个线程来调用trimToSize()函数的。
private void trimToSize() throws IOException {
while (size > maxSize) {
// Map.Entry<String, Entry> toEvict = lruEntries.eldest();
final Map.Entry<String, Entry> toEvict = lruEntries.entrySet().iterator().next();
remove(toEvict.getKey());
}
}
2.3 日志文件
Cache目录下有一个文件名称是journal,前5行是固定的,已经标注在代码中: * libcore.io.DiskLruCache //固定字符串
* 1 //缓存的版本号
* 100 //存储的key-value个数
* 2 //应用的版本号
* //空行
* CLEAN 3400330d1dfc7f3f7f4b8d4d803dfcf6 832 21054
* DIRTY 335c4c6028171cfddfbaae1a9c313c52
//有效建立或更新的Key-Value的Key,更新包括删除
* CLEAN 335c4c6028171cfddfbaae1a9c313c52 3934 2342
* REMOVE 335c4c6028171cfddfbaae1a9c313c52 //最新被删除的Key-Value的Key
* DIRTY 1ab96a171faeeee38496d8b330771a7a
* CLEAN 1ab96a171faeeee38496d8b330771a7a 1600 234
* READ 335c4c6028171cfddfbaae1a9c313c52 //最不常使用的Key-Value的Key
* READ 3400330d1dfc7f3f7f4b8d4d803dfcf
日志前5行的读取是通过下面函数进行的。
private void readJournal() throws IOException {
InputStream in = new BufferedInputStream(new FileInputStream(journalFile), IO_BUFFER_SIZE);
try {
String magic = readAsciiLine(in);
String version = readAsciiLine(in);
String appVersionString = readAsciiLine(in);
String valueCountString = readAsciiLine(in);
String blank = readAsciiLine(in);
if (!MAGIC.equals(magic)
|| !VERSION_1.equals(version)
|| !Integer.toString(appVersion).equals(appVersionString)
|| !Integer.toString(valueCount).equals(valueCountString)
|| !"".equals(blank)) {
throw new IOException("unexpected journal header: ["
+ magic + ", " + version + ", " + valueCountString + ", " + blank + "]");
}
while (true) {
try {
readJournalLine(readAsciiLine(in));//读取剩余日志
} catch (EOFException endOfJournal) {
break;
}
}
} finally {
closeQuietly(in);
}
}
剩余日志是什么含义和作用呢?回到刚才的completeEdit(Editoreditor, boolean success)函数,查看下面的代码段可以得知,当正确提交时,写入了CLEAN,读取出错时写入的是REMOVE:
private synchronized void completeEdit(Editor editor, boolean success) throws IOException {
……省略无关代码……
if (entry.readable | success) {
entry.readable = true;
journalWriter.write(CLEAN + ' ' + entry.key + entry.getLengths() + '\n');
if (success) {
entry.sequenceNumber = nextSequenceNumber++;
}
} else {
lruEntries.remove(entry.key);
journalWriter.write(REMOVE + ' ' + entry.key + '\n');
}
……省略无关代码……
}
2.4 Key-Value出队
根据Key返回的是DisKLruCache内部快照类Snapshot的对象,Snapshot可以返回Key对应的Entry的输入流和Editor对象,不存在就返回null。有了输入流,就可以恢复出来Value。
/**
* A snapshot of the values for an entry.
*/
public final class Snapshot implements Closeable {
private final String key;
private final long sequenceNumber;
private final InputStream[] ins;
private Snapshot(String key, long sequenceNumber, InputStream[] ins) {
this.key = key;
this.sequenceNumber = sequenceNumber;
this.ins = ins;
}
/**
* Returns an editor for this snapshot's entry, or null if either the
* entry has changed since this snapshot was created or if another edit
* is in progress.
*/
public Editor edit() throws IOException {
return DiskLruCache.this.edit(key, sequenceNumber);
}
/**
* Returns the unbuffered stream with the value for {@code index}.
*/
public InputStream getInputStream(int index) {
return ins[index];
}
/**
* Returns the string value for {@code index}.
*/
public String getString(int index) throws IOException {
return inputStreamToString(getInputStream(index));
}
@Override public void close() {
for (InputStream in : ins) {
closeQuietly(in);
}
}
}
3. 使用
DisKLruCache可用于大文件对象的硬盘缓存,典型应用是BitMap。这里以BitMap为例走一下DisKLruCache的构造和存取过程。抽取的代码位于Android6.0 SDK中的DisplayingBitmaps工程(路径为:sdk\samples\android-23\ui\)。
3.1 DisKLruCache的构造
构造过程其实就1句话,这里加入了同步锁,以及一些必要条件。
private static final int DISK_CACHE_INDEX = 0;
private final Object mDiskCacheLock = new Object(); //用于同步硬盘缓存的同步锁
private boolean mDiskCacheStarting = true;
private DiskLruCache mDiskLruCache;
/**
* Initializes the disk cache. Note that this includes disk access so this should not be
* executed on the main/UI thread. By default an ImageCache does not initialize the disk
* cache when it is created, instead you should call initDiskCache() to initialize it on a
* background thread.
*/
public void initDiskCache() {
// Set up disk cache
synchronized (mDiskCacheLock) {
if (mDiskLruCache == null || mDiskLruCache.isClosed()) {
File diskCacheDir = mCacheParams.diskCacheDir; //文件路径
if (mCacheParams.diskCacheEnabled && diskCacheDir != null) {
if (!diskCacheDir.exists()) {
diskCacheDir.mkdirs();
}
//可用空间大于硬盘缓存大小
if (getUsableSpace(diskCacheDir) > mCacheParams.diskCacheSize) {
try {
mDiskLruCache = DiskLruCache.open(
diskCacheDir, 1, 1, mCacheParams.diskCacheSize);//构造
} catch (final IOException e) {
mCacheParams.diskCacheDir = null;
Log.e(TAG, "initDiskCache - " + e);
}
}
}
}
mDiskCacheStarting = false;
mDiskCacheLock.notifyAll();
}
}
3.2 Key-Value入队
入队的过程就是通过Key获得快照,通过快照查看是否已经存储过。如果没有存储过,再获得Editor,从Editor获得Key对应文件的输出流,最后将BitMap写入输出流。
/**
* Adds a bitmap to both memory and disk cache.
* @param data Unique identifier for the bitmap to store
* @param value The bitmap drawable to store
*/
public void addBitmapToCache(String data, BitmapDrawable value) {
synchronized (mDiskCacheLock) {//同步锁
// Add to disk cache
if (mDiskLruCache != null) {
final String key = hashKeyForDisk(data);
OutputStream out = null;
try {
DiskLruCache.Snapshot snapshot = mDiskLruCache.get(key);//快照
if (snapshot == null) {//表明还没有存储该key
final DiskLruCache.Editor editor = mDiskLruCache.edit(key);
if (editor != null) {
out = editor.newOutputStream(DISK_CACHE_INDEX);//输入流
value.getBitmap().compress(
mCacheParams.compressFormat, mCacheParams.compressQuality, out);//bitmap写入输入流,即写入到了临时文件
editor.commit();//临时文件写入正式文件
out.close();
}
} else {//表明存储过该key
snapshot.getInputStream(DISK_CACHE_INDEX).close();
}
} catch (final IOException e) {
Log.e(TAG, "addBitmapToCache - " + e);
} catch (Exception e) {
Log.e(TAG, "addBitmapToCache - " + e);
} finally {
try {
if (out != null) {
out.close();
}
} catch (IOException e) {}
}
}
}
}
3.3 Key-Value出队
出队过程相对简单,就是获得快照,从快照获得输入流,从输入流恢复出Value。
/**
* Get from disk cache.
*
* @param data Unique identifier for which item to get
* @return The bitmap if found in cache, null otherwise
*/
public Bitmap getBitmapFromDiskCache(String data) {
final String key = hashKeyForDisk(data);
Bitmap bitmap = null;
synchronized (mDiskCacheLock) {
while (mDiskCacheStarting) {
try {
mDiskCacheLock.wait();
} catch (InterruptedException e) {}
}
if (mDiskLruCache != null) {
InputStream inputStream = null;
try {
final DiskLruCache.Snapshot snapshot = mDiskLruCache.get(key);
if (snapshot != null) {
if (BuildConfig.DEBUG) {
Log.d(TAG, "Disk cache hit");
}
inputStream = snapshot.getInputStream(DISK_CACHE_INDEX);
if (inputStream != null) {
FileDescriptor fd = ((FileInputStream) inputStream).getFD();
// Decode bitmap, but we don't want to sample so give
// MAX_VALUE as the target dimensions
bitmap = ImageResizer.decodeSampledBitmapFromDescriptor(
fd, Integer.MAX_VALUE, Integer.MAX_VALUE, this);
}
}
} catch (final IOException e) {
Log.e(TAG, "getBitmapFromDiskCache - " + e);
} finally {
try {
if (inputStream != null) {
inputStream.close();
}
} catch (IOException e) {}
}
}
return bitmap;
}
4. 总结
DisKLruCache相对LruCache来讲,有多个内部类,所以较LruCache复杂。就使用来讲,也需要考虑同步操作,更增添了复杂性。