一个LRUMap的实现——来自apache common-collections框架

最新推荐文章于 2024-04-12 14:13:55 发布

buster2014

最新推荐文章于 2024-04-12 14:13:55 发布

阅读量1.1k

点赞数

分类专栏： Java基础 java缓存机制-----memcache、ehcache J2EE(web项目) java-jsp编码问题 LRU

Java基础同时被 3 个专栏收录

57 篇文章 0 订阅

订阅专栏

J2EE(web项目)

48 篇文章 0 订阅

订阅专栏

java缓存机制-----memcache、ehcache

8 篇文章 0 订阅

订阅专栏

30. 下面哪个Map最适合用来实现LRU Cache?
A. Hashtable
B. TreeMap
C. HashMap
D. IdentityHashMap
E. WeakHashMap

网上给的答案是A。为什么？
大家一般如何实现这个最近最少使用 cache？

今天主要分享一个LRUmap的实现。我们经常会用到需要使用map来保存数据的时候，由于map本身的映射高效，最适合做随机读取的存储结构。当然LRU算法是在有限大小的存储集合下的一种调度算法，即最近最少使用。对于一个给定大小限制的容器，如何分配资源就涉及到调度，而LRU就是一种经典的调度了，在容器中定义一个最后使用时间，当容器满时，再来新的元素，那么淘汰最近最少使用的元素，把新的元素替换之，这是最直接的思想。

apache common-collections框架里有一个LRUMap的实现，其继承自抽象的linkedmap和抽象的hashmap。下面给出一段测试代码，来看看LRU的直观效果：

 
public static void main(String[] args) { 
        // TODO Auto-generated method stub 
        LRUMap map = new LRUMap(3); 
        map.put("1", 1); 
        map.put("2", 2); 
        //map.get("1"); 
        map.put("3", 3); 
        map.put("4", 4); 
         
        for(Iterator it = map.entrySet().iterator();it.hasNext();){ 
            System.out.println(it.next()); 
        } 
    } 
 

使用的版本是common-collections3.2.1，直接执行，结果会显示

2=2

3=3

4=4

如果把第6行的注释打开，那么执行结果会是

1=1

3=3

4=4

这样就符合了LRU的原理，在调用了一次get后，1对应的数据不再是最近最少使用。

具体实现也颇有趣，LRUmap继承linkedmap，维护了一个linked list来保存内部数据

 
/** Header in the linked list */ 
    protected transient LinkEntry header;

linkentry又是一个循环双向的链表，且继承了hashentry，hashentry虽然也是commons框架自己实现，但是与jdk的实现同理，也是利用链接桶来预防冲突

 
protected static class LinkEntry extends HashEntry { 
        /** The entry before this one in the order */ 
        protected LinkEntry before; 
        /** The entry after this one in the order */ 
        protected LinkEntry after; 
         
        /** 
         * Constructs a new entry. 
         *  
         * @param next  the next entry in the hash bucket sequence 
         * @param hashCode  the hash code 
         * @param key  the key 
         * @param value  the value 
         */ 
        protected LinkEntry(HashEntry next, int hashCode, Object key, Object value) { 
            super(next, hashCode, key, value); 
        } 
    } 
 

当进行put操作时，LRUMap会调用abstracthashmap的put方法，与传统一样，计算hashcode，在对应的hashcode桶定位index，然后做一个addmapping操作。本来在抽象类中的addmapping是传统的，等同于jdk中hashmap的addentry，但是LRUMap这里重写了addmapping，主要进行了map是否已满的判断，如果map未满，那么直接插入，否则，LRU将会定位到将被替换掉的entry的位置，然后做一个reuseMapping的操作，将该替换掉的entryremove，将新加进来的entry放到队尾。这样就完成了put操作。

进行get操作时，首先依据hashmap的原则找到entry，在返回value之前进行了LRU调整moveToMRU操作。该操作判断这个entry是否是队尾，如果是，那么什么都不用干，它就是最近被使用的，如果不是，那么调整它到队尾。

全部的源码见下：

 
/* 
 *  Licensed to the Apache Software Foundation (ASF) under one or more 
 *  contributor license agreements.  See the NOTICE file distributed with 
 *  this work for additional information regarding copyright ownership. 
 *  The ASF licenses this file to You under the Apache License, Version 2.0 
 *  (the "License"); you may not use this file except in compliance with 
 *  the License.  You may obtain a copy of the License at 
 * 
 *      http://www.apache.org/licenses/LICENSE-2.0 
 * 
 *  Unless required by applicable law or agreed to in writing, software 
 *  distributed under the License is distributed on an "AS IS" BASIS, 
 *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 
 *  See the License for the specific language governing permissions and 
 *  limitations under the License. 
 */ 
package org.apache.commons.collections.map; 
   
import java.io.IOException; 
import java.io.ObjectInputStream; 
import java.io.ObjectOutputStream; 
import java.io.Serializable; 
import java.util.Map; 
   
import org.apache.commons.collections.BoundedMap; 
   
/** 
 * A <code>Map</code> implementation with a fixed maximum size which removes 
 * the least recently used entry if an entry is added when full. 
 * <p> 
 * The least recently used algorithm works on the get and put operations only. 
 * Iteration of any kind, including setting the value by iteration, does not 
 * change the order. Queries such as containsKey and containsValue or access 
 * via views also do not change the order. 
 * <p> 
 * The map implements <code>OrderedMap</code> and entries may be queried using 
 * the bidirectional <code>OrderedMapIterator</code>. The order returned is 
 * least recently used to most recently used. Iterators from map views can  
 * also be cast to <code>OrderedIterator</code> if required. 
 * <p> 
 * All the available iterators can be reset back to the start by casting to 
 * <code>ResettableIterator</code> and calling <code>reset()</code>. 
 * <p> 
 * <strong>Note that LRUMap is not synchronized and is not thread-safe.</strong> 
 * If you wish to use this map from multiple threads concurrently, you must use 
 * appropriate synchronization. The simplest approach is to wrap this map 
 * using {@link java.util.Collections#synchronizedMap(Map)}. This class may throw  
 * <code>NullPointerException</code>'s when accessed by concurrent threads. 
 * 
 * @since Commons Collections 3.0 (previously in main package v1.0) 
 * @version $Revision: 646777 $ $Date: 2008-04-10 13:33:15 +0100 (Thu, 10 Apr 2008) $ 
 * 
 * @author James Strachan 
 * @author Morgan Delagrange 
 * @author Stephen Colebourne 
 * @author Mike Pettypiece 
 * @author Mario Ivankovits 
 */ 
public class LRUMap 
        extends AbstractLinkedMap implements BoundedMap, Serializable, Cloneable { 
     
    /** Serialisation version */ 
    private static final long serialVersionUID = -612114643488955218L; 
    /** Default maximum size */ 
    protected static final int DEFAULT_MAX_SIZE = 100; 
     
    /** Maximum size */ 
    private transient int maxSize; 
    /** Scan behaviour */ 
    private boolean scanUntilRemovable; 
   
    /** 
     * Constructs a new empty map with a maximum size of 100. 
     */ 
    public LRUMap() { 
        this(DEFAULT_MAX_SIZE, DEFAULT_LOAD_FACTOR, false); 
    } 
   
    /** 
     * Constructs a new, empty map with the specified maximum size. 
     * 
     * @param maxSize  the maximum size of the map 
     * @throws IllegalArgumentException if the maximum size is less than one 
     */ 
    public LRUMap(int maxSize) { 
        this(maxSize, DEFAULT_LOAD_FACTOR); 
    } 
   
    /** 
     * Constructs a new, empty map with the specified maximum size. 
     * 
     * @param maxSize  the maximum size of the map 
     * @param scanUntilRemovable  scan until a removeable entry is found, default false 
     * @throws IllegalArgumentException if the maximum size is less than one 
     * @since Commons Collections 3.1 
     */ 
    public LRUMap(int maxSize, boolean scanUntilRemovable) { 
        this(maxSize, DEFAULT_LOAD_FACTOR, scanUntilRemovable); 
    } 
   
    /** 
     * Constructs a new, empty map with the specified initial capacity and 
     * load factor.  
     * 
     * @param maxSize  the maximum size of the map, -1 for no limit, 
     * @param loadFactor  the load factor 
     * @throws IllegalArgumentException if the maximum size is less than one 
     * @throws IllegalArgumentException if the load factor is less than zero 
     */ 
    public LRUMap(int maxSize, float loadFactor) { 
        this(maxSize, loadFactor, false); 
    } 
   
    /** 
     * Constructs a new, empty map with the specified initial capacity and 
     * load factor. 
     * 
     * @param maxSize  the maximum size of the map, -1 for no limit, 
     * @param loadFactor  the load factor 
     * @param scanUntilRemovable  scan until a removeable entry is found, default false 
     * @throws IllegalArgumentException if the maximum size is less than one 
     * @throws IllegalArgumentException if the load factor is less than zero 
     * @since Commons Collections 3.1 
     */ 
    public LRUMap(int maxSize, float loadFactor, boolean scanUntilRemovable) { 
        super((maxSize < 1 ? DEFAULT_CAPACITY : maxSize), loadFactor); 
        if (maxSize < 1) { 
            throw new IllegalArgumentException("LRUMap max size must be greater than 0"); 
        } 
        this.maxSize = maxSize; 
        this.scanUntilRemovable = scanUntilRemovable; 
    } 
   
    /** 
     * Constructor copying elements from another map. 
     * <p> 
     * The maximum size is set from the map's size. 
     * 
     * @param map  the map to copy 
     * @throws NullPointerException if the map is null 
     * @throws IllegalArgumentException if the map is empty 
     */ 
    public LRUMap(Map map) { 
        this(map, false); 
    } 
   
    /** 
     * Constructor copying elements from another map. 
     * <p/> 
     * The maximum size is set from the map's size. 
     * 
     * @param map  the map to copy 
     * @param scanUntilRemovable  scan until a removeable entry is found, default false 
     * @throws NullPointerException if the map is null 
     * @throws IllegalArgumentException if the map is empty 
     * @since Commons Collections 3.1 
     */ 
    public LRUMap(Map map, boolean scanUntilRemovable) { 
        this(map.size(), DEFAULT_LOAD_FACTOR, scanUntilRemovable); 
        putAll(map); 
    } 
   
    //----------------------------------------------------------------------- 
    /** 
     * Gets the value mapped to the key specified. 
     * <p> 
     * This operation changes the position of the key in the map to the 
     * most recently used position (first). 
     *  
     * @param key  the key 
     * @return the mapped value, null if no match 
     */ 
    public Object get(Object key) { 
        LinkEntry entry = (LinkEntry) getEntry(key); 
        if (entry == null) { 
            return null; 
        } 
        moveToMRU(entry); 
        return entry.getValue(); 
    } 
   
    //----------------------------------------------------------------------- 
    /** 
     * Moves an entry to the MRU position at the end of the list. 
     * <p> 
     * This implementation moves the updated entry to the end of the list. 
     *  
     * @param entry  the entry to update 
     */ 
    protected void moveToMRU(LinkEntry entry) { 
        if (entry.after != header) { 
            modCount++; 
            // remove 
            entry.before.after = entry.after; 
            entry.after.before = entry.before; 
            // add first 
            entry.after = header; 
            entry.before = header.before; 
            header.before.after = entry; 
            header.before = entry; 
        } else if (entry == header) { 
            throw new IllegalStateException("Can't move header to MRU" + 
                " (please report this to commons-dev@jakarta.apache.org)"); 
        } 
    } 
     
    /** 
     * Updates an existing key-value mapping. 
     * <p> 
     * This implementation moves the updated entry to the top of the list 
     * using {@link #moveToMRU(AbstractLinkedMap.LinkEntry)}. 
     *  
     * @param entry  the entry to update 
     * @param newValue  the new value to store 
     */ 
    protected void updateEntry(HashEntry entry, Object newValue) { 
        moveToMRU((LinkEntry) entry);  // handles modCount 
        entry.setValue(newValue); 
    } 
     
    /** 
     * Adds a new key-value mapping into this map. 
     * <p> 
     * This implementation checks the LRU size and determines whether to 
     * discard an entry or not using {@link #removeLRU(AbstractLinkedMap.LinkEntry)}. 
     * <p> 
     * From Commons Collections 3.1 this method uses {@link #isFull()} rather 
     * than accessing <code>size</code> and <code>maxSize</code> directly. 
     * It also handles the scanUntilRemovable functionality. 
     *  
     * @param hashIndex  the index into the data array to store at 
     * @param hashCode  the hash code of the key to add 
     * @param key  the key to add 
     * @param value  the value to add 
     */ 
    protected void addMapping(int hashIndex, int hashCode, Object key, Object value) { 
        if (isFull()) { 
            LinkEntry reuse = header.after; 
            boolean removeLRUEntry = false; 
            if (scanUntilRemovable) { 
                while (reuse != header && reuse != null) { 
                    if (removeLRU(reuse)) { 
                        removeLRUEntry = true; 
                        break; 
                    } 
                    reuse = reuse.after; 
                } 
                if (reuse == null) { 
                    throw new IllegalStateException( 
                        "Entry.after=null, header.after" + header.after + " header.before" + header.before + 
                        " key=" + key + " value=" + value + " size=" + size + " maxSize=" + maxSize + 
                        " Please check that your keys are immutable, and that you have used synchronization properly." + 
                        " If so, then please report this to commons-dev@jakarta.apache.org as a bug."); 
                } 
            } else { 
                removeLRUEntry = removeLRU(reuse); 
            } 
             
            if (removeLRUEntry) { 
                if (reuse == null) { 
                    throw new IllegalStateException( 
                        "reuse=null, header.after=" + header.after + " header.before" + header.before + 
                        " key=" + key + " value=" + value + " size=" + size + " maxSize=" + maxSize + 
                        " Please check that your keys are immutable, and that you have used synchronization properly." + 
                        " If so, then please report this to commons-dev@jakarta.apache.org as a bug."); 
                } 
                reuseMapping(reuse, hashIndex, hashCode, key, value); 
            } else { 
                super.addMapping(hashIndex, hashCode, key, value); 
            } 
        } else { 
            super.addMapping(hashIndex, hashCode, key, value); 
        } 
    } 
     
    /** 
     * Reuses an entry by removing it and moving it to a new place in the map. 
     * <p> 
     * This method uses {@link #removeEntry}, {@link #reuseEntry} and {@link #addEntry}. 
     *  
     * @param entry  the entry to reuse 
     * @param hashIndex  the index into the data array to store at 
     * @param hashCode  the hash code of the key to add 
     * @param key  the key to add 
     * @param value  the value to add 
     */ 
    protected void reuseMapping(LinkEntry entry, int hashIndex, int hashCode, Object key, Object value) { 
        // find the entry before the entry specified in the hash table 
        // remember that the parameters (except the first) refer to the new entry, 
        // not the old one 
        try { 
            int removeIndex = hashIndex(entry.hashCode, data.length); 
            HashEntry[] tmp = data;  // may protect against some sync issues 
            HashEntry loop = tmp[removeIndex]; 
            HashEntry previous = null; 
            while (loop != entry && loop != null) { 
                previous = loop; 
                loop = loop.next; 
            } 
            if (loop == null) { 
                throw new IllegalStateException( 
                    "Entry.next=null, data[removeIndex]=" + data[removeIndex] + " previous=" + previous + 
                    " key=" + key + " value=" + value + " size=" + size + " maxSize=" + maxSize + 
                    " Please check that your keys are immutable, and that you have used synchronization properly." + 
                    " If so, then please report this to commons-dev@jakarta.apache.org as a bug."); 
            } 
             
            // reuse the entry 
            modCount++; 
            removeEntry(entry, removeIndex, previous); 
            reuseEntry(entry, hashIndex, hashCode, key, value); 
            addEntry(entry, hashIndex); 
        } catch (NullPointerException ex) { 
            throw new IllegalStateException( 
                    "NPE, entry=" + entry + " entryIsHeader=" + (entry==header) + 
                    " key=" + key + " value=" + value + " size=" + size + " maxSize=" + maxSize + 
                    " Please check that your keys are immutable, and that you have used synchronization properly." + 
                    " If so, then please report this to commons-dev@jakarta.apache.org as a bug."); 
        } 
    } 
     
    /** 
     * Subclass method to control removal of the least recently used entry from the map. 
     * <p> 
     * This method exists for subclasses to override. A subclass may wish to 
     * provide cleanup of resources when an entry is removed. For example: 
     * <pre> 
     * protected boolean removeLRU(LinkEntry entry) { 
     *   releaseResources(entry.getValue());  // release resources held by entry 
     *   return true;  // actually delete entry 
     * } 
     * </pre> 
     * <p> 
     * Alternatively, a subclass may choose to not remove the entry or selectively 
     * keep certain LRU entries. For example: 
     * <pre> 
     * protected boolean removeLRU(LinkEntry entry) { 
     *   if (entry.getKey().toString().startsWith("System.")) { 
     *     return false;  // entry not removed from LRUMap 
     *   } else { 
     *     return true;  // actually delete entry 
     *   } 
     * } 
     * </pre> 
     * The effect of returning false is dependent on the scanUntilRemovable flag. 
     * If the flag is true, the next LRU entry will be passed to this method and so on 
     * until one returns false and is removed, or every entry in the map has been passed. 
     * If the scanUntilRemovable flag is false, the map will exceed the maximum size. 
     * <p> 
     * NOTE: Commons Collections 3.0 passed the wrong entry to this method. 
     * This is fixed in version 3.1 onwards. 
     *  
     * @param entry  the entry to be removed 
     */ 
    protected boolean removeLRU(LinkEntry entry) { 
        return true; 
    } 
   
    //----------------------------------------------------------------------- 
    /** 
     * Returns true if this map is full and no new mappings can be added. 
     * 
     * @return <code>true</code> if the map is full 
     */ 
    public boolean isFull() { 
        return (size >= maxSize); 
    } 
   
    /** 
     * Gets the maximum size of the map (the bound). 
     * 
     * @return the maximum number of elements the map can hold 
     */ 
    public int maxSize() { 
        return maxSize; 
    } 
   
    /** 
     * Whether this LRUMap will scan until a removable entry is found when the 
     * map is full. 
     * 
     * @return true if this map scans 
     * @since Commons Collections 3.1 
     */ 
    public boolean isScanUntilRemovable() { 
        return scanUntilRemovable; 
    } 
   
    //----------------------------------------------------------------------- 
    /** 
     * Clones the map without cloning the keys or values. 
     * 
     * @return a shallow clone 
     */ 
    public Object clone() { 
        return super.clone(); 
    } 
     
    /** 
     * Write the map out using a custom routine. 
     */ 
    private void writeObject(ObjectOutputStream out) throws IOException { 
        out.defaultWriteObject(); 
        doWriteObject(out); 
    } 
   
    /** 
     * Read the map in using a custom routine. 
     */ 
    private void readObject(ObjectInputStream in) throws IOException, ClassNotFoundException { 
        in.defaultReadObject(); 
        doReadObject(in); 
    } 
     
    /** 
     * Writes the data necessary for <code>put()</code> to work in deserialization. 
     */ 
    protected void doWriteObject(ObjectOutputStream out) throws IOException { 
        out.writeInt(maxSize); 
        super.doWriteObject(out); 
    } 
   
    /** 
     * Reads the data necessary for <code>put()</code> to work in the superclass. 
     */ 
    protected void doReadObject(ObjectInputStream in) throws IOException, ClassNotFoundException { 
        maxSize = in.readInt(); 
        super.doReadObject(in); 
    } 
     
} 
 