内存替换算法:LRU和LFU超详细!!!

在我们内存或者cache中有两种替换算法来保证内存或者cache中都是“热点”的数据,一个是LRU和LFU。下面会先分析再给出代码!

在这里的get和set操作的时间复杂度都是O(1),因为这两种速度都是很快的,不能要求O(n)。

一、LRU

首先我们搞一个双向链表。

我们定义这个规则:如果有节点被访问(get或者set),那么直接放入到末尾,每一个队列都有一个capacity值表示容量,如果新元素加入,现在达到了容量值,就需要淘汰最近最少使用的,就是头节点。例如:现在node1被访问。

如果现在又来了一个新元素noden+1,超过了容量值。所以:剔除node2,noden+1条件到末尾

这就是最近最少使用的算法,热点数据都在队尾。删除只删除头节点。

基础数据结构Node

    public static class Node<V> {
        public V value;
        public Node<V> pre;
        public Node<V> next;

        public Node(V value) {
            this.value = value;
        }
    }

双向队列

基本属性和构造函数

    public static class DoubleLinkedList<V> {
        private Node<V> head;
        private Node<V> tail;

        public DoubleLinkedList() {
            this.head = null;
            this.tail = null;
        }

下面我们来看方法:

        public void addNode(Node<V> newNode) {
            if (newNode == null) {
                return;
            }
            if (this.head == null) {
                this.head = newNode;
                this.tail = newNode;
            } else {
                this.tail.next = newNode;
                newNode.pre = this.tail;
                this.tail = newNode;
            }
        }

添加新节点,如果当前是第一个元素,头指针和尾指针都指向它,如果不是,放在队尾

        public void moveNodeToTail(Node<V> node) {
            if (this.tail == null) {
                return;
            }
            if (this.head == node) {
                this.head = node.next;
                this.head.pre = null;
            } else {
                node.pre.next = node.next;
                node.next.pre = node.pre;
            }
            this.tail.next = node;
            node.pre = this.tail;
            this.tail = node;
            this.tail.next = null;
        }

将一个节点移动到队尾,分为该节点是不是头节点,如果是头节点指向下一个,如果不是,就直接拿出来,然后放到队尾。

        public Node<V> removeHead() {
            if(this.head == null){
                return this.head;
            }
            Node<V> res = this.head;
            if(this.head == this.tail){
                this.head = null;
                this.tail = null;
            }else{
                this.head = res.next;
                res.next = null;
                this.head.pre = null;
            }
            return res;
        }

 删除头节点并返回。

Cache数据结构

基本属性和构造方法

    public static class MyCache<K,V>{
        private HashMap<K,Node<V>>  keyNodeMap;
        private HashMap<Node<V>,K>  nodeKeyMap;
        private DoubleLinkedList<V> nodeList;
        private int capacity;

        public MyCache(int capacity){
            if(capacity < 1){
                throw new RuntimeException();
            }
            this.nodeKeyMap = new HashMap<>();
            this.keyNodeMap = new HashMap<>();
            this.nodeList = new DoubleLinkedList<>();
            this.capacity = capacity;
        }

这里面用到了两个HashMap,为啥?因为hashmap的put和get的时间复杂度都是O(1),如果不用,遍历链表,那么我们的操作就是时间复杂度O(n),所以用到它俩主要是以空间换时间的方式提升效率!!!

get方法,获取value,判断key是否在map里面,如果再然后拿出节点,再移动到末尾。

        public V get(K key){
            if(this.keyNodeMap.containsKey(key)){
                Node<V> res = this.keyNodeMap.get(key);
                this.nodeList.moveNodeToTail(res);
                return res.value;
            }
            return null;
        }

set方法,判断是否有,如果有取出,移动到末尾,如果没有新建节点,加入,然后判断是否达到了容量,如果达到了容量就剔除头部节点。

        public void set(K key,V value){
            if(this.keyNodeMap.containsKey(key)){
                Node<V> target = keyNodeMap.get(key);
                target.value = value;
                this.nodeList.moveNodeToTail(target);
            }else{
                Node<V> node = new Node<>(value);
                this.keyNodeMap.put(key,node);
                this.nodeKeyMap.put(node,key);
                this.nodeList.addNode(node);
                if(this.keyNodeMap.size() == this.capacity + 1){
                    removeMostCache();
                }
            }
        }
        private void removeMostCache(){
            Node<V> node = this.nodeList.removeHead();
            K removeKey = this.nodeKeyMap.get(node);
            this.keyNodeMap.remove(removeKey);
            this.nodeKeyMap.remove(node);
        }

这样就做到了get和set方法的时间复杂度O(1)。

二、LFU

最近最少使用算法,这个算法相比于LRU就有难度了。我们来看思路:两个数据结构。

小节点类型

    /**
     * 小节点类型
     */
    public static class Node {
        public Integer key;
        public Integer value;
        public Integer times;
        public Node up;
        public Node down;

        public Node(int key, int value, int times) {
            this.key = key;
            this.value = value;
            this.times = times;
        }
    }

大节点类型 

   /**
     * 大节点类型
     */
    public static class NodeList {
        public Node head;
        public Node tail;
        public NodeList pre;
        public NodeList next;

        public NodeList(Node node) {
            this.head = node;
            this.tail = node;
        }

实际上就是一个下面的结构,矩形表示NodeList,圆形表示Node。 

我们的数据是字母,上面矩形表示访问(get和set方法)的次数。先解释一下我们的规则:

(1)NodeList只有在下面有节点的时候才会被创建,如果下面的节点被移除,那么删除该NodeList。

(2)NodeList中head和tail分别指向下面圆形双向链表的头跟尾部。他本身也是一个双向链表。

(3)如果现在有一个新节点W被访问。插入到1下面的双向链表的首部

如果此时A被访问:就到2下面的双向链表的首部。

如果A再次被访问:

如果此时2被访问。2下面没有元素了,就删除2,然后c放到3上。

如果此时A再次被访问,那么创建4的NodeList,然后插入A:

如果这个时候来了一个H,但是现在达到了最大容量,就要剔除一个,我们剔除1下面双向链表的尾节点。

这个NodeList的头节点的node双向链表的尾节点就是最近最少使用的节点。以上就是整个LFU的过程,下面我们来看代码!

   /**
     * 大节点类型
     */
    public static class NodeList {
        public Node head;
        public Node tail;
        public NodeList pre;
        public NodeList next;

        public NodeList(Node node) {
            this.head = node;
            this.tail = node;
        }

        /**
         * 新到的节点放在头部位置
         */
        public void addNodeFromHead(Node newNode) {
            newNode.down = this.head;
            this.head.up = newNode;
            head = newNode;
        }

        public boolean isEmpty() {
            return head == null;
        }

        /**
         * 移除节点到下一个NodeList。
         */
        public void deleteNode(Node node) {
            if (this.head == this.tail) {
                this.head = null;
                this.tail = null;
            } else {
                if (node == head) {
                    head = node.down;
                    head.up = null;
                } else if (node == tail) {
                    tail = node.up;
                    tail.down = null;
                } else {
                    node.up.down = node.down;
                    node.down.up = node.up;
                }
            }
            node.up = null;
            node.down = null;
        }
    }

在NodeList里面我们提供了三个方法:

(1)addNodeFromHead:将新加入的节点放在头部位置,保证无论进入哪个NodeList都必须保证在头部位置。

(2)isEmpty:判断当前NodeList是否为null。

(3)deleteNode:就是将访问的节点,从该NodeList移除,这个节点可以是头尾节点或者中间节点。

    public static class LFUCache {
        private int capacity;
        private int size;
        //key => node
        private HashMap<Integer, Node> records;
        //存储node的nodelist节点
        private HashMap<Node, NodeList> heads;
        //nodelist头节点
        private NodeList headList;

        public LFUCache(int capacity) {
            this.capacity = capacity;
            this.size = 0;
            this.records = new HashMap<>();
            this.heads = new HashMap<>();
            this.headList = null;
        }

这里面有几个属性:

(1)capacity:表示最大容量

(2)size:表示当前大小。

(3)records:表示key和node,这里的key是Integer,value是node类型

(4)heads:表示的是Node的上层NodeList节点是哪个。

(5)headList:表示所有NodeList的双向链表的头一个。

下面看方法:

判断如果有这元素,就更新数据同时times次数+1。然后执行move方法移动到下一个节点。move方法后面会有,如果没在并且达到了容量,就剔除一个节点,然后放入一个新节点

        public void set(int key, int value) {
            if (records.containsKey(key)) {
                Node node = records.get(key);
                node.value = value;
                node.times++;
                NodeList curNodeList = heads.get(node);
                //讲目标节点移动到下一个NodeList
                move(node, curNodeList);
            } else {
                //达到容量
                if (size == capacity) {
                    Node node = headList.tail;
                    headList.deleteNode(node);
                    //是否删除这个头
                    modifyHeadList(headList);
                    records.remove(node.key);
                    heads.remove(node);
                    size--;
                }
                Node node = new Node(key, value, 1);
                //刚开始加入第一个元素
                if (headList == null) {
                    headList = new NodeList(node);
                } else {
                    if (headList.head.times.equals(node.times)) {
                        headList.addNodeFromHead(node);
                    } else {
                        NodeList newList = new NodeList(node);
                        newList.next = headList;
                        headList.pre = newList;
                        headList = newList;
                    }
                }
                records.put(key, node);
                heads.put(node, headList);
                size++;
            }
        }

剔除元素之后,要判断这个NodeList是不是没有node了,入股没有就删除,否则留下。

        /**
         * 如果当前nodelist为空,就更换headList,否则就不
         */
        private boolean modifyHeadList(NodeList nodeList) {
            if (nodeList.isEmpty()) {
                if (headList == nodeList) {
                    headList = nodeList.next;
                    if (headList != null) {
                        headList.pre = null;
                    }
                } else {
                    nodeList.pre.next = nodeList.next;
                    if (nodeList.next != null) {
                        nodeList.next.pre = nodeList.pre;
                    }
                }
                return true;
            }
            return false;
        }

移动节点到新的NodeList

        private void move(Node node, NodeList oldNodeList) {
            oldNodeList.deleteNode(node);
            //拿到前一个节点的NodeList
            NodeList preList = modifyHeadList(oldNodeList) ? oldNodeList.pre : oldNodeList;
            NodeList nextList = oldNodeList.next;
            if (nextList == null) {
                NodeList newList = new NodeList(node);
                if (preList != null) {
                    preList.next = newList;
                }
                newList.pre = preList;
                if (headList == null) {
                    headList = newList;
                }
                heads.put(node, newList);
            } else {
                if (nextList.head.times.equals(node.times)) {
                    nextList.addNodeFromHead(node);
                    heads.put(node, nextList);
                } else {
                    NodeList newList = new NodeList(node);
                    if (preList != null) {
                        preList.next = newList;
                    }
                    newList.pre = preList;
                    newList.next = nextList;
                    nextList.pre = newList;
                    if (headList == nextList) {
                        headList = newList;
                    }
                    heads.put(node, newList);

                }
            }
        }

get方法:

        public int get(int key) {
            if (!records.containsKey(key)) {
                return -1;
            }
            Node node = records.get(key);
            node.times++;
            NodeList curNodeList = heads.get(node);
            move(node, curNodeList);
            return node.value;
        }

完整代码:

LFU:

public class LFU {

    /**
     * 小节点类型
     */
    public static class Node {
        public Integer key;
        public Integer value;
        public Integer times;
        public Node up;
        public Node down;

        public Node(int key, int value, int times) {
            this.key = key;
            this.value = value;
            this.times = times;
        }
    }

    /**
     * 大节点类型
     */
    public static class NodeList {
        public Node head;
        public Node tail;
        public NodeList pre;
        public NodeList next;

        public NodeList(Node node) {
            this.head = node;
            this.tail = node;
        }

        /**
         * 新到的节点放在头部位置
         */
        public void addNodeFromHead(Node newNode) {
            newNode.down = this.head;
            this.head.up = newNode;
            head = newNode;
        }

        public boolean isEmpty() {
            return head == null;
        }

        /**
         * 移除节点到下一个NodeList。
         */
        public void deleteNode(Node node) {
            if (this.head == this.tail) {
                this.head = null;
                this.tail = null;
            } else {
                if (node == head) {
                    head = node.down;
                    head.up = null;
                } else if (node == tail) {
                    tail = node.up;
                    tail.down = null;
                } else {
                    node.up.down = node.down;
                    node.down.up = node.up;
                }
            }
            node.up = null;
            node.down = null;
        }
    }

    public static class LFUCache {
        private int capacity;
        private int size;
        //key => node
        private HashMap<Integer, Node> records;
        //存储node的nodelist节点
        private HashMap<Node, NodeList> heads;
        //nodelist头节点
        private NodeList headList;

        public LFUCache(int capacity) {
            this.capacity = capacity;
            this.size = 0;
            this.records = new HashMap<>();
            this.heads = new HashMap<>();
            this.headList = null;
        }

        public void set(int key, int value) {
            if (records.containsKey(key)) {
                Node node = records.get(key);
                node.value = value;
                node.times++;
                NodeList curNodeList = heads.get(node);
                //讲目标节点移动到下一个NodeList
                move(node, curNodeList);
            } else {
                //达到容量
                if (size == capacity) {
                    Node node = headList.tail;
                    headList.deleteNode(node);
                    //是否删除这个头
                    modifyHeadList(headList);
                    records.remove(node.key);
                    heads.remove(node);
                    size--;
                }
                Node node = new Node(key, value, 1);
                //刚开始加入第一个元素
                if (headList == null) {
                    headList = new NodeList(node);
                } else {
                    if (headList.head.times.equals(node.times)) {
                        headList.addNodeFromHead(node);
                    } else {
                        NodeList newList = new NodeList(node);
                        newList.next = headList;
                        headList.pre = newList;
                        headList = newList;
                    }
                }
                records.put(key, node);
                heads.put(node, headList);
                size++;
            }
        }

        public int get(int key) {
            if (!records.containsKey(key)) {
                return -1;
            }
            Node node = records.get(key);
            node.times++;
            NodeList curNodeList = heads.get(node);
            move(node, curNodeList);
            return node.value;
        }

        /**
         * 如果当前nodelist为空,就更换headList,否则就不
         */
        private boolean modifyHeadList(NodeList nodeList) {
            if (nodeList.isEmpty()) {
                if (headList == nodeList) {
                    headList = nodeList.next;
                    if (headList != null) {
                        headList.pre = null;
                    }
                } else {
                    nodeList.pre.next = nodeList.next;
                    if (nodeList.next != null) {
                        nodeList.next.pre = nodeList.pre;
                    }
                }
                return true;
            }
            return false;
        }

        private void move(Node node, NodeList oldNodeList) {
            oldNodeList.deleteNode(node);
            //拿到前一个节点的NodeList
            NodeList preList = modifyHeadList(oldNodeList) ? oldNodeList.pre : oldNodeList;
            NodeList nextList = oldNodeList.next;
            if (nextList == null) {
                NodeList newList = new NodeList(node);
                if (preList != null) {
                    preList.next = newList;
                }
                newList.pre = preList;
                if (headList == null) {
                    headList = newList;
                }
                heads.put(node, newList);
            } else {
                if (nextList.head.times.equals(node.times)) {
                    nextList.addNodeFromHead(node);
                    heads.put(node, nextList);
                } else {
                    NodeList newList = new NodeList(node);
                    if (preList != null) {
                        preList.next = newList;
                    }
                    newList.pre = preList;
                    newList.next = nextList;
                    nextList.pre = newList;
                    if (headList == nextList) {
                        headList = newList;
                    }
                    heads.put(node, newList);

                }
            }
        }
    }
}

LRU:

package day_05;

import java.util.HashMap;

/**
 * @ClassName LRU
 * @Description
 * @Author 戴书博
 * @Date 2020/6/13 20:18
 * @Version 1.0
 **/
public class LRU {

    public static class Node<V> {
        public V value;
        public Node<V> pre;
        public Node<V> next;

        public Node(V value) {
            this.value = value;
        }
    }

    public static class DoubleLinkedList<V> {
        private Node<V> head;
        private Node<V> tail;

        public DoubleLinkedList() {
            this.head = null;
            this.tail = null;
        }

        public void addNode(Node<V> newNode) {
            if (newNode == null) {
                return;
            }
            if (this.head == null) {
                this.head = newNode;
                this.tail = newNode;
            } else {
                this.tail.next = newNode;
                newNode.pre = this.tail;
                this.tail = newNode;
            }
        }

        public void moveNodeToTail(Node<V> node) {
            if (this.tail == null) {
                return;
            }
            if (this.head == node) {
                this.head = node.next;
                this.head.pre = null;
            } else {
                node.pre.next = node.next;
                node.next.pre = node.pre;
            }
            this.tail.next = node;
            node.pre = this.tail;
            this.tail = node;
            this.tail.next = null;
        }

        public Node<V> removeHead() {
            if(this.head == null){
                return this.head;
            }
            Node<V> res = this.head;
            if(this.head == this.tail){
                this.head = null;
                this.tail = null;
            }else{
                this.head = res.next;
                res.next = null;
                this.head.pre = null;
            }
            return res;
        }
    }

    public static class MyCache<K,V>{
        private HashMap<K,Node<V>>  keyNodeMap;
        private HashMap<Node<V>,K>  nodeKeyMap;
        private DoubleLinkedList<V> nodeList;
        private int capacity;

        public MyCache(int capacity){
            if(capacity < 1){
                throw new RuntimeException();
            }
            this.nodeKeyMap = new HashMap<>();
            this.keyNodeMap = new HashMap<>();
            this.nodeList = new DoubleLinkedList<>();
            this.capacity = capacity;
        }

        public V get(K key){
            if(this.keyNodeMap.containsKey(key)){
                Node<V> res = this.keyNodeMap.get(key);
                this.nodeList.moveNodeToTail(res);
                return res.value;
            }
            return null;
        }

        public void set(K key,V value){
            if(this.keyNodeMap.containsKey(key)){
                Node<V> target = keyNodeMap.get(key);
                target.value = value;
                this.nodeList.moveNodeToTail(target);
            }else{
                Node<V> node = new Node<>(value);
                this.keyNodeMap.put(key,node);
                this.nodeKeyMap.put(node,key);
                this.nodeList.addNode(node);
                if(this.keyNodeMap.size() == this.capacity + 1){
                    removeMostCache();
                }
            }
        }
        private void removeMostCache(){
            Node<V> node = this.nodeList.removeHead();
            K removeKey = this.nodeKeyMap.get(node);
            this.keyNodeMap.remove(removeKey);
            this.nodeKeyMap.remove(node);
        }


    }
}

 

相关推荐
©️2020 CSDN 皮肤主题: 技术黑板 设计师:CSDN官方博客 返回首页