LRU cache in Java with Generics and O(1) operations

Question

This is a question that comes up a lot in job interviews. The idea is to define a data structure instead of using Java\'s built in LinkedHashMap.

An LRU cache delete

Answer 1

/*Java implementation using Deque and HashMap    */

interface Cache<K,V> {
    V get(K key) ;
    void set(K key, V value);
}

class Node <K,V>{
    K key;
    V value;

    public Node (K key, V value) {
        this.key = key;
        this.value = value;
    }
}

public class LRUCache <K,V> implements Cache<K,V>{
    Deque<Node<K,V>> dq = new LinkedList<>();
    Map<K, Node<K, V>> map = new HashMap<>();
    static int SIZE;

    public LRUCache(int size) {
        this.SIZE = size;
    }

    public V get(K key) {
        Node<K,V> result = map.get(key);
        if(result != null) {
            dq.remove(result);
            dq.addFirst(result);
            System.out.println("Get " +key +" : " +result.value);
            return result.value;
        }
        else {
            System.out.println("Cache miss");
            return null;
        }
    }

    public void set(K key, V value) {
        System.out.println("Set " +key +" : " +value);
        Node<K,V> result = map.get(key);
        if(result != null) {
            result.value = value;
            map.put(key, result);
            dq.remove(result);
            dq.addFirst(result);
            System.out.println("Updated frame " +key+" as " + value);
        }
        else {
            if(dq.size() == SIZE) {
                Node<K,V> toRemove = dq.removeLast();
                map.remove(toRemove);
                System.out.println("Frame removed " +toRemove.key +" : " +toRemove.value);
            }
            Node<K,V> newNode = new Node(key, value);
            dq.addFirst(newNode);
            map.put(key, newNode);
            System.out.println("Frame added " + key + " : " +value);
        }
    }

    public static void main(String[] args) {
        Cache<Integer, Integer> lru = new LRUCache<>(5);
        lru.get(2);
        lru.set(1, 11);
        lru.set(2, 22);
        lru.get(2);
        lru.set(3, 33);
        lru.set(4, 44);
        lru.set(5, 55);
        lru.get(2);
        lru.get(1);
        lru.set(6, 66);
    }
}

Answer 2

From the question itself, we can see that the problem of O(n) operations arises when querying the linked list. Therefore, we need an alternative data structure. We need to be able to update the items' last access time from the HashMap without searching.

We can keep two separate data structures. A HashMap with (Key,Pointer) pairs and a doubly linked list which will work as the priority queue for deletion and store the Values. From the HashMap, we can point to an element in the doubly linked list and update its' retrieval time. Because we go directly from the HashMap to the item in the list, our time complexity remains at O(1)

For example, our doubly linked list can look like:

least_recently_used  -> A <-> B <-> C <-> D <-> E <- most_recently_used

We need to keep a pointer to the LRU and MRU items. The entries' values will be stored in the list and when we query the HashMap, we will get a pointer to the list. On get(), we need to put the item at the right-most side of the list. On put(key,value), if the cache is full, we need to remove the item at the left-most side of the list from both, the list and the HashMap.

The following is an example implementation in Java:

public class LRUCache<K, V>{

    // Define Node with pointers to the previous and next items and a key, value pair
    class Node<T, U> {
        Node<T, U> previous;
        Node<T, U> next;
        T key;
        U value;

        public Node(Node<T, U> previous, Node<T, U> next, T key, U value){
            this.previous = previous;
            this.next = next;
            this.key = key;
            this.value = value;
        }
    }

    private HashMap<K, Node<K, V>> cache;
    private Node<K, V> leastRecentlyUsed;
    private Node<K, V> mostRecentlyUsed;
    private int maxSize;
    private int currentSize;

    public LRUCache(int maxSize){
        this.maxSize = maxSize;
        this.currentSize = 0;
        leastRecentlyUsed = new Node<K, V>(null, null, null, null);
        mostRecentlyUsed = leastRecentlyUsed;
        cache = new HashMap<K, Node<K, V>>();
    }

    public V get(K key){
        Node<K, V> tempNode = cache.get(key);
        if (tempNode == null){
            return null;
        }
        // If MRU leave the list as it is
        else if (tempNode.key == mostRecentlyUsed.key){
            return mostRecentlyUsed.value;
        }

        // Get the next and previous nodes
        Node<K, V> nextNode = tempNode.next;
        Node<K, V> previousNode = tempNode.previous;

        // If at the left-most, we update LRU 
        if (tempNode.key == leastRecentlyUsed.key){
            nextNode.previous = null;
            leastRecentlyUsed = nextNode;
        }

        // If we are in the middle, we need to update the items before and after our item
        else if (tempNode.key != mostRecentlyUsed.key){
            previousNode.next = nextNode;
            nextNode.previous = previousNode;
        }

        // Finally move our item to the MRU
        tempNode.previous = mostRecentlyUsed;
        mostRecentlyUsed.next = tempNode;
        mostRecentlyUsed = tempNode;
        mostRecentlyUsed.next = null;

        return tempNode.value;

    }

    public void put(K key, V value){
        if (cache.containsKey(key)){
            return;
        }

        // Put the new node at the right-most end of the linked-list
        Node<K, V> myNode = new Node<K, V>(mostRecentlyUsed, null, key, value);
        mostRecentlyUsed.next = myNode;
        cache.put(key, myNode);
        mostRecentlyUsed = myNode;

        // Delete the left-most entry and update the LRU pointer
        if (currentSize == maxSize){
            cache.remove(leastRecentlyUsed.key);
            leastRecentlyUsed = leastRecentlyUsed.next;
            leastRecentlyUsed.previous = null;
        }

        // Update cache size, for the first added entry update the LRU pointer
        else if (currentSize < maxSize){
            if (currentSize == 0){
                leastRecentlyUsed = myNode;
            }
            currentSize++;
        }
    }
}

Answer 3

The LinkedHashMap designed with that in mind

From the javadocs:

A special constructor is provided to create a linked hash map whose order of iteration is the order in which its entries were last accessed, from least-recently accessed to most-recently (access-order). This kind of map is well-suited to building LRU caches. Invoking the put, putIfAbsent, get, getOrDefault, compute, computeIfAbsent, computeIfPresent, or merge methods results in an access to the corresponding entry (assuming it exists after the invocation completes). The replace methods only result in an access of the entry if the value is replaced. The putAll method generates one entry access for each mapping in the specified map, in the order that key-value mappings are provided by the specified map's entry set iterator. No other methods generate entry accesses. In particular, operations on collection-views do not affect the order of iteration of the backing map.

The removeEldestEntry(Map.Entry) method may be overridden to impose a policy for removing stale mappings automatically when new mappings are added to the map.

Answer 4

Using a Stack and a HashMap:

import java.util.HashMap;
import java.util.Stack;
public class LRU {
    private HashMap<String,Object> lruList;
    private Stack<String> stackOrder;
    private int capacity;
    LRU(int capacity){
      this.capacity = capacity;
      this.lruList = new HashMap<String, Object>(capacity);
      this.stackOrder = new Stack<String>();
    }
    public void put(String key, Object value){
      if(lruList.containsKey(key) || this.capacity < lruList.size() + 1) {
        if(lruList.containsKey(key)){
            String remove = key;
        }else{
            String remove = this.stackOrder.firstElement();
        }
        this.stackOrder.removeElement(remove);
        this.lruList.remove(remove);
      }
      this.lruList.put(key, value);
      this.stackOrder.add(key);
    }
    public Stack<String> get(){
      return this.stackOrder;
    }
    public Object get(String key){
      Object value = lruList.get(key);
      put(key, value);
      return value;
    }
}

public static void main(String[] args) {
    LRU lru = new LRU(3);
    lru.put("k1","v1");
    lru.put("k2","v2");
    lru.put("k3","v3");
    System.out.println(lru.get());
    lru.get("k1");
    System.out.println(lru.get());
    lru.put("k4","v4");
    System.out.println(lru.get());
}

Output:

[k1, k2, k3]

[k2, k3, k1]

[k3, k1, k4]

Answer 5

Here is the java implementation

import java.util.HashMap;
import java.util.Map;

import com.nadeem.app.dsa.adt.Cache;
// Kind of linkedHashMap
public class LRUCache <K, V> implements Cache<K, V> {

private int capacity;
private Node<K, V> head, tail;
private Map<K, Node<K, V>> map;

private static final int DEFAULT_CAPACITY = 10;

public LRUCache(int capacity) {
    this.capacity = capacity;
    this.map = new HashMap<K, Node<K,V>>();
}

public LRUCache() {
    this(DEFAULT_CAPACITY);
}

@Override
public V get(K key) {
    V result = null;
    Node<K, V> node = this.map.get(key);
    if (node != null) {
        result = node.value;
        remove(node);
        addAsHead(node);
    }
    return result;
}

@Override
public void set(K key, V value) {
    Node<K, V> node = this.map.get(key);
    if (node == null) {
        Node<K, V> temp = new Node<K, V>(key, value);
        if (this.map.size() < this.capacity) {
            addAsHead(temp);
        } else {
            this.map.remove(this.tail.key);
            remove(this.tail);
            addAsHead(temp);
        }
        this.map.put(key, temp);
    } else {
        node.value = value;
        remove(node);
        addAsHead(node);
    }
}

private void remove(Node<K, V> node) {

    if (node.pre == null) {
        this.head = node.next;
    } else {
        node.pre.next = node.next;
    }

    if (node.next == null) {
        this.tail = node.pre;
    } else {
        node.next.pre = node.pre;
    }       
}

private void addAsHead(Node<K, V> node) {
    if (this.head == null) {
        this.head = node;
        this.tail = node;
    } else {
        this.head.pre = node;
        node.next = this.head;
        this.head = node;
    }
}

@Override
public void remove(K key) {
    Node<K, V> node = this.map.get(key);
    if (node != null) {
        this.remove(node);
    }       
}

private static class Node <S, T> {
    public S key;
    public T value;
    Node<S, T> pre;
    Node<S, T> next;

    Node(S key, T value) {
        this.key = key;
        this.value = value;
    }       
}

@Override
public int size() {
    return this.map.size();
}

}

Here is the unit test

public class LRUCacheTest {

private LRUCache<Integer, Integer> cache;

@Before
public void doBeforeEachTestCase() {
    this.cache = new LRUCache<Integer, Integer>(2);
}

@Test
public void setTest() {
    this.cache.set(1, 1);
    assertThat(this.cache.size(), equalTo(1));
    assertThat(this.cache.get(1), equalTo(1));

    this.cache.set(2, 2);
    assertThat(this.cache.size(), equalTo(2));
    assertThat(this.cache.get(2), equalTo(2));

    this.cache.set(3, 3);
    assertThat(this.cache.size(), equalTo(2));
    assertThat(this.cache.get(3), equalTo(3));

    this.cache.set(1, 3);
    assertThat(this.cache.size(), equalTo(2));
    assertThat(this.cache.get(1), equalTo(3));

    assertThat(this.cache.get(4), equalTo(null));
}

}

Answer 6

Here is the java implementation with Generics and LinkedHashMap, and complexity of O(1) on get() and put().

import java.util.LinkedHashMap;

public class LRUCache<K, V> {

    private LinkedHashMap<K, V> lruCacheMap;
    private final int capacity;
    private final boolean SORT_BY_ACCESS = true;
    private final float LOAD_FACTOR = 0.75F;

    public LRUCache(int capacity){
        this.capacity = capacity;
        this.lruCacheMap = new LinkedHashMap<>(capacity, LOAD_FACTOR, SORT_BY_ACCESS);
    }

    public V get(K k){
        return lruCacheMap.get(k);
    }

    public void put(K k, V v){
        if(lruCacheMap.containsKey(k)){
            lruCacheMap.remove(k);
        }else if(lruCacheMap.size() >= capacity){
            lruCacheMap.remove(lruCacheMap.keySet().iterator().next());
        }
        lruCacheMap.put(k, v);
    }

    public void printSequence(){
        System.out.println(lruCacheMap.keySet());
    }
}

This is the code for testing it:

import java.util.Arrays;
import java.util.HashMap;
import java.util.Map;

public class TestLruCache {

    static class MyHardDrive {
        Map<String, Object> resources = new HashMap<>();

        MyHardDrive(){
            for(Character i='A'; i<='Z'; i++){
                resources.put(i.toString(), new Object());
            }
        }

        Object loadResource(String name){
            return resources.get(name);
        }
    }

    public static void main(String[] args) {
        MyHardDrive hd = new MyHardDrive();
        LRUCache<String,Object> cache = new LRUCache<>(4);

        for(String key: Arrays.asList("A","B","C","A","D","E","F","E","F","G","A")){
            Object object = cache.get(key);
            if(object==null){
                object = hd.loadResource(key);
                cache.put(key, object);
            }
            cache.printSequence();
        }
    }
}