How can I implement decrease-key functionality in Python's heapq?

Question

I know it is possible to realize decrease-key functionality in O(log n) but I don\'t know how?

Answer 1

Imagine you are using a heap as a priority queue, where you have a bunch of tasks represented by strings and each task has a key. For concreteness, look at: task_list = [[7,"do laundry"], [3, "clean room"], [6, "call parents"]] where each task in task_list is a list with a priority and description. If you run heapq.heapify(task_list), you get your array to maintain the heap invariant. However, if you want to change the priority of "do laundry" to 1, you have no way of knowing where "do laundry" is in the heap without a linear scan through the heap (hence can't do decrease_key in logarithmic time). Note decrease_key(heap, i, new_key) requires you to know the index of the value to change in the heap.

Even if you maintain a reference to each sublist and actually change the key, you still can't do it in log time. Since a list is just a reference to a bunch of mutable objects, you could try doing something like remember the original order of the task: (in this case that "do laundry" was the 0th task in your original task_list):

task_list = [[7, "do laundry"], [3, "clean room"], [6, "call parents"]]
task_list_heap = task_list[:] # make a non-deep copy
heapq.heapify(task_list_heap)
# at this point:
# task_list = [[7, 'do laundry'], [3, 'clean room'], [6, 'call parents']]
# task_list_heap = [3, 'clean room'], [7, 'do laundry'], [6, 'call parents']]
# Change key of first item of task_list (which was "do laundry") from 7 to 1.
task_list[0][0] = 1
# Now:
# task_list = [[1, 'do laundry'], [3, 'clean room'], [6, 'call parents']]
# task_list_heap = [3, 'clean room'], [1, 'do laundry'], [6, 'call parents']]
# task_list_heap violates heap invariant at the moment

However, you now need to call heapq._siftdown(task_list_heap, 1) to maintain the heap invariant in log time (heapq.heapify is linear time), but unfortunately we don't know the index of "do laundry" in task_list_heap (the heap_index in this case is 1).

So we need to implement our heap keeps track of the heap_index of each object; e.g., have an list (for the heap) and a dict mapping each object to its index in the heap/list (that gets updated as the heap positions get swapped adding a constant factor to each swap). You could read through heapq.py and implement yourself as the procedure is straightforward; however, others have implement this sort of HeapDict already.