Chapter 27: Heaps and Priority Queues
27.06Intermediate / Advanced6 topics21 min2 interactive animations

Specialized heaps

Heaps beyond the binary heap: d-ary heaps, binomial heaps, Fibonacci heaps, pairing heaps, the indexed priority queue and decrease-key.

Learning goals

  • Tune a d-ary heap's branching factor to the workload
  • Describe how binomial, Fibonacci and pairing heaps merge in less than O(n)
  • Implement an indexed priority queue with decrease-key
  • Compare the operation costs of the heap family

Prerequisites

Heap operationsDijkstra's algorithm

The binary heap is the default, but algorithms like Dijkstra and Prim call decrease-key far more often than they extract, and mergeable heaps are needed for some parallel and incremental algorithms. Other heaps trade simplicity for those operations.

27.06.1

D-ary heap

A **d-ary heap** gives each node up to d children: the parent of i is (i - 1) / d and the children start at d · i + 1. The tree is shallower (height log base d of n), so **sift-up** is cheaper, but **sift-down** must compare up to d children per level, so it is dearer. Workloads with many insertions or decrease-keys favour a larger d; workloads with many extractions favour d = 2. A 4-ary heap is also friendlier to the cache, since the children are adjacent in the array.

The animation inserts and then extracts the 14 values 14 down to 1: d = 2 uses 31 comparisons for the insertions and 42 for the extractions (73 in all), d = 3 uses 24 and 45 (69), and d = 4 uses 22 and 54 (76). The ascending-input preset shows the cheapest case for insertion: each of the 12 values needs one comparison, 11 in all, since the first insertion needs none.

OperationBinary heap (d = 2)d-ary heap
Heightlog2 nlog_d n
insert / decrease-keyO(log2 n)O(log_d n)
extract-minO(log2 n)O(d · log_d n)
Java
1static class DaryHeap {
2 private final int d;
3 private int[] a = new int[16];
4 private int size;
5 DaryHeap(int d) { this.d = d; }
6 void add(int x) {
7 if (size == a.length) a = Arrays.copyOf(a, 2 * size);
8 a[size] = x;
9 int i = size++;
10 while (i > 0 && a[(i - 1) / d] > a[i]) { // sift-up: one comparison per level
11 int p = (i - 1) / d;
12 int t = a[p]; a[p] = a[i]; a[i] = t;
13 i = p;
14 }
15 }
16 int poll() {
17 int min = a[0];
18 a[0] = a[--size];
19 int i = 0;
20 while (true) { // sift-down: up to d comparisons per level
21 int m = i;
22 for (int c = d * i + 1; c < Math.min(size, d * i + 1 + d); c++) if (a[c] < a[m]) m = c;
23 if (m == i) break;
24 int t = a[i]; a[i] = a[m]; a[m] = t;
25 i = m;
26 }
27 return min;
28 }
29 boolean isEmpty() { return size == 0; }
30}

d-ary heap: how the branching factor changes sift-up and sift-down cost

Interactive · edit the input

Your input

Try

heap array (parent of i is ⌊(i - 1) / 4⌋)

none
Setup

A 4-ary heap gives each node up to 4 children, so the tree is shallower (height ⌊log4 n⌋) but a sift-down must compare 4 children per level. Insert all values, then extract them all.

Step 1 / 25
Java
1static int d = 4; // branching factor
2static int parent(int i) { return (i - 1) / d; }
3static int firstChild(int i) { return d * i + 1; }
4static void siftUp(int[] a, int i) {
5 while (i > 0 && a[parent(i)] > a[i]) { // one comparison per level
6 swap(a, parent(i), i);
7 i = parent(i);
8 }
9}
10static void siftDown(int[] a, int n, int i) {
11 while (true) {
12 int m = i;
13 for (int c = firstChild(i); c < Math.min(n, firstChild(i) + d); c++) // up to d comparisons per level
14 if (a[c] < a[m]) m = c;
15 if (m == i) return;
16 swap(a, i, m);
17 i = m;
18 }
19}

Live state

0 sift-up comparisons0 sift-down comparisons0 swaps

d

4

size

0

No condition evaluated on this step
Execution trace (dry run table)Show 25 steps
StepLineStateWhat happens
1L1d=4 size=0A 4-ary heap gives each node up to 4 children, so the tree is shallower (height ⌊log4 n⌋) but a sift-down must compare 4 children per level. Insert all values, then extract them all.
2L4d=4 size=1Insert 9: it moved up 0 levels. Sift-up costs one comparison per level.
3L4d=4 size=2Insert 7: it moved up 1 level. Sift-up costs one comparison per level.
4L4d=4 size=3Insert 8: it moved up 0 levels. Sift-up costs one comparison per level.
5L4d=4 size=4Insert 3: it moved up 1 level. Sift-up costs one comparison per level.
6L4d=4 size=5Insert 5: it moved up 0 levels. Sift-up costs one comparison per level.
7L4d=4 size=6Insert 2: it moved up 2 levels. Sift-up costs one comparison per level.
8L4d=4 size=7Insert 6: it moved up 0 levels. Sift-up costs one comparison per level.
9L4d=4 size=8Insert 1: it moved up 2 levels. Sift-up costs one comparison per level.
10L4d=4 size=9Insert 4: it moved up 0 levels. Sift-up costs one comparison per level.
11L4d=4 size=10Insert 10: it moved up 0 levels. Sift-up costs one comparison per level.
12L4d=4 size=11Insert 12: it moved up 0 levels. Sift-up costs one comparison per level.
13L4d=4 size=12Insert 11: it moved up 0 levels. Sift-up costs one comparison per level.
14L10d=4 size=11Extract 1: the last leaf 11 went to the root and sank 2 levels. Each level costs up to 4 comparisons.
15L10d=4 size=10Extract 2: the last leaf 12 went to the root and sank 2 levels. Each level costs up to 4 comparisons.
16L10d=4 size=9Extract 3: the last leaf 10 went to the root and sank 2 levels. Each level costs up to 4 comparisons.
17L10d=4 size=8Extract 4: the last leaf 12 went to the root and sank 1 level. Each level costs up to 4 comparisons.
18L10d=4 size=7Extract 5: the last leaf 11 went to the root and sank 2 levels. Each level costs up to 4 comparisons.
19L10d=4 size=6Extract 6: the last leaf 10 went to the root and sank 1 level. Each level costs up to 4 comparisons.
20L10d=4 size=5Extract 7: the last leaf 11 went to the root and sank 1 level. Each level costs up to 4 comparisons.
21L10d=4 size=4Extract 8: the last leaf 12 went to the root and sank 1 level. Each level costs up to 4 comparisons.
22L10d=4 size=3Extract 9: the last leaf 10 went to the root and sank 0 levels. Each level costs up to 4 comparisons.
23L10d=4 size=2Extract 10: the last leaf 11 went to the root and sank 0 levels. Each level costs up to 4 comparisons.
24L10d=4 size=1Extract 11: the last leaf 12 went to the root and sank 0 levels. Each level costs up to 4 comparisons.
25L1d=4Sorted output [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]. With d = 4: 13 comparisons for the insertions and 48 for the extractions, 61 in all.
27.06.2

Binomial heap

A **binomial heap** is a list of **binomial trees**, at most one of each degree, like the binary representation of n. A tree B_k has 2^k nodes and is built by linking two B_(k-1) trees, the smaller key becoming the root. **Merging** two heaps works like binary addition, linking equal-degree trees as carries: O(log n). Insert is a merge with a one-node heap; extract-min removes the smallest root, and its children (which are binomial trees of smaller degrees) are merged back in.

The reward is the fast **meld** (union) that a binary heap lacks, which costs O(n) there. The price is more complex code and more pointers per node.

OperationBinomial heap
insertO(log n) worst, O(1) amortised
find-minO(log n) scanning the roots
extract-minO(log n)
unionO(log n)
decrease-keyO(log n)
Java
1static class Node {
2 int key, degree;
3 Node child, sibling;
4 Node(int key) { this.key = key; }
5}
6static Node head; // roots, in increasing order of degree
7static void link(Node child, Node parent) { // both trees have the same degree
8 child.sibling = parent.child;
9 parent.child = child;
10 parent.degree++;
11}
12static Node mergeLists(Node a, Node b) {
13 Node dummy = new Node(0), t = dummy;
14 while (a != null && b != null) {
15 if (a.degree <= b.degree) { t.sibling = a; a = a.sibling; } else { t.sibling = b; b = b.sibling; }
16 t = t.sibling;
17 }
18 t.sibling = a != null ? a : b;
19 return dummy.sibling;
20}
21static Node union(Node a, Node b) { // like adding two binary numbers
22 Node h = mergeLists(a, b);
23 if (h == null) return null;
24 Node prev = null, x = h, next = x.sibling;
25 while (next != null) {
26 if (x.degree != next.degree || (next.sibling != null && next.sibling.degree == x.degree)) { prev = x; x = next; }
27 else if (x.key <= next.key) { x.sibling = next.sibling; link(next, x); }
28 else { if (prev == null) h = next; else prev.sibling = next; link(x, next); x = next; }
29 next = x.sibling;
30 }
31 return h;
32}
33static void insert(int key) { head = union(head, new Node(key)); }
34static int extractMin() {
35 Node best = head, bestPrev = null, prev = null;
36 for (Node x = head; x != null; prev = x, x = x.sibling) if (x.key < best.key) { best = x; bestPrev = prev; }
37 if (bestPrev == null) head = best.sibling; else bestPrev.sibling = best.sibling;
38 Node rev = null, c = best.child; // the children, reversed into increasing degree order
39 while (c != null) { Node next = c.sibling; c.sibling = rev; rev = c; c = next; }
40 head = union(head, rev);
41 return best.key;
42}
27.06.3

Fibonacci heap

A **Fibonacci heap** is a lazy binomial-style heap. Insert just adds a tree to a circular root list: O(1). Union concatenates two root lists: O(1). Only **extract-min** does real work: it moves the minimum's children to the root list and **consolidates**, linking trees of equal degree until all degrees are distinct. **Decrease-key** cuts the node from its parent when heap order breaks, and a **cascading cut** removes marked ancestors to keep the trees bushy, so each costs O(1) amortised.

That makes Dijkstra and Prim run in O(E + V log V), the best known bound, but the constants and pointer overhead are large, so binary heaps (or d-ary heaps) are usually faster in practice. Fibonacci heaps are the best illustration of amortised analysis. The code below is a compact complete version.

Java
1static class Node {
2 int key, degree;
3 boolean mark; // lost a child since it last became a child itself
4 Node parent, child, left = this, right = this;
5 Node(int key) { this.key = key; }
6}
7static Node min;
8static int size;
9static void addRoot(Node x) { // splice x into the circular root list next to min
10 x.parent = null; x.mark = false;
11 if (min == null) { x.left = x; x.right = x; min = x; return; }
12 x.left = min; x.right = min.right; min.right.left = x; min.right = x;
13 if (x.key < min.key) min = x;
14}
15static Node insert(int key) { Node x = new Node(key); addRoot(x); size++; return x; }
16
17static int extractMin() {
18 Node z = min;
19 if (z.child != null) { // children become roots
20 List<Node> kids = new ArrayList<>();
21 Node c = z.child;
22 do { kids.add(c); c = c.right; } while (c != z.child);
23 for (Node k : kids) addRoot(k);
24 }
25 z.left.right = z.right; z.right.left = z.left;
26 if (z == z.right) min = null;
27 else { min = z.right; consolidate(); }
28 size--;
29 return z.key;
30}
31static void consolidate() {
32 Node[] table = new Node[64];
33 List<Node> roots = new ArrayList<>();
34 Node c = min;
35 do { roots.add(c); c = c.right; } while (c != min);
36 for (Node w : roots) {
37 Node x = w;
38 int d = x.degree;
39 while (table[d] != null) { // two trees of degree d: link them into one of degree d + 1
40 Node y = table[d];
41 if (y.key < x.key) { Node t = x; x = y; y = t; }
42 link(y, x);
43 table[d++] = null;
44 }
45 table[d] = x;
46 }
47 min = null;
48 for (Node r : table) if (r != null) addRoot(r);
49}
50static void link(Node y, Node x) { // y becomes a child of x
51 y.parent = x; y.mark = false;
52 if (x.child == null) { x.child = y; y.left = y; y.right = y; }
53 else { y.left = x.child; y.right = x.child.right; x.child.right.left = y; x.child.right = y; }
54 x.degree++;
55}
56static void decreaseKey(Node x, int newKey) {
57 x.key = newKey;
58 Node p = x.parent;
59 if (p != null && x.key < p.key) { cut(x, p); cascadingCut(p); } // heap order broke: cut x out
60 if (x.key < min.key) min = x;
61}
62static void cut(Node x, Node p) {
63 if (x.right == x) p.child = null;
64 else { x.left.right = x.right; x.right.left = x.left; if (p.child == x) p.child = x.right; }
65 p.degree--;
66 addRoot(x);
67}
68static void cascadingCut(Node y) {
69 Node z = y.parent;
70 if (z == null) return;
71 if (!y.mark) y.mark = true; // the first lost child: just mark it
72 else { cut(y, z); cascadingCut(z); } // the second: cut it too, and continue upward
73}
27.06.4

Pairing heap

A **pairing heap** is a heap-ordered tree with a simple rule set: **meld** two heaps by making the one with the larger root a child of the other; insert is a meld with a one-node heap; **delete-min** removes the root and melds its children in **two passes**: first in pairs from left to right, then the results from right to left. It has tiny code and excellent practical speed, often beating Fibonacci heaps, although its exact amortised cost for decrease-key is still an open research question (known bounds are O(log n) amortised for delete-min and insert and meld in O(1)).

The complete implementation fits in a few lines because every operation is a meld.

Java
1static class Node {
2 int key;
3 Node child, sibling; // the first child and the next sibling
4 Node(int key) { this.key = key; }
5}
6static Node meld(Node a, Node b) {
7 if (a == null) return b;
8 if (b == null) return a;
9 if (b.key < a.key) { Node t = a; a = b; b = t; }
10 b.sibling = a.child; // the larger root becomes the first child of the smaller
11 a.child = b;
12 return a;
13}
14static Node insert(Node root, int key) { return meld(root, new Node(key)); }
15static Node deleteMin(Node root) { // the root's key was the minimum; returns the new root
16 List<Node> pairs = new ArrayList<>();
17 Node first = root.child;
18 while (first != null) { // pass 1: meld the children in pairs, left to right
19 Node a = first, b = a.sibling;
20 Node next = b == null ? null : b.sibling;
21 a.sibling = null;
22 if (b != null) b.sibling = null;
23 pairs.add(meld(a, b));
24 first = next;
25 }
26 Node result = null;
27 for (int i = pairs.size() - 1; i >= 0; i--) result = meld(pairs.get(i), result); // pass 2: right to left
28 return result;
29}
27.06.5

Indexed priority queue

Algorithms such as Dijkstra and Prim need to **change the priority of an item that is already in the queue**. An **indexed priority queue** gives each item a fixed index 0..n-1 and keeps two extra arrays: pq (the heap, holding indexes) and qp (the inverse: qp[i] is the heap position of item i). With qp the position of any item is found in O(1), so decreaseKey(i, key) is a sift-up from qp[i], O(log n), and contains(i) is qp[i] != -1.

It replaces Java's 'push a duplicate and skip stale entries' trick with a true decrease-key, keeping the heap at most n entries, which is better when the number of decrease-keys is large. The cost is the extra bookkeeping: every swap must update qp.

Java
1static class IndexedMinPQ {
2 private int n; // number of items in the queue
3 private final int[] pq, qp, key; // pq: heap of item indexes (1-based), qp[i]: position of item i or -1
4 IndexedMinPQ(int capacity) {
5 pq = new int[capacity + 1]; qp = new int[capacity]; key = new int[capacity];
6 Arrays.fill(qp, -1);
7 }
8 boolean contains(int i) { return qp[i] != -1; }
9 void insert(int i, int k) { key[i] = k; pq[++n] = i; qp[i] = n; swim(n); }
10 void decreaseKey(int i, int k) { key[i] = k; swim(qp[i]); }
11 int delMin() { // returns the index of the smallest key
12 int min = pq[1];
13 swap(1, n--);
14 sink(1);
15 qp[min] = -1;
16 return min;
17 }
18 int minKey() { return key[pq[1]]; }
19 boolean isEmpty() { return n == 0; }
20 private void swim(int p) { while (p > 1 && key[pq[p / 2]] > key[pq[p]]) { swap(p, p / 2); p /= 2; } }
21 private void sink(int p) {
22 while (2 * p <= n) {
23 int c = 2 * p;
24 if (c < n && key[pq[c + 1]] < key[pq[c]]) c++;
25 if (key[pq[p]] <= key[pq[c]]) break;
26 swap(p, c);
27 p = c;
28 }
29 }
30 private void swap(int x, int y) { int t = pq[x]; pq[x] = pq[y]; pq[y] = t; qp[pq[x]] = x; qp[pq[y]] = y; }
31}

Best

O(1)

Average

O(log n)

Worst

O(log n)

Space

O(n)

27.06.6

Decrease-key

**Decrease-key** lowers the priority value of an item already in the heap. In a binary heap the new key can only move **up**, so it is a sift-up from the item's position: O(log n), but you must first **know the position**, which needs the index map of the previous lesson. In the animation's “Decrease a key” preset, the key at index 4 of [1, 3, 5, 7, 9, 8, 6] drops from 9 to 2: it swaps with its parent 3 and stops below the root, 2 comparisons and 1 swap, giving [1, 2, 5, 7, 3, 8, 6].

The heaps differ most in how they handle this operation, which is why Fibonacci heaps matter in theory: decrease-key is O(1) amortised, and it is the operation Dijkstra and Prim perform up to E times.

Heapinsertextract-mindecrease-keymeld
Binary heapO(log n)O(log n)O(log n)O(n)
d-ary heapO(log_d n)O(d log_d n)O(log_d n)O(n)
Binomial heapO(1) amortisedO(log n)O(log n)O(log n)
Fibonacci heapO(1)O(log n) amortisedO(1) amortisedO(1)
Pairing heapO(1)O(log n) amortisedo(log n) amortised, exact bound openO(1)

Binary heap: sift-up, sift-down, insert, extract, replace, decrease-key and build-heap

Interactive · edit the input

Your input

Try

min-heap as a tree · 8 nodes

9

[0]

7

[1]

8

[2]

3

[3]

5

[4]

2

[5]

6

[6]

1

[7]

Being comparedJust swappedSettled in placeHeap order holds

heap array (index = position in the tree)

90
71
82
33
54
25
66
17
Setup

Heapify 8 values in place: the last 4 entries are leaves, so start at the last internal node and sift each one down, moving toward the root.

Step 1 / 24
Java
1static void siftUp(int[] a, int i) {
2 while (i > 0) {
3 int p = (i - 1) / 2; // parent
4 if (a[p] <= a[i]) break; // heap order restored
5 swap(a, p, i);
6 i = p;
7 }
8}
9static void siftDown(int[] a, int n, int i) {
10 while (true) {
11 int l = 2 * i + 1, r = l + 1, m = i;
12 if (l < n && a[l] < a[m]) m = l; // find the smallest of the node and its children
13 if (r < n && a[r] < a[m]) m = r;
14 if (m == i) break; // both children are larger: done
15 swap(a, i, m);
16 i = m;
17 }
18}
19static int size;
20static void insert(int[] a, int x) { a[size] = x; siftUp(a, size++); }
21static int extractMin(int[] a) {
22 int min = a[0];
23 a[0] = a[--size]; // move the last leaf to the root
24 siftDown(a, size, 0);
25 return min;
26}
27static void buildHeap(int[] a, int n) {
28 for (int i = n / 2 - 1; i >= 0; i--) siftDown(a, n, i); // the last n/2 entries are leaves already
29}
30static int replaceRoot(int[] a, int x) { // pop and push in one sift
31 int min = a[0];
32 a[0] = x;
33 siftDown(a, size, 0);
34 return min;
35}
36static void decreaseKey(int[] a, int i, int x) { a[i] = x; siftUp(a, i); }

Live state

0 comparisons0 swaps

size

8

No condition evaluated on this step
Execution trace (dry run table)Show 24 steps
StepLineStateWhat happens
1L1size=8Heapify 8 values in place: the last 4 entries are leaves, so start at the last internal node and sift each one down, moving toward the root.
2L28size=8 index=3Sift down the node at index 3 (3). Everything below it is already a heap.
3L12size=8 node=3Compare 3 with its only child 1. The smaller child is 1.
4L15size=8 node=3Swap 3 with 1: the smaller value moves up.
5L12size=8 node=3Compare 3 with its children: none, it is a leaf. Neither is smaller: heap order holds here.
6L28size=8 index=2Sift down the node at index 2 (8). Everything below it is already a heap.
7L12size=8 node=8Compare 8 with its children 2 and 6. The smaller child is 2.
8L15size=8 node=8Swap 8 with 2: the smaller value moves up.
9L12size=8 node=8Compare 8 with its children: none, it is a leaf. Neither is smaller: heap order holds here.
10L28size=8 index=1Sift down the node at index 1 (7). Everything below it is already a heap.
11L12size=8 node=7Compare 7 with its children 1 and 5. The smaller child is 1.
12L15size=8 node=7Swap 7 with 1: the smaller value moves up.
13L12size=8 node=7Compare 7 with its only child 3. The smaller child is 3.
14L15size=8 node=7Swap 7 with 3: the smaller value moves up.
15L12size=8 node=7Compare 7 with its children: none, it is a leaf. Neither is smaller: heap order holds here.
16L28size=8 index=0Sift down the node at index 0 (9). Everything below it is already a heap.
17L12size=8 node=9Compare 9 with its children 1 and 2. The smaller child is 1.
18L15size=8 node=9Swap 9 with 1: the smaller value moves up.
19L12size=8 node=9Compare 9 with its children 3 and 5. The smaller child is 3.
20L15size=8 node=9Swap 9 with 3: the smaller value moves up.
21L12size=8 node=9Compare 9 with its only child 7. The smaller child is 7.
22L15size=8 node=9Swap 9 with 7: the smaller value moves up.
23L12size=8 node=9Compare 9 with its children: none, it is a leaf. Neither is smaller: heap order holds here.
24L1size=8 heap order=holds[1, 3, 2, 7, 5, 8, 6, 9]: heap order holds. 11 comparisons and 7 swaps.

Practice ladder

Now try it yourself

Easy

  • Last Stone Weight (with a d-ary heap)
  • Kth Smallest Element in a Sorted Matrix

Medium

  • Network Delay Time (indexed queue)
  • Path With Minimum Effort

Advanced

  • Implement a Fibonacci heap
  • Minimum Spanning Tree with an indexed priority queue

Key takeaways

  • A d-ary heap trades cheaper sift-up for dearer sift-down; larger d suits decrease-key-heavy workloads.
  • Binomial heaps meld in O(log n) by adding binary-number-style trees.
  • Fibonacci heaps give O(1) amortised insert, meld and decrease-key, with large constants.
  • Pairing heaps are tiny and fast in practice; every operation is a meld.
  • An indexed priority queue keeps a position map so decrease-key is a sift-up from a known position.