Chapter 21: Greedy Algorithms and Exchange Arguments
21.06Intermediate4 topics16 min4 interactive animations

Greedy with sorting/heaps

Greedy algorithms driven by sorting and heaps: connecting ropes, optimal merge patterns, scheduling with deadlines and maximising the tasks finished before their deadlines.

Learning goals

  • Minimise total merge cost with a min-heap and relate it to Huffman coding
  • Generalise the merge pattern to k-way merges
  • Check whether tasks can all meet their deadlines by earliest-deadline-first
  • Maximise the number of tasks finished using a max-heap and replacement

Prerequisites

Greedy fundamentalsString and frequency greedy

When the greedy choice depends on values that change as you go, a heap is the data structure that keeps the best candidate at the top.

21.06.1

Connect ropes minimum cost

Joining two ropes costs the sum of their lengths. To join them all at minimal total cost, always join the **two shortest** ropes, and push the joined rope back into a min-heap. The “4 3 2 6” preset costs 5 + 9 + 15 = 29. A long rope is best joined late, because every join it takes part in charges its whole length again.

This is exactly the Huffman merge step with lengths instead of frequencies. The exchange argument is the same: in an optimal order the two shortest ropes are joined first.

Java
1static int connectRopes(int[] ropes) {
2 PriorityQueue<Integer> heap = new PriorityQueue<>();
3 for (int r : ropes) heap.add(r);
4 int cost = 0;
5 while (heap.size() > 1) {
6 int joined = heap.poll() + heap.poll(); // the two shortest ropes
7 cost += joined;
8 heap.add(joined);
9 }
10 return cost;
11}

Best

O(n log n)

Average

O(n log n)

Worst

O(n log n)

Space

O(n) for the heap

Connect ropes with minimum total cost

Interactive · edit the input

Your input

Try

ropes (shortest first)

2
3
4
6
Setup

Joining two ropes costs the length of the result, and that rope may be joined again later. So long ropes should be joined as late as possible: always join the two shortest.

Step 1 / 5
Java
1static int connectRopes(int[] ropes) {
2 PriorityQueue<Integer> pq = new PriorityQueue<>();
3 for (int r : ropes) pq.add(r);
4 int cost = 0;
5 while (pq.size() > 1) {
6 int merged = pq.poll() + pq.poll(); // always join the two shortest
7 cost += merged; // the cost of a join is the length of the result
8 pq.add(merged);
9 }
10 return cost;
11}

Live state

0 joins

ropes

4

cost

0

No condition evaluated on this step
Execution trace (dry run table)Show 5 steps
StepLineStateWhat happens
1L3ropes=4 cost=0Joining two ropes costs the length of the result, and that rope may be joined again later. So long ropes should be joined as late as possible: always join the two shortest.
2L6joined=2 + 3 merged=5 cost=5Join 2 and 3 into 5 (cost 5, total 5).
3L6joined=4 + 5 merged=9 cost=14Join 4 and 5 into 9 (cost 9, total 14).
4L6joined=6 + 9 merged=15 cost=29Join 6 and 9 into 15 (cost 15, total 29).
5L10return=29Minimum total cost 29. This is exactly the Huffman merge rule, with lengths in place of frequencies.
21.06.2

Merge files optimal pattern

Merging two sorted files of sizes a and b costs a + b record moves, and merging all files pairwise forms a binary tree whose internal nodes are the intermediate files. Total cost is the sum of every internal node's weight, which equals each file's size times its depth in the merge tree. That is the same quantity Huffman minimises, so the optimal merge pattern is again: merge the two smallest files first.

For sizes 20, 30, 10, 5, 30 the merges cost 15, 35, 60 and 95, a total of 205. Merging in the order given instead (20 and 30, then add 10, 5 and 30 in turn) would cost 50 + 60 + 65 + 95 = 270.

A **k-way** merge combines k files at once. The greedy rule becomes: merge the k smallest. If the file count does not satisfy (n - 1) mod (k - 1) = 0, pad with zero-size files at the front first so every merge is full; this is the standard fix for k-ary Huffman codes too.

Java
1static int mergeCost(int[] sizes, int k) {
2 PriorityQueue<Integer> heap = new PriorityQueue<>();
3 for (int s : sizes) heap.add(s);
4 while ((heap.size() - 1) % (k - 1) != 0) heap.add(0); // pad so every merge combines exactly k files
5 int cost = 0;
6 while (heap.size() > 1) {
7 int merged = 0;
8 for (int i = 0; i < k; i++) merged += heap.poll();
9 cost += merged;
10 heap.add(merged);
11 }
12 return cost;
13}

Best

O(n log n)

Average

O(n log n)

Worst

O(n log n)

Space

O(n) for the heap

Connect ropes with minimum total cost

Interactive · edit the input

Your input

Try

ropes (shortest first)

2
3
4
6
Setup

Joining two ropes costs the length of the result, and that rope may be joined again later. So long ropes should be joined as late as possible: always join the two shortest.

Step 1 / 5
Java
1static int connectRopes(int[] ropes) {
2 PriorityQueue<Integer> pq = new PriorityQueue<>();
3 for (int r : ropes) pq.add(r);
4 int cost = 0;
5 while (pq.size() > 1) {
6 int merged = pq.poll() + pq.poll(); // always join the two shortest
7 cost += merged; // the cost of a join is the length of the result
8 pq.add(merged);
9 }
10 return cost;
11}

Live state

0 joins

ropes

4

cost

0

No condition evaluated on this step
Execution trace (dry run table)Show 5 steps
StepLineStateWhat happens
1L3ropes=4 cost=0Joining two ropes costs the length of the result, and that rope may be joined again later. So long ropes should be joined as late as possible: always join the two shortest.
2L6joined=2 + 3 merged=5 cost=5Join 2 and 3 into 5 (cost 5, total 5).
3L6joined=4 + 5 merged=9 cost=14Join 4 and 5 into 9 (cost 9, total 14).
4L6joined=6 + 9 merged=15 cost=29Join 6 and 9 into 15 (cost 15, total 29).
5L10return=29Minimum total cost 29. This is exactly the Huffman merge rule, with lengths in place of frequencies.
21.06.3

Scheduling with deadlines

Given tasks with a duration and a deadline, can they all be finished on one machine in time? Sort by **deadline** (earliest deadline first, EDF) and run them back to back: the schedule is feasible exactly when every task's finish time is at most its deadline. EDF is optimal because swapping two adjacent tasks that are out of deadline order never makes the later of the two finish later than its deadline.

The same rule also minimises the **maximum lateness** (how far past a deadline any task finishes) when all tasks must run: EDF gives the smallest possible maximum lateness. The next lesson animates the harder variant where some tasks must be dropped.

Java
1// tasks[i] = {duration, deadline}
2static boolean canFinishAll(int[][] tasks) {
3 int[][] a = tasks.clone();
4 Arrays.sort(a, Comparator.comparingInt(x -> x[1])); // earliest deadline first
5 int time = 0;
6 for (int[] t : a) {
7 time += t[0];
8 if (time > t[1]) return false;
9 }
10 return true;
11}
12
13static int maxLateness(int[][] tasks) {
14 int[][] a = tasks.clone();
15 Arrays.sort(a, Comparator.comparingInt(x -> x[1]));
16 int time = 0, worst = 0;
17 for (int[] t : a) {
18 time += t[0];
19 worst = Math.max(worst, time - t[1]); // negative lateness counts as zero
20 }
21 return worst;
22}

Best

O(n log n)

Average

O(n log n)

Worst

O(n log n)

Space

O(1) beyond the sorted copy

Most tasks before their deadlines (sort + max-heap)

Interactive · edit the input

Your input

Try

tasks by deadline

10:15T4
10:20T1
20:50T2
30:60T3

chosen durations (max first) · time 0

none
Setup

Consider tasks in order of deadline and tentatively take each one. If the total time then overshoots the current deadline, drop the longest task taken so far: that frees the most time while losing only one task.

Step 1 / 7
Java
1static int maxTasks(int[][] tasks) { // tasks[i] = {duration, deadline}
2 Arrays.sort(tasks, Comparator.comparingInt(a -> a[1])); // earliest deadline first
3 PriorityQueue<Integer> chosen = new PriorityQueue<>(Collections.reverseOrder()); // durations of the chosen tasks
4 int time = 0;
5 for (int[] t : tasks) {
6 chosen.add(t[0]); time += t[0];
7 if (time > t[1]) time -= chosen.poll(); // too late: drop the longest chosen task
8 }
9 return chosen.size();
10}

Live state

tasks

4

No condition evaluated on this step
Execution trace (dry run table)Show 7 steps
StepLineStateWhat happens
1L3tasks=4Consider tasks in order of deadline and tentatively take each one. If the total time then overshoots the current deadline, drop the longest task taken so far: that frees the most time while losing only one task.
2L6task=T4 time=10 deadline=15Take T4 (duration 10): total time 10 against deadline 15.
3L6task=T1 time=20 deadline=20Take T1 (duration 10): total time 20 against deadline 20.
4L6task=T2 time=40 deadline=50Take T2 (duration 20): total time 40 against deadline 50.
5L6task=T3 time=70 deadline=60Take T3 (duration 30): total time 70 against deadline 60.
6L7dropped=30 time=40Too late: drop the longest chosen task (duration 30). Time is back to 40.
7L9return=33 tasks can be finished in time.
21.06.4

Max tasks before deadlines

When not every task fits, which largest set of tasks can finish by their deadlines? Process tasks in deadline order and keep a **max-heap of the durations** taken so far with their total time. Add each task; if the total exceeds the current deadline, drop the longest task taken so far (which might be the new one). Dropping the longest frees the most time while losing only one task.

The “Swap out the longest task” preset (duration:deadline pairs 10:20, 20:50, 30:60, 10:15) finishes 3 of 4: after the 30-minute task joins, the total of 70 exceeds its deadline of 60, so the longest task taken (the 30) is swapped out. “Classic” finishes 4 of 5, and “Nothing fits” finishes 0.

Java
1// courses[i] = {duration, lastDay}
2static int scheduleCourse(int[][] courses) {
3 int[][] a = courses.clone();
4 Arrays.sort(a, Comparator.comparingInt(x -> x[1])); // by deadline
5 PriorityQueue<Integer> taken = new PriorityQueue<>(Collections.reverseOrder());
6 int time = 0;
7 for (int[] c : a) {
8 taken.add(c[0]);
9 time += c[0];
10 if (time > c[1]) time -= taken.poll(); // drop the longest task taken so far
11 }
12 return taken.size();
13}

Best

O(n log n)

Average

O(n log n)

Worst

O(n log n)

Space

O(n) for the heap

Most tasks before their deadlines (sort + max-heap)

Interactive · edit the input

Your input

Try

tasks by deadline

10:15T4
10:20T1
20:50T2
30:60T3

chosen durations (max first) · time 0

none
Setup

Consider tasks in order of deadline and tentatively take each one. If the total time then overshoots the current deadline, drop the longest task taken so far: that frees the most time while losing only one task.

Step 1 / 7
Java
1static int maxTasks(int[][] tasks) { // tasks[i] = {duration, deadline}
2 Arrays.sort(tasks, Comparator.comparingInt(a -> a[1])); // earliest deadline first
3 PriorityQueue<Integer> chosen = new PriorityQueue<>(Collections.reverseOrder()); // durations of the chosen tasks
4 int time = 0;
5 for (int[] t : tasks) {
6 chosen.add(t[0]); time += t[0];
7 if (time > t[1]) time -= chosen.poll(); // too late: drop the longest chosen task
8 }
9 return chosen.size();
10}

Live state

tasks

4

No condition evaluated on this step
Execution trace (dry run table)Show 7 steps
StepLineStateWhat happens
1L3tasks=4Consider tasks in order of deadline and tentatively take each one. If the total time then overshoots the current deadline, drop the longest task taken so far: that frees the most time while losing only one task.
2L6task=T4 time=10 deadline=15Take T4 (duration 10): total time 10 against deadline 15.
3L6task=T1 time=20 deadline=20Take T1 (duration 10): total time 20 against deadline 20.
4L6task=T2 time=40 deadline=50Take T2 (duration 20): total time 40 against deadline 50.
5L6task=T3 time=70 deadline=60Take T3 (duration 30): total time 70 against deadline 60.
6L7dropped=30 time=40Too late: drop the longest chosen task (duration 30). Time is back to 40.
7L9return=33 tasks can be finished in time.

Practice ladder

Now try it yourself

Easy

  • Last Stone Weight
  • Minimum Cost to Connect Sticks (premium)

Medium

  • Course Schedule III
  • Task Scheduler
  • Furthest Building You Can Reach

Advanced

  • IPO
  • Minimum Cost to Hire K Workers

Key takeaways

  • Connecting ropes and merging files are Huffman's algorithm on lengths: always merge the two smallest.
  • k-way merges pad with zero-size files so every merge combines exactly k.
  • Earliest-deadline-first decides feasibility and minimises maximum lateness.
  • To maximise the number of tasks, process by deadline and drop the longest task whenever time runs over.