Link

Quicksort Reading

Table of contents

  1. Partitioning problem
  2. Naive quicksort
  3. Binary search tree analogy
  4. Real-world runtime
    1. In-place partitioning
    2. Median selection

Merge sort is an example of a divide-and-conquer algorithm: an algorithm that solves a problem by breaking down the problem into two or more smaller subproblems, solving each recursive subproblem, and then combining the results to solve the original problem.

It turns out that Java uses a variant of the quicksort algorithm for sorting primitive data types. Quicksort is another divide-and-conquer sorting algorithm.

  1. Select a pivot item from the array.
  2. Partition around the pivot to yield two subproblems: items less than the pivot and items greater than the pivot.
  3. Recursively quicksort the partitions.

While the runtime for merge sort depended only on the merge operation, the runtime for quicksort depends on two algorithm design decisions: pivot selection and partitioning.

Partitioning problem

Given an array a, partitioning around pivot x = a[i] rearranges the items in a as follows.

  • Pivot x is in its sorted, final position in the array index a[j].
  • All items to the left of x are less than or equal to x.
  • All items to the right of x are greater than or equal to x.

Identify Valid Partitions

Which of the above partitions are valid?

Options A, B, and C are valid partitions. Option D is invalid because the entry to the immediate right of x is 4, which is less than x. Note that the relative order of items to the left of x does not need to stay the same as the input. The same applies to the relative order of items to the right of x.

Naive quicksort

Naive quicksort makes the following algorithm design decisions.

Pivot selection
Leftmost item in the current subproblem, x = a[0].
Partitioning
  1. Create a new output array.
  2. Scan across the input array once to add all items a[i] < x to the output array.
  3. Add the pivot x to the output array.
  4. Scan across the input array again to add all items a[i] >= x to the output array.
Is naive quicksort stable?

Yes, naive quicksort is stable because the partitioning algorithm is stable since it maintains the relative ordering of equal items. We’ll see later that switching to a faster but unstable in-place partitioning algorithm will result in an unstable quicksort.

Binary search tree analogy

The runtime analysis for naive quicksort is analogous to spindly vs. bushy binary search trees. In the same way that each node in a BST partitions the space left and right, quicksort partitions the array into items less-than (or equal-to) the pivot and items greater-than the pivot.

Binary Search Tree Analogy for Quicksort

Give an input that will result in worst-case runtime if the leftmost item is always chosen as the pivot.

Any input values that would form a spindly tree would work. An array sorted in ascending or descending order, for example. Or, an array whose values alternate between the next-smallest and next-largest values as another example.

Give a tight asymptotic runtime bound for naive quicksort assuming unique items.

The runtime in the best case is in Theta(N log N), using a similar analysis to merge sort. If every pivot splits down the approximate middle, then we’ll have log2 N levels where each level does roughly N work.

The runtime in the worst case is in Theta(N2), following the same analysis as selection sort.

Real-world runtime

Why use quicksort if its asymptotic runtime is no better than merge sort? Consider the following running time estimates to sort one billion random items.

ComputerInsertion sortMerge sortNaive quicksortJava quicksort
Laptop317 years18 min38 min12 min
Supercomputer1 weekinstantinstantinstant

Quicksort is an example of how algorithm design decisions impact real-world runtime. Even though quicksort has a worst-case quadratic runtime, the constant factors vary significantly depending on the pivot selection and partitioning algorithms.

In-place partitioning

Naive quicksort is slow in part due to the time it takes to scan across the input and copy items into a new output array. We can improve runtime by designing a in-place partitioning that swaps items in the input array instead of copying them into a new output array.

Hoare partitioning works by creating int L and int G pointers at initialized to the left and right sides of the array.

  • The L pointer loves small items and hates large or equal items.
  • The G pointer loves large items and hates small or equal items.
  • Walk the pointers towards each other, stopping on a hated item. When both pointers stop, swap and move both pointers by one.
  • When the pointers cross (G is to the immediate left of L), swap the pivot item with the item at G.

Median selection

In-place partitioning improves runtime in the typical case, but doesn’t help with avoiding worst-case quadratic runtime.

Select a better pivot
Select the exact or approximate median as the pivot.
Randomness
Select a pivot at random or shuffle the input before sorting.
Introspection
Switch to a safer sorting algorithm based on recursive depth.

We will study methods of selecting a better pivot.

Give a tight asymptotic runtime bound for finding the exact median item in an array.

O(N log N). Insert all items into a priority queue and then remove N / 2 of the items to find the median. Or, sort the array using merge sort and pick the item at index N / 2.

It turns out that it’s possible to find the exact median item in worst-case linear time using a combination of algorithm ideas.1

The unfortunate reality is that quicksort with exact median selection is about 5 times slower than merge sort in practice. To save time, consider how we might compute an approximate median item in constant time with the median-of-3 algorithm.

Median-of-3
Sample 3 items from the array and take the median of the 3 items. Java picks 3 evenly-spaced indices of the array.
/** Return the median of 3 numbers in constant time. */
static int median3(int a, int b, int c) {
    if (a < b) {
        if      (b < c) return b; // a b c
        else if (a < c) return c; // a c b
        else            return a; // c a b
    } else {
        if      (a < c) return a; // b a c
        else if (b < c) return c; // b c a
        else            return b; // c b a
    }
}

Median-of-3 Decision Tree

What is the worst-case runtime for naive quicksort if we use median-of-3 pivot selection?

If Java picks items from 3 evenly-spaced indices, then we can still construct a worst-case where the median of the sample still results in a relatively spindly tree and quadratic runtime. But this provides a bit more robustness against the basic worst-case of ascending or descending inputs without significantly slowing down quicksort.

  1. Manuel Blum, Robert W. Floyd, Vaughan Pratt, Ronald L. Rivest, and Robert E. Tarjan. 1973. Time Bounds for Selection. Journal of Computer and System Sciences 7, 448–461. DOI: https://doi.org/10.1016%2FS0022-0000%2873%2980033-9