Quicksort
Quicksort as a partition-sorting algorithm, understanding its worst-case behavior, and designing real-world optimizations.
Kevin Lin, with thanks to many others.
1
Ask questions anonymously on Piazza. Look for the pinned Lecture Questions thread.
Feedback from the Reading Quiz
2
Naive Quicksort
Partition around a pivot item, e.g. leftmost item.
Quicksort left side, all keys ≤ pivot.
Quicksort right side, all keys ≥ pivot.
3
Demo
32
15
2
17
19
26
41
17
17
15
2
17
19
26
17
17
32
41
Partition(32)
≤ 32
≥ 32
32’s final position
4
Quicksort Case Analysis
Give the tight asymptotic time complexity of naive quicksort assuming no duplicate keys.
5
Beyond Worst-Case Runtime Analysis
Naive Quicksort. Ω(N log N), O(N2) time complexity.
Mergesort. Θ(N log N) time complexity.
Java Quicksort. Ω(N), O(N2) time complexity?
Suppose a supercomputer is 10,000 times faster than a home computer.
6
Algorithms (Robert Sedgewick, Kevin Wayne/Princeton)
7
Sort
Best-Case
Worst-Case
Space
Stable
Notes
Selection Sort
Θ(N2)
Θ(N2)
Θ(1)
No
Heapsort
Θ(N)
Θ(N log N)
Θ(1)
No
Slow in practice.
Merge Sort
Θ(N log N)
Θ(N log N)
Θ(N)
Yes
Fastest stable sort.
Insertion Sort
Θ(N)
Θ(N2)
Θ(1)
Yes
Best for small or almost sorted inputs.
Naive Quicksort
Θ(N log N)
Θ(N2)
Θ(N)
Yes
2x or more slower than merge sort.
Java Quicksort
Ω(N)
O(N2)
?
No
Fastest comparison sort.
Argument 1: 10% Case
Suppose the pivot is always at least 10% from either edge (not to scale).
8
N
N/10
9N/10
N/100
9N/100
9N/100
81N/100
Work at each level is in O(N).
Height is about log10/9 N ∈ O(log N).
Overall: O(N log N).
Partition-sorting
9
5
3
2
1
8
4
6
7
3
2
1
4
7
8
6
5
3
2
1
4
5
7
8
6
2
1
3
4
5
6
7
8
Argument 2: Binary Search Tree Analogy
Random insertion into a binary search tree is expected to take O(N log N) time.
10
5
3
2
1
8
4
6
7
3
2
1
4
7
8
6
2
1
4
6
8
5
3
5
7
5
3
7
2
4
6
8
1
Optimizing Quicksort
Naive Quicksort.
Recursive Depth. Ω(log N), O(N).
Pivot choice. Leftmost item. Θ(1)
Common worst-case: sorted array!
Partitioning. Allocate a new array. Θ(N)
Slow but stable.
Common worst-case: all duplicates!
Java Quicksort–5x or more faster.
Recursive Depth. Ω(log N), O(N).
Pivot choice. Approximate median. Θ(1)
Resilient to worst-case inputs.
Partitioning. Long-distance swaps. Θ(N)
In-place, fast, but unstable.
3-way partition to handle duplicates.
11
Pivot Choice
12
Median-Finding
Goal. Find the median item in O(N) time.
Reduces to the selection problem.
Selection. Given an array of N items, find item of rank K.
For median, find K = N / 2.
13
Algorithms (Robert Sedgewick, Kevin Wayne/Princeton)
Median-Finding
Goal. Find the median item in O(N) time.
Reduces to the selection problem.
Selection. Given an array of N items, find item of rank K.
For median, find K = N / 2.
How difficult is this problem?
Why is the time complexity of selection in Ω(N)?
Describe an O(N log N) runtime algorithm for selection with any K.
Describe an O(N) runtime algorithm for selection with K = 0, 1, 2.
14
Algorithms (Robert Sedgewick, Kevin Wayne/Princeton)
Q
Median-Finding
Why is the time complexity of selection lower bounded by N?
Need to consider all of the elements in the array at least once.
Describe an O(N log N) runtime algorithm for selection with any K. By reduction to sorting.
No transformation of inputs.
Sort the input array with merge sort in Θ(N log N) time.
Return the item at index K of the sorted array.
Describe an O(N) runtime algorithm for selection with K = 0, 1, 2.
K = 0 is the min item. Scan for the minimum item. K = 1 is the next min. Scan twice. K = 2 is the next-next min. Scan three times…
15
A
If selection reduces to sorting, which of the following statements about problem difficulty is true?
16
Quickselect: Partition-selection with leftmost item as pivot
17
550
14
10
330
5
4
817
913
14
10
330
550
5
4
817
913
6
5
9
550
14
10
330
817
913
9
550
14
6
10
5
330
817
913
14
10
330
5
4
4
4
5
10
14
330
5
4
4
4
5
Median can’t be here
Quickselect Runtime
Worst case to find median, K = N / 2. Poor pivot choices (i.e. spindly tree).
Similar analysis as running the first half of selection sort.
Best case to find median, K = N / 2. Good pivot choices (i.e. bushy tree).
18
Approximate Median-Finding
Unfortunate reality: Quicksort with quickselect pivots is significantly slower than merge sort.
Goal. Find the approximate median item in Θ(1) time.
Median-of-3. Pick 3 items and take the median of the sample.
19
if (a < b)
if (b < c) return b;
else if (a < c) return c;
else return a;
else
if (a < c) return a;
else if (b < c) return c;
else return b;
20
Median-of-3 Decision Tree
a ⩻ b
b ⩻ c
No
Yes
a ⩻ c
a c b
Yes
c a b
No
a ⩻ c
b ⩻ c
b a c
b c a
c b a
Yes
No
No
Yes
No
Yes
a b c
Partitioning
21
Hoare Partitioning
Hoare partitioning. In-place, unstable partitioning algorithm. Initialize an int L and an int G.
L. Left pointer that loves small items < pivot.
G. Right pointer that loves big items > pivot.
Idea. Walk towards each other, swapping anything they don’t like.
End result is that things on left are “small” and things on the right are “large”.
Hoare partitioning improves real-world runtime and space complexity.
Asymptotic time complexity still depends on pivot choice!
22
Demo
Optimizing Quicksort
Naive Quicksort.
Recursive Depth. Ω(log N), O(N).
Pivot choice. Leftmost item. Θ(1)
Common worst-case: sorted array!
Partitioning. Allocate a new array. Θ(N)
Slow but stable.
Common worst-case: all duplicates!
Java Quicksort–5x or more faster.
Recursive Depth. Ω(log N), O(N).
Pivot choice. Approximate median. Θ(1)
Resilient to worst-case inputs.
Partitioning. Long-distance swaps. Θ(N)
In-place, fast, but unstable.
3-way partition to handle duplicates.
23
Dual-Pivot Quicksort
24
Dual-Pivot Quicksort: Data Structure Analogy
If classic quicksort is analogous to BSTs, then dual-pivot quicksort is analogous to 2-3 trees.
25
2
6
4
1
3
5
7
3 5
1 2
6 7
4
“Classic Quicksort”
Dual-Pivot Quicksort
Dual-Pivot Quicksort
Use two partitioning keys p1 and p2 and partition into three subarrays:
Keys less than p1.
Keys between p1 and p2.
Keys greater than p2.
Recursively quicksort the three subarrays (skip middle subarray if p1 = p2).
Now widely used. Java 8, Python unstable sort, Android, …
26
Algorithms (Robert Sedgewick, Kevin Wayne/Princeton)
27
Sort
Best-Case
Worst-Case
Space
Stable
Notes
Selection Sort
Θ(N2)
Θ(N2)
Θ(1)
No
Heapsort
Θ(N)
Θ(N log N)
Θ(1)
No
Slow in practice.
Merge Sort
Θ(N log N)
Θ(N log N)
Θ(N)
Yes
Fastest stable sort.
Insertion Sort
Θ(N)
Θ(N2)
Θ(1)
Yes
Best for small or almost sorted inputs.
Naive Quicksort
Θ(N log N)
Θ(N2)
Θ(N)
Yes
2x or more slower than merge sort.
Java Quicksort
Θ(N)
Θ(N2)
O(log N)
No
Fastest comparison sort.
In Sep 2009, on the JDK developers’ mailing list…
Replacement of quicksort in java.util.Arrays with new dual-pivot quicksort
The invariant of classical Quicksort is:
[ <= p | >= p ]
The new Dual-Pivot Quicksort uses *two* pivots elements in this manner:
Pick an elements P1, P2, called pivots from the array.
Assume that P1 <= P2, otherwise swap it.
Reorder the array into three parts: those less than the smaller pivot, those larger than the larger pivot, and in between are those elements between (or equal to) the two pivots.
Recursively sort the sub-arrays.
The invariant of the Dual-Pivot Quicksort is:
[ < P1 | P1 <= & <= P2 | > P2 ]
28
Replacement of quicksort in java.util.Arrays with new dual-pivot quicksort (Vladimir Yaroslavskiy/Oracle Corporation)
What is your burning question from today’s lecture?
29