Comparison Sorts
Optimizing selection sort, analyzing sorting algorithm stability, and quantifying sortedness.
Kevin Lin, with thanks to many others.
1
Ask questions anonymously on Piazza. Look for the pinned Lecture Questions thread.

Feedback from the Reading Quiz
2

Performance Definitions
Time complexity. Characterizations of algorithm runtime efficiency.
Dijkstra’s has time complexity O(E log V).

Space complexity. Characterizations of added memory usage of an algorithm.
Dijkstra’s has space complexity Θ(V) for the fringe, distTo, and edgeTo.

Note that the graph takes up space Θ(V + E), but we don’t count this as part of the space complexity of Dijkstra since the graph itself already exists and is an input to Dijkstra’s.
3

Selection Sort
Selection sort. Repeatedly select the smallest remaining item and swap it to its proper index.
Find the smallest item in the array, and swap it with the first item.
Find the next smallest item in the array, and swap it with the next item.
Continue until all items in the array are sorted.

Θ(N2) time complexity.
Θ(1) space complexity.

We look through entire remaining array every time to find the minimum.
4
Demo

Idea. Instead of rescanning entire array for min, maintain a heap so that finding min is fast!

Heapsort. Selection sort with a max-oriented heap–neat trick for saving memory later.
O(1). Create output array.
O(N log N). Insert all items into a new max-heap (separate array for the heap).
Repeat N times:
O(log N). Delete max item from the max-heap.
O(1). Put max item at the end of the unused part of the output array.
O(N log N) time complexity.
Θ(N) space complexity to create Θ(N) separate array heap and Θ(N) output array.
Naive Heapsort with a Max-Heap
5
Demo

In-place Heapsort
Idea. Save ~2N memory by treating the input array as a heap. Avoid extra copies of data.

Bottom-up heapification. Efficient heap construction by sinking nodes in reverse level order.
Once heap-ified, algorithm is almost the same as naive heap sort.
6
Demo
32
15
2
17
19
26
41
17
17
41
19
32
17
15
26
2
17
17
Heapification

Idea. Save ~2N memory by treating the input array as a heap. Avoid extra copies of data.

Bottom-up heapification. Efficient heap construction by sinking nodes in reverse level order.
Once heap-ified, algorithm is almost the same as naive heap sort.
Bottom-up heapification.
Repeat N times:
O(log N). Delete max item from the max-heap.
O(1). Put max item at the end the array.

Give the tight asymptotic time complexity of in-place heapsort in big-O notation.
In-place Heapsort Runtime
7
Q

Give the tight asymptotic time complexity of in-place heapsort in big-O notation.
8

Bottom-up heapification.
Repeat N times:
O(log N). Delete max item from the max-heap.
O(1). Put max item at the end the array.

O(N log N) time overall. Θ(1) space complexity for extra heapification variables, e.g. size.
Bottom-up heapification is N sink operations, each taking no more than O(log N) time, so overall runtime for heapification is O(N log N).
Extra: Show that bottom-up heapification is Θ(N) in the worst case.
Extra: Show heapsort is Θ(N log N) in the worst case.
In-place Heapsort Runtime
9
A

Heapsort Stability
Bottom-up heapification.
Repeat N times:
Delete max item from the max-heap.
Put max item at the end the array.

Heapsort is not stable. Give an example.
10
Q

Heapsort Stability
Bottom-up heapification.
Repeat N times:
Delete max item from the max-heap.
Put max item at the end the array.

Heapsort is not stable. Give an example.
11
A
1B
1A
1D
1C
1A
1B
1C
1D

Heapsort Stability
Bottom-up heapification.
Repeat N times:
Delete max item from the max-heap.
Put max item at the end the array.

Heapsort is not stable. Give an example.
12
A
1D
1B
1C
1B
1D
1C
1A

Selection Sort Stability
Selection sort. Repeatedly select the smallest remaining item and swap it to its proper index.
Find the smallest item in the array, and swap it with the first item.
Find the next smallest item in the array, and swap it with the next item.
Continue until all items in the array are sorted.

Selection sort is not stable. Give an example.
13
Q

Selection Sort Stability
Selection sort. Repeatedly select the smallest remaining item and swap it to its proper index.
Find the smallest item in the array, and swap it with the first item.
Find the next smallest item in the array, and swap it with the next item.
Continue until all items in the array are sorted.

Selection sort is not stable. Give an example.
14
2A
2B
2C
1
A

Selection Sort Stability
Selection sort. Repeatedly select the smallest remaining item and swap it to its proper index.
Find the smallest item in the array, and swap it with the first item.
Find the next smallest item in the array, and swap it with the next item.
Continue until all items in the array are sorted.

Selection sort is not stable. Give an example.
15
1
2B
2C
2A
A

16
Sort
Best-Case
Worst-Case
Space
Stable
Notes
Selection Sort
Θ(N2)
Θ(N2)
Θ(1)
No
Heapsort
Θ(N)
Θ(N log N)
Θ(1)
No
Slow in practice.

Merge Sort
Merge sort.
If array is of size 1, return.
Merge sort the left half.
Merge sort the right half.
Merge the two sorted halves.

Θ(N log N) time complexity.
Θ(N) space complexity.
Stable!
17
Demo
N=64
N=32
N=32
~32
~64
~32
M
M
M
16
16
16
16
M
M
M
M
~16
8
8
M
M
~8
···
···

18
Sort
Best-Case
Worst-Case
Space
Stable
Notes
Selection Sort
Θ(N2)
Θ(N2)
Θ(1)
No
Heapsort
Θ(N)
Θ(N log N)
Θ(1)
No
Slow in practice.
Merge sort
Θ(N log N)
Θ(N log N)
Θ(N)
Yes
Faster than heapsort.

Insertion Sort
Unstable sorting algorithms (heapsort, selection sort) use long-distance swaps.
Merge sort, a stable sort, uses the fact that left-half items come before right-half items.

Idea. Build a sorted subarray (like selection sort) by using left-neighbor swaps for stability.
Insertion sort. Scan from left to right…
If an item is out of order with respect to its left-neighbor, swap left.
Keep on swapping left until the item is in order with respect to its left-neighbor.
19
Demo

20
Insertion Sort Examples
P O T A T O
P O T A T O  (0 swaps)
O P T A T O  (1 swap )
O P T A T O  (0 swaps)
A O P T T O  (3 swaps)
A O P T T O  (0 swaps)
A O O P T T  (3 swaps)
S O R T E X A M P L E
S O R T E X A M P L E  (0 swaps)
O S R T E X A M P L E  (1 swap )
O R S T E X A M P L E  (1 swap )
O R S T E X A M P L E  (0 swaps)
E O R S T X A M P L E  (4 swaps)
E O R S T X A M P L E  (0 swaps)
A E O R S T X M P L E  (6 swaps)
A E M O R S T X P L E  (5 swaps)
A E M O P R S T X L E  (4 swaps)
A E L M O P R S T X E  (7 swaps)
A E E L M O P R S T X  (8 swaps)
Purple	Item that we’re swapping left.
Black	Item swapped with purple item.
Grey		Not considered this iteration.
Algorithms (Robert Sedgewick, Kevin Wayne/Princeton)

Give the tight asymptotic time complexity of insertion sort.
21

Quantifying Sortedness
Inversion. A pair of keys that are out of order.



Partially sorted. An array of length N is partially sorted if the number of inversions is in O(N).
Sorted array has 0 inversions.
Sorted subarray with 10 random items at the end has at most 10N + 45 inversions.

Each insertion sort local swap fixes 1 inversion. Θ(N + K) where K is number of inversions.
22
A E E L M O T R X P S
1
2
3
4
5
6
T–R
T–P
T–S
R–P
X–P
X–S
COS 226: Elementary Sorts (Robert Sedgewick, Kevin Wayne/Princeton)

23
Sort
Best-Case
Worst-Case
Space
Stable
Notes
Selection Sort
Θ(N2)
Θ(N2)
Θ(1)
No
Heapsort
Θ(N)
Θ(N log N)
Θ(1)
No
Slow in practice.
Merge sort
Θ(N log N)
Θ(N log N)
Θ(N)
Yes
Fastest stable sort.
Insertion Sort
Θ(N)
Θ(N2)
Θ(1)
Yes
Best for small or almost sorted inputs.

What is your burning question from today’s lecture?
24

Optimizing Insertion Sort (Optional)
Idea. Fix multiple inversions at once.
Stride. Instead of comparing adjacent items, compare items that are h apart.



Shellsort. Perform multiple, partial insertion sorts with different strides.
Start with large stride and decrease towards 1, e.g. h = 7, then h = 3, and then h = 1.

Insertion sort is shellsort with one stride, h = 1.
25
M O R T E X A S P L E
h = 7
Algorithms (Robert Sedgewick, Kevin Wayne/Princeton)
S O R T E X A M P L E
h = 3

26
Shellsort: Three Iterations of Partial Insertion Sort
Algorithms (Robert Sedgewick, Kevin Wayne/Princeton)
S O R T E X A M P L E
M O R T E X A S P L E  (1 swap,  >1 inversions)
M O R T E X A S P L E  (0 swaps,  0 inversions)
M O L T E X A S P R E  (1 swap,  >1 inversions)
M O L E E X A S P R T  (1 swap,  >1 inversions)
M O L E E X A S P R T
E O L M E X A S P R T  (1 swap,  >1 inversions)
E E L M O X A S P R T  (1 swap,  >1 inversions)
E E L M O X A S P R T  (0 swaps,  0 inversions)
A E L E O X M S P R T  (2 swaps, >1 inversions)
A E L E O X M S P R T  (0 swaps,  0 inversions)
A E L E O P M S X R T  (1 swaps, >1 inversions)
A E L E O P M S X R T  (0 swaps,  0 inversions)
A E L E O P M S X R T  (0 swaps,  0 inversions)
h = 7
h = 3
A E L E O P M S X R T
A E L E O P M S X R T
A E L E O P M S X R T
A E L E O P M S X R T
A E E L O P M S X R T
A E E L O P M S X R T
A E E L O P M S X R T
A E E L M O P S X R T
A E E L M O P S X R T
A E E L M O P S X R T
A E E L M O P R S X T
A E E L M O P R S T X
h = 1

27
Sort
Best-Case
Worst-Case
Space
Stable
Notes
Selection Sort
Θ(N2)
Θ(N2)
Θ(1)
No
Heapsort
Θ(N)
Θ(N log N)
Θ(1)
No
Slow in practice.
Merge sort
Θ(N log N)
Θ(N log N)
Θ(N)
Yes
Fastest stable sort.
Insertion Sort
Θ(N)
Θ(N2)
Θ(1)
Yes
Best for small or almost sorted inputs.
Shellsort
Θ(N)
Ω(N log N)
Θ(1)
No
Optional for this course.