Comparison Sorts
Optimizing selection sort, analyzing sorting algorithm stability, and quantifying sortedness.
Kevin Lin, with thanks to many others.
1

Selection Sort Stability
Find the smallest item in the array, and swap it with the first item.
Find the second smallest item in the array, and swap it with the second item.
Continue until all items in the array are sorted.
Selection sort is not stable. Give an example.
2
1
Q
?: What does it mean to have a stable sorting algorithm?






Q1: Selection sort is not stable. Give an example.

Selection Sort Stability
Find the smallest item in the array, and swap it with the first item.
Find the second smallest item in the array, and swap it with the second item.
Continue until all items in the array are sorted.
Selection sort is not stable. Give an example.
3
1
A

Heapifying
Give the result of bottom-up max-heapifying the array [9, 1, 1, 3, 5, 5, 6, 8].
4
Q
Q1: Give the result of bottom-up max-heapifying the array [9, 1, 1, 3, 5, 5, 6, 8].

Heapifying
Give the result of bottom-up max-heapifying the array [9, 1, 1, 3, 5, 5, 6, 8].
5
A
9
1
1
3
5
5
6
8
1
9
3
1
5
5
6
8

Recursive Merge Sorting
Give the two arrays that will be merged by the final step of merge sort on [9, 1, 1, 3, 5, 5, 6, 8].
6
Q
Q1: Give the two arrays that will be merged by the final step of merge sort on [9, 1, 1, 3, 5, 5, 6, 8].

Recursive Merge Sorting
Give the two arrays that will be merged by the final step of merge sort on [9, 1, 1, 3, 5, 5, 6, 8].
7
A

Insertion Sort
Unstable sorting algorithms (selection sort, heapsort) use long-distance swaps.
Merge sort, a stable sort, uses the fact that left-half items come before right-half items.

Idea. Build a sorted subsection but use only left-neighbor swaps to maintain stability.
Insertion sort. Scan from left to right…
If an item is out of order with respect to its left-neighbor, swap left.
Keep on swapping left until the item is in order with respect to its left-neighbor.
8
32,15,2,17,19,26,41,17,17

9
Insertion Sort Examples
P O T A T O
P O T A T O  (0 swaps)
O P T A T O  (1 swap )
O P T A T O  (0 swaps)
A O P T T O  (3 swaps)
A O P T T O  (0 swaps)
A O O P T T  (3 swaps)
S O R T E X A M P L E
S O R T E X A M P L E  (0 swaps)
O S R T E X A M P L E  (1 swap )
O R S T E X A M P L E  (1 swap )
O R S T E X A M P L E  (0 swaps)
E O R S T X A M P L E  (4 swaps)
E O R S T X A M P L E  (0 swaps)
A E O R S T X M P L E  (6 swaps)
A E M O R S T X P L E  (5 swaps)
A E M O P R S T X L E  (4 swaps)
A E L M O P R S T X E  (7 swaps)
A E E L M O P R S T X  (8 swaps)
Purple	Item that we’re swapping left.
Black	Item swapped with purple item.
Grey		Not considered this iteration.
Algorithms (Robert Sedgewick, Kevin Wayne/Princeton)

Inversions: Quantifying Sortedness
Inversion. A pair of keys that are out of order.



Partially sorted. An array of length N is partially sorted if the number of inversions is in O(N).
A sorted array has 0 inversions.
A reverse-sorted array has about ½N2 inversions.

Each local swap fixes 1 inversion. Insertion sort runtime is in Θ(N + K) with K inversions.
10
A E E L M O T R X P S
1
2
3
4
5
6
T–R
T–P
T–S
R–P
X–P
X–S
COS 226: Elementary Sorts (Robert Sedgewick, Kevin Wayne/Princeton)

Worst-Case Inversions
If we add 10 unknown items to the end of a sorted array, at most how many inversions can there be in the array?
11
Q
Q1: If we add 10 unknown items to the end of a sorted array, at most how many inversions can there be in the array?

Worst-Case Inversions
If we add 10 unknown items to the end of a sorted array, at most how many inversions can there be in the array?
12
A

Monotonically-Improving
For each algorithm, can the inversion count increase at some point in its execution?
Insertion sort.
Selection sort.
Bottom-up max-heapification.
Bottom-up min-heapification.
Merge sort.
13
Q
For each algorithm, can the inversion count increase at some point in its execution?

Q1: Insertion sort




Q2: Selection sort




Q3: Bottom-up max-heapification




Q4: Bottom-up min-heapification




Q5: Merge sort

Monotonically-Improving
For each algorithm, can the inversion count increase at some point in its execution?
Insertion sort.
Selection sort.
Bottom-up max-heapification.
Bottom-up min-heapification.
Merge sort.
14
A

15
Sort
Best-Case
Worst-Case
Space
Stable
Notes
Selection Sort
Θ(N2)
Θ(N2)
Θ(1)
No
Heapsort
Θ(N)
Θ(N log N)
Θ(1)
No
Slow in practice.
Merge Sort
Θ(N log N)
Θ(N log N)
Θ(N)
Yes
Fastest stable sort.
Insertion Sort
Θ(N)
Θ(N2)
Θ(1)
Yes
Best for small or almost sorted inputs.

Shellsort: Optimizing Insertion Sort (Optional)
Idea. Fix multiple inversions at once.
Stride. Instead of comparing adjacent items, compare items that are h apart.



Shellsort. Perform multiple, partial insertion sorts with different strides.
Start with large stride and decrease towards 1, e.g. h = 7, then h = 3, and then h = 1.

Insertion sort is shellsort with one stride, h = 1.
16
M O R T E X A S P L E
h = 7
Algorithms (Robert Sedgewick, Kevin Wayne/Princeton)
S O R T E X A M P L E
h = 3

17
Shellsort: Three Iterations of Partial Insertion Sort
Algorithms (Robert Sedgewick, Kevin Wayne/Princeton)
S O R T E X A M P L E
M O R T E X A S P L E  (1 swap,  >1 inversions)
M O R T E X A S P L E  (0 swaps,  0 inversions)
M O L T E X A S P R E  (1 swap,  >1 inversions)
M O L E E X A S P R T  (1 swap,  >1 inversions)
M O L E E X A S P R T
E O L M E X A S P R T  (1 swap,  >1 inversions)
E E L M O X A S P R T  (1 swap,  >1 inversions)
E E L M O X A S P R T  (0 swaps,  0 inversions)
A E L E O X M S P R T  (2 swaps, >1 inversions)
A E L E O X M S P R T  (0 swaps,  0 inversions)
A E L E O P M S X R T  (1 swaps, >1 inversions)
A E L E O P M S X R T  (0 swaps,  0 inversions)
A E L E O P M S X R T  (0 swaps,  0 inversions)
h = 7
h = 3
A E L E O P M S X R T
A E L E O P M S X R T
A E L E O P M S X R T
A E L E O P M S X R T
A E E L O P M S X R T
A E E L O P M S X R T
A E E L O P M S X R T
A E E L M O P S X R T
A E E L M O P S X R T
A E E L M O P S X R T
A E E L M O P R S X T
A E E L M O P R S T X
h = 1

18
Sort
Best-Case
Worst-Case
Space
Stable
Notes
Selection Sort
Θ(N2)
Θ(N2)
Θ(1)
No
Heapsort
Θ(N)
Θ(N log N)
Θ(1)
No
Slow in practice.
Merge Sort
Θ(N log N)
Θ(N log N)
Θ(N)
Yes
Fastest stable sort.
Insertion Sort
Θ(N)
Θ(N2)
Θ(1)
Yes
Best for small or almost sorted inputs.
Shellsort
Θ(N)
Ω(N log N)
Θ(1)
No
Optional for this course.