Earlier in the quarter we discovered that there are some advantages to keeping a list sorted. In particular, if access to individual items in the list is fast (O(1)), we can look for an item in the list using binary search in O(log n) time, instead of needing to use sequential search, which uses O(n) time. In assignment 1, we looked at one way to do this: whenever an item was inserted in the list of strings, we made sure to place it in the correct place so the list always remained sorted.
But what if we start with an unordered collection of data? How can we rearrange the data so it is sorted? There are many ways to do this and it's worth taking a few moments to think about possible strategies. To make it more concrete, let's look at the following problem. We are visiting Uncle Luddite's house for Thanksgiving dinner. After dinner he suggests looking through his list of people who he has sent holiday cards to in the past to see if there are names to add to or delete from the list. Having no use for modern gadgets like PDAs, cell phones, or iPods, Luddite writes each name and address on a separate 3x5 index card, and keeps the list of cards sorted alphabetically in a box in the closet.
Unfortunately, in the process of getting the list down from the top shelf of the closet, the cards are dropped and wind up in the floor in an unordered pile. Somehow they need to be put back into alphabetical order. Fortunately, Luddite's precocious niece is getting bored and looking for something to do, so it seems like a perfect job for her. Your job is to come up with specific instructions describing how to sort the cards. Being young and precocious, the niece needs very specific instructions, like "look at the next two cards and do ...", but if you can give her specific instructions, she can be counted on to do the job reliably. What instructions do you give her? In other words, what is her strategy or algorithm for sorting the cards?
WARNING: do not read the rest of this handout without trying the problem first, particularly if you've never thought about sorting algorithms before. It spoils the fun (not to mention that it reduces what you learn from the exercise).
Your answer (strategy, algorithm) here:
There are many classic algorithms for sorting. The problem has been around long before the advent of modern computers, but with the appearance of fast digital computers in the mid 20th century, sorting algorithms received much more intensive study. These days sorting remains an important problem, with applications ranging from sorting large collections of data to sorting lists of results in a web search or sorting the names in the contact list on your cell phone.
Obviously we only have time to take a look at a couple of illustrative examples. To make things concrete, we'll assume that we are sorting the entire contents of an array of integers. It's simple to generalize this to sorting other kinds of objects, or sorting parts of an array or list. About the only requirement is that we can compare two array elements to determine their ordering. It also helps to have constant-time (O(1)) access to the array elements, although the particular algorithms we'll look at actually can be made to work equally well on data structures like linked lists where access to adjacent elements is fast, even if access to random elements is not.
Insertion Sort
The idea behind insertion sort is that at each step, we pick an element that has not been sorted and insert it into the proper place in the sorted part of the list. It is the same idea behind the algorithms we used to maintain a sorted list in assignment 1; the difference is that instead of starting with an empty list and inserting items one at a time, we start with a full list, and go through it one item at a time, moving successive items to the correct place in the partially sorted part of the list.
To develop the algorithm, it helps to draw pictures of both the initial and final conditions (both trivial - random order vs. sorted), and, much more importantly, what the situation is like when part of the array is sorted. The later diagram gives us a way of visualizing how we can proceed to move one more item into the right place, which is the step we need to repeat to sort the entire array. Pictures:
Initial: Final:
Partly sorted:
The key step is to move the next unsorted item into the right place by shifting larger items over to make room, exactly as we did in assignment 1. That gives us the following algorithm:
void insertionSort(int[] items) {
for (int k = _____________ ; ____________________ ; _______________ ) {
// insert items[k] into the correct location in items[0..k-1]
}
}
Running time: How fast is it? First, what is the problem size?
Then, how much work do we do? How many times around the outer loop? For each trip around the outer loop, how many compare-and-shift operations in the inner loop?
Total time:
Can we do better?
Merge Sort
We'd like to see if there's some way to sort a list faster than insertion sort (or selection sort or any of the other similar algorithms). If we remember binary vs. sequential search, the key in slipping from O(n) running time to O(log n) is that at each step we were able to reduce the size of the remaining problem by half, instead of one item at a time. Maybe there's a way to do something similar to get a faster sorting algorithm.
It turns out there is. The idea is to use a strategy known as divide and conquer: to solve the problem (sorting a list), we'll split it into two half-size subproblems, solve the subproblems, then combine the results to get a solution to the original problem. Quicksort and Mergesort are two of the best known sorting algorithms using this strategy; we'll look at mergesort here.
The idea behind mergesort is very simple: to sort a large list, divide the list into two, sort each of the halves (first half and second half of the original list), then combine the two sorted halves to produced a sorted list with all of the values from the original list. It is almost as straightforward as it sounds. But there are a couple of subtle points we need to be sure to get right:
So the algorithm for merge sort looks like this:
public void msort(int[] items) {
// base case: quit if nothing to do
if (items.length < 2) return;
// allocate two half-size arrays and copy the halves of items to them
int mid = items.length/2;
int[] left = new int[_____________________ ]
int[] right = new int[_____________________ ]
// copy first half into left, second half into right
for (int k = 0; k _______________ ; k++) {
left[k] = items[k];
}
for (int k = __________ ; k < ______________ ; k++) {
right[ __________________ ] = items[ _______________ ];
}
// sort left and right half-arrays
// merge left and right back into the original array
merge(left, right, items);
}
Now the remaining operation is the merge. The basic idea is to keep track of how much has been copied from the left and right half-arrays. At each step, we look at the first uncopied item in each half-array and copy the smaller one back to the original array. We keep this up as long as there is at least one more item in each of the half-arrays that has not been copied. When that is done, we finish up by copying the remaining items from the half that has not been completely copied to the original array. In pictures, the strategy looks roughly like this:
public void merge(int[] left, int[] right, int[] items) {
int k = 0; // left[k] is the first uncopied item in left
int j = 0; // right[j] is the first uncopied item in right
int p = 0; // items[p] is where the next item should be copied
while ( ______________________________________________________ ) {
if (left[k] < right[j]) {
items[p] = left[k];
p++; k++;
} else {
items[p] = right[j];
p++; j++;
}
}
// copy remainder of left into items
while ( ____________________________ ) {
items[p] = left[k];
p++; k++;
}
// copy remainder of right into items
while ( ____________________________ ) {
items[p] = right[j];
p++; j++;
}
}
Running time: As before, we take the problem size n to be the number of items in the original array. The analysis needs to take into account the total amount of work done directly plus the amount of work done by the recursive calls to msort. Each level of recursive calls splits the problem into twice as many subproblems, each of which has half the size of the subproblems at the previous level. We can think of it like this:
The total work, then, is the amount of work done at each recursive level times the number of recursive levels to reach the base case.
Total work done at each level: Depth of recursion: Total cost of mergesort: