When we studied sorting several weeks ago, we discovered that simple algorithms like insertion and selection sort required O(n2) time. Mergesort, on the other hand, was O(n log n), which gave us a huge performance improvement, particularly when sorting medium to large collections of data. The one downside to mergesort was that it required extra space for a second copy of the data to perform the merge step. Since often we are trying to sort large collections of data, and since access to main storage is orders of magnitude faster than access to data on a disk drive or network, it would be nice if we could sort as much data in main storage as possible. So the question is, can we find an equally efficient, O(n log n) algorithm to sort data that does not require substantial amounts of extra space beyond that needed for a single copy of the data itself.
The answer is yes. The classic example is quicksort, which is another O(n log n) sorting algorithm invented by C. A. R. Hoare in the early 1960s. The idea behind quicksort is similar to mergesort: we partition the data into two halves, then recursively sort the halves. The difference is that when we divide the data, we rearrange it so that all the data in the first half of the list is smaller than the data in the second half. Then after recursively sorting the two halves, we're done - no merge step is required since all of the (sorted) small values are in the first half of the list and already precede the (sorted) larger values in the second half.
The key step, then, is to partition the array. We want to partition a[lo..hi] by picking a pivot value x from the values in a[lo..hi]. We then rearrange the array by moving all values not greater than x to the left and all greater values to the right, finally putting x in the middle and returning its location as the result. Here's the specification of the partition step:
// partition a[lo..hi] and return an index mid such that // a[lo..mid-1] <= a[mid] < a[mid+1..hi] public int parition(int[] a, int lo, int hi)
It's worth drawing a picture to make the definition clear:
Leaving aside the implementation of partition for now, we can develop quicksort
as a main method that sorts an array and a recursive helper method that does
the actual work.
// Sort array a
public void quicksort(int[] a) {
quicksort(a, 0, a.length-1);
}
// Sort a[lo..hi]
public void quicksort(int[] a, int lo, int hi) {
if ( _____________________ ) {
// base case - nothing to sort
return;
} else {
// general recursive case
}
Obviously the partition step contains the real work. For simplicity we'll choose the first value in the array segment as the partition value x. Then we rearrange the array so that larger values are to the right and smaller values are to the left. Finally, when all the other values are rearranged, we move x to the middle by swapping it with a suitable value and return its final location.
As usual, before coding, its often helpful to draw a picture to visualize what the partition looks like while it is in progress. Assuming that we've partitioned part of the array, but not all of it, here's the picture:
In words, a[lo] is the pivot value, x; the values in a[lo+1..L-1] are <= x, the values in a[R+1..hi] are > x, and the values in a[L..R] remain to be processed. Given that picture, we can write the partition method. The final detail is that once we're done, we need to swap a[lo] (i.e., the pivot value x) with a[L-1] (the "last" value that is <= x) to get x in the middle.
// partition a[lo..hi] and return an index mid such that
// a[lo..mid-1] <= a[mid] < a[mid+1..hi]
public int parition(int[] a, int lo, int hi) { }
How fast is it? Assuming that the partition step divides the array into roughly equal halves, the analysis is basically the same as mergesort:
Amount of work done at each recursive level:
Depth of recursion:
Total work needed for quicksort:
But will that always happen? What if partition consistently makes lousy choices, always picking either the largest or smallest value in the array as the pivot. What is the performance then?
Amount of work at each recursive level:
Depth of recursion:
Total work needed:
So quicksort has good performance if partition does a good job of picking a pivot, but this performance is not guaranteed. If partition does a poor job picking pivots, performance can be as bad as insertion or selection sort. So we need to do a good job of picking the pivot. A classic strategy is to pick 3 or 5 values scattered throughout the array being partitioned and use the median value of these as the pivot. Another strategy that works well is to randomly pick a location in the array and use the value contained there as the pivot. While a random choice could pick a bad value, it almost always does a good job. In any case, practical implementations of quicksort run in O(n log n) time and require no additional space for a merge step. It is the method of choice for sorting values that fit in main memory.
One final aside: while mergesort and quicksort perform much better than insertion or selection sort on large arrays, they do involve overhead for the merge or partition steps, plus the recursive sorts. For very small arrays (say a half dozen items), the overhead of the fancier sorting algorithms exceeds the cost of using a simple quadratic sort. So the base case of most practical implementations of mergesort or quicksort is not when the size of the subproblem is 1 or smaller. Instead, once the number of items to be sorted is smaller than some threshold, insertions sort is typically used to sort the remaining items.