Recursive Algorithm Analysis

Table of contents

1. Binary search
2. Merge sort
3. Common recurrences

Binary search

Let’s analyze the runtime for binary search on a sorted array. (Trivia: a bug in Java’s binary search was discovered in 2006.) The binarySearch method below returns the index of the search item, x, or -1 if x is not in the sorted array.

static int binarySearch(int[] sorted, int x) {
    return binarySearch(sorted, x, 0, sorted.length);
}

static int binarySearch(int[] sorted, int x, int lo, int hi) {
    if (lo > hi)
        return -1;
    int mid = (lo + hi) / 2;
    if (x < sorted[mid])
        return binarySearch(sorted, x, lo, mid - 1);
    else if (x > sorted[mid])
        return binarySearch(sorted, x, mid + 1, hi);
    else
        return mid;
}

Our goal is to prove that the worst-case order of growth of the runtime of binarySearch is in Theta(log₂ N). In order to solve this, we’ll need a new idea known as a recurrence relation, which models the runtime of a recursive algorithm using mathematics.

Goal

Give the runtime in terms of N = hi - lo + 1, the size of the current recursive subproblem.

There are two parts to every recurrence relation: the time spent on non-recursive work, and the time spent on recursive work. We define T(N) as the maximum number (worst case) of key compares (cost model) to search a sorted subarray of length N (current recursive subproblem). We can define the runtime of binary search using the following recurrence. (Assume floor division for N / 2 to keep the math simple.)

Binary search

T(N) = T(N / 2) + c for N > 1; T(1) = d

c represents the constant time spent on non-recursive work, such as comparing lo < hi, computing mid, and comparing x with sorted[mid]. T(1) = d represents the base case, which takes a different amount of constant time to compare lo < hi and immediately return -1. The time spent on recursive work is modeled by T(N / 2) because a recursive call to binarySearch will examine either the lower half or upper half of the remaining N items.

We can solve this recurrence relation and find a closed-form solution by unrolling the recurrence: plugging the recurrence back into itself until the base case is reached.

• T(N) = T(N / 2) + c
• T(N) = T(N / 4) + c + c
• T(N) = T(N / 8) + c(3)
• …
• T(N) = T(N / N) + c(log₂ N)
• T(N) = d + c(log₂ N)

Merge sort

Merge sort is an alternative sorting algorithm to selection sort. It relies on the merge operation, which takes two sorted arrays and returns a sorted result containing all of the items in both input arrays.

Using the merge operation can give us a good speed-up over regular selection sort.

1. Selection sort the left half.
2. Selection sort the right half.
3. Merge the two sorted halves.

The final version of this algorithm is known as merge sort.

1. If the array is of size 1, return.
2. Recursively merge sort the left half.
3. Recursively merge sort the right half.
4. Merge the two sorted halves.

Assuming perfect floor division to keep the math simple, we can define the runtime of merge sort using the following recurrence. Each call to merge sort makes two recursive calls to subproblems of half the size each and then spends linear time merging the sorted halves.

Merge sort

T(N) = T(N / 2) + T(N / 2) + c₁N + c₀ for N > 1; T(1) = d

Unrolling this recurrence is a bit trickier since there are two recursive branches. Instead, we can refer back to the visualization above for the small example where N = 64. The top layer takes about 64 units of time merging two sorted halves of 32 items each. The second layer also takes about 64 units to time merging four sorted halves of 16 items each. By identifying the pattern in the recursion tree diagram, we can see that each level will take about 64 units of time.

Since the entire runtime of merge sort is represented by the recursion tree, we can find the total time spent by multiplying the number of levels in the tree by the time spent on each level. Each level divides the input size in half until the base case of 1 item is reached so there are log₂ N levels. Each level takes about N time to merge many small arrays. Overall, the order of growth of the runtime for merge sort is in N log₂ N.

Common recurrences

Recurrences can be used to model all kinds of algorithms, both recursive and iterative. Each of the common recurrences below has a geometric visualization that can be used to provide a visual proof of its corresponding order of growth. We’ll develop some of this visual intuition in class, but it’s helpful to walk through each of these recurrences. It’s sometimes necessary to use one of the familiar summations introduced earlier to simplify a solution.