# Recursive Algorithm Analysis

## Table of contents

## Binary search

Let’s analyze the runtime for binary search on a sorted array. (Trivia: a bug in Java’s binary search was discovered in 2006.) The `binarySearch`

method below returns the index of the search item, `x`

, or -1 if `x`

is not in the `sorted`

array.

```
static int binarySearch(int[] sorted, int x) {
return binarySearch(sorted, x, 0, sorted.length);
}
static int binarySearch(int[] sorted, int x, int lo, int hi) {
if (lo > hi)
return -1;
int mid = (lo + hi) / 2;
if (x < sorted[mid])
return binarySearch(sorted, x, lo, mid - 1);
else if (x > sorted[mid])
return binarySearch(sorted, x, mid + 1, hi);
else
return mid;
}
```

## What is a good cost model for binary search?

We can count the number of compares. The number of calls to `binarySearch`

also works. Or the number of times we compute `mid`

.

## What is the best-case order of growth of the runtime?

In the best case, `x`

is the immediate middle item in the sorted array. This takes constant time since it’s not necessary to recurse on either the left or right sides.

## What is the worst-case order of growth of the runtime?

We don’t really have the tools to be precise, but we know that each recursive call operates on half the problem. The problem continues to be divided in half until we’re only examining one item.

This pattern of repeatedly dividing in half is mathematically defined by the base-2 logarithmic function, log_{2} (sometimes written as lg). When no log base is specified, computer scientists typically assume base 2. This assumption doesn’t affect asymptotic analysis because all log bases are within a constant multiplicative factor of each other.

Our goal is to prove that the worst-case order of growth of the runtime of `binarySearch`

is in Theta(log_{2} N). In order to solve this, we’ll need a new idea known as a **recurrence relation**, which models the runtime of a recursive algorithm using mathematics.

- Goal
- Give the runtime in terms of N = hi - lo + 1, the size of the current recursive subproblem.

There are two parts to every recurrence relation: the time spent on non-recursive work, and the time spent on recursive work. We define **T(N)** as the maximum number (worst case) of key compares (cost model) to search a sorted subarray of length N (current recursive subproblem). We can define the runtime of binary search using the following recurrence. (Assume floor division for N / 2 to keep the math simple.)

- Binary search
- T(N) = T(N / 2) + c for N > 1; T(1) = d

c represents the constant time spent on non-recursive work, such as comparing `lo < hi`

, computing `mid`

, and comparing `x`

with `sorted[mid]`

. T(1) = d represents the base case, which takes a different amount of constant time to compare `lo < hi`

and immediately `return -1`

. The time spent on recursive work is modeled by T(N / 2) because a recursive call to `binarySearch`

will examine either the lower half or upper half of the remaining N items.

We can solve this recurrence relation and find a closed-form solution by **unrolling the recurrence**: plugging the recurrence back into itself until the base case is reached.

- T(N) = T(N / 2) + c
- T(N) = T(N / 4) + c + c
- T(N) = T(N / 8) + c(3)
- …
- T(N) = T(N / N) + c(log
_{2}N) - T(N) = d + c(log
_{2}N)

## Merge sort

Merge sort is an alternative sorting algorithm to selection sort. It relies on the **merge** operation, which takes two **sorted** arrays and returns a sorted result containing all of the items in both input arrays.

## What is a good cost model for merge assuming input arrays of the same length?

Counting the number of comparisons works since we need to make between about 0.5N to N comparisons. Counting the number of array assignments is also a good cost model since each item in both of the input arrays will need to be copied over once to the sorted output.

## What is the runtime of merge in terms of N, the total number of items in both input arrays?

Theta(N) since each item in both input arrays needs to be copied over to the sorted output.

Using the merge operation can give us a good speed-up over regular selection sort.

- Selection sort the left half.
- Selection sort the right half.
- Merge the two sorted halves.

The final version of this algorithm is known as merge sort.

- If the array is of size 1, return.
- Recursively merge sort the left half.
- Recursively merge sort the right half.
- Merge the two sorted halves.

Assuming perfect floor division to keep the math simple, we can define the runtime of merge sort using the following recurrence. Each call to merge sort makes two recursive calls to subproblems of half the size each and then spends linear time merging the sorted halves.

- Merge sort
- T(N) = T(N / 2) + T(N / 2) + c
_{1}N + c_{0}for N > 1; T(1) = d

Unrolling this recurrence is a bit trickier since there are two recursive branches. Instead, we can refer back to the visualization above for the small example where N = 64. The top layer takes about 64 units of time merging two sorted halves of 32 items each. The second layer also takes about 64 units to time merging four sorted halves of 16 items each. By identifying the pattern in the recursion tree diagram, we can see that each level will take about 64 units of time.

Since the entire runtime of merge sort is represented by the recursion tree, we can find the total time spent by multiplying the number of levels in the tree by the time spent on each level. Each level divides the input size in half until the base case of 1 item is reached so there are log_{2} N levels. Each level takes about N time to merge many small arrays. Overall, the order of growth of the runtime for merge sort is in N log_{2} N.

## Common recurrences

Recurrences can be used to model all kinds of algorithms, both recursive and iterative. Each of the common recurrences below has a geometric visualization that can be used to provide a visual proof of its corresponding order of growth. We’ll develop some of this visual intuition in class, but it’s helpful to walk through each of these recurrences. It’s sometimes necessary to use one of the familiar summations introduced earlier to simplify a solution.

Recurrence | Order of Growth |
---|---|

T(N) = T(N / 2) + c | log N |

T(N) = 2T(N / 2) + c | N |

T(N) = T(N - 1) + c | N |

T(N) = T(N - 1) + cN | N^{2} |

T(N) = 2T(N - 1) + c | 2^{N} |

T(N) = T(N / 2) + cN | N |

T(N) = 2T(N / 2) + cN | N log N |