Algorithm Analysis I Study Guide

Runtime Minimization. One of the most important properties of a program is the time it takes to execute. One goal as a programmer is to minimize the time (in seconds) that a program takes to complete.

Algorithm Scaling. While we ultimately care about the runtime of an algorithm in seconds, we’ll often say that one algorithm is better than another simply because of how it scales. By scaling, we mean how the runtime of a piece of code grows as a function of its input size. For example, inserting at the beginning of ArrayList on an old computer might take R(N) = 0.0001N seconds, where N is the size of the list.

Simplfying Algebraic Runtime. We utilize four simplifications to make runtime analysis simpler.

• Pick an arbitrary option to be our cost model, such as the number of array accesses.
• Focus on the worst case. If the number of operations is between 1 and 2N + 1, consider only the 2N + 1.
• Ignore small inputs. Treat 2N + 1 just like 2N.
• Ignore constant scaling factor. Treat 2N just like N.

The cost model is simply an operation that we’re picking to represent the entire piece of code. Make sure to pick an appropriate cost model! If we had chosen the number of increment operations as our cost model, we’d mistakenly determine that the runtime was proportional to N. This is incorrect since, for large N, the comparisons will vastly outnumber the increments.

Order of Growth. The result of applying our last 3 simplifications gives us the order of growth of a function. So for example, suppose R(N) = 4N² + 3N + 6, we’d say that the order of growth of R(N) is N².

The terms constant, linear, and quadratic are often used for algorithms with order of growth 1, N, and N² respectively. For example, we might say that an algorithm with runtime 4N² + 3N + 6 is a quadratic-time algorithm.

Simplified Analysis. We can apply our simplifications in advance. Rather than computing the number of operations for all operations, we can pick a specific operation as our cost model and count only that operation. After selecting a cost model, we have two options for analyzing the runtime.

• Compute the exact expression that counts the number of operations.
• Use intuition and inspection to find the order of growth of the number of operations.

One common intuitive/inspection-based approach is use geometric intuition. For example, if we have nested for loops where i goes from 0 to N, and j goes from i + 1 to N, we observe that the runtime is effectively given by a right triangle of side length N. Since the area of a such a triangle grows quadratically, the order of growth of the runtime is quadratic.

Big Theta. To formalize our intuitive simplifications, we introduce Big-Theta notation. Big-Theta is essentially equivalent to our order of growth analysis. That is, if a function R(N) has order of growth exactly N², then we also have that R(N) is in Theta(N²).

Recommended Problems

1. Textbook 1.4 Exercise 6
2. Suppose we have a function with runtime R(N) in Theta(N²). Which of the following can we say?
   • R(N) is in Theta(N²) for any inputs.
   • R(N) is in Theta(N²) for best case inputs.
   • R(N) is in Theta(N²) for worst case inputs.
   • For large N, if we run the function on an input of size N and a second input of size 10N, we will have to wait roughly 100 times as long for the larger input.
   • If we run the function on an input of size 1000, and an input of size 10000, we will have to wait roughly 100 times as long for the larger input.
3. For each of the functions shown, give a tight asymptotic runtime bound using asymptotic notation in terms of N.¹

   public static int f1 (int n) {
       int x = 0;
       for (int i = 0; i < n; i++)
           x++;
       return x;
   }
   

   public static int f2(int n) {
       int x = 0;
       for (int i = 0; i < n; i++)
           for (int j = 0; j < i*i; j++)
               x++;
           return x;
   }
   

   public static int f3 (int n) {
       if (n <= 1) return 1;
       return f3(n-1) + f3(n-1)
   }
   

   public static int f4 (int n) {
       if (n <= 1) return 1;
       return f4(n/2) + f4(n/2);
   }
   

   public static int f5 (int n) {
       if (n <= 1) return 1;
       return f1(n) + f5(n/2) + f5(n/2);
   }