Algorithm Analysis Study Guide

Runtime Minimization. One of the most important properties of a program is the time it takes to execute. One goal as a programmer is to minimize the time (in seconds) that a program takes to complete.

Algorithm Scaling. While we ultimately care about the runtime of an algorithm in seconds, we’ll often say that one algorithm is better than another simply because of how it scales. By scaling, we mean how the runtime of a piece of code grows as a function of its input size. For example, inserting at the beginning of ArrayList on an old computer might take R(N) = 0.0001N seconds, where N is the size of the list.

Simplifying Algebraic Runtime.

• Pick a representative option to be our cost model, such as the number of array accesses.
• Focus on the worst case. If the number of operations is between 1 and 2N + 1, consider only the 2N + 1.
• Ignore small inputs. Treat 2N + 1 just like 2N.
• Ignore constant scaling factor. Treat 2N just like N.

The cost model is an operation that we’re picking to represent the entire piece of code. Not all cost models effectively represent the runtime of a program. Use your understanding of the program to identify the places where most of the time is spent, such as in for loops or while loops.

Order of Growth. The result of applying our last 3 simplifications gives us the order of growth of a function. So for example, suppose R(N) = 4N² + 3N + 6, we’d say that the order of growth of R(N) is N².

The terms constant, linear, and quadratic are often used for algorithms with order of growth 1, N, and N² respectively. For example, we might say that an algorithm with runtime 4N² + 3N + 6 is a quadratic-time algorithm.

Simplified Modeling Process. We can apply our simplifications in advance. Rather than computing the number of operations for all operations, we can pick a specific operation as our cost model and count only that operation. After selecting a cost model, we have two options for analyzing the runtime.

• Compute the exact expression that counts the number of operations.
• Use intuition and inspection to find the order of growth of the number of operations.

One common intuitive/inspection-based approach is use geometric intuition. For example, if we have nested for loops where i goes from 0 to N, and j goes from i + 1 to N, we observe that the runtime is effectively given by a right triangle of side length N. Since the area of a such a triangle grows quadratically, the order of growth of the runtime is quadratic.

Big Theta. To formalize our intuitive simplifications, we introduce Big-Theta notation. Big-Theta is essentially equivalent to our order of growth analysis. That is, if a function R(N) has order of growth exactly N², then we also have that R(N) is in Theta(N²).

Recommended Problems

1. Textbook 1.4 Exercise 6
2. Suppose we have an algorithm with runtime R(N) in Theta(N²). Which of the following can we say?
   • R(N) is in Theta(N²) for any inputs.
   • R(N) is in Theta(N²) for best case inputs.
   • R(N) is in Theta(N²) for worst case inputs.
   • For large N, if we run the function on an input of size N and a second input of size 10N, we will have to wait roughly 100 times as long for the larger input.
   • If we run the function on an input of size 1000, and an input of size 10000, we will have to wait roughly 100 times as long for the larger input.
3. Suppose we have an algorithm with runtime R(N) in Theta(N² log N). For each statement about the algorithm runtime, determine if it is definitely true, definitely false, or not enough information to know for sure.
   • R(N) is in O(N² log N).
   • R(N) is in Omega(N² log N).
   • R(N) is in O(N³).
   • R(N) is in Theta(N² log₄ N).
4. Suppose we have an algorithm with runtime R(N) in O(N³). For each statement about the algorithm runtime, determine if it is definitely true, definitely false, or not enough information to know for sure.
   • There exists some inputs for which R(N) is in Theta(N²).
   • There exists some inputs for which R(N) is in Theta(N³).
   • There exists some inputs for which R(N) is in Theta(N⁴).
   • The worst case runtime R(N) is in O(N³).
   • The worst case runtime has order of growth N³.
5. Q4a from CSE 373 19au MT (Solution)
6. Q3d, Q3e from CSE 373 19au Final (Solution)
7. Q4b from CS 61BL 17su MT1 (Solution)