Reductions and Decomposition Reading

Complete the Reading Quiz by noon before lecture.

In the previous lecture, we briefly resurfaced a quote about The Manner in Which Learning Occurs.

programming is […] fundamentally about the iterative process of refining mental representations of computational problems and solutions and expressing those representations as code¹

Programming is difficult partly because complex systems are difficult to understand. Building abstractions with data structures can help to manage some of the cognitive load involved in designing large systems. However, this quote also applies to the study of algorithms. Near the beginning of the course, we introduced the Algorithm Design Process: a 6-step process for improving our data structures and algorithms.

1. Hypothesize. How do invariants affect the behavior for each operation?
2. Identify. What strategies have we used before? What examples can we apply?
3. Plan. Propose a new way from findings.
4. Analyze. Does the plan do the job? What are potential problems with the plan?
5. Create. Implement the plan.
6. Evaluate. Check implemented plan.

Data structures and algorithms courses are often considered foundational computer science courses, and that they teach much of what you need to solve standard interview questions at tech companies. Why do companies seek candidates with this specific knowledge?

One major reason is that many real world problems can be formulated in such a way that they’re solvable with the data structures and algorithms we’ve learned. This process of turning a real-world problem to a computable, abstract representation is called problem decomposition. We’ve been learning how to write programs efficiently and write programs that run efficiently by studying examples of problem decomposition and applying them in the homework. Understanding these classical examples helps us to learn how to solve new problems by remixing patterns, strategies, and techniques from past problems.

Reductions are a special type of problem decomposition. Whereas our current strategies for solving problems involve identifying the limitations of our algorithms (e.g. BFS on weighted graphs, the case for linear-height BSTs), reductions do not involve modifying the algorithm itself. Instead, reductions follow a three-step process.

1. Transform the input so that it can be solved by the standard algorithm.
2. Run the standard algorithm as-is on the transformed input.
3. Transform the output of the algorithm to solve the original problem.

Not all reductions use all three steps. But since we’re able to use the standard algorithm as-is, this allows us to make statements about the relative difficulty of problems, where difficulty can be (informally) thought of as runtime. Using an algorithm for Problem Q to solve Problem P, then we say that P reduces to Q. This implies that P cannot be more difficult than Q, assuming that our transformations do not significantly increase the runtime.

Reading Quiz