Post-Lecture Review

One possible ordering: - 5, 4, 2, 0, 3, 1

Another possible ordering: - 5, 2, 4, 3, 0, 1

Using BFS to solve topological sort is a reduction because it only modifies the input/outputs without modifying the underlying BFS algorithm itself. Topological sort via BFS works by taking an input graph, converting it to a graph with a single start vertex, running unmodified BFS on it, and then removing that vertex in the ordering created by BFS.

By contrast, using a modified Dijkstra’s for Prim’s is not a reduction because it involves modifying the underlying algorithm, not feeding a modified input into the unmodified algorithm. While modifying an algorithm to get the result is still a useful skill, it’s not quite the same as a reduction. The latter’s power comes from converting inputs/outputs and therefore can be done even without knowledge of the underlying algorithm solving the problem internally.

Quick Sort

Quick sort will run quickly on a set of primitives like the ID numbers, and because the list will stay mostly in sorted order you can select a pivot from the middle which will keep the algorithm efficient. Since the ID numbers are unique, it doesn’t matter that quick sort isn’t stable.

Best Case Runtime: Theta(nlogn)

Worst Case Runtime: Theta(n^2)

Merge Sort

Merge sort will provide stability to maintain relative order based on names. It will also have the same runtime regardless of how sorted the list is- which is good because you can’t predict how student scores will distribute based on names so the “sortedness” of the scores won’t impact the runtime.

Best Case Runtime: Theta(nlogn)

Worst Case Runtime: Theta(nlogn)

Insertion Sort

Insertion sort will enable you to maintain the leader board as the scores roll in. Also because the runners are released in heats based on their performance the scores will likely be in mostly sorted order as they roll in which will enable insertion sort to run more quickly.

Best Case Runtime: Theta(n)

Worst Case Runtime: Theta(n^2)

Heap Sort

If you maintain a min heap of the experiment runtimes as the data rolls in you can efficiently surface the shortest time. Then once all the data has been collected you can run heap sort, which as a more efficient sorting algorithm will help with the volume of data you are sorting. Also, it’s possible that the quicker a lab is able to execute the experiment the sooner they will report the runtimes, so it’s possible that you will receive data in a mostly sorted order which will cause fewer percolations and better runtime.

Best Case Runtime: Theta(n)

Worst Case Runtime: Theta(nlogn)

Selection Sort

As you only need to sort the first set of items until you reach your budget you can use selection sort to find the first X cheapest items and then end the algorithm early once you’ve maxed out your budget. It’s also likely this is a pretty small set of items, so an n^2 runtime of items you can afford will likely still run very quickly.

Best Case Runtime: Theta(n^2)

Worst Case Runtime: Theta(n^2)

1.1 Initial Contents: [-8, 0, 0, 2, 0, 4, 4, 6] 1.2 find: [-8, 0, 0, 2, 0, 4, 0, 6]. Note that we update node G to point directly to the root, but we do not update H because there’s no easy way to tell which children G has (or more accurately which children point to G). Path compression is useful because it doesn’t require any additional “scanning” through the data structure even though it improves things for future: it only modifies the nodes we already have to visit during that call to find.

2.1: Ordering Relations

One ordering relation could be using the built-in integer ordering relation for the length of the ingredients list. That could be helpful if we wanted to find the recipes that use the fewest ingredients, for example.

Another ordering relation could be using the built-in String ordering relation for the name of each recipe. That would be suitable if we were sorting the recipes to put into the alphabetized index of a cookbook.

2.2: Stable Sorts

The following could be a stable sort result: [Cookies, Carrot Cake, Sourdough]. However, this could NOT be a stable sort result: [Carrot Cake, Cookies, Sourdough]. Note that it is a correct sort by the ordering relation of bake time, but it is not stable in that it does not preserve the original order of equivalent elements.

In general, there is exactly 1 stable sort result for any given array. This is because all elements must be in order, and any “ties” between equivalent elements must be exactly in their original ordering relative to each other.

1.1 Minimum Calls to Find: Since each call to find() compresses every node along the path to the root, in general the minimum number of calls to find() to fully compress the tree would be the same as the number of leaf nodes that are not already direct descendants of their root. In this case, there are 6 nodes requiring 6 calls to find(): 13, 14, 12, 9, 7, and 16.

1.2 Maximum Calls to Find: There is no maximum number of calls to find() before the tree is guaranteed to be fully compressed. This is because we have no guarantee that every vertex will eventually be passed as an argument to find(). For example, you could call find() an unlimited number of times, only passing “0” as an argument, and never have an effect on the tree. This is one of the key reasons why path compression has different worst-case and in-practice runtimes.

The runtime of Kruskal’s algorithm is Theta(|E| log|E|), which is equivalent to Theta(|E| log|V|) because they are within a constant factor of each another. The runtime is dominated by the time it takes to sort the edges, which requires |E| log|E| time to complete. Later in the course, we’ll explore sorting algorithms in more depth.

Prim’s algorithm is typically preferable when we have a dense graph, where we have a lot of edges relative to the number of vertices, because Prim’s tends to do work vertex-by-vertex. Kruskal’s algorithm is typically preferable when we have a sparse graph, or when the edges are already sorted for some reason, because it thinks edge-by-edge and must sort the edges so it can save quite a bit of time if the edges are already sorted. Note that the differences between dense and sparse graphs aren’t reflected in a worst-case asymptotic analysis, but would manifest in “in practice” time or the constant factors that we observe.

The DisjointSets ADT represents, as the title suggests, a collection of disjoint sets: meaning that elements within the DisjointSets structure are organized into any number of different sets, but every element must belong to exactly one set (so the sets cannot share elements and are disjoint). You might imagine using a DisjointSets ADT to model the ingredients that you slowly combine during a recipe: when you add eggs to a bowl containing flour and stir them together, you’ve created a combined set that shows both ingredients are now inseparable.

We chose to use trees (specifically, up-trees) exactly because they require only a small change to combine two sets together, and do not require iterating through every element in one set to add to the other. Tree-based implementations make the union operation easier because we only have to modify the pointer of the tree’s root in order to change the set membership of the entire tree!

The height of each tree in WeightedQuickUnion is bounded to log(n) where n is the number of elements in the tree. As illustrated in the slides, this happens because WeightedQuickUnion always ensures that when unioning two trees together, the smaller tree goes under the larger tree and this prevents a long degenerate tree from forming. However, the worst case would still be where we union a tree with another tree of equal size, in which case WeightedQuickUnion still has to place one underneath the other, and the resulting tree ends up with a height that is 1 greater than either of the trees it unioned. This leads to the same pattern as binary trees, where doubling the number of nodes would increase the height by 1: so we have a log(n) bound on the height.

Therefore, in the worst case, find() has a runtime of log(n) for a particular set, where again n is the number of elements in that set.

The following edges make up the MST for this graph: (0, 3) (2, 4) (3, 5) (0, 1) (1, 4) (4, 6).

The main difference between SPT and MST is that MST represents the edge weights that sum up to the total minimum cost. There is no notion of a specific start point, it only matters about the total weight. An SPT, on the other hand, cares about the shortest path (minimum distance cost) not for the entire graph but for individual paths from the start vertex to all the other vertices. So SPTs really care who their starting vertex is since the whole point is to find the shortest paths from that start.

Another important property to remember is that an MST makes no effort to minimize the total weight of the path between any two vertices. That is, an MST minimizes the total sum of all edge weights, but it’s possible that following the edges in an MST might not be the minimum path from any vertex A to vertex B.

In general, Dijkstra’s and Prim’s are very similar algorithms, except at each step Dijkstra’s is concerned with choosing the vertex that has the minimum distance from the start, while Prim’s is concerned with choosing the vertex that has the minimum weight to add it just to the growing MST. Dijkstra’s takes the distance of each vertex in the “known cloud” into account when choosing which one to connect a candidate new vertex to, while Prim’s only cares to choose the smallest edge from anywhere in the “known cloud” to any candidate vertex.

Dijkstra’s algorithm produces a SPT while Prim’s algorithm produces a MST. Dijkstra’s algorithm solves the Weighted Shortest Paths Problem (and unweighted), while Prim’s algorithm solves the Minimum Spanning Tree problem.

1.1 Shortest Path Tree

The following edges would be in the shortest path tree: (B,F) (B,G) (B,E) (B,I) (B,D) (E,A) (I,C) (I,H). For each of these edges, direction matters: we’re specifying that the second vertex has a backpointer to the first vertex.

1.2

To find a single shortest path, say from B to C, we first start at the destination vertex (C), and then work backward by following the “backpointers” stored in the shortest path tree until we hit the origin vertex. In this example, we would see the backpointer stored for vertex C that points to vertex I, so we would add I to the path we’re building up. Then, I has a backpointer to B, so we add B and finish our iteration. Reversing the built-up path, we get that the shortest path is B → I → C. Note that this algorithm to recover a shortest path from the shortest path tree does not take constant time to complete!

We can’t use BFS to compute the shortest path on a weighted graph because one key property necessary for BFS to give us the correct result is for it to iterate in order from closest to farther nodes. Since we have changed the definition of “closest” for a weighted graph but not changed BFS’s algorithm, it no longer has a guarantee that it will examine the closest nodes first, so the “First Try Phenomenon” is no longer reliable because we don’t know the order BFS will consider the shortest path. For example, consider a graph with 3 nodes: A, B, and C, where (A,B) has weight 3, (A,C) has weight 3, and (B,C) has weight 10. If we ran BFS to find the shortest path from B to C, we would get (B,C) as our result because it is one edge, but that’s incorrect because it has a higher total weight.

In Dijkstra’s algorithm, a graph with negative edge weights may fool the algorithm into avoiding what at first seems to be a higher weight path because of a high initial weight, when that path would really have been better because of a negative edge weight hiding behind it that would have made it faster. Once again, the “First Try Phenomenon” becomes unreliable because we can’t greedily guarantee that we’ve seen the best path to something by the time we add it to “known”. Consider a graph with A, B, and C and weights (A,B): 10, (A,C): 4, (B,C): -6. The shortest path from A to C is through B, but Dijkstra’s wouldn’t discover that.

There are many possible orderings! BFS specifies an approach to traversing nodes, but there are many orderings that could be considered BFS orderings. One example:

2, 1, 5, 3, 4, 6

In general, any ordering that starts with 2, then has 1, 3, and 5 in any order, then has 4 and then 6 will be valid.

BFS traverses the graph level by level from a particular start node, and travels to the outer levels (neighbors, then neighbors of neighbors, then neighbors of neighbors of neighbors, etc.). BFS by itself doesn’t produce any output since it’s just a traversal, but you can modify it and do something along the way to process data in the graph. A common use is to track distances and predecessor edges to be able to easily find the shortest paths to each vertex in an unweighted graph, as we saw in LEC 15.

The difference between BFS and DFS is that BFS searches breadth first and visits all of the shallow nodes (1 hop away) before venturing deeper. DFS on the other hand, will go as far deep as possible before returning to revisit shallow nodes, and even when it does come back to an earlier point/level, it’ll try to go as deep as possible again until a dead end.

There are so many correct answers here! In general, BFS and DFS are helpful when we want to look at every vertex in a graph (an “exhaustive search”). You might imagine a graph of UW course prerequisites: we could perform a BFS to help us list out all possible courses that require taking a specific class. Even cooler, since BFS proceeds level-by-level, if you are aiming to take a particular upper-level, you could perform a BFS from an early prerequisite and discover what class you need to take each quarter to aim for it!

This graph is undirected (because the edges aren’t arrows), and simple (because it contains no parallel edges or self-loops). It is not connected, because it is not possible to reach any node from any other node.

Since there are 4 edges and this is an undirected graph, we know there will be 8 entries filled in for our adjacency matrix (for each edge (u,v), one from u to v and one from v to u). The following row,column pairs will be 1s:

• 0,1
• 0,2
• 0,3
• 1,0
• 1,3
• 2,0
• 3,0
• 3,1

Vertex A is Stevie (because there are edges from Stevie to David and Jake, as shown in the list) Vertex B is Jake Vertex C is David Vertex D is Patrick

a) The 1 is now at the root because it’s the minimum value (its children are 2 and 6). 2, 3, and 10 have all shifted down one level when they were swapped as part of percolateUp of that new 1 node.

b) [2 3 6 8 10 15 18 25] —> add (1) —> [2 3 6 8 10 15 18 25 1] —> do the percolateUp() —> [1 2 6 8 3 15 18 20 25 10]

The height of a binary heap is Theta(log(n)) because it’s complete. So the worst case runtime could happen if you have to do log(n) swaps when you percolateUp/Down from a root to leaf or vice versa.

a) RemoveMin consists of 3 steps: (1) removing the root node (which is always the minimum in the structure, according to the Heap invariant); (2) replacing it with the rightmost node of the bottommost layer (filling the hole left by the root); and (3) performing percolateDown() in which we recursively swap the new root node with its smallest child until the heap invariant is satisfied.

b) In this heap, first 13 gets removed and 11 is moved in to take its place. Then, we swap 11 (which is currently in the root node position) with 4, and then with 5, because at each step those are the smallest child. After the second swap, the heap invariant is restored (and the tree is still complete). In the final heap, 4 is the root, 5 is its left child, and 11 is 5’s left child.

We’re able to enforce a complete tree for heaps because the heap invariant is so much looser than the BST invariant. Since so few restrictions are placed on each node under the heap invariant, it becomes maintainable to keep a complete tree at all times with percolation operations that are fast and do not require changing much of the tree at once.

Remember that removeMin would need to work on an unsorted array by scanning through elements to identify what the minimum element is. Because we have no ordering property that can help us determine what element that is, we’re forced to scan through all of them, a linear time (O(n)) operation. Then, we need to remove that element, which can also be a linear time operation (O(n)) because we may have to shift over every element in the array if the min was found at the very front. Combined, this becomes O(n + n), or an O(n) linear runtime. Note that it is not O(n^2) – that would be the case only if we were to do a linear-time operation for each element in another linear time operation. Since here we do one after the other, we can simply drop the constant factor of 2.

a) RAM is “Random Access Memory”, which is where the majority of data our programs operate on is stored. RAM is a physical chip in the computer, but as computer scientists we typically think of it through a higher-level abstraction as being a giant array, where each index is a memory address. Importantly, RAM is so named because it allows us to “random access” any index, without having to iterate from one end.

b) Contiguous Memory refers to memory that is a continuous chunk of locations next to one another. An array is an example of contiguous memory, because all of the indices are stored next to one another. This has huge implications for spatial locality!

c) The CPU is the “Central Processing Unit”, which is the “brain” of our program and where all operations are carried out. Note that values are only stored here very temporarily: normally, the CPU pulls from a cache or RAM to grab the inputs and writes its output to those locations as well.

d) A cache is a smaller, faster, “intermediate” memory that is located between the CPU and RAM. The purpose of a cache is to provide fast access to a small subset of the addresses we might be interested in, because cache access is much faster than RAM but it is limited in size. We pull whole blocks of RAM into the cache at once, in the hopes that the CPU will next want a nearby address. The cache is represented by a mini-fridge in our bubble tea analogy :)

The first code snippet (the one that uses an array) will exhibit better use of caching and typically run faster in practice. This is because the array elements will be next to each other in memory (contiguous), while the linked list elements may be all over the place, and therefore we likely won’t be able to use the values stored in the cache as much as we complete the second code snippet.

Even though accessing a leaf node of the B+ tree might require scanning through a number of key/value pairs, we typically consider that runtime to be “insignificant” against the massive in-practice runtime of going out to disk. Since a B+ tree is designed to minimize disk accesses, for huge amounts of data it is going to typically be faster (by taking better advantage of the memory layout) than something like an AVL tree. Note that asymptotic analysis still applies and can still be helpful to characterize the number of disk accesses, but that category of operation vastly dominates any analysis we would do on smaller operations that only happen in memory.

a) No, this is not an AVL Tree. The AVL invariant is violated for node 17, which has a left subtree height of 0 and a right subtree height of 2.

b) Yes, this is an AVL Tree. It follows both the BST and AVL invariants.

A BST has no built-in way of regulating its balance, so the performance of the data structure is very dependent on the order that elements are inserted. If we know that elements will be inserted in an order that preserves the balance of the tree, then BSTs can achieve the same complexity class as AVL trees (and avoid some additional constant-time operations!). However, in situations where we can’t predict the insertion order of the elements or know that their ordering will cause a degenerate BST (effectively a linked list), we prefer an AVL tree that can “protect us” from hitting the linear worst-case complexity class for the runtime of containsKey.

The AVL Invariant allows the height of subtrees to differ by 1 so it can be flexible enough to be maintainable as we perform inserts. As we explored in lecture, invariants that are too strict can make it impossible or computationally expensive to maintain them as we perform insertions in our data structure. By allowing a difference of 1, we don’t have to rebalance the tree perfectly every time, an operation that could potentially affect a huge number of nodes. Instead, this invariant hits the “sweet spot” where it is strong enough to maintain logarithmic runtime, but lenient enough that only a single rotation can always restore the property.

We could insert the elements in the order 1, 2, 3, 4, 5. That would cause the BST to become degenerate (effectively a linked list).

a) The worst-case runtime of finding a key in a BST is still n, because we might have a degenerate tree and have to look through every node.

b) Even though the BST invariant doesn’t improve the worst-case runtime of finding a key (over a plain ol’ binary tree), it is still useful! Note that it does help improve the worst possible runtime for finding a key in the case where we assume a nice balanced tree. If we are in a situation where we can make that assumption, the BST invariant gives us a better worst-case runtime – and even in a “normal” situation without a strict guarantee, the BST invariant can help improve the runtime somewhat for most trees that are reasonably balanced (just not in the worst case). As we’ll see, however, where the BST invariant really shines is when we also apply the AVL invariant!

Recall that deterministic means the hash function will always return the same output when the same input is given. Hash functions need to be deterministic so we make sure we look in the same place when we insert an element and when we later look up that element. We apply the hash function for essentially every operation on a Hash Map: we hash the key each time when we insert, and later when we want to look up a key to get it or remove it. Those operations need to come out with the same value, so we can mod it by the number of buckets and end up with a “consistent” place to look to get that confident constant-time “jumping-to-the-right-bucket” behavior!

The hash function is what determines how evenly buckets will be distributed. A uniform hash function will distribute keys more or less evenly across the range of possible integers (then, after applying mod, across the range of possible Hash Map buckets).

a) Suppose we call put(28, "I"). What is the index of the bucket where (28, I) ends up? 3 (because 28 % 5 = 3).

b) Compute the load factor for the Hash Map after inserting I. 1.8 (because there are 9 elements and 5 buckets, so 9 / 5 = 1.8).

c) How many elements would be stored in a new bucket after re-distributing elements during the resizing? 4 elements change position during the resizing: (105, B), (186, C), (77, G), and (28, I). The reason these elements change position is because they have different values % 5 (the old Hash Map) and % 10 (the new Hash Map).

d) Compute the new load factor for the Hash Map after resizing. 0.9 (because there are 9 elements and 10 buckets, so 9 / 10 = 0.9).

(B) “The average chain length of each bucket”. Remember that the load factor can be thought of as beign the length of the chains if things were evenly distributed, which is why it’s so useful in the “In Practice” case. It is not the runtime to determine which bucket a specific key falls into – that is determined by the time it takes the hash function to run (and afterward, array indexing is a constant time operation).

We want to resize every so often in hashing so we can redistribute and spread out our current values over a larger array. This will help keep the average chain length low and help us achieve constant runtime in terms of the size of the map.

1) What is the size of the input on level i? $n/9^i$

2) What is the work done by each node on the i-th recursive level? $n/9^i$

3) What is the number nodes at level i? $3^i$

4) What is the total work done at the i-th recursive level? (number of nodes at that level * work done per node at that level): $3^i * n/9^i = n/3^i$

5) What value of i does the last level of the tree occur? $i = log_9(n)$

6) What is the total work across the base case level? (number of nodes at that last level of the tree * work done per node in the base case): $3^log_9(n) * 1 = n^(1/2)$

7) What is the big theta for this recurrence? Per master theorem: a = 3, b = 9, c = 1. $\log_9(3) = 1/2 <= 1$, so $\Theta(n^c) = \Theta(n)$

C. “An equation that defines a sequence based on a rule that gives the next term as a function of the previous term(s)”.

First, we figure out the values of constants a, b, and c in the Master Theorem. Here, a=4, b=2, and c=2. Since $\log_b(a) = \log_2(4) = 2$, which is equal to c, we’re in Case 2. The theta bound is therefore $\Theta(n^2 * \log(n))$.