CSE143X Notes for Wednesday, 11/6/13

We continued our discussion of recursion by looking at several more examples. We started by exploring how to write a method called stutter that will take an integer value like 348 and return the integer 334488 (the value you get by replacing each digit with two of that digit). How do we write this? We again started with simple cases. What are easy numbers to stutter? Someone suggested 0, although that's actually a hard number to stutter. We can't really return 00, although we'll consider it acceptable to return 0. So what are some easy numbers to stutter? 1, 2, any one digit number.

So how do we stutter one-digit numbers? To turn a digit n into nn, we'd need to do something like this:

        n0  10 of n
         n  plus 1 of n
        --
        nn
So we need a total of 11 times n. Some people saw this without thinking about individual digits and it's pretty obvious when you think about it that 1 times 11 is 11, 2 times 11 is 22, 3 times 11 is 33, and so on. I said to assume for now a precondition that n is greater than or equal to 0. So we have our base case:

        //pre : n >= 0
        public int stutter(int n) {
            if (n < 10)
                return 11 * n;
            else
                return ...;
        }
So what do we do with numbers like 348? We need some way to break it up into smaller chunks, a way to split off one or more digits. Someone suggested that division by 10 would help:

        n = 348

             34 | 8
         -------+-------
         n / 10 | n % 10
What next? Here you have to think recursively. The original number was 348. We've split it up into 34 and 8. What if we stuttered those values? You have to make the leap of faith and believe that the method actually works. If it does, then it turns 34 into 3344 and 8 into 88. How do we combine them to get 334488? We can't do simple addition:

        3344
          88
        ----
        wrong
We need to shift the 88 over to the right relative to the 3344. We do this by multiplying 3344 by 100:

        334400
            88
        ------
        334488
But that means we've solved the problem:

        //pre : n >= 0
        public int stutter(int n) {
            if (n < 10)
                return 11 * n;
            else
                return 100 * stutter(n / 10) + stutter(n % 10);
        }
I mentioned that the final version of the method in the handout includes an extra case for negative numbers. So we remove the precondition and add a case for negative n. For a number like -348, we return -334488 by stuttering the positive number (stutter 348) and then negating the result.

Then we turned to the problem of writing the binary representation of a number. I pointed out that when you count in binary, you only have two digits to work with. So in base 10, you don't run out of digits and switch to 10 for a while (0, 1, 2, ..., up to 9). But in base 2, you run out of digits almost right away (0, 1, out of digits). So in base 2, counting goes like this:

        base 10    base 2
        -----------------
         0          0
         1          1
         2          10
         3          11
         4          100
         5          101
         6          110
         7          111
         8          1000
I said that I didn't have time to go through this example in detail, but that I'd explain the basic idea. It's a nice application of recursion, because we've seen that recursion just requires you to be able to do a small bit of the overall problem, to nibble away at the problem. So I said consider a number like 345,926. What is its base-2 representation? Nobody in the class was geeky enough to know the answer (or at least they weren't willing to let people know that they were that geeky). But I asked people if they could tell me anything about the base-2 representation of 345,926. Someone said it ends in 0. How do you know? Because it's an even number. Even numbers end in 0 in base-2 and odd numbers end in 1 (just look at the list above and see how the final digit alternates).

This doesn't seem like much, but that small observation is enough to solve the overall task. It requires some insight into the problem that I didn't have time to develop in detail, but I made an analogy. I reminded people that when we worked on stutter and were considering a base-10 number like 342, we used division by 10 to divide it into two pieces:

        n = 342

             34 | 2
         -------+-------
         n / 10 | n % 10
It turns out that the same kind of trick works in base-2. Division by 10 gives you the leading digits in base-10 versus the final digit in base-10. The corresponding rule in base-2 would be:

n = 345,927 <leading base-2 digits of 345,927> | 1 -----------------------------------+------ n / 2 | n % 2 So to write out the base-2 representation of n, we simply write the base-2 representation of (n / 2) and then write the final digit (n % 2). The base case is for numbers that are the same in base-2 and base-10 (0 and 1). Throwing in a case for negatives, we end up with the solution from handout #13:

        public void writeBinary(int n) {
            if (n < 0) {
                System.out.print("-");
                writeBinary(-n);
            } else if (n < 2)
                System.out.print(n);
            else {
                writeBinary(n/2);
                System.out.print(n % 2);
            }
        }
This relatively short and simple method will write out the binary representation of any number. And if you change the three occurrences of "2" to "3", then it writes out the base-3 representation of a number. If you change the 2's to 4's, it writes out the base-4 representation. And so on.

We then considered the task of writing a method sum that takes an array of integers as a parameter and that returns the sum of the numbers in the array. Even though this is a fairly simple task, this turns out to be challenging to implement using recursion. We could make a base case for an empty array (an array of length 0):


        public int sum(int[] list) {
            if (list.length == 0)
                return 0;
            else {
                ...
            }
        }
But what would we use for a recursive case? Suppose, for example, that we are asked to work with an array of length 10. How do we simplify an array of length 10 so that it gets closer to being an empty array? The only way to do that would be to construct a new array and we're specifically told we can't do that.

It is useful to consider how you would solve this iteratively. We'd use a classic cumulative sum approach:

        int sum = 0;
        for (int i = 0; i < list.length; i++)
            sum += list[i];
The key to the iterative approach is to work with an index variable that starts at 0 and goes until it becomes the length of the array. We can use this same idea for a recursive solution. Remember that you want to think about how to pick off some small piece of the problem. If we're allowed to refer to indexes, then we can say that:

        sum(entire list) = list[0] + sum(list starting at 1)
In other words, we know that we have to add in list[0]. And what does that leave us? That leaves us with the task of adding up the remaining values of list (the values starting at index 1). And how would we do that? We can similarly say that:

        sum(list starting at 1) = list[1] + sum(list starting at 2)
This would lead to a series of recursive calls, each of which handle one element of the array:

        sum(entire list) = list[0] + sum(list starting at 1)
            sum(list starting at 1) = list[1] + sum(list starting at 2)
                sum(list starting at 2) = list[2] + sum(list starting at 3)
                    sum(list starting at 3) = list[3] + sum(list starting at 4)
                        sum(list starting at 4) = list[4] + sum(list starting at 5)
And when does this process end? We could end when we get to the last element of the array:

        sum(list starting at list.length-1) = list[list.length - 1]
This led to the following code:

        if (index == list.length - 1)
            return list[list.length - 1];
        else
            return list[index] + sum(list, index + 1);
Unfortunately, this assumes that a last element exists. That won't be true of an array of length 0. An easier approach is to end when we have reached an index beyond the last element of the array. The for loop version of this executes as long as the index is less than the length of the array. Once the index becomes equal to the length of the array, it stops. Our recursion can similarly stop when the index becomes equal to the length of the array. And what does that add up to? It adds up to 0:

        if (index == list.length)
            return 0;
        else
            return list[index] + sum(list, index + 1);
Returning 0 in this case is the recursive equivalent of initializing the cumulative sum to 0.

We are almost ready to write the recursive method, but to express this idea as a method, we would have to pass the method both the array and the starting index. So we want our method to look like this:

        // computes the sum of the list starting at the given index
        public int sum(int[] list, int index) {
            ...
        }
If we are allowed to have those two parameters, then the code is fairly simple to write using the approach described before:

        // computes the sum of the list starting at the given index
        public int sum(int[] list, int index) {
            if (index == list.length)
                return 0;
            else
                return list[index] + sum(list, index + 1);
        }
Of course, there's the issue of that extra parameter. We were asked to write a method that is passed just the array. We could complain that we were asked to solve the wrong problem, but that's a bad attitude to have. We should allow a client to call methods using parameters that are convenient. We need the second parameter to write our recursive solution, but we shouldn't burden the client with the need to understand that detail.

This is a common situation. We often find ourselves wanting to write a recursive method that has extra parameters relative to the public method we are being asked to implement. The solution is to introduce a private method that has the parameter passing we want. So the client gets the convenient public method with the kind of parameter passing the client wants and we get our convenient private method with the kind of parameter passing we want. Here is what it would look like in this case:

        // returns the sum of the numbers in the given array
        public int sum(int[] list) {
            return sum(list, 0);
        }

        // computes the sum of the list starting at the given index
        private int sum(int[] list, int index) {
            if (index == list.length)
                return 0;
            else
                return list[index] + sum(list, index + 1);
        }
The public method simply calls the private method passing it an appropriate value for the second parameter. Both methods can be called sum because they have different signatures (one parameter versus two). We make the second method a private method because we don't want to clutter up the public interface seen by the client just because we found it useful to have this method available to us in implementing the public method.

There is a final benefit to this two method approach. Sometimes there are actions you want to perform once either before or after the recursive method executes. By dividing this into a public and a private method, we can include this one-time code in the public method either just before or just after the call on the private method.

Then I went to the computer to consider a problem that will be more like the next homework assignment. The idea is to prompt the user for the name of a file or directory and to print out that name along with any files that are underneath it. In particular, if it's a directory, we want to include everything inside the directory.

Java has a class called File that is part of the java.io package that provides the functionality we need to solve this problem. I put up this starter code for us:

        import java.io.*;
        import java.util.*;
        
        public class Crawler {
            public static void main(String[] args) {
                Scanner console = new Scanner(System.in);
                System.out.print("directory or file name? ");
                String name = console.nextLine();
                File f = new File(name);
                if (!f.exists()) {
                    System.out.println("That file or directory does not exist");
                } else {
                    print(f);
                }
            }
        
            public static void print(File f) {
                System.out.println(f.getName());
            }
        }
This is the boring prompting part of the program. The real work is in writing the print method that is supposed to print everything. In the starter code, we simply print the name of the given file. This worked fairly well. For example, when I ran it and typed in "/Users/reges/143/crawler/Crawler.java", it printed:

        Crawler.java
This is the name of the file we were writing and it's what we want it to do for a file name. Then I typed in the directory name "/Users/reges/143/crawler" it printed:

        crawler
This is good in that it is printing the name of the crawler directory, but it wasn't showing any of the files and subdirectories inside the directory. One thing to keep in in mind is that the File class in Java is used to store information about both files and directories. In the first case, we were given a File object that was linked to a file. But in this second case, we have a File object linked to a directory.

I mentioned that there is a method called listFiles that will return a list of the files in a directory. The return type is File[]. So we might be tempted to write code like this:

        public static void print(File f) {
            System.out.println(f.getName());
            File[] files = f.listFiles();
            for (int i = 0; i < f.length; i++) {
                // do something with files[i]
            }
        }
That approach works, but I pointed out that this is a good place to use a for-each loop. We don't particularly care that the files are stored in an array. We only care that they are in some kind of list structure that we can iterate over. The for-each loop allows us to do this with simpler syntax:

        public static void print(File f) {
            System.out.println(f.getName());
            for (File subF : f.listFiles()) {
                // do something with subF
            }
        }
When we describe a for-each loop like this, we read it as, "for each File subF in f.listFiles()...". You need to come up with a variable name in the for-each loop. I couldn't use "f" because I was already using that for the name of the File passed to the method as a parameter. I could have used any name I want ("x", "foo", etc). I used the name "subF" because it seemed to me like a good way to indicate that this is a subfile of the original File object.

What we want to do with this is to print each subfile name with some indentation. So we can complete this with a simple println:

        public static void print(File f) {
            System.out.println(f.getName());
            for (File subF : f.listFiles()) {
                System.out.println("    " + subF.getName());
            }
        }
This version worked nicely. When I typed in "/Users/reges/143/crawler", it showed the name of the directory along with a list of the files and subdirectories inside the directory

crawler
    Crawler.class
    Crawler.java
    Crawler.java~
    dirA
    dirB
    dirC
    DirectoryCrawler.class
    DirectoryCrawler.java
    Fun.java
In fact, this is the same list we saw in the Mac finder window. So far so good.

But when we ran it again and asked it to print information for "/Users/reges/143/crawler/Crawler.java", we got a NullPointerException. Remember that sometimes the method is going to be asked to print a File object tied to a directory and sometimes it's going to be asked to print a File object that is tied to a simple file. For the simple file case, when we call f.listFiles(), we get the value null returned. So then when we try to use it in a for-each loop, we get the NullPointerException thrown.

We fixed this by including a test on whether or not the File object is a directory. We found just what we needed in the File class: a method called isDirectory that returns a boolean result. So we rewrote print to be:

        public static void print(File f) {
            System.out.println(f.getName());
            if (f.isDirectory()) {
                for (File subF : f.listFiles()) {
                    System.out.println("    " + subF.getName());
                }
            }
        }
At this point we had a pretty good solution using iterative techniques. But we're still far from a working solution. I ran the program again and typed in "/Users/reges/143/crawler". It printed the "crawler" name and printed the contents of the directory, but it didn't show everything. For example, I have a subdirectory called "dirA" and it wasn't showing the contents of that. Why not? Because there are files and subdirectories inside that directory (or folders inside that folder if you prefer the desktop terminology). Our code doesn't handle directories inside of directories, so we have to fix it.

This is a very crucial point in the development of this program. In fact, if there is anything that I would recommend going over several times, it is what is about to happen right here. This is the crucial insight as to how to apply recursion here and it will be the most useful guide to help you understand what you'll need to do for the next programming assignment.

So let's review where we are at. We have a method called print that is passed a File parameter called f. If f is a directory, we use a for-each loop to iterate over its contents, storing each one in a File variable called subF. The problem is that some of the subF objects are themselves directories that need to be expanded. Your instinct, then, might be to do something like this:

        public static void print(File f) {
            System.out.println(f.getName());
            if (f.isDirectory()) {
                for (File subF : f.listFiles()) {
                    System.out.println(subF.getName());
                    if (subF.isDirectory()) {
                        ...
                    }
                }
            }
        }
This is not the right way to think about the problem. And it won't work. What if we had an inner for-each loop with a subsubF variable? Would that solve the problem? No, because one of those subsubF objects might be a directory. So we'd need another test and another loop with a subsubsubF variable. Would that work? No, because one of those might be a directory. We can't solve this problem easily with simple iterative techniques. The easier way to solve it is to think about this problem recursively.

To think about it recursively, think about the method we are writing. It is supposed to print information about either a file or a directory. If it is printing information about a directory, then it prints the contents of the directory. Our problem is that those contents might be either files or directories, which have to be handled differently.

Often with a recursive problem, the answer will be staring you right in the face if you can just connect the dots. We find ourselves inside the for-each loop needing to print information about either a file or a directory. As I keep saying in lecture (to make fun of how easy this is when you can just look at it the right way), "If only we had a method that would print information about a file or directory, we could call it...But we do have such a method...it's the print method we're writing."

We can almost make this code work by replacing the println with a call on print:

        public static void print(File f) {
            System.out.println(f.getName());
            if (f.isDirectory()) {
                for (File subF : f.listFiles()) {
                    print(subF);
                }
            }
        }
We ran this version and when we asked for the contents of "/Users/reges/143/crawler" we saw a long list as it printed every single file and directory that is found inside that directory.

I've been using an almost joking tone when I say, "If only we had a method that would..." I don't mean to imply that this is somehow obvious. It takes time to master recursion, so you should expect that you'll struggle as you work to understand this. The point I am trying to make is that the answer is probably simpler than you think it is. There is an elegance to recursion that allows you to solve very complex problems with just a few lines of code. So when you struggle to understand this, be sure to consider the possibility that the answer is actually much simpler than you realize if you can just think about it the right way.

I mentioned earlier that this solution is almost right. There are two problems. First, it should indent things to indicate what is inside of what. The problem is that we need different levels of indentation. We solved this by adding a new parameter called level to the method and using it in a for loop to print indentation:

        public static void print(File f, int level) {
            for (int i = 0; i < level; i++) {
                System.out.print("    ");
            }
            System.out.println(f.getName());
            ...
        }
This required us to change the call in main to include a level:

        print(f, 0);
We also had to change the recursive call inside of print to include a value for the level parameter:

        print(subF, level + 1);
It is important to use an expression like (level + 1) versus saying level++. In writing iterative solutions, we often use expressions like level++, but they tend not to work well in recursive solutions. In this case, it would cause successive elements of a directory to be printed with increasing indentation, moving diagonally to the right rather than lining up.

When we ran the new version with "/Users/reges/143/crawler" we saw this list of files and subdirectories, with indentation indicating what is inside of what:

        crawler
            Crawler.class
            Crawler.java
            Crawler.java~
            dirA
                data.txt
                dirA1
                    fun.txt
                    fun2.txt
                dirA2
                    dirA2a
                        deep.txt
                        deep2.txt
                    help.txt
            dirB
                dirB1
                    Fun.class
                    Fun.java
                dirB2
                dirB3
                    dirB3a
                        answers.txt
                        howdy.txt
                    hw4.txt
                paper.doc
                sample.txt
            dirC
                special.txt
            DirectoryCrawler.class
            DirectoryCrawler.java
Because the code is written recursively, the crawler can handle any level of nesting that we encounter.


Stuart Reges
Last modified: Thu Nov 8 10:12:16 PST 2018