I began by discussing the idea of exhaustive search. With an exhaustive search, we generate all possible choices for some problem. For example, suppose you want to generate list all of the 3-digit numbers that are composed of the digits 1, 2, and 3. People started calling out numbers:
123
111
333
321
...
We discussed how you could write code to accomplish this, and someone mentioned
that we could use a triply-nested for loop:
for (int d1 = 1; d1 <= 3; d1++) {
for (int d2 = 1; d2 <= 3; d2++) {
for (int d3 = 1; d3 <= 3; d3++) {
System.out.println("" + d1 + d2 + d3);
}
}
}
I mentioned that this isn't a bad way of thinking about what is involved with
exhaustive search. We are exploring a set of choices to be made (1st digit,
2nd digit, 3rd digit). I mentioned that we often want to see a diagram of the
different choices and where they lead. We refer to this as a decision tree.
Here is the decision tree for the triply nested for loop above:
In backtracking, there is an additional idea that we don't tend to apply in exhaustive search. Instead of exploring every possible choice, we stop exploring when we hit a dead end. A dead end is a state where you know that there is no hope of finding a solution beyond a certain choice. For example, suppose that you are dealing with the Cartesian plane and you want to think about all possible paths from the origin (0, 0) to a particular point. Suppose, for example, that you want to get to the point (1, 2):
^
|
| * (1, 2)
|
<---------------+--------------->
|
|
|
V
Suppose that you are limited to just three kind of moves:
choice #1
|
+----------------+-------+--------+----------------+ ...
| | | |
1st possibility 2nd possibility 3rd possibility 4th possibility
Each choice we might make at the top of the tree leaves us in a different part
of the solution space. From there, we consider all possible second choices we
might make. Below is a complete decision tree for two possible moves:
What happens in backtracking is that we explore specific choices until we reach a dead end. Once we find that some path is not going to work out, we back up to where we last made a choice and make a different choice. That's where the idea of "backtracking" comes from (backing up to the last place we made a choice and moving on to the next choice).
Below is a diagram in which we explore all possible choices trying to move from the origin to the point (1, 2). We stop exploring when we get to an x-coordinate or y-coordinate that is beyond the value we are searching for because that is a dead-end:
Then I said that I wanted to solve a classic problem known as "8 queens". The challenge is to place eight queens on a chessboard so that no two queens threaten each other. Queens can move horizontally, vertically or diagonally, so it is a real challenge to find a way to put 8 different queens on the board so that no two of them are in the same row, column or diagonal.
I began by considering the simpler problem of placing 4 queens on a 4-by-4 board. I started us off by putting a queen in the upper-left corner:
Q - - -
- - - -
- - - -
- - - -
I asked people to place more queens and we were able to get 3 queens on this
board with no two threatening each other, but not four. That's because I
purposely led us down a bad path. There is no solution with a queen in the
corner. I allowed people to make their own suggestions and we got this one:
- - Q -
Q - - -
- - - Q
- Q - -
I pointed out that people are really smart about solving problems like these.
Computers are not. So the computer will solve this by exploring all sorts of
possibilities.The simplest way to think of this in terms of a decision tree is to imagine all the places you might put a first queen. For 8 queens, there are 64 of them because the chess board is an 8 by 8 board. So at the top of the tree, there are 64 different choices you could make for placing the first queen. Then once you've placed one queen, there are 63 squares left to choose from for the second queen, then 62 squares for the third queen and so on.
The backtracking technique we are going to use involves an exhaustive search of all possibilities. Obviously this can take a long time to execute, because there are lots of possibilities to explore. So we need to be as smart as we can about the choices we explore. In the case of 8 queens, we can do better than to consider 64 choices followed by 63 choices followed by 62 choices and so on. We know that a whole bunch of these aren't worth exploring.
One approach is to observe that if there is any solution at all to this problem, then the solution will have exactly one queen in each row and exactly one queen in each column. That's because you can't have two in the same row or two in the same column and there are 8 of them on an 8 by 8 board. So we can search more efficiently if we go row by row or column by column. It doesn't matter which choice we make, so let's explore column by column.
In this new way of looking at the search space, the first choice is for column 1. We have 8 different rows where we could put a queen in column 1. At the next level we consider all of the places to put a queen in column 2. And so on. So at the top of our decision tree we have:
choice for column 1?
|
|
+------+------+------+---+--+------+------+------+
| | | | | | | |
row 1 row 2 row 3 row 4 row 5 row 6 row 7 row 8
So there are 8 different branches. Under each of these branches, we have 8
branches for each of the possible rows where we might place a queen in column
2. For example, if we think just about the possibility of row 5 for the queen
in column 1, we'll end up exploring:
choice for column 1?
|
|
+------+------+------+---+--+------+------+------+
| | | | | | | |
row 1 row 2 row 3 row 4 row 5 row 6 row 7 row 8
|
choice for column 2?
|
+------+------+------+---+--+------+------+------+
| | | | | | | |
row 1 row 2 row 3 row 4 row 5 row 6 row 7 row 8
Again, the pictures I can draw in these notes aren't very good because the tree
is so big. There are 8 branches at the top. From each of these 8 branches
there are 8 branches. And from each of those branches there are 8 branches.
And so on, eight levels deep (one level for each column of the board).For example, we might choose to explore placing a queen in row 1 for column 1. So we go down that branch. We will explore all of the possibilities in that branch. If they all lead to dead ends, then we'll come back to the top level and move on to the possibility of having a queen in row 2 of column 1. We explore all of those possibilities. And if they all fail, we try row three for column 1. And so on. As long as we have more choices to explore, we keep trying the next possibility. If they all fail, we conclude that this is a dead end.
It's clear that the 8 choices could be coded fairly nicely in a for loop, something along the lines of:
for (int row = 1; row <= 8; row++)
But what we need for backtracking is something more like a deeply nested for
loop:
for (int row = 1; row <= 8; row++) // to explore column 1
for (int row = 1; row <= 8; row++) // to explore column 2
for (int row = 1; row <= 8; row++) // to explore column 3
for (int row = 1; row <= 8; row++) // to explore column 4
....
That's not a bad way to think of what backtracking does, but we're going to use
recursion to write this in a more elegant way.Before we looked at the recursive solution to the problem, I mentioned something I learned from an old colleague of mine named Steve Fisher who used to teach the cse143 equivalent at Stanford in Pascal back in the 1980's. Steve pointed out to me that these recursive backtracking problems are much easier to write and to understand if you separate off the low-level details of the problem. For the 8 queens problem he created supporting code for what we would today recognize as a board object to keep track of a lot of the details of the chess board.
So I began by discussing what kind of methods we'd want to have for a board object. Obviously it would need some kind of constructor. I mentioned that we want to pass it an integer n so we could solve the more general "n queens" problem with an n-by-n board. So we want:
public Board(int size)
I asked for more suggestions. Someone said we need to be able to test whether
it's safe to place a queen at a particular location:
public boolean safe(int row, int col)
What else? Someone said we need a way to place a queen on the board:
public void place(int row, int col)
Someone else mentioned that we'd need a way to remove a queen because the
backtracking involves trying different possibilities. So we need:
public void remove(int row, int col)
Anything more? Someone suggested that we need a way to find out where the
queens are placed. I said that for our purposes, let's settle for a method
that just prints out the entire board, queens and all:
public void print()
Anything else? Perhaps I'm just a lazy programmer, but I'd like to be able to
ask the board what size it is:
public int size()
So I told people to assume that we have a Board class that implements all of
those methods. But that's not the interesting code. The interesting code is
the backtracking code which, given this class, we can now write in a very
straightforward manner.I mentioned that in the program I wrote, I prompt the user for a board size and I construct a board of that size. Then I call a method called "solve", passing it the Board object:
public static void solve(Board b) {
...
}
I mentioned that one of the issues that comes up with recursive programming is
that we are often asked to write a method like this and we discover that we
want to have the recursion work in a different way. For example, we might want
more parameters or we might want to have a different return type. So it's
common with recursive programming to introduce a "helper" method that does the
actual recursion. That's what we'll do here.So how is the recursion going to work? I pointed out that there are different ways to approach a backtracking problem. You can decide that you want to find all solutions or you might decide that one is enough. In this case, we are going to print all of them.
I mentioned that a good name for our recursive method would be "explore". It is a helper method for the public method, so we want to make it private. So what we know so far is that it looks like this:
private static void explore(...) {
...
}
What parameters will it need? Someone fairly quickly pointed out that it needs
the Board object. Anything else? This seemed to stump people for a while.
That's not surprising, because we're trying to understand how this recursion is
supposed to implement the backtracking.I said that each level of the decision tree will be handled by a different invocation of the method. So one method invocation will handle column 1. A second invocation will handle column 2 and so on. So what will the method need to know besides where the Board is? Someone said it needs to know what column to work on. So that leaves us with this header:
private static void explore(Board b, int col) {
...
}
I mentioned that we don't want to waste our time exploring dead ends. So we
want to think in terms of what kind of precondition we'd want for this method.
For example, suppose that I've placed queens in columns 1, 2 and 3 and I'm
thinking of calling the method to explore column 4. Would that make sense if
the first three queens have not been placed safely? The answer is no. If the
first three queens already threaten each other, then why bother placing any
more? Once something becomes a dead end, we should stop exploring it. So I
mentioned that the method should have the following precondition:
// pre: queens have been safely placed in previous columns
I told people that this precondition will turn out to be very helpful to us as
we think through the different cases. We're using recursion to solve this, so
we have to think about base cases and recursive cases. So I asked people,
"What would be a nice column to get to?" Some people said "8" and I said it
would be nice to get to column 8 because it would mean that 7 of the 8 queens
have been placed properly. But other people had an even better answer. It
would be even better to get to column 9 because the precondition tells us that
if we ever reach column 9, then queens have been safely placed in each of the
first eight columns.This turns out to be our base case. What do we do if we get to column 9? If we get there, we'd know that we've found an answer, so we can just print it:
if (col > b.size()) {
b.print();
} else {
...
}
What about the recursive case? For example, maybe 4 queens have already been
placed and we've been called up to place a queen in column 5. We have 8
possibilities to explore (the 8 rows of this column where we might place a
queen). A for loop works nicely to go through the different row numbers.What do we do with each one? First we'd want to make sure it's safe to place a queen in that row of our column. If not, then it's a dead end that's not worth exploring. So our code will look like this:
for (int row = 1; row <= b.size(); row++) {
if (b.safe(row, col)) {
...
}
}
And how do we explore a particular row and column? First we place a queen
there. And then we'd explore the other choices. In this case, the other
choices involve other columns that come after our column. That's where the
recursion comes in:
for (int row = 1; row <= b.size(); row++) {
if (b.safe(row, col)) {
b.place(row, col);
explore(b, col + 1);
...
}
}
I pointed out that people often use an expression like "col++" in their
recursive calls, but it's not a good idea. In this case, because we're in a
for loop, we'd be using different column numbers for each row.But what if it doesn't work? What if it turns out to be a dead end? Then we want to move on to the next possibility and explore it. That means just coming around the for loop to try the next row. But it's not quite that simple, because we have placed a queen in the current row. We have to "undo" the current choice before we move on to the next choice:
for (int row = 1; row <= b.size(); row++) {
if (b.safe(row, col)) {
b.place(row, col);
explore(b, col + 1);
b.remove(row, col);
}
}
This pattern of making a choice (placing a queen), then making a recursive call
to explore later choices, and then undoing the choice (removing the queen) is
common in recursive backtracking. It is almost a backtracking mantra: choose,
explore, unchoose.If we get all the way through this for loop without finding a solution, then we simply return and allow the backtracking to proceed if there are any other choices to be explored.
It turns out this is the entire recursive backtracking code. Putting it all together we get:
private static void explore(Board b, int col) {
if (col > b.size()) {
b.print();
} else {
for (int row = 1; row <= b.size(); row++) {
if (b.safe(row, col)) {
b.place(row, col);
explore(b, col + 1);
b.remove(row, col);
}
}
}
}
We need some code in the "solve" method that starts the recursion in column 1
and then we're done:
public static void solve(Board solution) {
explore(solution, 1);
}
I brought up a full version of the program and ran it to show people that it
finds a solution to 8 queens, 4 queens, and so on. In the case of 2 queens, it
finds there are no solutions.I then brought up a variation of the program that includes an animation. The animation is generated by the backtracking itself. I can't in these notes capture what that dynamic animation shows, so I'll just say that if you weren't in class to see it, I highly recommend that you download the queens zip file from the calendar and compile and run the Queens2 program.