CSE 331: Whole program assertions

lecture

Course motto: Your job is to deliver code that you know works.

What does this program do?

The program below is just twenty-two lines long. Read it carefully before you read anything else. Then answer one question: what does it do?

Your answer should describe what a person who runs the program sees, not what each line does, as you would tell a friend who needed to run the program. Write your answer down. As we go through the notes for this lecture we are going to hold your "user manual" to a high standard you may find surprising. Since the code here is fixed, your description is "right" only if it is true on every run of the program. To figure that out, you have to have some way of coming to know the code will behave as the user manual describes (again, on every run of the program!).

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

import java.util.ArrayList;
import java.util.List;
import java.util.Scanner;

public class Sum {
    public static void main(String[] args) {
        Scanner in = new Scanner(System.in);
        System.out.println("Enter some numbers.");
        List<Integer> list = new ArrayList<>();
        while (in.hasNextInt()) {
            list.add(in.nextInt());
        }
        // (1)
        System.out.println("Got " + list);
        int sum = 0;
        for (int i = 0; i < list.size(); i++) {
            sum += list.get(i);
        }
        // (2)
        System.out.println("The sum of your list was: " + sum);
    }
}

The comments marked (1) and (2) do nothing; they mark two places we will come back to.

To run the program yourself, save it as Sum.java, then compile and run it in a terminal:

javac Sum.java
java Sum

Type some numbers and then end the input, with Control-D on macOS and Linux or with Control-Z and Enter on Windows. One run looks like this:

Enter some numbers.
3 4 5
Got [3, 4, 5]
The sum of your list was: 12

Every run

Most readers answer with something like:

It reads a list of numbers from the user and prints them and their sum.

For the single run above, that answer is exactly right. Whether it is right about the program in general is a different question.

A run of a program is a single execution of it, from the first line of main to the exit, on one particular input. The run above is one example. If instead we run again and type 1 2 3, that would give another. Typing nothing at all would give a third example run.

In general, a program that reads input has as many runs as there are things a user might type, which is to say infinitely many possible runs! Our "user manual" sentence is a claim about every possible run. A single run of the program can only confirm the claim for that run alone. On the other hand, a single run on which the "user manual" sentence is false refutes the claim. Such a run is a counterexample: it shows that the claim does not hold.

OK, so let's try a run that our "user manual" sentence did not have in mind. Try typing a word among the numbers:

Enter some numbers.
1 2 three 4
Got [1, 2]
The sum of your list was: 3

The user typed four things, and the program read two of them, printed those two with their sum, and said nothing about the rest. Our sentence never mentioned this possibility, and a user who typed three by accident and got a sum of 3 would probably feel misled.

Next, try typing nothing and ending the input immediately. Now the program prints Got [] and a sum of 0. Our "user manual" sentence is true of this run, if we grant that an empty list has a sum of 0.

Finally, let's try typing two numbers that are each unremarkable on their own:

Enter some numbers.
2147483647 1
Got [2147483647, 1]
The sum of your list was: -2147483648

The two numbers add to 2147483648, but the program printed a negative number! A Java int holds only the whole numbers from \(-2^{31}\) to \(2^{31} - 1\), that is, from -2147483648 to 2147483647, and an addition that passes the top wraps around to the bottom. Our sentence promised the sum, and on this run the program did not deliver it.

What happened here happens for most of us on most programs. The "user manual" we write after a first reading is true of the runs we pictured while writing it and silent or wrong about the runs we did not. Unfortunately, the runs we did not imagine are where programs tend to fail once other people start using them. If your job is to deliver code that you know works, that habit is the first thing standing in your way: you cannot know that a program works until you can say, for every run, what working means.

The specification

The overflow run refutes our "user manual" sentence (i.e., it shows that it is false, that it does not actually hold). We could change the program, so that it reads long values (to avoid overflow on the earlier example), or stops with a message at the first word, or refuses input whose sum would overflow. None of those would make our sentence true, though, and each would need a different one. So for now, we keep the program and repair the "user manual" sentence, which means we need to add what it left out.

The program asks for numbers. It reads whole numbers between \(-2^{31}\) and \(2^{31} - 1\) from the user, in order, until the input ends or the next thing typed is not such a number; anything from that point on is ignored. It prints the numbers it read, then their sum, wrapped around into that same range if the true sum falls outside it.

That is longer and uglier than the sentence we began with, but it has the virtue of being true on every run.¹

Check it against the four runs above: it says what happens on the word, on the empty input, and on the overflow, and a user who read it first would be surprised by none of them. Even the range clause is needed, because typing 2147483648 alone makes the program read nothing at all, that number being too large for an int, and the sentence says so. The clauses about the range are unpleasant, and their unpleasantness is information: they tell you that this program is probably not the one you want, and they point at what to change.

On every run is the part our first version got wrong and our second version gets right.

Because a specification makes a claim about all possible runs of the program, a single run where it is false refutes it. If there are infinitely many possible runs (the common case), no number of runs where the specification happens to be true can establish it.

As seen from outside is why the specification never mentions list, sum, hasNextInt, or line 15: a user manual is written for someone who will never see the code, and a good one lets that person predict the program's behavior without it.

Must is there because a specification does not describe what the code happens to do; it states what the code is required to do, and the code either meets that requirement or fails it. When we repaired our sentence above we were describing, and the result was a specification this program meets. More often the order is reversed: someone writes the specification first, and the program has to live up to it. Homework 1 will ask you to work in both directions.

What is true at a point

The specification speaks from outside the program: it must be written in terms a user can check.

In order to know that a program meets its specification, we have to look inside it, at the state of its variables at various points during execution. OK, so now let's step inside the code, to the comment marked (1) on line 13. The loop has just finished, nothing has been printed since "Enter some numbers.", and the program is about to print list. What do we know about list at that moment?

We know quite a lot, and we know it without running anything. Whatever the user typed, the loop took numbers from the input one at a time, in order, and stopped at the end of the input or at the first thing that was not an int; so at line 13, list holds exactly the numbers the user typed before that point, in the order they typed them. That is true on the run with 3 4 5, on the run with three in it, on the empty run, and on the overflow run, and it would be true on any run we have not tried.

A fact like that differs in kind from the specification. The user cannot see it, since nothing has been printed yet, and the user does not know that a variable called list exists. It is a claim about the inside of the program, at one particular point. We call such a claim an assertion. This one is true at that point on every run.

We write an assertion in double braces at the point it describes, so that the claim sits beside the code it is about:

        // (1)
        {{ list holds, in order, every number the user typed before
           the input ended or something that was not an int appeared }}
        System.out.println("Got " + list);

If you have ever written a comment like // list is sorted here to remind yourself of something you were sure of, you have written an assertion. Three things it is not, though:

• It is not Java: the program never executes the braces, and Java's own assert statement, which does execute when assertions are turned on, checks a condition on the one run that reaches it.
• It is not a print statement, which would likewise show you only what particular runs do.
• It is not a test, which checks the runs you chose to try.

An assertion, like a specification, claims something about all the runs at once, the ones you tried and the ones you did not.

Sum with assertions

Point (2) on line 19 sits just after the second loop. The same kind of reasoning applies there. sum started at zero on line 15, and the loop added each element of list to sum exactly once. So at line 19, sum is the sum of the elements of list, with the same qualification the specification needed: if the true sum does not fit in an int, then sum holds the wrapped-around value instead.

That is the claim most people write at (2). However, it is not the only claim that line 20 depends on. The sum the user sees on line 20 only makes sense as the sum of the numbers they were shown on line 14. So a second claim belongs at (2): list still holds exactly what it held at (1). This second claim holds, because the summing loop only reads list and never writes to it. It is the kind of fact that often goes unstated, because it seems too obvious to state. But a program that changed list while summing it could print two lines that do not agree with each other, so the claim is worth writing down.

Here is Sum with both points filled in:

        while (in.hasNextInt()) {
            list.add(in.nextInt());
        }
        // (1)
        {{ list holds, in order, every number the user typed before
           the input ended or something that was not an int appeared }}
        System.out.println("Got " + list);
        int sum = 0;
        for (int i = 0; i < list.size(); i++) {
            sum += list.get(i);
        }
        // (2)
        {{ sum is the sum of the elements of list, wrapped into the
           int range if the true sum falls outside it }}
        {{ list still holds, in order, every number the user typed
           before the input ended or something that was not an int
           appeared }}
        System.out.println("The sum of your list was: " + sum);

Now let's set the specification beside the assertions. The specification makes two promises about what happens after the prompt: the program prints the numbers it read, and then the program prints their sum (wrapped if need be). Line 14 keeps the first promise because of the assertion at (1): line 14 prints list, and list holds the numbers read. Line 20 keeps the second promise because of the two assertions at (2): line 20 prints sum, sum is the wrapped sum of list, and list still holds the numbers read. In other words, the specification says what the user will see, and the assertions say why the code delivers it.

Reasoning between assertions

We have now made two arguments about Sum. The first argued from what the input loop does to the claim at (1). The second argued from the claim at (1), through the summing loop, to the two claims at (2). Both arguments have the same shape: a claim before a piece of code, the code itself, and a claim after the code. This shape is the pattern of nearly every argument about a program that we will make this quarter.

{{ P }}
some code
{{ Q }}

Here is the rule. Suppose P holds at its point. Suppose also that running the code from any state in which P is true ends in a state in which Q is true. Then Q holds at its point. This rule is powerful because of what it leaves out. To establish Q, we needed only P and the few lines of code between P and Q. We needed nothing about how the program arrived at P, because whatever happened earlier is summarized by P. That is why a large program can be understood a few lines at a time.

Our two arguments above were made in prose, at the level of "the loop added each element once." That level of argument is fine for a loop whose behavior is obvious. Making the argument precise, so that it stands up for any loop, is the work of the lectures on reasoning later in the course.

The edge of the program

Everything a program knows about the world arrives through its input. At the edge of the program, where the input comes in, there is no one to make promises. The person at the keyboard may type a word where the program wanted a number, or nothing at all, or paste in a whole file. They did not read the code, and they owe the program nothing. Therefore, a specification for a whole program must not begin with "assuming the user types numbers." Instead, it has to say what the program does on every input: the inputs its author hoped for and the inputs its author did not. That is why our specification had to mention the word, the empty input, and the overflow.

Sum's answer to a word is to stop reading and print what it has so far. Other answers would be reasonable too: Sum could complain about the word, or skip the word and keep going. What is not reasonable is having no answer at all. Here is what having no answer looks like. Suppose line 10 used hasNext, which checks only that more input is waiting, instead of hasNextInt, which checks that the next thing is an int. Now suppose the user typed a word:

Enter some numbers.
3 4 hello
Exception in thread "main" java.util.InputMismatchException
    at java.base/java.util.Scanner.throwFor(Scanner.java:977)
    at java.base/java.util.Scanner.next(Scanner.java:1632)
    at java.base/java.util.Scanner.nextInt(Scanner.java:2297)
    at java.base/java.util.Scanner.nextInt(Scanner.java:2251)
    at Sum.main(Sum.java:11)

A stack trace is a message from the language to the programmer. It shows the exception, the method that threw it, and the line in our code that led there. To someone who can read the code, a stack trace is the start of a repair. To the person who typed hello, it is none of that. The stack trace does not say what went wrong in terms they understand. It does not say what to do next. And it ends the run without the output they came for. The stack trace reached the user, as stack traces usually do, because the author never decided what should happen on a word. The specification promised an answer, and the user got a message meant for someone else.

We hold your programs to a standard all quarter: every input a user can produce gets a response the specification describes, and a stack trace is never that response.

How do you come to know

This lecture began by asking what a program does. It arrived at two kinds of claim about every run of the program: a specification, which speaks from outside the program, and assertions, which speak from inside it. What this lecture has not yet said is how anyone comes to know that such a claim holds. That question, more than any definition, is what the rest of the course is about.

There are three ways to come to know, and we have already used all three.

• Testing: choosing runs on purpose (the ordinary runs, and the runs a user manual might forget) and checking each run against the claim. That is what we were doing when we ran 3 4 5, then typed a word, then nothing, then a number at the top of the int range. A run tells you what happened on that run, and nothing about any other run. Testing is how our first sentence was found wanting.
• Tools: letting another program check yours. The compiler did that when we ran javac: its type checker looked at every run without running any, but it checked only that each value has the right type.
• Reasoning: arguing from the code itself that the claim is true on every run at once. That is what we did at (1) and (2), when we said what the loops must have left behind.

Testing can refute a claim about every run and never confirm it. A tool can confirm a claim about every run, but only the one claim it was built to check. Reasoning can confirm a claim about every run, if the argument is right. An argument can be wrong, and a test that refutes the claim refutes the argument too. The lectures ahead take up testing first and reasoning after.

The first reading asks what it means to know that code works, and it offers three answers that do not agree with each other. This lecture gives you the vocabulary to read those three answers with.

Finally, let's return to the course motto: your job is to deliver code that you know works. This lecture has given each of its words an exact meaning.

• Your job means that the compiler will not do this for you. Neither will a teammate or a tool that wrote some of the code. The claim is yours, and so is coming to know that it holds.
• Deliver means the code goes to people who will type things you did not imagine. Those people are owed what the specification promises, not what you hoped the code would do.
• Know means having reasons, from tests, from tools, and from arguments, that the code works.
• Works means the code meets its specification, on every run and not only on the runs you tried.

Exercises

Each answer is folded under its question. Write your own answer first.

1. Go back to the sentence you wrote before reading past the program.

• Test your sentence against the runs in this lecture. Where is it wrong, and where is it silent?

• Predict what Sum prints for each of these two inputs, then run Sum to check.

  3 4
  5
  

  2.5 3
  

• Repair your sentence until it is true of every run you have seen. Only then compare your sentence with ours.

Enter some numbers.
3 4
5
Got [3, 4, 5]
The sum of your list was: 12

Enter some numbers.
2.5 3
Got []
The sum of your list was: 0

2. Here are four statements about Sum. Which of them are specifications, and which of those does Sum meet? For each specification that Sum does not meet, give a run that shows it does not.

• A. It prints the numbers it read, and then their sum.
• B. It prints the numbers it read, and then a whole number.
• C. It prints every whole number the user typed.
• D. It keeps the numbers in a list and adds them up in a loop.

3. Compare statement B from exercise 2 with the specification we wrote for Sum. Which of the two tells a user more, and does that make the other one useless? Then ask the same two questions of these two assertions, both of which hold at point (2):

{{ sum is the sum of the elements of list, wrapped into
   the int range if the true sum falls outside it }}

{{ if list is empty, then sum is 0 }}

4. Change line 15 of Sum from int sum = 0; to int sum = 1;.

• Which of the three assertions in Sum with assertions still hold, and which do not?
• Does the changed program still meet our specification? If not, give a run that shows it does not.

5. The rule in Reasoning between assertions takes a claim before some code to a claim after it. Use the rule on the two lines between point (1) and the summing loop:

        // (1)
        {{ list holds, in order, every number the user typed before
           the input ended or something that was not an int appeared }}
        System.out.println("Got " + list);
        {{ ? }}
        int sum = 0;
        {{ ? }}

• What holds after line 14, and why?
• What holds after line 15, and why?

6. Here is Sum changed to read long values. The highlighted lines are the four that changed. A long has 64 bits where an int has 32, so a long holds the whole numbers from \(-2^{63}\) to \(2^{63} - 1\), that is, from -9223372036854775808 to 9223372036854775807.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

import java.util.ArrayList;
import java.util.List;
import java.util.Scanner;

public class Sum {
    public static void main(String[] args) {
        Scanner in = new Scanner(System.in);
        System.out.println("Enter some numbers.");
        List<Long> list = new ArrayList<>();
        while (in.hasNextLong()) {
            list.add(in.nextLong());
        }
        // (1)
        System.out.println("Got " + list);
        long sum = 0;
        for (int i = 0; i < list.size(); i++) {
            sum += list.get(i);
        }
        // (2)
        System.out.println("The sum of your list was: " + sum);
    }
}

• Predict which runs in this lecture now go differently, then run the program to check.
• Write the specification of this changed program.

7. Here are two more statements, each offered as a specification for the hasNext version of Sum from The edge of the program. For each statement, what does it promise a user who types a word, and does the hasNext version meet it? Which of the two statements, if either, does this course accept?

• E. The user must type only whole numbers between \(-2^{31}\) and \(2^{31} - 1\). The program asks for them, reads them until the input ends, then prints them and their sum, wrapped around into that range if need be.
• F. The program asks for numbers and reads whole numbers between \(-2^{31}\) and \(2^{31} - 1\) until the input ends, then prints them and their sum, wrapped around into that range if need be. If anything else is typed before the input ends, it stops there with a stack trace.

Further reading

• Specifications. Barbara Liskov and John Guttag, Program Development in Java (2001). Chapter 3 specifies procedures much as this course does, and Chapter 9 judges specifications by how restrictive, how general, and how clear they are.
• Assertions. Robert Floyd, "Assigning Meanings to Programs" (1967), and C. A. R. Hoare, "An Axiomatic Basis for Computer Programming" (Communications of the ACM, 1969), the classic papers on attaching claims to points in a program and reasoning from each to the next. The P, code, Q shape above is Hoare's.
• Testing. Brian Kernighan and Rob Pike, The Practice of Programming (1999). Chapter 6 recalls Dijkstra's point that testing can show the presence of bugs but never their absence, which this lecture made about runs. It then turns to the boundaries of a program's input, such as an empty input or a single item, where bugs cluster.