CSE 331 Reading 1

Due Friday, October 2 at 11:59pm (late policy).

Our motto for this course is: Your job is to deliver code that you know works. In this reading, we will look at what "know" means. Does knowing require a proof? Does testing count? When you say you know your code works, what do you actually know, and how did you come to know it?

The three articles below each give a different perspective on "knowing". Willison says you know what you have tested and watched run. Fishman reports on the team that wrote the space shuttle's software. Their code has almost no errors, and their process is very strict. Hoare asks how software got so reliable when almost none of it was ever proven correct. His answer is careful engineering, and testing is a large part of it.

Your task: Choose one article, read it, and write a reflection. See the readings page for information on what you need to submit and how reflections are graded. When you are done, submit on Gradescope as described in the "Submit" section below.

Each title below is a link to the original. If for any reason a link does not work, you can also use the "cached" links, which point to copies stored on UW servers: a PDF of each article and, for some, an HTML version that is easier to read with a screen reader or on a phone. (Cached links require UW NetID login).

Articles

Willison 2025, Your job is to deliver code you have proven to work

The title is nearly our motto, but Willison says "proven" where we say "know". By proof he means evidence you can show someone else. First you run the code yourself and watch it do the right thing. Then you write an automated test that fails if you undo your change. He says neither step is optional. He wrote this about programmers who send large, untested, AI-written changes to code review and expect the reviewer to catch the problems. He says the same two steps apply when a coding agent writes the code, and the human is still the one accountable. About 5 minutes to read (cached: PDF, HTML).

Fishman 1996, They Write the Right Stuff

Fishman reports on the 260 people who wrote the software that flew the space shuttle. In 1996, their three most recent versions, each 420,000 lines long, had one error each. He asks how they do it. The answer is in how the group plans, checks, and records its work, and what it does when it finds a mistake. The group's own manager says the process stifles creativity. As you read, ask what these programmers know about their code that most programmers do not, and what that costs. About 20 minutes to read (cached: PDF, HTML).

Hoare 1996, How Did Software Get So Reliable Without Proof?

Some computer scientists hoped programmers would prove their programs correct, with a mathematical proof like the ones in geometry. Hoare helped invent the logic for doing that, and we will use it later in this course. But almost no software is ever proven, and most of it works anyway (at least most of the time...). In this paper Hoare asks why. He looks at how working engineers manage, test, and debug large programs, and he has a surprising view of what a test really checks. This is an academic paper, and its last section is addressed to researchers. About an hour to read (cached: PDF, HTML).

Extras

Here are some additional papers that are relevant to the themes explored by the papers above, in case you are curious and want to know more. These are completely optional, but you are welcome to mention them in your reflection if you connect it explicitly to one of the papers above.

Cook 1998, How Complex Systems Fail

Cook is a physician who writes about safety in hospitals and other hazardous systems. This is a list of eighteen short claims about how such systems fail, and why they usually do not. He never mentions software, but his claims bear on all three articles. He says such systems are always partly broken, and keep working because people catch problems as they come up. He also says that blaming an accident on one "root cause" is a mistake. About 10 minutes to read (cached PDF).

Leveson and Turner 1993, An Investigation of the Therac-25 Accidents

The Therac-25 was a radiation therapy machine controlled by software. In the 1980s it gave six patients massive overdoses.¹ Leveson, then a professor here at UW, and Turner investigate what went wrong. They find real bugs, and they argue that blaming the bugs misses the point. As you read, ask what the people who built and ran this machine thought they knew, and why. About two hours to read (cached PDF). The Wikipedia article covers the same events in about 20 minutes (cached PDF).

Thompson 1984, Reflections on Trusting Trust

Thompson is one of the creators of Unix. In this short lecture he shows how to hide a back door in a compiler so that it persists even after it is removed from the compiler's source code. His point is that reading a program's source cannot tell you everything the program does. You also have to trust the tools that built it, and the people who wrote those tools. That pushes hard on the word "know" in our motto. The examples are in C, and the last few paragraphs are about hackers in the 1984 news. About 10 minutes to read, longer if you work through the code (cached: PDF, HTML).

Submit

Submit your reflection on Gradescope by Friday, October 2 at 11:59pm. The Gradescope assignment will ask you which article you read and whether you used AI for editing, and it has spaces for you to write your reflection. See the readings page for more information.