Crater Lake

Advanced Topics in Software Systems: Testing and debugging

CSE P 504, Spring 2024, University of Washington

Logistics

Class meetings

Mon 6:30pm-9:20pm, in room CSE2 G10

Please bring a laptop; the in-class activities will require use of a computer.

Instructor

Michael Ernst

Teaching Assistant

James Yoo

Communication

Syllabus

Course description

This course teaches on the principles of dynamic and static program analysis, emphasizing state-of-the-art approaches to effective software testing and debugging. Lectures and discussions contribute to theoretical understanding. In-class activities and written homeworks provide a hands-on experience.

The goal of the class is for you to come away with a new perspective on testing, debugging, and verification. You already know how to perform these tasks, and you probably do so as part of your job. This class will introduce you to new ideas, methodologies, and tools from the state of the art, including from research. Seeing new ways of working and different viewpoints will enable you to take advantage of these ideas, or to adapt them to your own work environment, or to understand that their limitations make them impractical for you to use.

Many of the tools you will use are in active production use at forward-looking companies. Others have not yet hit the mainstream, and we will discuss why.

Course format

Student interaction during lecture is expected. There are homeworks, but no final project nor final exam.

Before each class meeting, do the assigned reading and do the setup for that week's assignment. This will make the class more productive.

Each class meeting consists of two parts. The first part is lecture and discussion. The second part is an in-class activity in which you will use a state-of-the-art program analysis tool. You can view the second part of the class meeting as a chance to start on the week's assignment, while the course staff are available to answer questions.

Some of the tools you will use are widely used (though not yet standard at every company), and others are rarely used. All of them address an important problem for software developers.

The class meets once a week in person. Fully remote attendance is not permitted. Missing more than two sessions without the instructor's permission will lead to a reduction in grade. If you have an emergency or unavoidable conflict, we understand: slides and other materials are linked from this webpage, and lecture recordings are available from Canvas, under "Panopto Recordings".

Grading

Grades will be based on assignments and participation:

Late policy

Assignments are generally due on Sunday nights, because previous students have wanted to work over the weekend. A downside of this due date is that you generally won't get feedback on the previous week's assignment before you start the next one.

Assignments must be submitted on Canvas by the due date and time. Unless otherwise noted, all times are given in PDT (Pacific Time). The submission site remains open for 48 hours after the deadline. Assignments submitted within 24 hours after the deadline will incur a 10% penalty; assignments submitted within 24 to 48 hours will incur a 20% penalty. Assignments will not be accepted more than 48 hours after the deadline, except in extraordinary circumstances (in which case, discuss your situation with the instructor).

You can find the general course policies here. Note that CSE P 504 explicitly permits groupwork by pairs of students.

Requirements and background

Students should be proficient programmers, but do not need to know any particular programming language. Some activites and homeworks provide code for you to manipulate or extend. That code is often in Java. (Java was the first widely-used memory-managed language, which made it an attractive target for research; and since so much previous used Java, subsequent research used Java to build upon it and to enable direct comparison.) For other assignments, you may use any programming language you like.

Course materials

Please do the reading before the corresponding lecture.

Also, perform the setup for each in-class activity before the corresponding class meeting. The setup is typically installing any required software and running it once to make sure it is operational.

Please bring a laptop; the in-class exercises will require use of a computer.

Date    Topic Materials Assignments
03/25 Introduction Slides
Reading 1: all of the CSE P 504 webpage; Reading 2 		Testing and debugging challenges (due 03/31); Ducks (due 04/01)
04/01 Best practices and version control Slides 1; Slides 2
Reading 1 (chapter 1 and appendix A); Reading 2; Reading 3; Reading 4; Reading 5. Think about what you agree and disagree with in all this advice, so that we can discuss in class. 		Git bisect (due 04/07)
04/08 Coverage-based testing Slides
Reading 1 (chapter 3, all but the section on DART); Reading 2; Reading 3 (talk)		 Code coverage (due 04/14)
04/15 Mutation-based testing Slides
Reading 1 (Sections 1-2); Reading 2 (Sections 1-4); Reading 3		 Mutation (due 04/21)
04/22 Delta debugging Slides (for demo code, see demo branch of delta-debugging repository)
Reading 1; Reading 2; Reading 3; Reading 4 		Delta debugging (due 04/28)
04/29 Invariants and partial oracles Slides
Reading 1; Reading 2 (read sections 1-4, 6-7, and 11; skim sections 5 and 8-10). 		Invariants and partial oracles (due 05/05)
05/06 Statistical fault localization Slides
Reading 1 and 2		 Fault localization (due 05/12)
05/13 Static analysis Slides
Reading 1 (sections 2.0-2.14); Reading 2 (sections 1.0-2.2; skim remainder of section 2).		 Abstract interpretation (due 05/26 and 06/02)
05/20 Abstract interpretation Slides
Reading 1 (rest of chapter 2); Reading 2 (chapters 1, 2, and 35).
05/27 No class due to Memorial Day
06/03 Automated theorem proving Slides Theorem proving (optionally due 06/09)