CSEP503: Principles of Software Engineering
David Notkin, University of Washington,
Computer Science & Engineering
Last updated: 11/13/2007 08:49 PM
Final Examination By University rule, an instructor is allowed
to dispense with a final examination at the scheduled time (6:30-8:20PM,
December 13, 2007) with unanimous consent of the class. If you prefer to
have a final examination for the entire class, you must let me know by
the 6:00PM before the second lecture (October 9, 2007).
Office Hours I will generally try to be in my office (CSE 542,
206-685-3798) from 5:00-6:00PM on Tuesdays before lectures. I am happy to
find other times to talk by phone or in person as needed. My email is
notkin at cs.washington.edu
Teaching Assistant Jonathan Beall (jibb at cs.washington.edu) is
our TA this quarter.
Assigned Work Barring the need for a final, there four major
assignments in the course, each worth 25% of the total grade. Some of the
assignments must be done in groups of more than one; in these cases, the very
strong expectation is that a single grade will be shared equally among all
participants in the group.
- Essay on software engineering (individually or in pairs). Due 6:00PM
on Tuesday October 23, 2007.
- Based on the three sets of source materials listed next, write a 5-10
page articulate, well-reasoned essay on what advances in key attributes
(productivity, quality, etc.) are within reach by 2020. I
recommend that you explicitly focus on specific attributes and perhaps
on specific software domains (e.g., databases, web-based systems,
...). The essay must not be "pie in the sky" -- what you
hope will happen -- nor primarily personal opinion (although this may
well drive much of your thinking). It should be scholarly (citing
appropriate sources). You will have some insights into my
own views through lectures and the document below that I co-authored,
but I am not interested in essays that largely mimic my point of view
- Assigned readings -- in general, you needn't read each and every
page in these documents, but the core arguments and reasoning are
- Fredrick P. Brooks, Jr. "No
Silver Bullet: Essence and Accidents of Software Engineering",
Computer, April 1987. This is also found in Brooks'
Mythical Man-Month (25th Anniversary Edition).
- The 1968 and 1969 NATO
Software Engineering Conferences (Naur and Randell, Randell
and Buxton, editors, respectively). These are long
documents, but quite easy (and fascinating) to read.
- The SWEBOK Guide, Chapter
1, "Introduction to the Guide" (found in several
formats at the site).
- David Notkin, Michael Gorlick, Mary Shaw. "An
Assessment of Software Engineering Body of Knowledge Efforts",
A Report to the ACM Council (May 2000).
- Your personal knowledge, experience, observations, and any
reading/research your may have done in the software engineering
- Three or more other pertinent documents that you find and read
- We will make these essays available to the entire class and request
comments upon them by your classmates. (Stay tuned for more
details about formats, how to submit, etc.)
- Two state-of-the-research reports (both of them in groups of 1-3 people). One of the reports is due at
6:00PM on Tuesday November 6, 2007, and another is due at 6:00PM on Tuesday
December 4, 2007. These are not "original" research reports,
but are rather secondary research reports. That is, they require the
definition of a topic, and approval by me, identification of pertinent
papers and materials (perhaps with help from me), perhaps some hands-on
experience for some kinds of topics, and finally a scholarly report on the
topic and your analysis of it, complete with citations, open questions,
- The list of potential topics is enormous, but perhaps a few examples
might help: state-of-the-research in ... symbolic execution, random
testing, code searching, reliability models, software inspections,
program slicing, automatic theorem proving, clone analysis, ...
- Many of you have asked for some samples of these research reports.
I finally decided that, on the whole, the following papers from an ICSE
2000 "future of software engineering" track are in roughly the
style I'd like -- http://www.informatik.uni-trier.de/~ley/db/conf/icse/future2000.html is
the whole list, and they are available from the ACM Digital Library.
Some are much better than others, and the part that looks to the future
could/should be reduced with an added focus on current status. But
still, the feel is reasonable. As for length, roughly the same as
these (10 conference pages) is fine, but I'm flexible within reason.
- As with the essays, we will make these reports available to the entire
class and request comments upon them by your classmates.
- Tool-based assignment (individual). Due at 6:00PM on Tuesday
November 20, 2007. Details forthcoming (likely to use Daniel Jackson's
Adapted (almost directly) from Kathi Fisler's course at WPI
Consider a software system for managing circulation of
books at a library. The system tracks books and patrons. Books may be
checked in or checked out. If a book is checked out, it may also be on
request by other patrons. When the book is returned, the software
automatically checks it out to the next patron waiting for the book. For
each patron, the system keeps track of which books they have checked out and
which they have on request.
The library system needs to satisfy the following
- If a patron requests a book, the book is eventually
checked out to that patron.
- The data on books and patrons is always consistent:
that is, a book shows as checked out to a patron if and only if that
patron's record indicates that the book is checked out to them. The same
relationship must hold for request information.
(50%) Model the structural aspects of this library
system in Alloy (the books, patrons, requests, etc). If you leave out
part of the system because you feel you can't model it well in Alloy,
provide a comment justifying your decision.
(25%) Model the check-out, check-in, and
make-request operations, and verify that checking-out followed by
checking-in the same book from the same patron restores the original
(25%) Try to verify that your Alloy model
satisfies the consistency property (the second property above). Describe any additional facts or invariants that you added to
the model to verify the property. If you can't get the property to
verify, provide a concrete counterexample that Alloy generates and
explain why you can't augment the model to eliminate the counterexample.
Contributions such as participation in class discussions (during lecture or
electronically), commenting on your classmates essays and such, etc. will be
considered as a minor -- but highly satisfying and appreciated -- amount of
Lecture Schedule The following schedule is subject to change.
slide downloads, AV archives and links to the live streamed lectures.
- October 2: Introduction and Overview
- October 9: Requirements and Specifications I
- October 16: Requirements and Specifications II
- October 23: Design I
- October 30: Design II
- November 6 (likely taught from the Microsoft site): Software Evolution
- November 13: Analysis and Tools
- November 20: Quality assurance/testing
- November 27: Mining software repositories
- December 4: TBD
Responses to first one-minute papers (10/3/2007,
Just to be clear, the following list does not respond to each and every
point and/or suggestion made in the papers; rather, I've tried to select points
that might have broader interest. But if your comment or idea isn't
listed, it doesn't mean I've ignored it! And, as of now, I have no
requests for a final, which several of you thought was indeed the most important
point on Tuesday evening!
- I will soon list some resources on this web page that may be useful for
defining and researching your research topics.
- How can we tap the experience of people in the class effectively?
Any great ideas? Send them to me or post them to the mailing list.
- How can I start on the 2020 essay? The first thing I'd suggest is to
do the readings, which may help you put some of your thoughts in order.
- Can we have a class wiki for people to start discussions for possible
group efforts? Sure, we'll get back to you on this.
- Can you suggest some topics? Sure, but how about spending a bit of
time on your own, or using the wiki a bit (when we set it up)?
- What do people really mean when they talk about "software
engineering"? I'm not sure it will really help, but here are some
relatively common definitions:
- The practical application of scientific knowledge to the design and
construction of computer programs and the associated documentation
required to develop, operate, and maintain them. [Boehm]
- The systematic approach to the development, operation, maintenance,
and retirement of software. [IEEE]
- The establishment and use of sound engineering principles (methods) in
order to obtain economically software that is reliable and works on real
- Multi-person construction of multi-version software. [Parnas]
- Software from womb to tomb. [unknown]
- Can't we have something other than a paper to write for homework?
And, related: "I would actually much rather write code than write an
essay or research paper. Often this is because code is easier to tell when
it has fully satisfied the requirements of the assignment and thus when it
is done. I hope that the writing assignments will be described in a way so
that I will know when I have done them correctly." First, you do
have one non-paper assignment -- the tool assignment will be more like
assignments you've likely had in the past. Second, I understand these
papers are likely pushing your comfort zone -- but at the same time, I
believe that this is where this class can provide you with the most
value. Please observe that the concern about a paper being done
correctly is a terrific example of an ill-defined, but still important,
requirement. Again, it's useful to push your comfort zone.
- "It seemed like a large part of the lecture was spent talking about a
'software crisis' fairly defensively. However, I'm not aware of any software
crisis, although I also don't read the news. Is the media and/or the
academic world proclaiming a software crisis?" I certainly made
that point in lecture, repeatedly -- I wasn't trying to be defensive,
although I apparently failed in that. I was rather trying to give a
different perspective that might help us improve more quickly.
And yes, to a large degree, many parts of industry, the media, and academia
believe that there is a software crisis.
- I would actually much rather write code than write an essay or research
paper. Often this is because code is easier to tell when it has fully
satisfied the requirements of the assignment and thus when it is done. I
hope that the writing assignments will be described in a way so that I will
know when I have done them correctly.
- Is there an expected format for the essay and research reports?
Almost anything reasonable is reasonable! I can provide a few
pointers, if you need.
- Even in the absence of "no best solution," can't you help us out
with ways to do the best with what we have and with comparisons among
approaches? I surely hope to do this to some degree.
- If many of the problems in software engineering practice are not indeed
technical problems per se, then why are you focusing on technical problems
primarily? There are at least two reasons. First, my expertise
is largely limited outside of the technical arena. Second, you might
be surprised at how useful technical approaches can be to helping crystallize
non-technical issues. I will try to, at least now and then, reinforce
- "You mentioned something about intellectual tools being better than
technical tools. Could you give me an example of what you meant? I
couldn’t find any mention of it in the slides ..." The example
I mentioned in class, I think, was design patterns. These are a way to
solve common problems that arise in object-oriented programming, as you may
know. To the best of my knowledge, few design patterns are encoded
into programming language constructs or supported by widely used tools or
environments; however, they change the lexicon in which design (at some
level) is done, and that's an incredible "intellectual"
tool. We'll see other examples.
- "Is the perfect the enemy of the good?" a key issue in
software? This isn't a simple question to answer, but I'd say,
absolutely. For some classes of software, there is no question that
"good enough" is a reasonable goal as opposed to
"perfect"; treating all classes of software as if they all shared
equal requirements and expectations can be very costly.
- How about more on software tools, and more spread out? I think this
will indeed take place to some degree -- but remind me again later on if
you're not satisfied.
- One unanswered question is "[w]hether there is any hope of
standardizing the way software is built, and whether doing so would actually
qualitatively impact the problem areas mentioned above?" I do not
believe that there is any hope of standardizing the construction of all
software; indeed, one of the exciting parts of the field is that we are far
less constrained than other engineering fields. At the same time, I
agree entirely with Michael Jackson that specific domains can (and indeed
have) become standardized -- and more and more of them will over time.
- Even though you are skeptical of quantitative data, will you show us
some? I was imprecise -- I am not against quantitative data.
However, I feel that they are often used as a proxy for a qualitative
measure that we really want. But I will indeed talk some about
empirical software engineering and what we seem to know and not know, to
But which of the papers that you find are any good? Well, that's not an
easy question, but here is some evaluation of various publications in software
The two topic journals are widely considered to be:
- IEEE Transactions on Software Engineering
- ACM Transactions on Software Engineering and Methodology [I am
editor-in-chief of TOSEM]
The top research conferences include:
- ACM SIGSOFT Symposium on the Foundations of Software Engineering (FSE),
which has been joint with the European Software Engineering Conference (ESEC)
in alternate years for about a decade.
- The International Conference on Software Engineering (ICSE)
- ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA)
There are a couple of magazines that often have useful material, although rarely
as deep technically, including:
- IEEE Software
- IEEE Computer
- Communications of the ACM (CACM) (until sometime in the early 1980’s or
so, this was one of the very leading journals in computing research and it
is soon to be revamped)
Some other forums that have material of more varied quality, but definitely
with some high quality papers, including:
- Empirical Software Engineering journal
- Automated Software Engineering (both a conference and a journal)
- International Conference on Software Maintenance
A number of publications in related areas sometimes have high-quality
publications touching on software engineering, including:
- ACM Transactions on Programming Languages and Systems (TOPLAS)
- OOPSLA and ECOOP (object-oriented programming systems conferences)
- ACM SIGPLAN PLDI (programming language design and implementation)
This is not a complete list, and it surely has both false positives and false
negatives (to varying degrees). But it’s something. Feel
free to run other stuff by me, if you want an assessment.
Some email and my responses
- A student (working at a major company on a large programming environment
comprised of 10's of millions of lines of code) writes:
While listening to the material relating to how define a requirements, as
well as the Z (“Zed!” J) approach to capturing these requirements, I had
a difficult time mapping this into reality. Why?
If I assume that Z can help clearly articulate requirements (and I am still
having a hard time swallowing this; I do hope the tool exercise helps me
to), it feels as though the amount of effort required to achieve this for
the amount of product we are expected to deliver is unreasonable. That
is, if I had to use Z to define my requirements for my current product as an
example, it feels as though it would take 10x the amount of time or more to
achieve this as opposed to delivering a “good enough” specification.
Now perhaps that is the problem and why we fail more often than we want to.
The mythical man month suggests that the engineering portion of software
development should be 1/6 of the time spent, while we are usually at about
1/3, and we usually spend only about 1/5 of the time for planning.
Perhaps this the paper I should be writing for October 23rd. My
inference is that to date you are suggesting that in our current industry we
are ignoring the complexities of software, which is misleading us. And once
we recognize this complexity, computer science will move further away from
an art form and closer to a true science.
Unfortunately, it doesn't surprise me that I was inarticulate in making my
points, and this question helps me "understand the
misunderstanding." Thank you. First, what I don't believe,
at least in general: (a) that industry inherently spends the
"wrong" amount of time on various phases of the software
lifecycle; and (b) that I intended to suggest that "our current
industry [ignores] the complexities of software, which is misleading
us." Well, maybe I believe (a) to some degree -- it's not
entirely inconsistent with a set of the problems you collectively listed in
my pre-course questionnaire. I absolutely reject (b), and the
suggested consequence. (I cannot speak for other software engineering
researchers, only for myself.)
So, what do I believe is the role of something like Z in reality? I
think it is analogous -- at least in most domains -- to the use of proofs
of program correctness. That is, in general, one may not need or
benefit from a Z specification -- the clarity is there (because of prior
experience, because of some other effort, or whatever) and thus the risk is
small. But there may be new features, existing and continually
confounding aspects of a system, or such, where the risk is instead (at
least the initial risk) in understanding what should be done with some
care. The examples I mentioned in lecture about using formalisms to
isolate and later replace key, complex pieces of code in an existing system
are indeed the kind of thing I have in mind.
Let me stress this one more time: I do not believe that the problems of
software engineering can be solved by "always writing formal
specifications" anymore than I believe that they can be solved by
"always using design patterns" or "always using agile
processes" or "always using statically-typed programming
languages" or such. Having more arrows in your quiver than you
presently do, however, might change your reality at times. To be
blunt, it's not good to assume that the "only" things you have at
your disposal is the programming language, the compiler, grep/find, and a
few other programming-oriented tools -- that's limiting, as well, in some
- Another student writes:
There was a question in last night’s
class that I think is worth revisiting. One of the students asked
whether writing unit tests to specify code functionality was equivalent to
using Z. I can’t remember the response exactly, but it seemed
that there might have been a misunderstanding of what was meant by the
I am pretty sure the student was referring to an Agile practice called
“Test Driven Development” or TDD. This is a design discipline
where the basic idea of this is to first specify functionality in the form
of a failing test or tests, then write the simplest code to exactly satisfy
the tests. This process is performed incrementally as the code is
created. This practice is similar to Z in which a person flushes
out pieces of the specification first and then writes the code according to
the Z spec later. As far as I can tell, writing program
specifications in a programming language is equivalent in every meaningful
way to writing them in a specification language like Z. The unit
tests, however, have three distinct advantages:
First, the specification is
created in a language already understood by the developers.
Second, the pre-written
unit tests provide real-time feedback as to how well the actual code meets
specification and when the programmer is “done”, which limits
superfluous (extra-specified) code.
Third, the tests also
stand as a barrier to preventing the code from drifting out of spec as
changes are made.
The application of
TDD has been shown in some studies to reduce bug counts by an _order of
magnitude_ over standard techniques where tests are written after the fact.
(I am referencing studies internal to [major company]). Knowing this,
I would like to hear more discussion about this in the class, because I
think it is highly relevant to what we talked about last night.
Thanks again, for allowing me to clarify what I meant to say. This
question came up, if I recall, in the context of the discussion of the
"small scope hypothesis." Andoni et al. (sdg.csail.mit.edu/pubs/2002/SSH.ps)
say: "The 'small scope hypothesis' argues that a high proportion
of bugs can be found by testing the program for all test inputs within some
small scope. ... If the hypothesis holds, it follows that it is more
effective to do systematic testing within a small scope than to generate
fewer test inputs of a larger scope." In essence, they argue that
exhaustive testing in a small scope dominates other approaches to testing
over a larger scope.
I certainly am aware of test-driven development (although I hadn't seen the
order of magnitude numbers), and I was trying to respond in that
context. Indeed, this specific paper compares the "small scope
hypothesis" to two specific forms of testing: code coverage and
mutation testing. It does not compare to test-driven development, so
in principle one cannot extrapolate to that approach. In some sense,
though, I don't think that comparison is necessarily the right thing to
think about. Instead, I might think that a combination of these two
powerful techniques (at least in some domains) might be attractive: that is,
write tests a priori and exhaustively test within bounded
scopes. Perhaps they each catch different kinds of errors; perhaps
not. I find it easy to believe that effective testing -- however
defined -- is not likely to be represented by a single approach.
Maybe no clearer, but I hope so.