CSEP503: Principles of Software Engineering

David Notkin, University of Washington, Computer Science & Engineering
Autumn 2007

Last updated: 11/13/2007 08:49 PM

Final Examination  By University rule, an instructor is allowed to dispense with a final examination at the scheduled time (6:30-8:20PM, December 13, 2007) with unanimous consent of the class.  If you prefer to have a final examination for the entire class, you must let me know by the 6:00PM before the second lecture (October 9, 2007).

Office Hours  I will generally try to be in my office (CSE 542, 206-685-3798) from 5:00-6:00PM on Tuesdays before lectures.  I am happy to find other times to talk by phone or in person as needed.  My email is notkin at cs.washington.edu

Teaching Assistant  Jonathan Beall (jibb at cs.washington.edu) is our TA this quarter.

Assigned Work  Barring the need for a final, there four major assignments in the course, each worth 25% of the total grade.  Some of the assignments must be done in groups of more than one; in these cases, the very strong expectation is that a single grade will be shared equally among all participants in the group.

  1. Essay on software engineering (individually or in pairs).  Due 6:00PM on Tuesday October 23, 2007.
  2. Two state-of-the-research reports (both of them in groups of 1-3 people).  One of the reports is due at 6:00PM on Tuesday November 6, 2007, and another is due at 6:00PM on Tuesday December 4, 2007.  These are not "original" research reports, but are rather secondary research reports.  That is, they require the definition of a topic, and approval by me, identification of pertinent papers and materials (perhaps with help from me), perhaps some hands-on experience for some kinds of topics, and finally a scholarly report on the topic and your analysis of it, complete with citations, open questions, etc.  
  3. Tool-based assignment (individual).  Due at 6:00PM on Tuesday November 20, 2007.  Details forthcoming (likely to use Daniel Jackson's Alloy tool).
    Adapted (almost directly) from Kathi Fisler's course at WPI

    Consider a software system for managing circulation of books at a library. The system tracks books and patrons. Books may be checked in or checked out. If a book is checked out, it may also be on request by other patrons. When the book is returned, the software automatically checks it out to the next patron waiting for the book. For each patron, the system keeps track of which books they have checked out and which they have on request.

    The library system needs to satisfy the following properties:

    1. (50%) Model the structural aspects of this library system in Alloy (the books, patrons, requests, etc). If you leave out part of the system because you feel you can't model it well in Alloy, provide a comment justifying your decision.

    2. (25%) Model the check-out, check-in, and make-request operations, and verify that checking-out followed by checking-in the same book from the same patron restores the original library system.

    3. (25%) Try to verify that your Alloy model satisfies the consistency property (the second property above). Describe any additional facts or invariants that you added to the model to verify the property. If you can't get the property to verify, provide a concrete counterexample that Alloy generates and explain why you can't augment the model to eliminate the counterexample.

Contributions such as participation in class discussions (during lecture or electronically), commenting on your classmates essays and such, etc. will be considered as a minor -- but highly satisfying and appreciated -- amount of extra credit.

Lecture Schedule  The following schedule is subject to change.  Lecture slide downloads, AV archives and links to the live streamed lectures. 

  1. October 2: Introduction and Overview
  2. October 9: Requirements and Specifications I
  3. October 16: Requirements and Specifications II
  4. October 23: Design I
  5. October 30: Design II
  6. November 6 (likely taught from the Microsoft site): Software Evolution
  7. November 13:  Analysis and Tools
  8. November 20:  Quality assurance/testing
  9. November 27: Mining software repositories
  10. December 4:  TBD

Responses to first one-minute papers (10/3/2007, 10/4/2007)

Just to be clear, the following list does not respond to each and every point and/or suggestion made in the papers; rather, I've tried to select points that might have broader interest.  But if your comment or idea isn't listed, it doesn't mean I've ignored it!  And, as of now, I have no requests for a final, which several of you thought was indeed the most important point on Tuesday evening!

  1. I will soon list some resources on this web page that may be useful for defining and researching your research topics.
  2. How can we tap the experience of people in the class effectively?  Any great ideas?  Send them to me or post them to the mailing list.
  3. How can I start on the 2020 essay?  The first thing I'd suggest is to do the readings, which may help you put some of your thoughts in order.
  4. Can we have a class wiki for people to start discussions for possible group efforts?  Sure, we'll get back to you on this.
  5. Can you suggest some topics?  Sure, but how about spending a bit of time on your own, or using the wiki a bit (when we set it up)?
  6. What do people really mean when they talk about "software engineering"?  I'm not sure it will really help, but here are some relatively common definitions:
  7. Can't we have something other than a paper to write for homework?  And, related: "I would actually much rather write code than write an essay or research paper. Often this is because code is easier to tell when it has fully satisfied the requirements of the assignment and thus when it is done. I hope that the writing assignments will be described in a way so that I will know when I have done them correctly."  First, you do have one non-paper assignment -- the tool assignment will be more like assignments you've likely had in the past.  Second, I understand these papers are likely pushing your comfort zone -- but at the same time, I believe that this is where this class can provide you with the most value.  Please observe that the concern about a paper being done correctly is a terrific example of an ill-defined, but still important, requirement.  Again, it's useful to push your comfort zone.
  8. "It seemed like a large part of the lecture was spent talking about a 'software crisis' fairly defensively. However, I'm not aware of any software crisis, although I also don't read the news. Is the media and/or the academic world proclaiming a software crisis?"  I certainly made that point in lecture, repeatedly -- I wasn't trying to be defensive, although I apparently failed in that.  I was rather trying to give a different perspective that might help us improve more quickly.   And yes, to a large degree, many parts of industry, the media, and academia believe that there is a software crisis.
  9. I would actually much rather write code than write an essay or research paper. Often this is because code is easier to tell when it has fully satisfied the requirements of the assignment and thus when it is done. I hope that the writing assignments will be described in a way so that I will know when I have done them correctly.
  10. Is there an expected format for the essay and research reports?  Almost anything reasonable is reasonable!  I can provide a few pointers, if you need.
  11. Even in the absence of "no best solution," can't you help us out with ways to do the best with what we have and with comparisons among approaches?  I surely hope to do this to some degree.  
  12. If many of the problems in software engineering practice are not indeed technical problems per se, then why are you focusing on technical problems primarily?  There are at least two reasons.  First, my expertise is largely limited outside of the technical arena.  Second, you might be surprised at how useful technical approaches can be to helping crystallize non-technical issues.  I will try to, at least now and then, reinforce this point.
  13. "You mentioned something about intellectual tools being better than technical tools. Could you give me an example of what you meant? I couldn’t find any mention of it in the slides ..."  The example I mentioned in class, I think, was design patterns.  These are a way to solve common problems that arise in object-oriented programming, as you may know.  To the best of my knowledge, few design patterns are encoded into programming language constructs or supported by widely used tools or environments; however, they change the lexicon in which design (at some level) is done, and that's an incredible "intellectual" tool.  We'll see other examples.
  14. "Is the perfect the enemy of the good?" a key issue in software?  This isn't a simple question to answer, but I'd say, absolutely.  For some classes of software, there is no question that "good enough" is a reasonable goal as opposed to "perfect"; treating all classes of software as if they all shared equal requirements and expectations can be very costly.
  15. How about more on software tools, and more spread out?  I think this will indeed take place to some degree -- but remind me again later on if you're not satisfied.
  16. One unanswered question is "[w]hether there is any hope of standardizing the way software is built, and whether doing so would actually qualitatively impact the problem areas mentioned above?"  I do not believe that there is any hope of standardizing the construction of all software; indeed, one of the exciting parts of the field is that we are far less constrained than other engineering fields.  At the same time, I agree entirely with Michael Jackson that specific domains can (and indeed have) become standardized -- and more and more of them will over time.
  17. Even though you are skeptical of quantitative data, will you show us some?  I was imprecise -- I am not against quantitative data.  However, I feel that they are often used as a proxy for a qualitative measure that we really want.  But I will indeed talk some about empirical software engineering and what we seem to know and not know, to some degree.

Research Resources

But which of the papers that you find are any good?  Well, that's not an easy question, but here is some evaluation of various publications in software engineering research.  

  The two topic journals are widely considered to be:

The top research conferences include:

  There are a couple of magazines that often have useful material, although rarely as deep technically, including:

Some other forums that have material of more varied quality, but definitely with some high quality papers, including:

A number of publications in related areas sometimes have high-quality publications touching on software engineering, including:

This is not a complete list, and it surely has both false positives and false negatives (to varying degrees).   But it’s something.  Feel free to run other stuff by me, if you want an assessment.

Some email and my responses

  1. A student (working at a major company on a large programming environment comprised of 10's of millions of lines of code) writes:

    While listening to the material relating to how define a requirements, as well as the Z (“Zed!” J) approach to capturing these requirements, I had a difficult time mapping this into reality.  Why?

    If I assume that Z can help clearly articulate requirements (and I am still having a hard time swallowing this; I do hope the tool exercise helps me to), it feels as though the amount of effort required to achieve this for the amount of product we are expected to deliver is unreasonable.  That is, if I had to use Z to define my requirements for my current product as an example, it feels as though it would take 10x the amount of time or more to achieve this as opposed to delivering a “good enough” specification.  Now perhaps that is the problem and why we fail more often than we want to.  The mythical man month suggests that the engineering portion of software development should be 1/6 of the time spent, while we are usually at about 1/3, and we usually spend only about 1/5 of the time for planning. 

    Perhaps this the paper I should be writing for October 23rd.  My inference is that to date you are suggesting that in our current industry we are ignoring the complexities of software, which is misleading us. And once we recognize this complexity, computer science will move further away from an art form and closer to a true science.

    Unfortunately, it doesn't surprise me that I was inarticulate in making my points, and this question helps me "understand the misunderstanding."  Thank you.  First, what I don't believe, at least in general: (a) that industry inherently spends the "wrong" amount of time on various phases of the software lifecycle; and (b) that I intended to suggest that "our current industry [ignores] the complexities of software, which is misleading us."  Well, maybe I believe (a) to some degree -- it's not entirely inconsistent with a set of the problems you collectively listed in my pre-course questionnaire.  I absolutely reject (b), and the suggested consequence.  (I cannot speak for other software engineering researchers, only for myself.)

    So, what do I believe is the role of something like Z in reality?  I think it is analogous -- at least in most domains -- to the use of proofs of  program correctness.  That is, in general, one may not need or benefit from a Z specification -- the clarity is there (because of prior experience, because of some other effort, or whatever) and thus the risk is small.  But there may be new features, existing and continually confounding aspects of a system, or such, where the risk is instead (at least the initial risk) in understanding what should be done with some care.  The examples I mentioned in lecture about using formalisms to isolate and later replace key, complex pieces of code in an existing system are indeed the kind of thing I have in mind.

    Let me stress this one more time: I do not believe that the problems of software engineering can be solved by "always writing formal specifications" anymore than I believe that they can be solved by "always using design patterns" or "always using agile processes" or "always using statically-typed programming languages" or such.  Having more arrows in your quiver than you presently do, however, might change your reality at times.  To be blunt, it's not good to assume that the "only" things you have at your disposal is the programming language, the compiler, grep/find, and a few other programming-oriented tools -- that's limiting, as well, in some situations.
  2. Another student writes:

    There was a question in last night’s class that I think is worth revisiting.  One of the students asked whether writing unit tests to specify code functionality was equivalent to using Z.   I can’t remember the response exactly, but it seemed that there might have been a misunderstanding of what was meant by the question.

      I am pretty sure the student was referring to an Agile practice called “Test Driven Development” or TDD.  This is a design discipline where the basic idea of this is to first specify functionality in the form of a failing test or tests, then write the simplest code to exactly satisfy the tests.  This process is performed incrementally as the code is created.   This practice is similar to Z in which a person flushes out pieces of the specification first and then writes the code according to the Z spec later.   As far as I can tell, writing program specifications in a programming language is equivalent in every meaningful way to writing them in a specification language like Z.   The unit tests, however, have three distinct advantages:

    ·         First, the specification is created in a language already understood by the developers.

    ·         Second, the pre-written unit tests provide real-time feedback as to how well the actual code meets specification and when the programmer is “done”, which limits superfluous (extra-specified) code.

    ·         Third, the tests also stand as a barrier to preventing the code from drifting out of spec as changes are made.  

    The application of TDD has been shown in some studies to reduce bug counts by an _order of magnitude_ over standard techniques where tests are written after the fact.  (I am referencing studies internal to [major company]).  Knowing this, I would like to hear more discussion about this in the class, because I think it is highly relevant to what we talked about last night.

    Thanks again, for allowing me to clarify what I meant to say.  This question came up, if I recall, in the context of the discussion of the "small scope hypothesis."  Andoni et al. (sdg.csail.mit.edu/pubs/2002/SSH.ps) say:  "The 'small scope hypothesis' argues that a high proportion of bugs can be found by testing the program for all test inputs within some small scope. ...  If the hypothesis holds, it follows that it is more effective to do systematic testing within a small scope than to generate fewer test inputs of a larger scope."  In essence, they argue that exhaustive testing in a small scope dominates other approaches to testing over a larger scope. 

    I certainly am aware of test-driven development (although I hadn't seen the order of magnitude numbers), and I was trying to respond in that context.  Indeed, this specific paper compares the "small scope hypothesis" to two specific forms of testing: code coverage and mutation testing.  It does not compare to test-driven development, so in principle one cannot extrapolate to that approach.  In some sense, though, I don't think that comparison is necessarily the right thing to think about.  Instead, I might think that a combination of these two powerful techniques (at least in some domains) might be attractive: that is, write tests a priori and exhaustively test within bounded scopes.  Perhaps they each catch different kinds of errors; perhaps not.  I find it easy to believe that effective testing -- however defined -- is not likely to be represented by a single approach.

    Maybe no clearer, but I hope so.