CSE403 Software Engineering, Autumn 1999

Gary Kimura


Lecture #26 Notes

When to Call it a Day


1)      Is it ready to ship?


a)      Who decides when it is ready to ship?

i)        The developer?

ii)       The customer?

iii)     The managers?

iv)     The executive?


b)      What factors are involved in the decision?

i)        Is the software ready?

ii)       Do we need the revenue?

iii)     Is there an external factor that is setting the completion date?


c)      It is not just the software being ready that determines when to ship

i)        Development and testing plays a big role in getting the product ready, but there is also…

ii)       Documentation

iii)     Manufacturing

iv)     Sales and marketing

v)      Support and more


d)      Does it meet the requirements


e)      Is the quality sufficient for the customer

i)        PC mentality

ii)       Mission critical systems

iii)     Financial, medical, imbedded systems in planes, factories, safety critical equipment


f)        Is the software ready and tested

i)        What are the remaining bugs like

(1)   Are there showstopper bugs (i.e., bugs that you cannot ship knowing it is in the product)

(2)   What is the cost of keeping a known bug in the product

(a)    Is it an obscure bug

(b)   Are there easy workarounds for the bug

(3)   Is the product “bug free”

ii)       In house testing

(1)   Is the daily pass rate acceptable?  The definition of acceptable open to discussion.

(2)   Has it successful run through its validation suite?

(3)   Is everyone happy with the product

iii)     Off-site testing through release candidates

(1)   Are the beta customers satisfied

iv)     Is the product stable enough for its intended market

v)      What do we do when we stop finding bugs


g)      Is the documentation done and accurate


h)      Is manufacturing ready for the release

i)        There is physical manufacturing and shipping of the release media and documentation

ii)       Filling the distribution channel

iii)     Even with internet distribution there needs to be well planned capacity


i)        Is product support ready

i)        Whether the support is “on-site”, “9 to 5”, “24 by 7”, “phone”, or “mail only”, the support staff needs to be trained and have adequate communication capacity.  


j)        Is sales and marketing ready


2)      Is the customer ready?


A lot of this probably depends on the type of customer


a)      The shrink warp market 


b)      ISV – More technical savvy than the regular shrink wrapped market, however also less tolerant of silly, stupid errors


c)      IHV – They have a different set of issues from ISV including automated setup and installation, and customization support


d)      Contract work – Various levels from delivering an entire turnkey system to only a small component that the buyer will integrate.


e)      Internal technical support (e.g., an internal development tools group)


3)      NT Death Marches (ship mode)


a)      All source changes must be reviewed and approved.  It must be for a specific showstopper bug.  Bugs are recorded in a bug database called “raid”.  Sometimes kept an open showstopper bug active just to check-in more “fixes”


b)      Daily builds and stress runs (yes, seven days a week including most holidays)


c)      A typical day near the end


i)        5 AM results are starting to be gathered for the previous night stress run

ii)       7 AM release of the previous nights stress results.  Developers then have until noon or so to debug all the crashed machines.  Sometimes you need to keep the machine a lot longer.

iii)     8 AM meeting of the development team managers looking at the nightly stress results and new bugs review (they decide which bugs need to be fixed, and when to ship the product).  Near the very end this becomes a twice a day meeting

iv)     10 AM to 3 PM the build lab is willing to accept any bug fixes for approved showstopper bugs

v)      5 PM dinner is served

vi)     6 PM the next build is released and everyone installs the new system and starts up stress, and those will showstoppers bugs continue to work.


d)      When it is finally decided to ship the product then the bits go into escrow as the golden media is produced and manufacturing starts ramping up.  Testing continues and if necessary the bits can be recalled from escrow and the release done over again.


e)      Work continues on the subsequent release for the various server editions and international language versions.


4)      Ancillary issues


a)      Media hype

i)        Setting expectations

ii)       Beta previews

iii)     Getting beta customer testimonials might be important  


b)      Competitive pressure

i)        Market share before quality

ii)       First one defines the market and grabs market share even with junk

iii)     The followers often play catch-up with mixed success (unless you are a monopoly)


c)      Timing the release


d)      When do we get paid and are we ready for the IPO?


e)      Major release vs. minor release

i)        Big delta or small delta

ii)       Customer perception based on version number

iii)     Some IHV contracts are based on version number


f)        Where to have the ship party