CSE 163, Spring 2019: Tips on testing code

Testing public behavior: Writing good unit tests

Unit tests are tests meant to verify the external behavior of some program. More precisely, a good suite of tests should decisively answer the following question: "Does some program X match the expected behavior of its specification Y?

Not only that, you should be testing to make sure your program breaks in the correct ways as well. If somebody tries using your program in an invalid way, does it crash correctly?

Here are some specific things you might want to try checking when testing:

  1. The happy cases:

    Test how your clients will normally use your code.

  2. Errors and exceptions:

    Do you throw the correct exceptions when bad input is given? Did you forget to implement a certain exception? Do you check for an error condition too late?

  3. Preconditions and postconditions:

    Try checking/stress-testing your preconditions. Are your postconditions always satisfied afterwards?

  4. Empty and null cases:

    What happens when you pass in zero? An empty string or list? A None object (Python's version of null)?

  5. Edge cases:

    Test for edge cases and off-by-one errors. Look for "boundaries" in your input data.

    (What do we mean by "boundaries"? As an example, suppose you're testing an a program that checks in the input size is 16 or more and does something differently in that case. We woulc consider that magic number "16" as a "boundary", since the behavior of your class changes significantly once you hit that point. What happens if you inputting 15 items? 16 items? 17 items?)

    Another strategy is to try generating randomized data to hunt for weird edge cases.

  6. Mixing multiple methods:

    Each method individually might work fine, but what happens if you try using multiple of them, called in all kinds of different orders?