Unit Testing

What is Testing?¶

We’ve probably all had this experience before: spending several hours writing some code, finally getting everything to compile, and then running the program; only to realize moments later that nothing works correctly, and spending another several hours disabling parts of the program to figure out what’s wrong. As it turns out, it takes a long time to figure out what’s wrong in a program, especially if we need to sequentially disable different components or write new mini-programs to check what our code is doing.

Instead of manually going through this process every time, it would be much easier to automate it—we’ll check our code by using more code! When we refer to testing, we’re referring to this process: we’re testing the correctness of our code.

You’ll see testing used extensively throughout both industry and the assignments in this course: all our assignments include some test code for the classes you’ll be implementing, our grading will similarly be automated using programmatic tests.

In particular, our tests focus on checking individual units of your code—usually individual instances of the class you write—to ensure that they behave as expected in each given scenario; this style of testing is called unit testing. Usually, each unit test follows a simple process:

1. Arrange: Instantiate the unit under test and set up some scenario (e.g., adding 3 items into a list).
2. Act: Invoke some method to test (e.g., remove(5)).
3. Assert: Check that the unit behaved correctly. This may involve checking its return value, or examining other state in the object (sometimes by calling other methods), or making sure the correct exception was thrown. In most testing framework libraries, the checking is done by calling an assertion method that compares some expected value to the actual value produced. In our assignments, for example, you might see code like assertThat(returnValue).isEqualTo(15).

In most unit testing frameworks (including the one we use in our assignments), each test is contained within a single method, and any incorrect assertion will cause, the assertion method to throw an exception to indicate that the test failed. On the other hand, test methods that return normally indicate that the test passed.

A couple final notes and definitions:

Test case

A particular scenario to test; essentially corresponds to the “arrange” and “act” steps above (although it wouldn’t be incorrect to include the “assert” step, since a program should only ever have a single correct behavior for a given scenario).

Test suite

A collection of test cases. Usually, we collect all tests for a particular class in another class, so we can refer to that test class as the “test suite.”

JUnit¶

JUnit is the unit testing framework we use in this class, and it’s also the most commonly used testing library for Java. Note that we’re using JUnit 5, which differs slightly from earlier versions of JUnit.

As described above, we write JUnit tests by simply adding test methods to a regular Java class. JUnit test methods must satisfy a few requirements:

• They must have JUnit’s @Test annotation to that JUnit should treat them as tests.
• They must have a void return type.
• They usually shouldn’t have any parameters. (There are some JUnit extensions that allow you to add parameters, e.g., to use a single method to define multiple test cases based on the parameter value, but you won’t need to use these for our assignments.)
• They must not be private. By convention and for simplicity, JUnit 5 tests are usually written with default visibility (often called package-private; with no public/private/protected access modifier).

Example Test File¶

package mypackage;

import edu.washington.cse373.BaseTest;
import org.junit.jupiter.api.Test;                  // Import @Test

import java.util.ArrayList;
import java.util.List;

public class SampleTests extends BaseTest {         // Extends `BaseTest`
    @Test                                           // Apply @Test
    void remove_afterAddFew_returnsCorrectItem() {  // Naming convention
        // Set up the unit to test.
        List<Integer> list = new ArrayList<>();
        list.add(3);
        list.add(7);
        list.add(9);

        // Execute the method on the unit under test.
        Integer output = list.remove(2);
        // Run an assertion method.
        assertThat(output).isEqualTo(9);            // AssertJ assertion
    }

    @Test
    void size_afterAddMany_returnsCorrectSize() {
        List<Integer> list = new ArrayList<>();
        int amount = 5134;
        for (int i = 0; i < amount; i++) {
            list.add(i);
        }

        int size = list.size();
        assertThat(size).isEqualTo(amount);
        assertThat(list).hasSize(amount);           // Specialized assertion
    }
}

Line 4

Here, we import JUnit’s test annotation.

Line 9

All of our provided tests extend BaseTest, from our root src module. BaseTest implements WithAssertions from AssertJ, providing all our test classes with the default assertThat methods without need for imports.

Line 11

Notice that our test naming convention roughly corresponds to the 3 steps in unit testing: we have 3 parts separated by underscores, where the first and last describe the “act” and “assert” steps, while the middle describes the “arrange” step. Sometimes, we also break up the middle segment into multiple underscore-separated parts to make long names easier to read.
(Note that underscores in method names are allowed in test files, but will trigger Checkstyle errors in main source files.)

Line 21

This syntax may seem a little unusual.

assertThat(output).isEqualTo(9);

The assertThat call creates an object that stores output and provides many assertion methods.
The isEqualTo assertion method is the one you’ll see most often, and essentially just checks that output.equals(9) is true.

Line 34

AssertJ also provides specialized versions of assertThat with additional for particular types—in this case, a List. This assert is equivalent to the one above it, but also provides extra feedback if it fails: the error message will include not only the expected and actual sizes, but also the contents of the list (which will be truncated if the list is too large).

AssertJ similarly allows us to add our own assertThat implementations for custom data types; future assignments will include assertion classes for these in their test modules.

Running Tests in IntelliJ¶

Now, lets try running some tests! Open the AssertJIntro test class in the root test module of our project. This file contains a few basic assertions showing off some capabilities of AssertJ. Notice that the file icon has a red-green diamond, indicating that IntelliJ recognizes it as a JUnit test suite. (The file icon may vary if you have a non-default IntelliJ theme applied.)

Since JUnit is such a popular testing framework, IntelliJ integrates support for running tests directly from its UI. You can run the unit tests by following the same procedure as we did for running code: click on the play button by the AssertJIntro class, and choose Run ‘AssertJIntro’. IntelliJ will compile your code and invoke JUnit, which will then scan through the test class and run all methods annotated with @Test. JUnit will report the test results back to IntelliJ in real time, which should get displayed in a tool window like this:

Also try running an individual test method (e.g., simpleExamples) by tapping the green play icon next to the method’s declaration (line 9).

Tests for This Assignment¶

For this assignment, we’ve provided all the tests you’ll be graded on; find them in the cse143review.test module. There’s one test class for each class in problems, each filled with many unit tests for the methods you’ll be implementing.

You can try running these now, but they’ll all fail unless you’ve already started on the assignment. Instead, our main goal here is to introduce one last thing:

The Debugger and the jGRASP Plugin¶

IntelliJ includes a built-in Java debugger, and we’ve enhanced it by installing the jGRASP plugin. We can run the program in debug mode by choosing Debug … instead of Run … when you click on a play icon. Like jGRASP, the IntelliJ debugger runs your program exactly like it would when running it normally, but it allows you to pause execution on breakpoints.

Let’s try this now: open datastructures.LinkedIntList and place a breakpoint on line 49 (the end of the array constructor). Afterwards, open LinkedIntListProblemsTests and start the debugger for reverse3_returnsCorrectList.

When the debugger stops at the specified breakpoint, the IntelliJ will bring up the Debug tool window. Here, you can see the paused state of the program. At the top of the Debug tool window are small icons for controlling the debugger, including the two most common operations to Step over and Step into the next statement. This is similar to how the jGRASP debugger works.

We can visualize the debugger output by tapping on the jGRASP button at the top of the Debug tool window. This will draw a box-and-pointer diagram representing the paused program state. You can step through the execution of the program, or use the green continue icon on the left side of the Debug tool window to resume normal execution.

Thanks for entertaining all of these instructions. Now, we have all the tools we need to work on the main assignment.