Contents:

How to generate a Test Report

Gradle generates a Test Report automatically when it runs the entire Test Suite. To do the same, run ./gradlew test (or gradlew.bat test from Windows), or run the test Gradle task from your IDE.

This will result in output such as: 

cse331-19wi-CSENetID $ ./gradlew test
Starting a Gradle Daemon (subsequent builds will be faster)
> Task :compileJava UP-TO-DATE
> Task :processResources UP-TO-DATE
> Task :classes UP-TO-DATE
> Task :copyHw5TestScripts
> Task :copyHw6TestScripts
> Task :compileTestJava
> Task :processTestResources
> Task :testClasses
> Task :test

BUILD SUCCESSFUL in 9s
7 actionable tasks: 5 executed, 2 up-to-date
cse331-19wi-CSENetID $

If you have tests that fail, your output should look like: 

cse331-19wi-CSENetID $ ./gradlew test
> Task :compileJava UP-TO-DATE
> Task :processResources UP-TO-DATE
> Task :classes UP-TO-DATE
> Task :copyHw5TestScripts UP-TO-DATE
> Task :copyHw6TestScripts
> Task :compileTestJava UP-TO-DATE
> Task :processTestResources UP-TO-DATE
> Task :testClasses UP-TO-DATE

> Task :test FAILED

hw6.ScriptFileTests > checkAgainstExpectedOutput[0] FAILED
    org.junit.ComparisonFailure at ScriptFileTests.java:193

hw6.SpecificationTests > hw6.ScriptFileTests.checkAgainstExpectedOutput[0] FAILED
    org.junit.ComparisonFailure at ScriptFileTests.java:193

44 tests completed, 2 failed

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':test'.
> There were failing tests. See the report at: file:///.../cse331-19wi-CSENetID/build/reports/tests/test/index.html

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 2s
7 actionable tasks: 2 executed, 5 up-to-date
cse331-19wi-CSENetID $

A failing build has the location to an html file as seen towards the bottom of the second output.

Finding and opening a Test Report

As we noticed in the previous step, a failing test creates a Test Report which is located in build/reports/tests/test/index.html. Every successful run of ./gradlew test also generates a test report, but that report is generally not very useful.

You can open your Test Report in any of the following ways:

If you cannot find the build directory or the report, try running ./gradlew test from your terminal and then look for the report.

Understanding the Test Report

A Test Report for a successful test suite looks as follows:

There isn't much of interest here as all tests pass. So let's move on to a Test Report for a suite with failing tests. For simplicity, we have just a single specificatino test that fails. Here is what the report looks like:

As you can see, a new section exists which shows the tests that have failed. Due to the Gradle setup of your repository, tests may be listed twice. To learn more about the reason a specific test failed, click on the test.

In this view, you can see the name of the test (LoopGraph.test) and also can see some information about why the test failed. Specifically, the assertEquals which was caused by the .actual and .expected files not being the same.

Getting more information about a failing test

Now that you know which test is failing, you might want to see the exact output of the test. A better option here would be to see what exactly was different. It could be a single space, or an easy to miss typo. To see the difference between the .expected and .actual files, run diff src/test/java/hwN/TEST_NAME.expected build/classes/java/test/hwN/TEST_NAME.actual. This will output a diff of the files. Below is an example: 

cse331-19wi-CSENetID $ diff src/test/java/hw5/LoopGraph.expected build/classes/java/test/hw5/LoopGraph.actual 
4,6c4,7
< added node bar to graph1
< added edge foo->foo from foo to foo in graph1
< graph1 contains: foo
< the children of foo in graph1 are: foo(foo->foo)
---
> added node bar to graph1
> added edge foo->bar from foo to bar in graph1
> graph1 contains: bar foo
> the children of foo in graph1 are: bar(foo->bar)

Here there seems to be a mistake in the edge creation. This is an easy fix and the test suite should be fixed in just a few minutes.