The goal of this assignment is to appreciate the difficulty of writing tests, and the strengths and weaknesses of automatic systems for doing so.
You will submit this assignment as a zip file (.zip) containing two
commands.shfile, then will read the README file. The TA will not perform any tasks manually, and will not run any further commands beyond running the
commands.shscript must create any files that you mention later. It should contain comments indicating what part of the assignment each part of it is relevant to, such as "# 1" for the part relevant to question 1; that part of the file is probably just a
git clone ...command to obtain your repository.
commands.sh, say where to look in the output, and how to interpret the information there.
commands.shfile) that contain the coverage before and after the change. Summarize the difference in coverage before and after adding your tests.
If the coverage report gives a truncated result (say, an integer coverage percentage), then your new tests do not need to improve the the truncated result (you don't need to improve the whole test suite's coverage by a full percentage point). Change the coverage tool's output to include more digits of precision, or indicate a report that shows that your addition really did improve coverage.
commands.sh should include commands that download and install Randoop before running it.
If you do not know what the Java classpath is, please spend a moment to learn about it. Briefly, it tells the JVM where to find classes. When you run Randoop, the JVM will need to load and run classes from Randoop itself, and also classes from the application under test (and any libraries it uses). Be sure that the classpath contains both.
You can typically find a
.jar file in a
build/libs/ in the project after you build
it. If that is a "fat jar" that also contains all the libraries that
the project uses, then it is enough. If that is a "skinny jar" that
contains only the project's classfiles, but not those of libraries it
uses, then you need to either make a fat jar (use
shade plugin or
the Gradle shodow
plugin) or include all the libraries on the classpath.
If Randoop crashes due to a bug (this is usually indicated by "Randoop failed in an unexpected way." and a stack trace), submit a bug report and skip the two questions about the tests that it produces.
If you are having trouble running Randoop, first make sure that you can run this command from its user manual:
java -Xmx3000m -classpath $RANDOOP_JAR randoop.main.Main gentests --testclass=java.util.TreeSet --time-limit=10
The invocation on your project will be rather different; this
invocation is just to make sure you installed Randoop correctly. If
you didn't set environment variable
RANDOOP_JAR, then use
the path directly in place of
$RANDOOP_JAR, which is
how bash refers to a variable.
For the first test that is not a false positive, fix the underlying bug
and add a regression test to the project's test suite. (Before you try
to understand the regression
it, or run Randoop in the first place with
Show the (minimized) Randoop test and give a URL to the commit that
fixes the bug and adds a test.
At the end of the document:
Upload your zip file to Canvas.
This assignment will be peer reviewed. There is a rubric on canvas which has a set of Yes/No questions. If your answer is "No" for any of the criteria, give a short description of what failed for you. Also give comments on whether the given documentation was sufficient and easy to follow.
Peer review rubric
commands.shrather than in your main