CSE 401/M501 MiniJava Testing Utility

To make it easier to write comprehensive JUnit unit tests, we've developed a set of testing utilities for you to test the overall behavior of your MiniJava compiler. The MiniJavaTestBuilder can be used to implement assertions in your JUnit tests to assert the output and error produced when running MiniJava's main method, as well as verify the compiler's exit status when the main method exits. This means that you still use ant test to run your tests, and can/should continue to use JUnit's family of assertXXX methods when testing implementation details of your compiler.

WARNING: A terminally deprecated method in java.lang.System has been called
WARNING: System::setSecurityManager has been called by CSE401TestUtils (...)
WARNING: Please consider reporting this to the maintainers of CSE401TestUtils
WARNING: System::setSecurityManager will be removed in a future release

References

• Installation
• MiniJavaTestBuilder API Overview
• Fixing JUnit Tests for IntelliJ IDEA
• Advanced Testing Utilities and Customizations

Why MiniJavaTestBuilder?

When testing the scanner part of the MiniJava compiler, you could make a JUnit test suite/class called something like TestMiniJavaScanner inside the test/junit/ folder, and reuse/copy the runScannerTestCase helper method from TestDemoLanguageScanner to assert that running MiniJava scanner on the testCaseName.java file will output the same content as stored in the testCaseName.expected file:

private void runScannerTestCase(String testCaseName) {
    try {
        FileInputStream input = new FileInputStream(TEST_FILES_LOCATION + testCaseName + TEST_FILES_INPUT_EXTENSION);
        String[] expected = new String(Files.readAllBytes(Paths.get(TEST_FILES_LOCATION, testCaseName + TEST_FILES_EXPECTED_EXTENSION)),
                Charset.defaultCharset()).split(" ");

        ComplexSymbolFactory sf = new ComplexSymbolFactory();
        Reader in = new BufferedReader(new InputStreamReader(input));
        scanner s = new scanner(in, sf);
        Symbol t = s.next_token();
        int i = 0;
        while (t.sym != sym.EOF){
            // verify each token that we scan
            assertEquals(expected[i], s.symbolToString(t));
            t = s.next_token();
            i++;
        }
    } catch (IOException e) {
        fail(e.getMessage());
    }
}

With MiniJavaTestBuilder, we can simplify this to just:

private void runScannerTestCase(String testCaseName) {
    try {
        new MiniJavaTestBuilder()
                .assertSystemOutMatchesContentsOf(
                        Path.of(TEST_FILES_LOCATION,
                                testCaseName + TEST_FILES_EXPECTED_EXTENSION))
                .testCompiler("-S", TEST_FILES_LOCATION + testCaseName + TEST_FILES_INPUT_EXTENSION);
    } catch (IOException e) {
        fail(e.getMessage());
    }
}

Not only does it make it easier to implement test helper methods, it also makes it easy to make another, similar helper method for testing the parser part of the project. The only change needed is passing in the new command line arguments, and specify the expected AST/pretty print output files:

private void runParserTestCase(String testCaseName) {
    String filename = TEST_FILES_LOCATION + testCaseName + TEST_FILES_INPUT_EXTENSION;
    try {
        new MiniJavaTestBuilder()
                .assertSystemOutMatchesContentsOf(
                        Path.of(TEST_FILES_LOCATION,
                                testCaseName + TEST_FILES_EXPECTED_AST_EXTENSION))
                .testCompiler("-A", filename);
        new MiniJavaTestBuilder()
                .assertSystemOutMatchesContentsOf(
                        Path.of(TEST_FILES_LOCATION,
                                testCaseName + TEST_FILES_EXPECTED_PRETTY_PRINT_EXTENSION))
                .testCompiler("-P", filename);
    } ...
}

Furthermore, it even allows checking if the contents of System.err matches some expected output:

private void runScannerTestCase(String testCaseName) {
    try {
        new MiniJavaTestBuilder()
                .assertSystemOutMatchesContentsOf(...)
                .assertSystemErrIsEmpty()
                .testCompiler("-S", TEST_FILES_LOCATION + testCaseName + TEST_FILES_INPUT_EXTENSION);
    } ...
}

as well as the exit status:

private void runScannerTestCase(String testCaseName) {
    try {
        new MiniJavaTestBuilder()
                .assertSystemOutMatchesContentsOf(...)
                .assertSystemErrIsEmpty()
                .assertExitSuccess()
                .testCompiler("-S", TEST_FILES_LOCATION + testCaseName + TEST_FILES_INPUT_EXTENSION);
    } ...
}

For more on what is possible to assert through MiniJavaTestBuilder, see the References section at the top of the page.