CSE 373, Spring 2019: Project Grading

Interpreting feedback

After we're done grading (typically sometime over the weekend), we'll push back to student repos a file containing feedback and scores for the assignment. Here's an example feedback file:

# Feedback

Group dragon: netid1, netid2

Commit hash: 89315958911ba3eed9a7e982447b83dd304a579c

Raw score: 18 / 40

## Checkstyle

Score: 0 / 5

-   FAIL: (weight=1.0) ClassName1.java:5:8 [UnusedImportsCheck]
        Unused import - misc.exceptions.EmptyContainerException.

## ClassName1

Score: 1 / 5

-   PASS: (weight=1.0) testName1
        Description: Sometimes, our tests will include descriptions of their contents.
-   FAIL: (weight=2.0) testName2
        Description: We try to include descriptions for tests with complex behavior or
            ambiguous names.
        java.lang.AssertionError: expected:<0> but was:<1>
            [stack trace truncated in this example...]
            at java.util.concurrent.FutureTask.run(FutureTask.java:266)
            at java.lang.Thread.run(Thread.java:748)

## ClassName2

Score: 10 / 10

-   PASS: (weight=1.0) testName3
-   PASS: (weight=1.0) testName4
-   PASS: (weight=0.5) testName5
-   PASS: (weight=1.0) testName6

## TestClassName1

Sometimes, we may also include some comments here for a section.
These are usually important, and you should read them.

Score: 0 / 10

-   FAIL: (weight=1.0) AllOk
        Test incorrectly reports correct implementation of ClassName1 as broken
        -   PASS: yourTestName1
        -   FAIL: yourTestName2
        -   PASS: yourTestName3

-   SKIP: (weight=1.0) BuggyCaseName1
-   SKIP: (weight=1.0) BuggyCaseName2

## TestClassName2

Score: 7 / 10

-   PASS: (weight=1.0) AllOk
-   PASS: (weight=1.0) BuggyCaseName3
-   FAIL: (weight=1.0) BuggyCaseName4
        Unable to find bug with ClassName2 with [bug description]

-   PASS: (weight=1.0) BuggyCaseName5

At the top of the file will be the group name and members, followed by the full commit hash of the graded commit. Afterwards are the scoring details. Note that the total score listed at the top of the file is only the raw score; in multi-part assignments, you will have to chance to earn points back by submitting a fixed version of code along with the next part.

Style grading is completely binary based on the output of Checkstyle. If there were any Checkstyle errors in the files you needed to change for the assignment, you will receive a 0 points; otherwise, you'll receive full credit. (You may ignore the weights listed in this section, as they aren't used for anything.)

Regular tests

The rest of the points will distributed between the different files in the assignment. Each section has some max score; if you pass all our provided tests and all our secret tests, you will get full credit; otherwise, you will lose points based on the tests you failed. The final score for each section is calculated as the weighted average of the tests for that section, scaled up to the maximum section score. In other words, the formula is as follows:

max_score * sum(weights_of_passing_tests) / sum(weights_of_all_tests)

This value is rounded down to the nearest integer.

For any tests you failed, the stack trace of the failure will also be included. Assersion errors will mean that your code failed some assertion in the test, whereas other errors and exceptions will show up if your code throws one when it shouldn't. If your test failed a timeout, however, this stack trace will probably not be very useful.

For secret tests, the test name and stack trace and will be all you have to go off of as you debug. We try to make sure our test names adequately describe the case being tested, but feel free to try asking at office hours if you need some more direction.

Test tests

If the assignment included a testing portion, it will show up below the regular class tests. The testing portion is slightly different in that the "tests" are really testing your tests by running different versions of code through your tests. The name of each test will be a brief description of the bug in our code, or AllOk if there is no bug.

Note that, as mentioned earlier, if the AllOk case does not pass, all other cases in the section will be skipped. The test results of our working code on the tests you submitted will be displayed instead. Also note that this section does not include stack traces for failures.

Other notes

Occasionally, we may need to make manual raw grade adjustments for groups; this won't happen very often, but when it does, we'll note the adjustment in a section at the top of the feedback file, including the score adjustment amount and reason.

Regrades

In multi-part projects (assignments that have two parts due on successive weeks), you will have the opportunity to regain up to half of the points lost in the first part by submitting a fixed version of them along with the second part. More precisely, your final part 1 grade will be calculated as follows:

max(raw_part_1_initial, mean(raw_part_1_initial, raw_part_1_regraded))

where raw_part_1_initial refers to the raw score for the initial version of the part 1 files, as written in the feedback file for part 1, and raw_part_1_regraded refers to the score for the version of those same files submitted when part 2 is turned in, as written in a separate regraded part 1 feedback file that will be pushed to your repo along with the part 2 feedback. This value is rounded up to the nearest integer.