Homework 9: Shell Scripting

wget https://courses.cs.washington.edu/courses/cse391/24sp/homework/hw9/hw9.tar.gz
tar -xzf hw9.tar.gz

Overview¶

This assignment focuses on Bash shell scripting. Electronically turn in file autograder.sh as described in this document.

For this assignment, you will write a script autograder.sh that “grades” solutions to a hypothetical homework assignment. You’re given a set of students’ solution files, each of which is supposed to be named task1.sh (though some students did not submit the assignment correctly…), and your script will attempt to run each of them one at a time, examining the output to see if it matches the expected output. Here is one example output (after calling the script by typing in ./autograder.sh 50 into your shell):

$ ./autograder.sh 50
Autograding CSE 391 Homework
Grading with a max score of 50

Processing joshue ...
joshue has correct output
joshue has earned a score of 50 / 50

Processing dravir ...
dravir has incorrect output (1 lines do not match)
dravir has earned a score of 45 / 50

Processing joeblow ...
joeblow has incorrect output (4 lines do not match)
joeblow has earned a score of 30 / 50

Processing oterod ...
oterod did not turn in the assignment
oterod has earned a score of 0 / 50

In this example, the students/ directory contained four students (joshue, dravir, joeblow, and oterod), with varying correctness of solutions. We’ll unpack these throughout the spec.

Scaffold Structure¶

To get started, download the hw9.tar.gz from the course web site which should contain the following:

• students/: This directory contains a set of fake student solutions to examine. Within the students directory there are student folders and files such as:

  students/joshue/task1.sh
  students/oterod/TAsk1.sh
  students/zorahf/task1.sh
  

  Each student’s directory is supposed to contain a file representing that student’s homework solution. Some students submitted the assignment incorrectly (e.g. oterod in the above list) and will not receive credit for the assignment.

  Note that you are not targeting the specific students shown above; your code should not specifically mention names like zorahf or joshue. Your code should process all students in the students directory, and we will test you with different students/ directories.

• expected.txt: This file contains the expected homework output. This is the text you’ll compare to each student’s output to see whether it is correct.

Input Arguments¶

You will write a script autograder.sh that gives each student a score on the assignment. Your program should accept the maximum score as a command-line argument. For example, to run it with a max score of 50 points, you would type:

$ ./autograder.sh 50

If the user does not pass a value for the maximum points, your script should instead print exactly the following error message and exit with a non-zero exit code:

$ ./autograder.sh
Usage: ./autograder.sh MAXPOINTS

You do not need to check this argument for validity. If an argument is passed, you may assume it is a positive integer.

If the user passes in an argument, you should then print a welcome message with this exact format (replacing the 50 with the value of their argument):

Autograding CSE 391 Homework
Grading with a max score of 50

Processing Each Student¶

If your script is passed an argument, it should assume that the current directory contains:

1. a students/ directory whose contents resemble those in the sample files you have been given (containing one directory per student, each named with the student’s id)
2. and, one file called expected.txt

For each student in the students/ folder, you should:

1. run the student’s task1.sh file (if it exists), using the student’s folder as the working directory. - you should manually invoke bash to run them, i.e. bash ./task1.sh
2. capture the file’s output and compare it against the expected output file using diff, ignoring whitespace.
3. count the number of lines that do not match to calculate the student’s score (see below for more).
4. output four lines for each student:
   1. a “processing” line
   2. a “correctness” line
   3. a score line
   4. a blank newline

Calculating Score¶

To simplify things, we will consider any line of diff (ignoring whitespace) output containing a < or a > to count as 1 line of unmatched content.

For each unmatched line, deduct 5 points from the student’s score. For example, if the diff output has 4 lines that have < or >, the student should lose 20 points. If the student loses more points than there are points in the assignment, the student should receive 0 points.

Output format: Processing¶

The first line you should output for each student is an informational message. The format should be Processing <student ID> .... Be sure to match the spacing exactly!

Processing joshue ...

Output Format: Correctness¶

The second line you should output for each student indicates the correctness of their solution. There are three possible cases.

If the student joshue has a fully correct solution (no issues), output the following in this exact format (including whitespace and capitalization):

joshue has correct output

If the student smith has an incorrect solution (i.e. at least one pair of lines do not match), output the following in this exact format (including whitespace and capitalization):

smith has incorrect output (8 lines do not match)

If the student did not turn in the program or incorrectly named the file, you should instead print a special message. For example, if the user lewis did not turn in a proper task1.sh file, you would output the following:

lewis did not turn in the assignment

Output Format: Score¶

The third line of the output should be the student’s score. The format for this is the same regardless of the student’s correctness; the only thing that changes is the score value.

For example, assume that the maximum score is 50.

If a student belcher has no differing lines of output, the score output for that student would be (in this exact format, including spacing):

belcher has earned a score of 50 / 50

If student davis has 3 differing lines of output (-15), the score output would be (in this exact format, including spacing):

davis has earned a score of 35 / 50

If the student smith did not turn in the program or incorrectly named the file, the score output would be (in this exact format, including spacing):

smith has earned a score of 0 / 50

Assumptions¶

You may assume that:

• if an argument is passed in, it is a positive integer
• the ID for each student consists of just ASCII alphabet characters (in particular: no whitespace, no < or >, or / or \), and is otherwise a valid folder name
• no student’s program tries to do anything evil to your computer, such as erasing all your files
• students’ code will not lock up and get stuck in any sort of infinite loop

You may not assume that:

• the students directory is not empty
• every student has submitted a file called task1.sh
• every student’s task1.sh is executable (however, manually invoking task1.sh with bash partially resolves this)
• each submission outputs the same number of lines as expected.txt
• each submission has correct, working shell code
• no student loses more points than there are available in the assignment
• the assignment’s total points is a multiple of 5

It is okay if your autograder.sh script creates temporary files while doing its work, though if so, you should remove them as you go along.

Development Strategy¶

There are two different suggested approaches - pick the one that works best for you!

“Sequential” / chaining (like adding commands on pipe-by-pipe):

• Make your script print the introduction with the max points command line argument.
• Make your script able to simply output the names of all of the students to be processed.
• Make your script able to simply run each student’s task1.sh program and show its output on the terminal.
• Make your script able to diff the expected output against each student’s task1.sh output.
• Make your script compute and print the score for each student.
• Make your print the suggested usage and exit when an argument is not passed for the max points.

Do one student first, then the rest:

• Make your script print the introduction with the max points command line argument.
• Make your print the suggested usage and exit when an argument is not passed for the max points.
• Make your script output the first student to be processed
• Then, make your script run that student’s task1.sh program and show its output to the terminal
• Then, diff the expected output against that program
• Then, compute and print the score for that student
• Finally, “scale this up” for all the students!

Hints¶

• Some tasks in your script will involve running Unix commands and capturing their output with $( ). Since such commands run silently as their output is being captured, consider simply running the command first to see that the command is running properly and producing results that you expect.
• Remember to use echo statements to output partial results, computations, commands, etc. to verify them.

Your script should work if placed in any directory containing expected.txt and a students/ directory. Do not hard code in a specific directory on your machine or your home directory on attu. This is important for grading!!

Each Linux/Unix box can be slightly different; for full credit, your commands must be able to work properly on any of the CSE virtual machine image, attu, on the CSE basement lab computers, or Gradescope.