PROJECT: Homework 3

Due: Friday, November 3, 2023 11:59pm

You may not drop this project assignment

Assignment Goal

In this assignment we will put our skills together. You will get more practice shell scripting and integrating different types of processes. You will additionally learn to build a useful test tool to help with HW 5.

Hints and Tips

Taken in its entirety this assignment may seem quite long. You are encouraged to read through the described approach which is designed to break the assignment into managable pieces. While it is a a good idea to read the entire description before you start, completing one section at a time will help you tackle this large project.

You are always encouraged to test your code earlier rather than later. Make sure it meets the basic big picture requirements before you move on to finishing all the details.

There are some references linked at the end of this page.

Implementation Details

Background

You want to build a script to help test a project you are working on. This script will allow you to specify different executables and test files, as well as an output file to which you will write results. This assignment specifies some basic functionality for this test script. While you can imagine many ways to improve this testing, the evaluation of HW3 will be based on the specifications laid out here.

You will create on bash script called tninetest with usage:

tninetest OUTPUT EXECUTABLE TESTFILE...

tninetest will test the EXECUTABLE by redirecting TESTFILE(s) into the executable. Results of the tests will be written to OUTPUT.

Specific things you must complete are highlighted below in purple.

The Executable (What you are testing....)

For this assignment you will be testing an executable called t9. Later in the quarter you will build your own version of t9, in HW 5. You can learn more about the details of the executable there. For this assignment you will be able to retrieve an executable and a required dictionary file from the HW3 folder in cse374-materials repository.

Running t9 is designed to be interactive, but, you can automate this with some shell redirects. For example, this line loads a dictionary and then exits:

[hw3]$ ./t9 smallDictionary.txt <<< $'exit'
Welcome to the T9 decoder
Enter "exit" to quit
Enter key sequence (or "#" for next word):
Thank you!

You may wish to play around with the executable for a little bit. Can you open it and query for a word? (Hint: t9 smallDictionary.txt will start the executable and load one dictionary. Try following the interface to query for any of the words you see in the smallDictionary file.

	    Enter "exit" to quit.
	    Enter Key Sequence (or "#" for next word):
	    > 76257
	    'rocks'
	    Enter Key Sequence (or "#" for next word):
	    > #
	    'socks'
	    Enter Key Sequence (or "#" for next word):
	    > exit

Thoroughly testing code is difficult. When we test your t9 executables for HW5 we use some sample test input. This input is in one or more files which are redirected as input to a running t9 process. For this assignment we will provide a couple of test files, and your shell script should be able to run with one or more of these files. More information about how to use these files is below.

tninetest Implementation Details

To start your script include statements to ensure proper handling. This means, test for input arguments and ensure that they meet the usage description above. If the arguments do not represent appropriate usage you should print a message to standard error specifiying the usage and exit 1.

Your message should look like the usage synopsis

Usage: tninetest OUTPUT EXECUTABLE TESTFILE...

The elements (and expected usage) of the message above are:

OUTPUT: the name of an output file. The results of tests will be written here.
- If the output file does not exist create it.
- If the output file does exist overwrite it with new information
- Write to the output file: Testing EXECUTABLE with the correct executable name.
- Redirect the current date to the output file
EXECUTABLE: this is the name of the executable you will test. For now, use t9, but in the future you could create your own executable and test it here.

If the executable does not exist, or is not executable, print a message to stderr and exit 1
TESTFILE: One or more files that contain a series of inputs to a t9 execution. This assignment supplies Testfile1 and Testfile2. Handling notes are below.

After you perform the basic checks and set up described above you will start testing t9. The idea is to perform a test, and then send a message of SUCCESS or FAILURE to the OUTPUT file. First, do the following basic tests:

You will want to extend your PATH so that it include ./. This way we can call any executable in our working directory by name.
Run the executable with no arguments. The executable should exit with a failure (non zero) in this case. (Hint: $?). Report the results to OUTPUT with a message such as
Test Usability ... SUCCESS
Check to see if the executable can handle a non-existent dictionary file. Report the results to OUTPUT with a message such as
Test Handle Bad Dictionary ... FAILURE
Check to see if the executable can load a dictionary and exit successfully. (Hint: Pipes, Here documents, and Here strings.) Report the results to OUTPUT with a message such as
Test Load Dictionary ... SUCCESS
Check to see if the executable can return one word of your choice. You will try to query for one of the words in the smallDictionary file and check to see if the output from the executable contains that word. Hint:
```
[mh75@attu4 hw3]$ ./t9 smallDictionary.txt << INPUT
> 53556
> exit
> INPUT
Welcome to the T9 decoder
Enter "exit" to quit
Enter key sequence (or "#" for next word):
Your word is: jello
Enter key sequence (or "#" for next word):
Thank you!
```
Hint: >> testoutput, grep. Be sure to report the results to OUTPUT with a message such as
Test Find Word ... SUCCESS

Now, your script will loop through each of the TESTFILES and do the following:

If the file does not exist in your directory, use to download it from the course website https://courses.cs.washington.edu/courses/cse374/23au/assignments/TESTFILE
If the file does not exist and you can not retrieve it, print the message Skipping TESTFILE to the output file, and proceed to the next TESTFILE.
If the file does exist or was retrieved, print the message Testing TESTFILE to the output file.
Then run the executable with the appropriate testfile. (Hints: cat TESTFILE | ./t9 smallDictionary.txt, input redirection may be combined with a variety of output redirections.)
For the purposes of this assignment, a test is successful if the executable exits normally. Be sure to report the results to OUTPUT with a message such as
Test TESTFILE ... SUCCESS
If (and only if) the file was retrieved using wget you will want to delete the TESTFILE
Proceed to the next TESTFILE

This is an example of running a testscript that matches the above descriptions. Testfile1 is in the directory, and Testfile2 is not.

[mh75@localhost hw3b]$ ./tninetest testrecording t9 Testfile1 'not a file' Testfile2 2> errredirect
Replacing testrecording
[mh75@localhost hw3b]$ more testrecording 
Testing t9
Wed Nov  1 12:25:14 PM PDT 2023
Test Usability ... SUCCESS
Test Handle Bad Dictionary ... SUCCESS
Test Load Dictionary ... SUCCESS
Test Find Word ... SUCCESS
Testing Testfile1
Test Testfile1 ... SUCCESS
Skipping not a file
Testing Testfile2
Test Testfile2 ... SUCCESS
[mh75@localhost hw3b]$ more errredirect 
[mh75@localhost hw3b]$

You should be sure to delete any temporary files that are created. You should not produce output other than what has been requested. If the script does not report any errors at any time during its run, it should exit 0

Documentation

In addition to the lecture notes, you may find "The Linux Pocket Guide" a useful reference for completing this assignment. You may also wish to look forward to HW 5 to better understand the C code we are testing.

Online manuals:

Your script may create temporary files if you want. The mktemp program produces unique file names for temporary files. If you create a temporary file, you should remove it before your script exits. Generally it is best to create temporary files like this in /tmp.
To suppress the output of a command, try to redirect its output to /dev/null. For example tryls > /dev/null. There is also often a -q option on processes that prints less information.

Part 2: Write-up

You should include a README with HW3. In this readme you will want to include:

Your name and date
A brief description of your program
A list of any resources you used during this assignment
The supplied TESTFILEs are designed to test specific potential failure points. For each file identify one thing that is being tested.
This sample test script does not thoroughly test the executable code. What is another test that you could run?

Assessment

This homework will be worth a total of 45 points. Your script will be the majority of the points, but your README will also be graded. Be thoughtful about this.
Identifying information including your name, CSE 374 and the date should appear as comments in each of your files
Your grade depends on turning in a correct script, etc., that runs with bash on our reference system (Seaside).
Each submission should be in good style, including indentation and line breaks. Refer to the CSE 374 guide for hints on style.
Scripts should handle arguments with spaces in them, and print error messages appropriately to stdout.
Aside from the OUTPUT file, your script should leave no extra files in the directory.
Your script should not produce output aside from that described in the specification.
You should pay attention to the details of the specification, as scripts will be evaluated against those standards.

Notice that this assignment is a PROJECT, which means it is not eligible to be dropped at the end of the quarter.

Turning In

Set up to turn this in by adding a folder called HW3 to your personal Gitlab repository. This folder should reside at the top leverl of your repository, and it should contain tninetest and README.

Please submit your files via Gradescope. You will do this by choosing to upload from Gitlab, and submitting your repository. Gitlab will ignore files not in HW3.

There will be an autograder in place that will do some of the evaluation, but there will be additional manual grading to evaluate style and your analysis in the write-up. You may resubmit your homework as many times as you like up to the initial due-date.