This is the Spring 2021 final website.

as5: Menus

Last revised: May 11, 2021

Assigned:

• 12 May, 2021

Due:

• Part 1-4 (programming part)
  - Due 20-May, 10:00pm
  - Lock 22-May, 10:00pm
• Part 5-6 (analysis part)
  - Due 27-May, 10:00pm
  - Lock 29-May, 10:00pm

Android Goals:

• Further practice using callbacks; `onDraw()`
• Further practice with translate
• Gain more experience working with callbacks.
• Further practice with state machines and event handling

HCI Goals:

• Record user study data
• Produce plausible experiment results (within expected ranges)
• Understand pie menus
• Design your own interactor
• Collect, parse, and interpret results collected from a study.

• GitGrade links
• Overview of assignment
• Part 1: Overview
• Part 2: Implement Normal and Pie Menus
• Part 3: Implement the Experiment
• Part 4: Implement a Custom Menu
• Part 5: Conduct a User Study
• Part 6: Write a Report
• Submission Instructions
• Grading (45 pts)

GitGrade links

• Accept the Assignment
• Turn-in the Assignment
• Review your Submissions

Overview of assignment

This assignment has two main components.

First you will be creating three menus – a linear menu, a pie menu, and a custom menu that you design. The programming re-uses many concepts from ColorPicker, but will have more pieces to it, so please start early.

After your application is turned in, you will test these menus in an experiment with other people. The user study will include the following:

• Write your consent form using our template. Print it out for participants you see in person, or be ready to email it to remote participants;
• Recruit and consent at least 3 participants to test your app;
• Send your app (APK) to your participants so they can run the tests;
• Collect the data from participants who tested your app. If participants do the test on your device or emulator you will download the data yourself. If participants test your application remotely they will have to download and send you the data they collect for you (details for how to do this are in Part 5
• Import the data into the spreadsheet we provide;
• Turn in a copy of this data (so we can create a very large data set);
• Analyze the data and write the report.

Note: If you are unable to finish parts 1-4, you can borrow a phone with working code installed we will send you an APK after the lock date to finish parts 5-6.

Video of completed assignment

Note the Toasts (pop up messages) are missing a space between the word “Selected” and the item that was selected. This has been fixed in the current version of the code, but the rest of the app works substantively the same.

Part 1: Overview

Tasks

• Your first step is to read this spec thoroughly
• Read through the code of the classes listed below. Note any variables or methods you will be using or overriding from parent class.
• Read and understand the experiment design.

You will be implementing a class that can display a linear menu, a class that can display a pie menu, and a custom menu of your design, for use in an experiment comparing which is faster for the user to select menu items from. By menu items we mean the individual items in a menu, such as the number 1 in the sample video.

Overview of code structure

Files that you will be changing

• ExperimentActivity.java
  • Activity that displays the option to navigate between sessions. Also adds the corresponding menu view to the view hierarchy.
• MenuExperimentView.java
  • View that can displays and handle input for a menu. This is where you implement the state machine that will work for all of your menus. This is the parent class of NormalMenuView and PieMenuView and CustomMenuView
• NormalMenuView.java
  • View that implements normal linear menus. This knows how to draw a linearmenu, and figure out where input is in it. It inherits from MenuExperimentView.
• PieMenuView.java
  • View that implements pie menus. Similar to NormalMenuView,
• CustomMenuView.java
  • View that implements your own personally designed menu. Similar to NormalMenuView

Classes that have functions and fields you may need (your classes inherit from them)

• AbstractMainActivity
  • Methods to handle app creation, switching modes (from test to experiment), and starting experiments.
• AbstractMenuExperimentView
  • A lot of very handy helper methods (accessors and mutators) for the menus, as well as the Listener methods. Most of the other methods in this class are abstract and overridden in the child classes.
• TrialListener
  • Definition of a listener (callback) interface that is called when a trial is finished.

The structure of this code is represented by the Unified Modeling Language (UML) diagram shown below.

classDiagram AbstractMainActivity <|-- ExperimentActivity AbstractMainActivity <|-- TestActivity AbstractMenuExperimentView <|-- MenuExperimentView MenuExperimentView <|-- PieMenuView MenuExperimentView <|-- NormalMenuView MenuExperimentView <|-- CustomMenuView class MenuExperimentView { essentialGeometry() onTouchEvent() startSelection() endSelection() updateModel() } class State { START SELECTING } class MenuType { NORMAL PIE CUSTOM } class TaskType { LINEAR RELATIVE UNCLASS }

Other files you may want to look at

• TestActivity.java
  • Activity that app starts out in. Lets you test your menus before running the experiment
• ExperimentSession.java
  • Class that sets up all the conditions for the experiment and creates an iterator with all the trials.
• ExperimentTrial.java
  • Class that stores all the data for a single trial.

Overview of Experiment

In this experiment we will have 9 conditions, each of which will include the number of trials (where totalTrials = NUM_REPEATS * ITEM_MAX trials). You can think of this as a three dimensional array

	Menus
Tasks	NORMAL	PIE	CUSTOM
LINEAR	totalTrials	totalTrials	totalTrials
RELATIVE	totalTrials	totalTrials	totalTrials
UNCLASS	totalTrials	totalTrials	totalTrials

The three menu types (specified in an Enum in the code) are NORMAL (linear), PIE (round), and CUSTOM (your custom menu).

The three task types (also an Enum) are LINEAR (menu items that have a natural order such as 1,2,4,8,16,32,64,128); RELATIVE (menu items that have a relationship to each other, such as Up/UpRight/Right/Down/etc…); and UNCLASS (menu items that have no relationship or ordering, such as Print/Cast/Bookmark/etc);

The trials are specific combinations of menu item, task type, and menu type. So for example, one trial might involve showing the user a pie menu, with the numbers 1..128 in it, and asking them to select number 4. The participant can select any option, no matter what as soon as they complete a selection, the result is recorded and the next trial begins.

For any given session (one user’s worth of trials), each combination of menu type, task type, and menu item, will be repeated NUM_REPEATS times (NUM_REPEATS is specified in ExperimentSession.java). In addition, for any specific combination of menu type and task type, at most ITEM_MAX items (a total of ITEM_MAX*NUM_REPEATS trials) will be shown to the user. If you set both of these to 1 for testing, you can test all your conditions very quickly. Make sure to set them to the required values for user testing: ITEM_MAX must be at least 4 and NUM_REPEATS must be 3.

After the code for this project is submitted, you will recruit at least 3 family, friends, or other students and have them complete one session each. Thus, you will have a total of 3 (participant sessions) * 9 (conditions) * ITEM_MAX * NUM_REPEATS, or at least 324, data points when you have completed this assignment. You will analyze the data from this experiment in Part 5 of this project.

We have implemented in ExperimentSession the code for you to generate all of the trials from a setup file called menuContents.csv found in the assets directory. Make sure that you understand the createTrials() method provided in ExperimentSession.java which sets up conditions for the whole experiment. ExperimentSession is an iterator, so to run the trials for a given session you just use session.next() as long as session.getNext() is true.

Note: that we give the user feedback two ways.

• For updates that the user can ignore, throughout the code, we use something called a Toast - a widget that appears at the bottom of the screen and shows some announcement text. Toasts are temporarily displayed and then disappear. See Toast and for an example of how it is used look in AbstractMainActivity.java
• For critical experimental information (i.e. what to select in a trial), we give the user instructions by changing the contents of the TextView at the bottom of the screen (by changing the contents of R.id.instructionTextView). This is a permanent change that will be visible until they finish the trial, so a Toast is not an appropriate choice.

Part 2: Implement Normal and Pie Menus

For this part, you will be working in MenuExperimentView.java, NormalMenuView.java and PieMenuView.java. Recall that NormalMenuView and PieMenuView extend MenuExperimentView.

Tasks

• Implement onTouchEvent() in MenuExperimentView including
  • startSelection()
  • endSelection()
  • updateModel()
• In NormalMenuView implement
  • setup()
  • essentialGeometry()
  • onDraw()
• In PieMenuView implement
  • setup()
  • startSelection()
  • essentialGeometry()
  • onDraw()

Implementing MenuExperimentView

The MenuExperimentView class includes several methods that you will need to implement. You will implement the state machine logic in onTouchEvent (similar to what you’ve done in ColorPicker).

Handling Touch Events

You will handle touch input by implementing the onTouchEvent method. This is the event handler that is invoked when a touch occurs in this view. The state machine defined in onTouchEvent should work, without changes, for every menu you implement. The only menu-specific code is in essentialGeometry() (it finds the current index based on the cursor location). Thus onTouchEvent() is implemented in the parent class, MenuExperimentView and is not changed in the child classes.

MenuExperimentView’s implementation of onTouchEvent makes use of your menu’s essentialGeometry() function to determine the relative position of the user’s finger. Essential geometry will be passed the finger’s position relative to the menu’s (0,0), i.e. where the user pressed. You must implement your linear menu so this is its top left corner, and your pie menu so that the user’s finger is in the center of the pie.

You MUST NOT perform any assignments within essentialGeometry(), as it violates the utility of that method. Instead, essentialGeometry() will return the index associated with the desired view, allowing the calling method to act on that value.

You need to keep track of two main states: START and SELECTING. When in the SELECTING state you need to distinguish between the event type to determine if the user has selected an option or if they are still in the middle of making a choice.

Relevant touch events include ACTION_DOWN, ACTION_MOVE, and ACTION_UP; think about how these touch events relate to the change and how the UI must respond to these events.

More details about essentialGeometry(), specific to each menu, are in descriptions for those menus below.

stateDiagram-v2 [*] --> Selecting: Press/
startSelection() Selecting --> [*]: Release/
endSelection() Selecting --> Selecting: Drag/
updateModel()

Details for startSelection(), endSelection() and updateModel() can be found in MenuExperimentView. Note that there are no guards (boolean logic regarding the essential geometry) on these transitions: once a menu appears (in startSelection()), any dragging causes updateModel() to be called no matter where the user’s finger is (even if it is outside of the menu). Similarly, the menu must be removed and the state machine reset when selection ends (when the user lifts their finger), even if the finger is outside of the menu. Ending the selection will also announce which item was selected.

Notes

• Your menu must ONLY BE VISIBLE when you are in the SELECTING state. You will ensure this by using setVisibility() in startSelection() and endSelection() to show and hide the menu.
• You should make your menu look like the one in the video, although the menu does not need to be a pixel perfect match of what you see.
• We have provided a method that you may use to announce messages (AbstractMenuExperimentView#announce(String)), or you may use the Toast class directly.

Related APIs

• Toast

Implementing the Normal and Pie Menus

One tricky thing about context menus – they can appear anywhere in your user interface. To support this, similar to Doodle you need to set their location and their width and height. AbstractMainActivity assumes the menu should appear where you click and sets this for you. For some menus, this works great (such as the NormalMenuView) but for something like a PieMenuView which needs to be centered around the cursor, you may need to update this. All menus also need their width and height set, which we do by calling setLayoutParams() with the requisite width and height.

You will also implement essentialGeometry() for each and determine what menu item the touch event maps to. You’ll have to come up with the math logic to map from touch event to an item index.

Finally, you will implement onDraw() which at a minimum draws the menus approximately as shown in the screenshots. If you want to do something different, you may, as long as the size and position of each menu item does not change. For example, you can override the paint properties defined in MenuExperimentView, position the text differently, or draw more decorations on the menus.

Be sure to check the AbstractMenuExperimentView starter code for any additional values that you must use when creating your menus. Also make sure you are not “shadowing” variables (redefining variables already defined in super class).

NormalMenuView

The Normal Menu will appear where the user touches the screen (as the top/left corner of the menu) and extend down and to the right.

For the NormalMenuView, essentialGeometry() will return -1 if the pointer is outside the bounds of the menu in any direction or has moved less than MIN_DIST (inherited from AbstractMenuExperimentView) since interaction started. Otherwise it must return the “item number” (position in the list) corresponding to the menu item that the pointer is currently inside of.

The width of each item in the normal menu is CELL_WIDTH and the height is CELL_HEIGHT.

PieMenuView

The Pie Menu will appear centered where the user touches the screen. You will need to handle this in setup()

For the PieMenuView, essentialGeometry() will return -1 if the pointer has moved less than MIN_DIST since selection started, and otherwise it will return the “item number” (position in the list) corresponding to the menu item currently being selected. In the case of the Pie menu, the item number is the menu item that a ray from the center of the menu to the pointer intersects, with the first item being at the top of the pie and increasing as you go clockwise around the circle. The pie menu has no maximum diameter, so anything outside of the radius MIN_DIST around the finger will return a selection.

For PieMenuView, the given RADIUS corresponds to the outer radius. The inner radius is defined by RADIUS - TEXT_SIZE * 2.

Some additional hints for the pie menu

• drawArc() will draw a pizza-pie shaped arc, so you can do things like highlight a menu item with a single method call.
• You will need a rotational offset to ensure the top menu item is at the top of the pie (both when drawing and in essential geometry) because angle is traditionally measured from cardinal east. You can add this in radians before converting from angle to index.
• Just as in ColorPicker, you should look at the atan or atan2 function to help you calculate your pie menu’s essentialGeometry() function.
  • Remember that atan/atan2 return a value between 0..Math.PI or 0 .. -Math.PI. 0 being cardinal east and Math.PI or -Math.PI at cardinal west. Also remember that the y direction is positive pointing down the canvas, not pointing up like a traditional cartesian coordinate system. This may impact the values returned from trigonometric functions.

   

• Your pie menu text does not need to be centered – as long as it is contained within its arc and the outer ring of the pie menu, you are fine.

Related APIs

• Canvas: See documentation on drawArc() and drawText().
• Path: For adding text along a curve
• atan2

Part 3: Implement the Experiment

For this part, you will be working in MenuExperimentView.java and ExperimentActivity.java. This section will entail defining the functions which power your menu experiment.

• In ExperimentActivity
  • Set up and implement a TrialListener
  • Register the TrialListener in showMenuForTrial()
• In MenuExperimentView implement
  • startTrialRecording()
  • endTrialRecording()

Implementing ExperimentActivity

In ExperimentActivity you will implement part of the method showMenuForTrial(ExperimentTrial trial). Your main job is to register a callback with it that knows what to do when a trial is completed.

This callback needs to implement the code to respond to onTrialCompleted(), a method of the TrialListener interface. This listener code must do the following

• check if the session is over (remember the session is an iterator),
• If the session is over, update the text of the InstructionTextView to say that the session is completed, and also display a Toast to the same effect. Again, you may use to announce this message (AbstractMenuExperimentView#announce(String)), or you may use the Toast class directly. Also remove the current menu being shown to prevent any further interaction.
• If the session is not over, call showMenuForTrial() with the next trial.

Note: This means that until the next session is started, the system should not display any menus if the user clicks.

Implementing code to record each trial

The MenuExperimentView class has two methods that still need to be implemented to make the experiment work: startTrialRecording() and endTrialRecording().

startTrialRecording() is called from startSelection() and will start the trial recording only if you are in experiment mode. endTrialRecording() is called from endSelection() and will record the trial data, no matter where the finger is. You do not need to check whether the user clicked on the correct menu item when you call ExperimentTrial#endTrial(). The data will be recorded automatically by ExperimentSession.

Hint: Remember it is always good practice to check for null references!!!

Part 4: Implement a Custom Menu

Create a new menu! Working off of the provided MenuExperimentView interface, build your own third menu. This menu must be different from the Normal/Pie menus. You are encouraged to explore user interactions here, and you will receive credit as long as your code shows significant effort. Your menu does not have to be great here – if you choose, you can aim to create the “worst menu ever” for this submission. Alternatively, you can aim to outcompete our provided menu views!

Be sure to make use of the same abstractions (essentialGeometry() and a state machine) in this implementation.

Tasks:

• Before you start, design your new custom menu by doing the following:
  • Sketch what you want your menus to look like.
  • In couple of sentences, describe how the user will interact with your menu, particularly what types of gestures will be needed to bring the menu up on the screen and how you will select an item with the menu.
  • Describe how you expect your custom menu to perform relative to the Normal and Pie menus.
  • Next, determine the essential geometry for your menu. What equations will you use to calculate which part of the menu the user is currently selecting?
• Add this design work to your menu report for Part 6.
• Finally, implement and test your new custom menu.

Part 5: Conduct a User Study

(not due at the same time as the code portion)

Tasks

• Conduct user study
  • Run a trial on yourself
  • Create a final consent form from this sample
  • Recruit 3 participants
  • Consent the participants, including having them sign your consent form
  • Send your APK to any participants who are testing your app remotely
  • Have them complete a session with ITEM_MAX set to at least 4 and NUM_REPEATS set to 3.
  • Download the recorded data
• Analyze recorded data
• Write a report

Conduct User Study

Start by testing your own program as a participant so you know how long it takes to run a session (this information is needed for your consent form). This is also a good time to double check that your data is not lost when you download it.

Make a copy of the consent form form template and modify it in all of the places marked as such (in italics). The language used in your consent form is key, so ensure that it does not come off as forceful or coercive.

Recruit participants and have them sign your consent form

Recruit your 3 participants from family and friends who can safely use your app on your device or your emulator in person (co-present), or from the random group you have been assigned from class. Please see https://edstem.org/us/courses/4985/discussion/ for a list of group members.

Consenting your participants will be very similar whether they are co-present or remote. First, set up a time when can speak to your participant (in person, over the phone, or over a Zoom or other video conferencing call).

• For a co-present tester: print out two copies of the consent form for each participant – one for them and one for you. Hand them a copy of the consent form before describing the study.
• For remote testers: send a copy of the consent form to the participant prior to your phone call or video meeting.

Briefly explain what your user study is about, and ensure they understand their participation in the study is voluntary. Do not coerce anyone into participating in your study. Make sure they know they have a choice, and have read the consent form.

Your participants must acknowledge their consent by “signing” via the Google form (linked at the bottom of the consent form template). This form should send you an anonymized copy of their consent in email, which you can save and turn in with the rest of your report.

Collect data

Once your participant has been consented, allow them to test the app.

Our implementation uses the ExperimentSession object to create, manage, and record experiment data to a .csv file that is stored on the phone or emulator. If you are interested, check out the recordResult function ExperimentSession.java to understand how this is being stored.

Testing can happen on either a phone or other native Android device or on an emulator. If on an emulator, the participant may use a mouse, trackpad or touch display to interact with the app. When collecting data, you should inform users that all interactions will begin with tapping on the center of the screen. This should help avoid cases where your menu is unusable due to the screen format.

Co-present Testing

• Use the hamburger menu to select Clear Result CSV before starting your study so any prior sample testing data does not contaminate your results. Make sure that you do not clear the results between submissions for a co-present study! This will erase your trial data.
• Ensure the app is in Experiment mode and a new session has started.
• You will likely want to point out what they are to select is written at the bottom of the screen.
• If you are doing 3 co-present tests in a row, the app will move to the next session automatically and store all of the data sequentially in the TestResults.csv file saved to your device. However, this data will be erased if you switch back to test mode, so be sure to download any data before you do that.

Remote Testing

• Send your APK to the participant so they can load it onto their own device or into their own emulator (as we did with Doodle earlier this quarter). In your menu project in Android Studio, select the Build -> Build Bundle(s)/APK(s) -> Build APK(s) menu item. When the Build APK(s) message popup appears in the lower right hand corner, click the locate link to find the app-debug.apk.
• Send the APK to your participant. Unfortunately you can’t email your APK through UW’s servers, instead we recommend you upload the APK to your Google Drive or One Drive and send your participant a link to download it for testing.
• Have your participant use the hamburger menu to select Clear Result CSV before starting your study so any prior sample testing data does not contaminate your results, then run one experiment session.
• Once the user study is complete, your participant will need to download their study data from their device and send it back to you for analysis.
• Combine the three participants’ data into one common .csv file.

Once all participants are done and you have one .csv file with all of the data, it should have at least ITEM_MAX * NUM_REPARTS * NUM_PARTICIPANTS (324) data points. You will be turning in this file (through Canvas) as part of your final report.

Downloading user testing data

You can use the Android Studio Device File Explorer to access the file. This allows you to directly access the files on your Android device (emulated or physical) through a GUI window. Detailed instructions on getting the data off your device can be found on this page.

The data will likely be in /storage/emulated/0/CSE340_Menus/TestResult.csv or /sdcard/CSE340_Menus/TestResult.csv.

Note that each user is given a participant ID when the session is running. These are randomly generated numbers from 0-9999. You may renumber these to identify your participants by the order in which they tested (1, 2, 3) when you retrieve the downloaded .csv file(s).

Part 6: Write a Report

Complete the user study by writing up brief report. We have provided a template for you to use here. Please reference this template as necessary, but the information in the report must be your own descriptions.

Your report will require you to attach screen captures of your menu in both an unselected and selected state. You can capture a screenshot of the menu in action in one of two ways:

• On a phone: Touch the screen to display a custom menu with one hand. With your other hand capture a screenshot. How you do this may depend on the device. The resulting screenshot will be stored in the File Manger on your phone.
• On the emulator: Touch the screen to display a custom menu with one hand. With your other hand press the control-S (for Windows) or command-s (for Mac). The resulting screenshot should be found on your desktop.

Submission Instructions

You will change ONLY the following files. These will be submitted to GitGrade.

Part 1-4:

• ExperimentActivity.java
• MenuExperimentView.java
• NormalMenuView.java
• CustomMenuView.java
• PieMenuView.java

Part 5-6:

Your report will be uploaded to Gradescope. This must include:

• Sample consent form
• Evidence of three “signed” consent forms (either three scanned forms with the “sun signature” or copies of the three auto reply emails you get from the google form consent acknowledgement.)
• Report

Your participants must also fill out the on line (Google) consent form acknowledging that they agree to be a participant and identifying you as a tester.

You must also turn in your combined csv file so we can create one large data set. This .csv will be turned in through Canvas and NOT be put turned in with the rest of your report.

Grading (45 pts)

This HW will be out of 45 points and will roughly (subject to small adjustments) be distributed as:

Code portion (25 points)

• State machine implementation works: (6 pts)
• Normal menu draws correctly (3 pts)
• Pie menu draws correctly (3 pts)
• Normal & Pie menus correctly implement essential geometry (3 pts)
• Custom menu works (2 pts)
• Trials & callbacks handled correctly (4 pts)
• onDraw translates correctly (1 pt)
• Code quality (3 pts)

Experiment Portion (20 points)

• Reasonable CSV output: 1pt
• Consent form: 1pt
• Three participants consented via form: 1pt
• Report:
  • Introduction: 1pt
  • Method: 8 pts
  • Demonstrate understanding of results: 3pts
  • Makes appropriate/good use of charts: 2pts
  • Draw appropriate conclusions about linear vs. pie menu vs custom menu: 3pts