This is the Winter 23 final website.

as4: Menus

Last revised: 3 Feb, 2023
Assigned:
  • Part 1-6 Fri 3-Feb
Due:
  • Part 1-2 (video checkpoint) Due Wed 8-Feb 11:59pm
  • Part 1-4 (programming part) Due Sat 11-Feb 10:00pm
  • Part 5-6 (analysis part) Due Thur 16-Feb 10:00pm

HCI Goals:

  • Utilize Propositional Production Systems to design interactive application features
  • Define event handlers and bubbling
  • Accurately implement various rectangular and non-rectangular interactors such as linear and pie menus
  • Design an original customer interactor
  • Apply understanding of effective testing and ethical practices to conduct and record user study data
  • Produce plausible experiment results (within expected ranges)
  • Collect, parse, and interpret results collected from a study.

Android Goals:

  • Use the Android event handling APIs available to handle touch inputs accordingly
  • Analyze the structure of implementing and producing output with `onDraw()` method
  • Define developer callbacks to apply and maintain separation of concerns.
  • Practice implementing state machines and event handling accordingly

Turn-in links

Overview of assignment

The primary goal of this assignment is for you to be a component developer by implementing three new menus, then testing them to see how they work with actual user interactions. The code base is rather large and you will be putting into practice some complicated concepts, so please start early and utilize the support of office hours and our discussion board frequently.

This assignment has two main components. First you will be creating three menus – a linear menu, a pie menu, and a custom menu that you design.

Approximately half way through the coding portion of this assignment you will turn in a video checkpoint of your progress to Gradescope.

After your application code is turned in, you will test these menus in an experiment with other people. The user study will include the following:

Note: If you are unable to finish parts 1-4, we will send you an APK after the lock date to finish parts 5-6.

Video of completed assignment

Note the Toasts (pop up messages) are missing a space between the word “Selected” and the item that was selected. This has been fixed in the current version of the code, but the rest of the app works substantively the same.

Part 1: Overview

Tasks

You will be implementing a class that can display a linear menu, a class that can display a pie menu, and a custom menu of your design, for use in an experiment comparing which is faster for the user to select menu items from. By menu items we mean the individual items in a menu, such as the number 1 in the sample video.

Overview of code structure

Files that you will be changing

Classes that have functions and fields you may need (your classes inherit from them)

The structure of this code is represented by the Unified Modeling Language (UML) diagram shown below. Note: not all of the details are shown here, so make sure you read through each of the classes and make note of what methods (particulary getters and setters) exist in each. Hint: there are many useful methods in AbstractMenuExperimentView.

classDiagram AbstractMainActivity <|-- ExperimentActivity AbstractMainActivity <|-- TestActivity AbstractMenuExperimentView <|-- MenuExperimentView MenuExperimentView <|-- PieMenuView MenuExperimentView <|-- NormalMenuView MenuExperimentView <|-- CustomMenuView class TrialListener { onTrialCompleted() } class AbstractMenuExperimentView { #mState -mItems #setup() #essentialGeometry() #onTouchEvent() #startSelection() #endSelection() #updateModel() #announce() +getItems() +getCurrentIndex() +setCurrentIndex() +setTrialListener() +getTrialListener() } class MenuExperimentView { #essentialGeometry() #onTouchEvent() #startSelection() #endSelection() #updateModel() } class State { START SELECTING } class MenuType { NORMAL PIE CUSTOM } class TaskType { LINEAR RELATIVE UNCLASS }

Other files you may want to look at

Overview of Experiment

In this experiment we will have 9 conditions, each of which will include the number of trials (where totalTrials = NUM_REPEATS * ITEM_MAX trials). You can think of this as a three dimensional array

  Menus    
Tasks NORMAL PIE CUSTOM
LINEAR totalTrials totalTrials totalTrials
RELATIVE totalTrials totalTrials totalTrials
UNCLASS totalTrials totalTrials totalTrials


The three menu types (specified in an Enum in the code) are NORMAL (linear), PIE (round), and CUSTOM (your custom menu).

The three task types (also an Enum) are LINEAR (menu items that have a natural order such as 1/2/4/8/16/32/64/128); RELATIVE (menu items that have a relationship to each other, such as Up/UpRight/Right/Down/etc…); and UNCLASS (menu items that have no relationship or ordering, such as Print/Cast/Bookmark/etc);

The trials are specific combinations of menu item, task type, and menu type. So for example, one trial might involve showing the user a pie menu, with the numbers 1..128 in it, and asking them to select number 4. The participant can select any option, no matter what as soon as they complete a selection, the result is recorded and the next trial begins.

For any given session (one user’s worth of trials), each combination of menu type, task type, and menu item, will be repeated NUM_REPEATS times (NUM_REPEATS is specified in ExperimentSession.java). In addition, for any specific combination of menu type and task type, at most ITEM_MAX items (a total of ITEM_MAX*NUM_REPEATS trials) will be shown to the user. If you set both of these to 1 for testing, you can test all your conditions very quickly. Make sure to set them to the required values for user testing: ITEM_MAX must be at least 4 and NUM_REPEATS must be 3.

After the code for this project is submitted, you will recruit at least 3 family, friends, or other students and have them complete one session each. Thus, you will have a total of 3 (participant sessions) * 9 (conditions) * ITEM_MAX * NUM_REPEATS, or at least 324, data points when you have completed this assignment. You will analyze the data from this experiment in Part 5 of this project.

We have implemented in ExperimentSession the code for you to generate all of the trials from a setup file called menuContents.csv found in the assets directory. Make sure that you understand the createTrials() method provided in ExperimentSession.java which sets up conditions for the whole experiment. ExperimentSession is an iterator, so to run the trials for a given session you just use session.next() as long as session.getNext() is true.

Note: that we give the user feedback two ways.

Part 2: Implement Normal and Pie Menus

For this part, you will be working in MenuExperimentView.java, NormalMenuView.java and PieMenuView.java. Recall that NormalMenuView and PieMenuView extend MenuExperimentView.

Tasks

Implementing MenuExperimentView

The MenuExperimentView class includes several methods that you will need to implement. You will implement the state machine logic in onTouchEvent.

Handling Touch Events

You will handle touch input by implementing the onTouchEvent method. This is the event handler that is invoked when a touch occurs in this view. The state machine defined in onTouchEvent should work, without changes, for every menu you implement. The only menu-specific code is in essentialGeometry() (it finds the current index based on the cursor location). Thus onTouchEvent() is implemented in the parent class, MenuExperimentView and is not changed in the child classes.

This state machine should assume that all interactions are limited to a single touch that is performing a combination of pressing, dragging, and releasing within the bounds of the screen. The menus will have undefined behavior for other interactions (i.e. you will not have to implement handling that breaks this assumption).

MenuExperimentView’s implementation of onTouchEvent makes use of your menu’s essentialGeometry() function to determine the relative position of the user’s finger. Essential geometry will be passed the finger’s position relative to the menu’s (0,0) coordinate, i.e. where the user pressed. You must implement your linear menu so this is its top left corner, and your pie menu so that the user’s finger is in the center of the pie.

You MUST NOT perform any assignments within essentialGeometry(), as it violates the utility of that method. Instead, essentialGeometry() will return the index associated with the desired view, allowing the calling method to act on that value.

You need to keep track of two main states: START and SELECTING. When in the SELECTING state you need to distinguish between the event type to determine if the user has selected an option or if they are still in the middle of making a choice.

Relevant touch events include ACTION_DOWN, ACTION_MOVE, and ACTION_UP; think about how these touch events relate to the change and how the UI must respond to these events.

More details about essentialGeometry(), specific to each menu, are in descriptions for those menus below.

stateDiagram-v2 [*] --> SELECTING: Press/
startSelection() SELECTING --> [*]: Release/
endSelection() SELECTING --> SELECTING: Drag/
updateModel()

Details for startSelection(), endSelection() and updateModel() can be found in MenuExperimentView.

There are no “guards” (boolean logic regarding the essential geometry) on transitions between the states in this PPS:

As you write the PPS, make sure to utilize proper coding style to ensure that the code is readable to someone not familiar with the project. For an example of how to translate PPS into code, see the PPS page.

Notes

Related APIs

Implementing the Normal and Pie Menus

One tricky thing about context menus – they can appear anywhere in your user interface. To support this, similar to Doodle you need to set their location and their width and height. AbstractMainActivity assumes the menu should appear where you click and sets this for you (see AbstractMainActivity#mainLayoutTouchListener(view, event)). For some menus, this works great (such as the NormalMenuView) but for something like a PieMenuView which needs to be centered around the cursor, you will need to update the menu view’s top left location.

All menus will also need their width and height to be set, which is done by setLayoutParams() with the requisite width and height. The height and width can be set upon construction of the menu (through a call to an abstract setup() method) because the screen size of the menu likely won’t change once the number of items have been set (even if the menu is moved around the screen).

You will also implement essentialGeometry() for each menu and determine what menu item the touch event maps to. This requires your figuring out the math logic to map a touch event in the menu to a menu item index.

Finally, you will implement the menu’s onDraw() method which at a minimum draws the menus approximately as shown in the sample screenshots below. If you want to do something different, you may, as long as the size and position of each menu item does not change. For example, you can override the paint properties defined in MenuExperimentView, position the text differently, or draw more decorations on the menus.

Be sure to check the AbstractMenuExperimentView starter code for any additional values that you must use when creating your menus. Also make sure you are not “shadowing” variables (redefining variables already defined in super class).

Normal Unselected Normal Selected Pie Unselected Pie Selected
The normal menu no selection The normal menu with the 4 selected The pie menu unselected The pie menu selected

NormalMenuView

The Normal Menu will appear where the user touches the screen (as the top/left corner of the menu) and extend down and to the right.

For the NormalMenuView, essentialGeometry() will return -1 if the pointer is outside the bounds of the menu in any direction or has moved less than MIN_DIST (inherited from AbstractMenuExperimentView) since interaction started. Otherwise it must return the “item number” (position in the list) corresponding to the menu item that the pointer is currently inside of.

The width of each item in the normal menu is CELL_WIDTH and the height is CELL_HEIGHT.

PieMenuView

The Pie Menu will appear centered where the user touches the screen. You will need to adjust the location of the view to account for this in setup().

For the PieMenuView, essentialGeometry() will return -1 if the pointer has moved less than MIN_DIST since selection started, and otherwise it will return the “item number” (position in the list) corresponding to the menu item currently being selected. In the case of the Pie menu, the item number is the menu item that a ray from the center of the menu to the pointer intersects, with the first item being at the top of the pie and increasing as you go clockwise around the circle. The pie menu has no maximum diameter, so anything outside of the radius MIN_DIST around the finger will return a selection.

For PieMenuView, the given RADIUS corresponds to the outer radius. The inner radius is defined by RADIUS - TEXT_SIZE * 2.

Some additional hints for the pie menu

Related APIs

Turn in a Video Checkpoint

Once you have implemented your normal and pie menus, record evidence of these working by using the TestActivity. Run the app and select Switch to Test mode and cancel Experiment - from there you can use Open Normal Menu, select a few items in the menu, then Open Pie Menu and select a few items there. See the sumbmission section for how to record your video.

Note: Your video checkpoint does not need to demonstrate perfect behavior - but it does need to show you are bringing “something” up on the screen for each menu type.

Part 3: Implement the Experiment

For this part, you will be working in MenuExperimentView.java and ExperimentActivity.java. This section will entail defining the functions which power your menu experiment.

Tasks

Implementing ExperimentActivity

In ExperimentActivity you will implement part of the method showMenuForTrial(ExperimentTrial trial). We will use a callback in order to receive and process the trial information when a selection is made by a menu. Your main job in this showMenuForTrial is to write the callback and register it a using MenuView#setTrialListener(listener).

The callback you write will implement the code to respond to onTrialCompleted(), a method of the TrialListener interface. This listener code must do the following:

Note: This means that until the next session is started, the system should not display any menus if the user clicks.

Implementing code to record each trial

The MenuExperimentView class has two methods that still need to be implemented to make the experiment work: startTrialRecording() and endTrialRecording().

startTrialRecording() is called from startSelection() and will start the trial recording only if you are in experiment mode. endTrialRecording() is called from endSelection() and will record the trial data, no matter where the finger is. You do not need to check whether the user clicked on the correct menu item when you call ExperimentTrial#endTrial(). The data will be recorded automatically by ExperimentSession.

Hint: Remember it is always good practice to check for null references!!!

You may notice that AbstractMenuExperiementView.java only keeps one (rather than a List of) TrialListener objects. In other applications, we might keep a List of listeners to allow for more flexiblity in notifying a lot of other objects when an needed. For more on custom listeners, see CodePath’s guide to creating custom listeners.

Part 4: Implement a Custom Menu

Create a new menu! Working off of the provided MenuExperimentView interface, build your own third menu. This menu must be substantially different from the Normal/Pie menus. You are encouraged to explore user interactions here, and you will receive credit as long as your code shows significant effort. Your menu does not have to be great here – if you choose, you can aim to create the “worst menu ever” for this submission. Alternatively, you can aim to outcompete our provided menu views!

Be sure to make use of the same abstractions (essentialGeometry() and a state machine) in this implementation.

Tasks:

Part 5: Conduct a User Study

(Note: this is not due at the same time as the code portion)

Tasks

Conduct User Study

Start by testing your own program as a participant so you know how long it takes to run a session (this information is needed for your consent form). This is also a good time to double check that your data is not lost when you download it.

Make a copy of the consent form form template and modify it in all of the places marked as such (in italics). The language used in your consent form is key, so ensure that it does not come off as forceful or coercive.

If you are interested in collecting demographic information about your participants, you must explicitly ask if the participant wishes to disclose their age/gender/background information in the consent form. If they do not consent, do not assume or discuss their background.

Recruit participants and have them sign your consent form

Recruit your 3 participants from family and friends who can safely use your app on your device or your emulator in person while in person/co-present with you.

Consenting your participants will require that you first print out two copies of the consent form for each participant. Then, for each participant, set up a time when you can speak in person. At the meeting, hand them a copy of the consent form before describing the study. Briefly explain what your user study is about, and ensure they understand their participation in the study is voluntary. Do not coerce anyone into participating in your study. Make sure they know they have a choice, and have read the consent form.

Your participants must acknowledge their consent by filling out the bottom of one of the copies of their form, then “signing” the bottom by writing the words “I consent” and and drawing a picture of a sun next to those words.

Collect data

Once your participant has been consented, allow them to test the app.

Our implementation uses the ExperimentSession object to create, manage, and record experiment data to a .csv file that is stored on the phone or emulator. If you are interested, check out the recordResult function ExperimentSession.java to understand how this is being stored.

Testing can happen on either a phone or other native Android device or on an emulator. If on an emulator, the participant may use a mouse, trackpad or touch display to interact with the app. When collecting data, you should inform users that all interactions will begin with tapping on the center of the screen. This should help avoid cases where your menu is unusable due to the screen format.

Testing Steps

Once all participants are done and you have one .csv file with all of the data, it should have at least ITEM_MAX * NUM_REPARTS * NUM_PARTICIPANTS (324) data points. You will be turning in this file (through Gradescope) as part of your final report.

Downloading user testing data

You can use the Android Studio Device File Explorer to access the file. This allows you to directly access the files on your Android device (emulated or physical) through a GUI window. Detailed instructions on getting the data off your device can be found on this page.

The data will likely be in /storage/emulated/0/CSE340_Menus/TestResult.csv or /sdcard/CSE340_Menus/TestResult.csv.

You may download the data from each individual participant separately and combine the data into one big .csv file, or you may run the experiments sequentially and have the app store all of the data in one file.

Note that each user is given a participant ID when the session is running. These are randomly generated numbers from 0-9999. You may renumber these to identify your participants by the order in which they tested (1, 2, 3) when you retrieve the downloaded .csv file(s).

Part 6: Write a Report

Complete the user study by writing up brief report. We have provided a template for you to use here. Please reference this template as necessary, but the information in the report must be your own descriptions.

Your report will require you to attach screen captures of your menu in both an unselected and selected state. You can capture a screenshot of the menu in action in one of two ways:

Submission Instructions

Video Checkpoint

Partway into this assignment you are to turn in a video (in lieu of code) to Gradescope to demonstrate that you have made a good faith effort to implement Parts 1 and 2. You will receive full credit if you can demonstrate that you can bring up your Normal and Pie menus and interact with them to select an item or two in Test mode. Partial credit will be given if you can demonstrate some of this functionality.

Completed code (Part 1-4)

Remember to continually commit your changes to GitLab (git add/git commit/git push), and then turn in your code the Menus assignment on Gradescope.

Note: we will ONLY be using your code in the following files, do not modify any other files beyond this list:

Testing and Report (Part 5-6)

Your report will be uploaded to Gradescope. This must include:

Grading (50 pts)

This HW will be out of 50 points and will roughly (subject to small adjustments) be distributed as:

Video checkpoint (2 points)

Code portion (24 points)

Data (1 point)

Experiment Portion (23 points)