as5: Menus

Last revised: 9:00pm, Sunday, May 10th, 2020
Assigned:
  • May 12th, 2020
Due:
  • Part 1-4 (programming part)
    - Due May 21th, 2020
    - Lock May 22th, 2020
  • Part 5-6 (analysis part)
    - Due May 25th, 2020
    - Lock May 27th, 2020

Android Goals:

  • Further practice using callbacks; `onDraw()`
  • Further practice with translate
  • Gain more experience working with callbacks.
  • Further practice with state machines and event handling

HCI Goals:

  • Record user study data
  • Produce plausible experiment results (within expected ranges)
  • Understand pie menus
  • Design your own interactor
  • Collect, parse, and interpret results collected from a study.

GitGrade links

Accept the Assignment / Turn-in the Assignment / Review your Submissions

Overview of assignment

This assignment has two main components.

First you will be creating three menus – a linear menu, a pie menu, and a custom menu that you design. The programming re-uses many concepts from ColorPicker, but will have more pieces to it. Please start early, and reach out if you find you are spending more than 10 hours.

After your application is turned in, you will test these menus in an experiment with others. The user study will include the following:

Note: If you are unable to finish parts 1-4, you can borrow a phone with working code installed we will send you an APK after the lock date to finish parts 5-6.

Video of completed assignment

Note the Toasts (pop up messages) are missing a space between the word “Selected” and the item that was selected. This has been fixed in the current version of the code, but the rest of the app works substantively the same.

Part 1: Overview

Tasks

You will be implementing a class that can display a pie menu, a class that can display a linear menu, and a custom menu of your design, for use in an experiment comparing which is faster for the user to select menu items from. By menu items we mean the individual items in a menu, such as the number 1 in the sample video.

Overview of code structure

Files that you will be changing

Classes that have functions and fields you may need (your classes inherit from them)

The structure of this code is represented by the Unified Modeling Language (UML) diagram shown below.

classDiagram class MenuExperimentView { onTouchEvent() startSelection() endSelection() updateModel() onDraw() } AbstractMenuExperimentView <|-- MenuExperimentView AbstractMainActivity <|-- MainActivity AbstractMainActivity <|-- TestActivity MenuExperimentView <|-- PieMenuView MenuExperimentView <|-- NormalMenuView MenuExperimentView <|-- CustomMenuView class TaskType { LINEAR RELATIVE UNCLASS } class State { START SELECTING } class MenuType { NORMAL PIE CUSTOM }

Other files you may want to look at

Overview of Experiment

In this experiment we will have 9 conditions, each of which will include the number of trials (where totalTrials = NUM_REPEATS * ITEM_MAX trials). You can think of this as a three dimensional array

  Menus    
Tasks PIE NORMAL CUSTOM
LINEAR totalTrials totalTrials totalTrials
RELATIVE totalTrials totalTrials totalTrials
UNCLASS totalTrials totalTrials totalTrials


The three menu types (specified in an Enum in the code) are PIE (round), NORMAL (linear) and CUSTOM (your custom menu).

The three task types (also an Enum) are LINEAR (menu items that have a natural order such as 1,2,4,8,16,32,64,128); RELATIVE (menu items that have a relationship to each other, such as Up/UpRight/Right/Down/etc…); and UNCLASS (menu items that have no relationship or ordering, such as Print/Cast/Bookmark/etc);

The trials are specific combinations of menu item, task type, and menu type. So for example, one trial might involve showing the user a pie menu, with the numbers 1..128 in it, and asking them to select number 4. The participant can select any option, no matter what as soon as they complete a selection, the result is recorded and the next trial begins.

For any given session (one user’s worth of trials), each combination of menu type, task type, and menu item, will be repeated NUM_REPEATS times (NUM_REPEATS is specified in ExperimentSession.java). In addition, for any specific combination of menu type and task type, at most ITEM_MAX items (a total of ITEM_MAX*NUM_REPEATS trials) will be shown to the user. If you set both of these to 1 for testing, you can test all your conditions very quickly. Make sure to set them to the required values for user testing: ITEM_MAX must be at least 4 and NUM_REPEATS must be 3.

After the code for this project is submitted, you will recruit at least 3 family, friends, or other students and have them complete one session each. Thus, you will have a total of 3 (participant sessions) * 9 (conditions) * ITEM_MAX * NUM_REPEATS, or at least 324, data points when you have completed this assignment. You will analyze the data from this experiment in Part 5 of this project.

We have implemented in ExperimentSession the code for you to generate all of the trials from a setup file called menuContents.csv found in the assets directory. Make sure that you understand the createTrials() method provided in ExperimentSession.java which sets up conditions for the whole experiment. ExperimentSession is an iterator, so to run the trials for a given session you just use session.next() as long as session.getNext() is true.

Note: that we give the user feedback two ways.

Part 2: Implement MenuExperimentView and MainActivity

Tasks

For this part, you will be working in MenuExperimentView.java and MainActivity.java. This section will entail defining the functions which power your menu experiment. The MenuExperimentView class includes several methods that you will need to implement. You will implement the state machine logic in onTouchEvent (similar to what you’ve done in ColorPicker).

One tricky thing about context menus – they can appear anywhere in your user interface. To support this, we set things up so that the menu view is going to MATCH_PARENT width and height (i.e. its bounding box is the whole screen). However, the menu itself should show up right where the user presses down. To make this easier for your menus, you will translate the canvas so that they can draw themselves in a logical location given where the finger pressed in onDraw(). Then you will call drawMenu() with this translated canvas, which is what each specific menu implements to draw itself properly.

Handling Touch Events

You will handle touch input by implementing the onTouchEvent method. This is the event handler that is invoked when a touch occurs in this view. The state machine defined in onTouchEvent should work, without changes, for every menu you implement. The only menu-specific code is in essentialGeometry() (it finds the current index). Thus onTouchEvent() is implemented in the parent class, MenuExperimentView and is not changed in the child classes.

MenuExperimentView’s implementation of onTouchEvent makes use of your menu’s essentialGeometry() function to determine the relative position of the user’s finger. Essential geometry will be passed the finger’s position relative to the menu’s (0,0), i.e. where the user pressed. You must implement your linear menu so this is its top left corner, and your pie menu so that the user’s finger is in the center of the pie.

You MUST NOT perform any assignments within essentialGeometry(), as it violates the utility of that method. Instead, essentialGeometry() will return the index associated with the desired view, allowing the calling method to act on that value.

You need to keep track of two main states: START and SELECTING. When in the SELECTING state you need to distinguish between the event type to determine if the user has selected an option or if they are still in the middle of making a choice.

Relevant touch events include ACTION_DOWN, ACTION_MOVE, and ACTION_UP; think about how these touch events relate to the change and how the UI must respond to these events.

More details about essentialGeometry(), specific to each menu, are in descriptions for those menus in Part 3.

graph LR S((.)) --> A((Start)) A -- "Press:startSelection()" --> I(Selecting) I -- "Release:endSelection()" --> E[End] I -- "Drag:updateModel()" --> I classDef finish outline-style:double,fill:#d1e0e0,stroke:#333,stroke-width:2px; classDef normal fill:#e6f3ff,stroke:#333,stroke-width:2px; classDef start fill:#d1e0e0,stroke:#333,stroke-width:4px; classDef invisible fill:#FFFFFF,stroke:#FFFFFF,color:#FFFFFF class S invisible class A start class E finish class I normal

Details for startSelection(), endSelection() and updateModel() can be found in MenuExperimentView. Note that there are no guards (boolean logic regarding the essential geometry) on these transitions – any click inside the MenuExperimentView causes a menu to appear, and once a menu appears, dragging causes updateModel() to be called no matter where the finger is. Similarly, the menu must be removed when selection ends, and the trial recorded, no matter where the finger is. You do not need to check whether the user clicked on the correct menu item when you call endTrial(). This data will be recorded automatically by ExperimentSession.

Drawing your Menu

Your menu must ONLY BE VISIBLE when you are in the SELECTING state. You can check this in onDraw(). In addition, it needs to redraw whenever something important changes (like the selected item). Be sure to call invalidate() in the appropriate methods of the state machine to make this happen.

For onDraw(), your will do some setup that will allow your subclasses to draw properly. First, you will need to check if you are in the proper state to draw (you must only draw when in the SELECTING state).

Next, you really want to draw from the (0,0) location of the menu, rather than the (0,0) of the parent. But remember that your width and height are set using MATCH_PARENT. To fix this, we need to translate the coordinate system of the canvas so that (0,0) is at the startPoint for this interaction.

Finally, since this is an abstract class, we need to call the real drawing method, which our children will implement. This is called drawMenu() and must be overridden by PieMenuView, NormalMenuView, and CustomMenuView. Therefore, your onDraw() implementation in MenuExperimentView.java will be pretty short.

Implementing MainActivity

In MainActivity you will implement part of the method showMenuForTrial(ExperimentTrial trial). Your main job is to register a callback with it that knows what to do when a trial is completed.

MainActivity also needs to implement the code to respond to onTrialCompleted(), a method of the TrialListener interface. In particular, this code must always remove the current menu being shown. Then it must check if the session is over (remember the session is an iterator), and if not call showMenuForTrial() with the next trial. If the session is over, it must update text of InstructionTextView to say that the session is completed, and also display a Toast to the same effect. We have provided a method that you may use to announce this message (AbstractMenuExperimentView#announce(String)), or you may use the Toast class directly.

Note: This means that until the next session is started, the system should not display any menus if the user clicks.

Related APIs

Part 3: Implement Pie and Normal Menus

Tasks

Both PieMenuView and NormalMenuView extend MenuExperimentView. You will implement essentialGeometry() for each and determine what menu item the touch event maps to. You’ll have to come up with the math logic to map from touch event to an item index.

You will also implement drawMenu() which at a minimum draws the menus as shown in the screenshots. If you want to do something different, you may, as long as the size and position of each menu item does not change. For example, you can override the paint properties defined in MenuExperimentView, position the text differently, or draw more decorations on the menus.

Be sure to check the AbstractMenuExperimentView starter code for any additional values that you must use when creating your menus. Also make sure you are not “shadowing” variables (redefining variables already defined in super class).

NormalMenuView

The Normal Menu will appear where the user touches the screen (as the top/left corner of the menu) and extend down and to the right.

For the NormalMenuView, essentialGeometry() will return -1 if the pointer is outside the bounds of the menu in any direction or has moved less than MIN_DIST (inherited from AbstractMenuExperimentView) since interaction started. Otherwise it must return the “item number” (position in the list) corresponding to the menu item that the pointer is currently inside of.

The width of each item in the normal menu is CELL_WIDTH and the height is CELL_HEIGHT.

Some additional hints for the normal menu

PieMenuView

The Pie Menu will appear centered where the user touches the screen.

For the PieMenuView, essentialGeometry() will return -1 if the pointer has moved less than MIN_DIST since selection started, and otherwise it will return the “item number” (position in the list) corresponding to the menu item currently being selected. In the case of the Pie menu, the item number is the menu item that a ray from the center of the menu to the pointer intersects, with the first item being at the top of the pie and increasing as you go clockwise around the circle. The pie menu has no maximum diameter, so anything outside of the radius MIN_DIST around the finger will return a selection.

For PieMenuView, the given RADIUS corresponds to the outer radius. The inner radius is defined by RADIUS - TEXT_SIZE * 2.

Some additional hints for the pie menu

Related APIs

Part 4: Implement a Custom Menu

Create a new menu! Working off of the provided MenuExperimentView interface, build your own third menu. This menu must be different from the Pie/Normal menus. You are encouraged to explore user interactions here, and you will receive credit as long as your code shows significant effort. Your menu does not have to be great here – if you choose, you can aim to create the “worst menu ever” for this submission. Alternatively, you can aim to outcompete our provided menu views!

Be sure to make use of the same abstractions (essentialGeometry() and a state machine) in this implementation.

Part 5: Conduct and Write Up User Study

(not due at the same time as the code portion)

Tasks

Conduct User Study

Start by testing your own program as a participant so you know how long it takes to run a session (this information is needed for your consent form). This is also a good time to double check that your data is not lost when you download it.

Make a copy of the consent form form template and modify it in all of the places marked as such (in italics). The language used in your consent form is key, so ensure that it does not come off as forceful or coercive.

Recruit participants and have them sign your consent form

Recruit your 3 participants from family and friends who can safely use your app on your device or your emulator in person (co-present), or from the random group you have been assigned from class. Please see this document for your group members.

Consenting your participants will be very similar whether they are co-present or remote. First, set up a time when can speak to your participant (in person, over the phone, or over a Zoom or other video conferencing call).

Briefly explain what your user study is about, and ensure they understand their participation in the study is voluntary. Do not coerce anyone into participating in your study. Make sure they know they have a choice, and have read the consent form.

Your participants must acknowledge their consent by “signing” via the Google form (linked at the bottom of the consent form template). This form should send you an anonymized copy of their consent in email, which you can save and turn in with the rest of your report and reflection.

Collect data

Once your participant has been consented, allow them to test the app.

Our implementation uses the ExperimentSession object to create, manage, and record experiment data to a .csv file that is stored on the phone or emulator. If you are interested, check out the recordResult function ExperimentSession.java to understand how this is being stored.

Testing can happen on either a phone or other native Android device or on an emulator. If on an emulator, the participant may use a mouse, trackpad or touch display to interact with the app. When collecting data, you should inform users that all interactions will begin with tapping on the center of the screen. This should help avoid cases where your menu is unusable due to the screen format.

Co-present Testing

Remote Testing

Once all participants are done and you have one .csv file with all of the data, it should have at least ITEM_MAX * NUM_REPARTS * NUM_PARTICIPANTS (324) data points. You will be turning in this file (through Canvas) as part of your final report.

Downloading user testing data

You can use the Android Studio Device File Explorer to access the file. This allows you to directly access the files on your Android device (emulated or physical) through a GUI window. Detailed instructions on getting the data off your device can be found on this page.

The data will likely be in /storage/emulated/0/CSE340_Menus/TestResult.csv or /sdcard/CSE340_Menus/TestResult.csv.

Note that each user is given a participant ID when the session is running. These are randomly generated numbers from 0-9999. You may renumber these to identify your participants by the order in which they tested (1, 2, 3) when you retrieve the downloaded .csv file(s).

Write a Report

Complete the user study by writing up brief report. We have provided a template for you to use here. Please reference this template as necessary, but the information in the report must be your own descriptions.

Your report will require you to attach screen captures of your menu in both an unselected and selected state. You can capture a screenshot of the menu in action in one of two ways:

Part 6: Reflection

Your reflection should be done as Microsoft Word, Google or other type of document. Copy the following questions (in italics below) into your refelction document and add your responses below each question. You can have more than one answer per page, but if you can, please try to avoid page breaks in the middle of a question. Insert page breaks between questions as needed.

  1. Attach two screenshots of your custom menu, one where it is in a selected state and one where it is in an unselected state. These images may be the same as the ones you included in this report. Then re-read our Properties of People Design Tips for Vision and Motor. Which of these design tips did you employ/not employ when creating your Custom menu? List at least one Visual design tip and one Motor design tip and be specific in describing what part(s) of your menu’s operation utilized these tips.

  2. Given the structure of our Pie/Normal menus, how would you create an accessible menu when working with the MenuExperimentView class? What could make ensuring the menu is accessible difficult? (Rephrasing that last question: “If you were trying to ensure that your menu is accessible, what aspects would be difficult?”)

  3. As written, these menus can potentially appear all or partially off the phone screen. When would this case happen, how might that appear, and how could you account or prevent this from occurring? Would your solution work for all three types of menus?

Submission Instructions

You will change ONLY the following files. These will be submitted to GitGrade.

Part 1-4:

Part 5-6:

Your reflection and report will be uploaded to Gradescope. This will include:

Your participants must also fill out the on line (Google) consent form acknowledging that they agree to be a participant and identifying you as a tester.

You must also turn in your combined csv file so we can create one large data set. This .csv will be turned in through Canvas and NOT be put turned in with the rest of your report.

Grading Rubric

This HW will be out of 68-70 points and will roughly (subject to small adjustments) be distributed as:

Code portion (33-35 points)

Experiment Portion (35 points)