Interaction Technique Design --Week 7, Wednesday--

name: inverse
layout: true
class: center, middle, inverse
---
layout:false

# Why is selection (and testing) important .red[*]
.left-column50[

![:img False alarm in hawaii that led everyone to think there wsa a nuclear event, 75%,width](img/menus/alert1.jpg)
]
--
.right-column50[
![:img Screen showing what led to this error, 85%,width](img/menus/alert.jpg)
]

.footnote[.red[*] [Washington Post Article](https://www.washingtonpost.com/news/morning-mix/wp/2018/01/16/that-was-no-wrong-button-in-hawaii-take-a-look/?utm_term=.1848969db923)
]
---
class: center, middle, inverse

# Interaction Technique Design

Lauren Bricker

CSE 340 Spring 2021
---
layout: false

[//]: # (Outline Slide)
# Today's Agenda
- Administrivia
  - ColorPicker due at 10pm Thursday
  - Menus to be released today (in a couple of hours)
- Introducing key concepts for menus assignment
- Using key properties of people to predict design outcomes

---
# Reminder: Menus Assignment

Goal: to compare pie menus, linear menus, and a custom menu you make!

We provide support for
- Testing your menus (`TestActivity`)
- Running the experiment (`ExperimentActivity`)

You implement
- a variety of menus
- some testing code

---
# Menus Assignment

.left-column-half[
## Class Hierarchy

<div class="mermaid">
classDiagram

AbstractMainActivity <|-- ExperimentActivity
AbstractMainActivity <|-- TestActivity

AbstractMenuExperimentView <|-- MenuExperimentView
MenuExperimentView <|-- PieMenuView
MenuExperimentView <|-- NormalMenuView
MenuExperimentView <|-- CustomMenuView

class MenuExperimentView {
  essentialGeometry()
  onTouchEvent()
  startSelection()
  endSelection()
  updateModel()
}

</div>
]
.right-column-half[
## Enums
Used for PPS in Menu assignment *and* for experimental conditions. [Documentation](https://docs.oracle.com/javase/tutorial/java/javaOO/enum.html)

<div class="mermaid">
classDiagram

class State {
  START
  SELECTING
}

class MenuType {
  NORMAL
  PIE
  CUSTOM
}

class TaskType {
  LINEAR
  RELATIVE
  UNCLASS
}

</div>

]
???

Enums
- Used for PPS in ColorPicker (PPS States; Essential Geometry)
- Used for PPS in Menu assignment *and* for experimental conditions
- Easy to inspect if you want to (can use to determine possible states, number items, etc)

---

# What is a Menu in theory?

???
- Supports selection of an item from a fixed set of options
- Options are set and determined in advance
- Typically used for “commands” (verbs, things to do)
- Occasionally for selecting a value (e.g., picking a font)

--
count: false
.left-column-half[
- Supports selection of an item from a fixed set of options
- Options are set and determined in advance
- Typically used for “commands” (verbs, things to do)
- Occasionally for selecting a value <br> (e.g., picking a font)

]

--
count: false

.right-column-half[
In ColorPicker we used a pre-defined "hamburger" menu and responded to user
selections through an `onOptionsItemSelected` callback
]
---
# What is a Menu in theory?
.left-column-half[
- Supports selection of an item from a fixed set of options
- Options are set and determined in advance
- Typically used for “commands” (verbs, things to do)
- Occasionally for selecting a value <br> (e.g., picking a font)

Now we will create our own menu interactors!!!
]
--
.right-column-half[
<div class="mermaid">
stateDiagram-v2

[*] --> Selecting : Press/<BR>startSelection()
Selecting --> [*]: Release/<BR>endSelection()
Selecting --> Selecting: Drag/<BR>updateModel()
</div>
]

???

Implemented in MenuExperimentView

Works in all Menus!

---

# What is a Menu in theory?

.left-column-half[
- Supports selection of an item from a fixed set of options
- Options are set and determined in advance
- Typically used for “commands” (verbs, things to do)
- Occasionally for selecting a value <br> (e.g., picking a font)

Implemented in MenuExperimentView

Works in all Menus!
]

.right-column-half[
<div class="mermaid">
stateDiagram-v2

[*] --> Selecting : Press/<BR>startSelection()
Selecting --> [*]: Release/<BR>endSelection()
Selecting --> Selecting: Drag/<BR>updateModel()
</div>
]

---
# Aside: Custom Listeners

- Used in ColorPicker and Menus

- Why create custom listeners?

--
 - Lets you execute code when the view's model has changed
 - No other way to know that has happened

---
# Review: How to implement custom listeners

The Custom View needs to
- define the custom interface
- keep track of listeners

--
Anything *using* the view needs to
- implement the interface (specifically, the method in that interface that will be called)
- register itself as a listener with the view

---
# Example Custom View -- ColorPicker

In ColorPicker, we setup the Custom View side for you

```java
    // Currently registered ColorListener instance or null.
    private List<ColorChangeListener> mColorChangeListeners;

// Class which defines a listener to be called when a new color is selected.
    public interface ColorChangeListener {
        void onColorSelected(@ColorInt int color);
    }

// Registers a new listener
    public final void addColorChangeListener(@NonNull ColorChangeListener colorChangeListener) {
        mColorChangeListeners.add(colorChangeListener);
    }
```

---
# Example Custom Listener -- ColorPicker:

You implemented this

```java
// TODO: Register callback to update {color,label}
// View when color changed.`
```
.left-column60[
What method do we call to register the callback?  <br>`addColorChangeListener()`

]
.right-column40[
<div class="mermaid">
classDiagram
MainActivity ..> ColorPickerView : addColorChangeListener()
MainActivity --> ColorChangeListener : Contains
class ColorChangeListener {
  onColorChanged()
}
class ColorPickerView {
  addColorChangeListener()
}
</div>
]
---
# Example Custom Listener -- ColorPicker:

You implemented this

```java
// TODO: Register callback to update {color,label}
// View when color changed.`
```
.left-column60[
What method do we call to register the callback?  `addColorChangeListener()`

What do we usually do in a callback? <br>Update the application's (`MainActivity`) model
]
.right-column40[
<div class="mermaid">
classDiagram
ColorPickerView ..> ColorChangeListener : onColorChanged()
ColorChangeListener ..> MainActivity : update model

class ColorChangeListener {
  onColorChanged()
}

class MainActivity {
  ColorPickerView view
}
</div>
]

---
# Registering a TrialListener in Menus

In `ExperimentActivity` to record the results of a trial

```java
// TODO: register a TrialListener with the menu:
// TODO: When the user completes a trial, the menu listener should take care of the following:
// TODO: 1. Write the result to your CSV file. This should be accomplished with
//          ExperimentSession.recordResult().
// TODO: 2. show the menu for the next trial, if there is another trial available.
// TODO: 3. If another trial is not available, announce with a Toast and change the text in the
//          instruction view to say the session is completed by using R.string.session_completed
//          from strings.xml. For completeness, don’t forget to reset the session (to null)
//          if you are done with all of the sessions.
```

---
class: center, middle, inverse

# Experimenting with Interfaces

Important part of building interfaces is experimenting

Need structured ways to decide what's better

---
# Experiments should be tied to *hypotheses* based on *theories* what will go better

- Fitts Law (compare distance and size; check for infinite size)
- Steering Law (distance and size over a path)
- Cognitive modeling (complex multi-step model of expert behavior)
 - Does someone have to 'check' something? More than once?
 - Do they have to move? More than once
- Gestalt Psychology (will they see it at all?)
- Errors (will they be reduced)

???
why theory and not intuition?

---
# Which is better and why?
.left-column50[
## A. Pie Menu
![:img Pie Menu, 80%,width](img/menus/pie.jpg)
]
.right-column50[
## B. Pull down Menu
![:img Traditional Menu inside a web browser, 80%,width](img/menus/menu.png)
]
???

What analysis methods can we use to predict?

- Fitts Law (compare distance and size; check for infinite size)
- *Steering Law* (distance and size over a path)
- Cognitive modeling (complex multi-step model of expert behavior)
 - Does someone have to 'check' something? More than once?
 - Do they have to move? More than once
- Gestalt Psychology (will they see it at all?)
- Errors (will they be reduced)

---
# Steering Law (based on Fitts' law)
.left-column50[
## A. Pie Menu
![:img Pie Menu, 80%,width](img/menus/pie.jpg)
]
.right-column50[
## B. Pull down Menu
![:img Traditional Menu inside a web browser, 80%,width](img/menus/menu.png)
]

---
# Which is better and why?
.left-column50[
## A. Pie Menu
![:img Pie Menu, 80%,width](img/menus/pie.jpg)
]
.right-column50[
## B. Marking Menu
![:youtube Video assigned before class, 8c58bN6ajJ4?t=30]
]
???

- Fitts Law (compare distance and size; check for infinite size)
- Steering Law (distance and size over a path)
- Cognitive modeling (complex multi-step model of expert behavior)
 - **Does someone have to 'check' something? More than once?**
 - **Do they have to move? More than once**
- Gestalt Psychology (will they see it at all?)
- Errors (will they be reduced)
---
# Cognitive modeling (less double checking)

.left-column50[
## A. Pie Menu
![:img Pie Menu, 80%,width](img/menus/pie.jpg)
]
.right-column50[
## B. Marking Menu
![:youtube Video assigned before class, 8c58bN6ajJ4?t=30]
]

---
# Which is better and why?

A: Tapping    B: Crossing
![:youtube Video of using crossing for selection, kp2Zl4ONuik]

???

- **Fitts Law (compare distance and size; check for infinite size)**
- Steering Law (distance and size over a path)
- Cognitive modeling (complex multi-step model of expert behavior)
 - Does someone have to 'check' something? More than once?
 - Do they have to move? More than once
- Gestalt Psychology (will they see it at all?)
- Errors (will they be reduced)

---
# Fitts' Law (infinite width)

.left-column60[
A: Tapping    B: Crossing

![:img screen capture of the video showing both a tap target and a swipe, 100%, width](img/menus/tapping-crossing.png)
]
.right-column40[
Examples of crossing
- [Fruit Ninja](https://www.halfbrick.com/games/fruit-ninja)
- Swipe keyboard interactions
- Swiping to skip something, like a song in Spotify
]
---
# Back to comparing menus

Kurtenbach: What did they study here?

![:youtube Illustration of advantages of marking menus,dtH9GdFSQaw]

---
# How do we prove our theories? Hypothesis

<div class="mermaid">
graph LR
S((.)) --> Hypothesis(Hypothesis)

classDef finish outline-style:double,fill:#d1e0e0,stroke:#333,stroke-width:2px;
classDef normal fill:#e6f3ff,stroke:#333,stroke-width:2px;
classDef start fill:#d1e0e0,stroke:#333,stroke-width:4px;
classDef invisible fill:#FFFFFF,stroke:#FFFFFF,color:#FFFFFF

linkStyle 0 stroke-width:4px;

class S invisible
class Hypothesis start
</div>

Discuss

Marking Menus are better than Pie Menus

--
Marking Menus are *faster* than Pie Menus

--
Pie Menus are faster than Linear Menus for less than 8 items that have no obvious order

---
.left-column-half[
# What conditions does this experiment have?

`MenuType`

`TaskType` (could be a condition)
]
.right-column-half[
# What might we measure? (from POP-Motor)
]
--
.right-column-half[
- Time on Task
- Accuracy
- How strenuous
- Recall
- Emotional Response

]

---
# How do we prove our theories? Method

<div class="mermaid">
graph LR
S((.)) --> Hypothesis(Hypothesis)
Hypothesis -- "Study Design" --> Method(Method)

linkStyle 0 stroke-width:4px;
linkStyle 1 stroke-width:4px;

class S invisible
class Hypothesis start
class Method normal
</div>

Compare specific conditions

| MenuType | 8 items | 12 items |
|----------|---------|----------|
| Linear   |         |          |
| Pie      |         |          |
| Marking  |         |          |

What if we wanted to do length AND ordered vs not?

Want to consider every combination, or run twice (menuType x length and menuType x ordering)

---
# How do we prove our theories? Data

<div class="mermaid">
graph LR
S((.)) --> Hypothesis(Hypothesis)
Hypothesis -- "Study Design" --> Method(Method)
Method -- "Run Study" --> Data(Data)

linkStyle 0 stroke-width:4px;
linkStyle 1 stroke-width:4px;
linkStyle 2 stroke-width:4px;

class S invisible
class Hypothesis start
class Method,Data normal
</div>

What might we want to measure?

--
count:false
- Time on task -- how long to complete basic tasks? (For example, find something to buy, create a new account, and order the item.)
--
count:false
- Accuracy -- How many mistakes did people make? (And were they fatal or recoverable with the right information?)
--
count:false
- How strenuous (e.g. gaze has lower throughput but is less strenuous than head pointing (Mackenzie 2018)
--
count:false
- Recall -- How much does the person remember afterwards or after periods of non-use?
--
count:false
- Emotional Response -- How does the person feel about the tasks completed? (Confident? Stressed? Would the user recommend this system to a friend?)

???
Build up a list of good UI design principals from these basics

Undo
Predictability
... What is missing? (e.g. fun)
---
# How to get such data?

- Video
- Timestamps
- Notes
- Can transcribe and analyze interviews with users
- Can look for patterns across users

---
# How do we prove our theories? Analysis

<div class="mermaid">
graph LR
S((.)) --> Hypothesis(Hypothesis)
Hypothesis -- "Study Design" --> Method(Method)
Method -- "Run Study" --> Data(Data)
Data -- "Clean and Prep" --> Analysis(Analysis)

linkStyle 0 stroke-width:4px;
linkStyle 1 stroke-width:4px;
linkStyle 2 stroke-width:4px;
linkStyle 3 stroke-width:4px;

class S invisible
class Hypothesis start
class Method,Data,Analysis normal
</div>

![:img sample data from the menus experiement, 80%,width](img/menus/menus-data.png)

---
# How do we prove our theories? Conclusions

<div class="mermaid">
graph LR
S((.)) --> Hypothesis(Hypothesis)
Hypothesis -- "Study Design" --> Method(Method)
Method -- "Run Study" --> Data(Data)
Data -- "Clean and Prep" --> Analysis(Analysis)
Analysis --> Conclusions(Conclusions)

linkStyle 0 stroke-width:4px;
linkStyle 1 stroke-width:4px;
linkStyle 2 stroke-width:4px;
linkStyle 3 stroke-width:4px;
linkStyle 4 stroke-width:4px;

class S invisible
class Hypothesis,Conclusions start
class Method,Data,Analysis normal
</div>

![:img sample data from the menus experiement, 30%,width](img/menus/histogram.png)

---
# Design an experiment

linkStyle 0 stroke-width:4px;
linkStyle 1 stroke-width:4px;
linkStyle 2 stroke-width:4px;
linkStyle 3 stroke-width:4px;
linkStyle 4 stroke-width:4px;

class S invisible
class Hypothesis,Conclusions start
class Method,Data,Analysis normal
</div>

.left-column50[
## A.
![:img Picture of the Graffiti gesture recognition alphabet from the Palm Pilot, 100%,width](img/menus/Grafitti.png)
]
.right-column50[
## B.
![:img Picture of the Edgewrite gesture recognition alphabet, 50%,width](img/menus/Edgewrite.png)
]

???
Hypothesis: Errors (will they be reduced)
Method:??
Data:??
Analysis??
Conclusions??

---
# Design an experiment

linkStyle 0 stroke-width:4px;
linkStyle 1 stroke-width:4px;
linkStyle 2 stroke-width:4px;
linkStyle 3 stroke-width:4px;
linkStyle 4 stroke-width:4px;

class S invisible
class Hypothesis,Conclusions start
class Method,Data,Analysis normal
</div>

.left-column50[
## A. Pie Menu
![:img Pie Menu, 80%,width](img/menus/pie.jpg)
]
.right-column50[
## B. Pull down Menu
![:img Traditional Menu inside a web browser, 80%,width](img/menus/menu.png)
]

???

Hypothesis: Errors will be reduced;
Hypothesis: Motion will be faster due to low level motor things
Hypothesis: fewer cognitive checks needed
Method:??
Data:??
Analysis??
Conclusions??