Interaction Technique Design --Week 7, Wednesday--

name: inverse
layout: true
class: center, middle, inverse
---
layout:false

# Why is selection (and testing) important .red[*]
.left-column50[

![:img False alarm in hawaii that led everyone to think there wsa a nuclear event, 75%](img/menus/alert1.jpg)
]
--
.right-column50[
![:img Screen showing what led to this error, 85%](img/menus/alert.jpg)
]

.footnote[.red[*] [Washington Post Article](https://www.washingtonpost.com/news/morning-mix/wp/2018/01/16/that-was-no-wrong-button-in-hawaii-take-a-look/?utm_term=.1848969db923)
]
---
class: center, middle, inverse

# Interaction Technique Design

Lauren Bricker

CSE 340 Spring 2020
---
layout: false

[//]: # (Outline Slide)
# Today's Agenda
- Administrivia
  - ColorPicker due at 10pm
  - Practice quizzes (there are 2) due Wednesday 10pm
  - Menus to be released tonight
  - Examlet Friday
- Introducing key concepts for menus assignment
- Using key properties of people to predict design outcomes

---
# Menus Assignment

Goal: to compare pie menus, linear menus, and a custom menu you make!

We provide support for running the experiment in MainActivity (and a testing harness in TestActivity)

You implement a variety of menus

---
# Menus Assignment

<div class="mermaid">
classDiagram

class MenuExperimentView {
  onTouchEvent()
  startSelection()
  endSelection()
  updateModel()
  onDraw()
}

AbstractMenuExperimentView <|-- MenuExperimentView

</div>

---

# What is a Menu in theory?

???
- Supports selection of an item from a fixed set of options
- Options are set and determined in advance
- Typically used for “commands” (verbs, things to do)
- Occasionally for selecting a value (e.g., picking a font)

--
.left-column-half[
- Supports selection of an item from a fixed set of options
- Options are set and determined in advance
- Typically used for “commands” (verbs, things to do)
- Occasionally for selecting a value <br> (e.g., picking a font)
]
--
<div class="mermaid">
graph TD
S((.)) --> A(Start)
A -- "Press?startSelection()" --> I(Selecting)
I -- "Release:endSelection()" --> E[End]
I -- "Drag:updateModel()" --> I

classDef finish outline-style:double,fill:#d1e0e0,stroke:#333,stroke-width:2px;
classDef normal fill:#e6f3ff,stroke:#333,stroke-width:2px;
classDef start fill:#d1e0e0,stroke:#333,stroke-width:4px;
classDef invisible fill:#FFFFFF,stroke:#FFFFFF,color:#FFFFFF

class S invisible
class A start
class E finish
class I normal
</div>

???

Implemented in MenuExperimentView

Works in all Menus!

---

# What is a Menu in theory?

.left-column-half[
- Supports selection of an item from a fixed set of options
- Options are set and determined in advance
- Typically used for “commands” (verbs, things to do)
- Occasionally for selecting a value <br> (e.g., picking a font)

Implemented in MenuExperimentView

Works in all Menus!
]

<div class="mermaid">
graph TD
S((.)) --> A(Start)
A -- "Press?startSelection()" --> I(Selecting)
I -- "Release:endSelection()" --> E[End]
I -- "Drag:updateModel()" --> I

class S invisible
class A start
class E finish
class I normal
</div>

---
# Aside: Enums

.left-column-half[
Group of named constants

- Used for PPS in ColorPicker (PPS States; Essential Geometry)

- Used for PPS in Menu assignment *and* for experimental conditions

- Easy to inspect if you want to (can use to determine possible states, number items, etc)

[Documentation](https://docs.oracle.com/javase/tutorial/java/javaOO/enum.html)
]
.right-column-half[
<div class="mermaid">
classDiagram

class TaskType {
  LINEAR
  RELATIVE
  UNCLASS
}

class State {
  START
  SELECTING
}

class MenuType {
  NORMAL
  PIE
  CUSTOM
}

</div>

]

---
# How do we draw a menu?

.left-column30[
<br>
<div class="mermaid">
classDiagram

MenuExperimentView <|.. PieMenuView :  onDraw translates  and calls drawMenu

class PieMenuView {
  Canvas: 0,0 at first touch point.
  Clipping rect: Whole screen
  drawMenu()
}

class MenuExperimentView {
   Canvas : 0,0 at top left of screen;
   Clipping rect: Whole screen
   onDraw() translate
}
</div>
]

.right-column60[
- Important! MenuExperimentView set to `MATCH_PARENT`
  - Makes drawing tricky!
- Translate (like a parent view does for children)
- `onDraw` in `MenuExperimentView` calls `drawMenu` in child classes.
]

--
.right-column60[

Do we have to rotate/translate the `Canvas` back?
]

.right-column60[
We don't strictly have to rotate or translate back because this is
an inheritance drawing trick, not an interactor hierarchy drawing implementation
]

---
# Aside: Custom Listeners

- Used in ColorPicker and Menus

- Why create custom listeners?

--
 - Lets you execute code when the view's model has changed
 - No other way to know that has happened

---
# Review: How to implement custom listeners

The Custom View needs to
- define the custom interface
- keep track of listeners

--
Anything *using* the view needs to
- implement the interface (specifically, the method in that interface that will be called)
- register itself as a listener with the view

---
# Example Custom View -- ColorPicker

In ColorPicker, we setup the Custom View side for you

```java
    // Currently registered ColorListener instance or null.
    private List<ColorChangeListener> mColorChangeListeners;

// Class which defines a listener to be called when a new color is selected.
    public interface ColorChangeListener {
        void onColorSelected(@ColorInt int color);
    }

// Registers a new listener
    public final void addColorChangeListener(@NonNull ColorChangeListener colorChangeListener) {
        mColorChangeListeners.add(colorChangeListener);
    }
```

---
# Example Custom Listener -- ColorPicker:

You implemented this

.left-column-half[
```
// TODO: Register callback to update {color,label}
// View when color changed.`
```

What method do we call to register the callback?  <br>`addColorChangeListener()`

]
.right-column-half[
<div class="mermaid">
classDiagram
MainActivity ..> ColorPickerView : addColorChangeListener()
MainActivity --> ColorChangeListener : Contains
class ColorChangeListener {
  onColorChanged()
}
class ColorPickerView {
  addColorChangeListener()
}
class MainActivity {
  ColorPickerView view
}

</div>
]
---
# Example Custom Listener -- ColorPicker:

You implemented this

.left-column-half[
```
// TODO: Register callback to update {color,label}
// View when color changed.`
```

What method do we call to register the callback?  `addColorChangeListener()`

What do we usually do in a callback? <br>update application's (`MainActivity`) model
]
.right-column-half[
<div class="mermaid">
classDiagram
ColorPickerView ..> ColorChangeListener : onColorChanged()
ColorChangeListener ..> MainActivity : update model

class ColorChangeListener {
  onColorChanged()
}

class MainActivity {
  ColorPickerView view
}
</div>
]

---
# You will do this yourself in Menus

```java
// TODO: register a new listener with the menu so that the application knows when a selection is made

// TODO: implement the listener. When the user completes a trial, the menu listener should store
// the results of the trial, and setup for the next trial
```

---
class: center, middle, inverse

# Experimenting with Interfaces

Important part of building interfaces is experimenting

Need structured ways to decide what's better

---
# Experiments should be tied to *hypotheses* based on *theories* what will go better

- Fitts Law (compare distance and size; check for infinite size)
- Steering Law (distance and size over a path)
- Cognitive modeling (complex multi-step model of expert behavior)
 - Does someone have to 'check' something? More than once?
 - Do they have to move? More than once
- Gestalt Psychology (will they see it at all?)
- Errors (will they be reduced)

???
why theory and not intuition?

---
# Which is better and which laws explain it?
.left-column50[
## A. Pie Menu
![:img Pie Menu, 80%](img/menus/pie.jpg)
]
.right-column50[
## B. Pull down Menu
![:img Traditional Menu inside a web browser, 80%](img/menus/menu.png)
]
???

What analysis methods can we use to predict?

- Fitts Law (compare distance and size; check for infinite size)
- *Steering Law* (distance and size over a path)
- Cognitive modeling (complex multi-step model of expert behavior)
 - Does someone have to 'check' something? More than once?
 - Do they have to move? More than once
- Gestalt Psychology (will they see it at all?)
- Errors (will they be reduced)

---
# Steering Law (based on Fitts' law)
.left-column50[
## A. Pie Menu
![:img Pie Menu, 80%](img/menus/pie.jpg)
]
.right-column50[
## B. Pull down Menu
![:img Traditional Menu inside a web browser, 80%](img/menus/menu.png)
]

---
# Which is better and which law explains it?
.left-column50[
## A. Pie Menu
![:img Pie Menu, 80%](img/menus/pie.jpg)
]
.right-column50[
## B. Marking Menu
![:youtube Video assigned before class, 8c58bN6ajJ4?t=30]
]
???

- Fitts Law (compare distance and size; check for infinite size)
- Steering Law (distance and size over a path)
- Cognitive modeling (complex multi-step model of expert behavior)
 - **Does someone have to 'check' something? More than once?**
 - **Do they have to move? More than once**
- Gestalt Psychology (will they see it at all?)
- Errors (will they be reduced)
---
# Cognitive modeling (less double checking)

.left-column50[
## A. Pie Menu
![:img Pie Menu, 80%](img/menus/pie.jpg)
]
.right-column50[
## B. Marking Menu
![:youtube Video assigned before class, 8c58bN6ajJ4?t=30]
]

---
# Which is better and which laws explain it?

A: Tapping    B: Crossing
![:youtube Video of using crossing for selection, kp2Zl4ONuik]

???

- **Fitts Law (compare distance and size; check for infinite size)**
- Steering Law (distance and size over a path)
- Cognitive modeling (complex multi-step model of expert behavior)
 - Does someone have to 'check' something? More than once?
 - Do they have to move? More than once
- Gestalt Psychology (will they see it at all?)
- Errors (will they be reduced)

---
# Fitts' Law (infinite width)

A: Tapping    B: Crossing
![:youtube Video of using crossing for selection, kp2Zl4ONuik]

---
# Which is better and which laws explain it?
.left-column50[
## A.

![:img Picture of the Graffiti gesture recognition alphabet from the Palm Pilot, 100%](img/menus/Grafitti.png)
]
.right-column50[
## B.

![:img Picture of the Edgewrite gesture recognition alphabet, 50%](img/menus/Edgewrite.png)

]
???

.left-column50[
## A.

![:img Picture of the Graffiti gesture recognition alphabet from the Palm Pilot, 100%](img/menus/Grafitti.png)
]
.right-column50[
## B.

![:img Picture of the Edgewrite gesture recognition alphabet, 50%](img/menus/Edgewrite.png)

]

---
# Which is better and which laws explain it?

.left-column50[
## A.
![:img Old facebook security page with icons and text mixed
up,80%](img/menus/facebook.png)
]
.right-column50[
## B.

![:img March 2019 facebook security page with icons and
text clearly aligned, 80%](img/menus/facebook2.png)
]
???

- Fitts Law (compare distance and size; check for infinite size)
- Steering Law (distance and size over a path)
- Cognitive modeling (complex multi-step model of expert behavior)
 - Does someone have to 'check' something? More than once?
 - Do they have to move? More than once
- **Gestalt Psychology (group?)**
- **Errors (will they be reduced)**
---
# Gestalt Psychology (better grouping strategies)

.left-column50[
## A.
![:img Old facebook security page with icons and text mixed
up,80%](img/menus/facebook.png)
]
.right-column50[
## B.

![:img March 2019 facebook security page with icons and
text clearly aligned, 80%](img/menus/facebook2.png)
]

---
# Back to comparing menus

Kurtenbach: What did they study here?

![:youtube Illustration of advantages of marking menus,dtH9GdFSQaw]

---
# How do we prove our theories? Hypothesis

<div class="mermaid">
graph LR
S((.)) --> Hypothesis(Hypothesis)

linkStyle 0 stroke-width:4px;

class S invisible
class Hypothesis start
</div>

Discuss

Marking Menus are better than Pie Menus

--
Marking Menus are *faster* than Pie Menus

--
Pie Menus are faster than Linear Menus for less than 8 items that have no obvious order

---
.left-column-half[
# What conditions does this experiment have?

`MenuType`

`TaskType` (could be a condition)
]
.right-column-half[
# What might we measure? (from POP-Motor)
]
--
.right-column-half[
- Time on Task
- Accuracy
- How strenuous
- Recall
- Emotional Response

]

---
# How do we prove our theories? Method

<div class="mermaid">
graph LR
S((.)) --> Hypothesis(Hypothesis)
Hypothesis -- "Study Design" --> Method(Method)

linkStyle 0 stroke-width:4px;
linkStyle 1 stroke-width:4px;

class S invisible
class Hypothesis start
class Method normal
</div>

Compare specific conditions

| MenuType | 8 items | 12 items |
|----------|---------|----------|
| Linear   |         |          |
| Pie      |         |          |
| Marking  |         |          |

What if we wanted to do length AND ordered vs not?

Want to consider every combination, or run twice (menuType x length and menuType x ordering)

---
# How do we prove our theories? Data

<div class="mermaid">
graph LR
S((.)) --> Hypothesis(Hypothesis)
Hypothesis -- "Study Design" --> Method(Method)
Method -- "Run Study" --> Data(Data)

linkStyle 0 stroke-width:4px;
linkStyle 1 stroke-width:4px;
linkStyle 2 stroke-width:4px;

class S invisible
class Hypothesis start
class Method,Data normal
</div>

What might we want to measure?

--
- Time on task -- how long to complete basic tasks? (For example, find something to buy, create a new account, and order the item.)

--
- Accuracy -- How many mistakes did people make? (And were they fatal or recoverable with the right information?)

--
- How strenuous (e.g. gaze has lower throughput but is less strenuous than head pointing (Mackenzie 2018)

--
- Recall -- How much does the person remember afterwards or after periods of non-use?

--
- Emotional Response -- How does the person feel about the tasks completed? (Confident? Stressed? Would the user recommend this system to a friend?)

???
Build up a list of good UI design principals from these basics

Undo
Predictability
... What is missing? (e.g. fun)
---
# How to get such data?

- Video
- Timestamps
- Notes
- Can transcribe and analyze interviews with users
- Can look for patterns across users

---
# How do we prove our theories? Analysis

<div class="mermaid">
graph LR
S((.)) --> Hypothesis(Hypothesis)
Hypothesis -- "Study Design" --> Method(Method)
Method -- "Run Study" --> Data(Data)
Data -- "Clean and Prep" --> Analysis(Analysis)

linkStyle 0 stroke-width:4px;
linkStyle 1 stroke-width:4px;
linkStyle 2 stroke-width:4px;
linkStyle 3 stroke-width:4px;

class S invisible
class Hypothesis start
class Method,Data,Analysis normal
</div>

![:img sample data from the menus experiement, 80%](img/menus/menus-data.png)

---
# How do we prove our theories? Conclusions

<div class="mermaid">
graph LR
S((.)) --> Hypothesis(Hypothesis)
Hypothesis -- "Study Design" --> Method(Method)
Method -- "Run Study" --> Data(Data)
Data -- "Clean and Prep" --> Analysis(Analysis)
Analysis --> Conclusions(Conclusions)

linkStyle 0 stroke-width:4px;
linkStyle 1 stroke-width:4px;
linkStyle 2 stroke-width:4px;
linkStyle 3 stroke-width:4px;
linkStyle 4 stroke-width:4px;

class S invisible
class Hypothesis,Conclusions start
class Method,Data,Analysis normal
</div>

![:img sample data from the menus experiement, 30%](img/menus/histogram.png)

---
# Design an experiment

linkStyle 0 stroke-width:4px;
linkStyle 1 stroke-width:4px;
linkStyle 2 stroke-width:4px;
linkStyle 3 stroke-width:4px;
linkStyle 4 stroke-width:4px;

class S invisible
class Hypothesis,Conclusions start
class Method,Data,Analysis normal
</div>

.left-column50[
## A.
![:img Picture of the Graffiti gesture recognition alphabet from the Palm Pilot, 100%](img/menus/Grafitti.png)
]
.right-column50[
## B.
![:img Picture of the Edgewrite gesture recognition alphabet, 50%](img/menus/Edgewrite.png)
]

???
Hypothesis: Errors (will they be reduced)
Method:??
Data:??
Analysis??
Conclusions??

---
# Design an experiment

linkStyle 0 stroke-width:4px;
linkStyle 1 stroke-width:4px;
linkStyle 2 stroke-width:4px;
linkStyle 3 stroke-width:4px;
linkStyle 4 stroke-width:4px;

class S invisible
class Hypothesis,Conclusions start
class Method,Data,Analysis normal
</div>

.left-column50[
## A. Pie Menu
![:img Pie Menu, 80%](img/menus/pie.jpg)
]
.right-column50[
## B. Pull down Menu
![:img Traditional Menu inside a web browser, 80%](img/menus/menu.png)
]

???

Hypothesis: Errors will be reduced;
Hypothesis: Motion will be faster due to low level motor things
Hypothesis: fewer cognitive checks needed
Method:??
Data:??
Analysis??
Conclusions??