Interaction Technique Design --Week 7, Wednesday--

name: inverse
layout: true
class: center, middle, inverse
---
layout:false

# Why is selection (and testing) important .red[*]
.left-column50[

![:img Image showing a false alarm in hawaii that led everyone to
think there wsa a nuclear event, 100%](img/menus/alert1.jpg)
]
.right-column50[
![:img Image showing the screen that led to this error, 100%](img/menus/alert.jpg)
]

.footnote[.red[*] [Washington Post Article](https://www.washingtonpost.com/news/morning-mix/wp/2018/01/16/that-was-no-wrong-button-in-hawaii-take-a-look/?utm_term=.1848969db923)
]
---
class: center, middle, inverse

# Interaction Technique Design

Jennifer Mankoff
CSE 340 Winter 2020
---
layout: false

[//]: # (Outline Slide)
# Today's goals
- Running example: [Pet gallery](https://gitlab.cs.washington.edu/cse340-20wi/cse340-petgallery)
- Introducing key concepts for menus assignment
- Using key properties of people to predict design outcomes

---
# Revisiting event records
Event record:
- Where
  - Where is the location of the cursor (x,y position on the screen)
  - For focus based events, where is often the target (Lauren's lecture covered this)
- Value is anything you might need to know (like whether it's a 1 or 2 finger swipe; or which key was pressed)

Why is this important? For *picking*: 
- When input is positional, *only* interactors whose bounding box includes (x,y) -- where -- are picked
- When input is focus based, *only* interactors in the focus list are picked. Where isn't used in dispatch

---
# Menus Assignment

Will compare pie menus to linear menus

We provide support for running the experiment in MainActivity (and a testing harness in TestActivity)

You implement a variety of menus

---
# Menus Assignment

<div class="mermaid">
classDiagram

class MenuExperimentView {
  onTouchEvent()
  startSelection()
  endSelection()
  updateModel()
  onDraw()
}

AbstractMenuExperimentView <|-- MenuExperimentView

</div>

---

# What is a Menu in theory?

???
- supports selection of an item from a fixed set – usually set determined in advance
- typically used for “commands”
- occasionally for selecting a value (e.g., picking a font)

--
.left-column-half[
- supports selection of an item from a fixed set – usually set determined in advance
- typically used for “commands”
- occasionally for selecting a value <br> (e.g., picking a font)
]
--
<div class="mermaid">
graph TD
S((.)) --> A(Start)
A -- "Press?startSelection()" --> I(Selecting)
I -- "Release:endSelection()" --> E[End]
I -- "Drag:updateModel()" --> I

classDef finish outline-style:double,fill:#d1e0e0,stroke:#333,stroke-width:2px;
classDef normal fill:#e6f3ff,stroke:#333,stroke-width:2px;
classDef start fill:#d1e0e0,stroke:#333,stroke-width:4px;
classDef invisible fill:#FFFFFF,stroke:#FFFFFF,color:#FFFFFF

class S invisible
class A start
class E finish
class I normal
</div>

???

Implemented in MenuExperimentView

Works in all Menus!

---

# What is a Menu in theory?

.left-column-half[
- supports selection of an item from a fixed set – usually set determined in advance
- typically used for “commands”
- occasionally for selecting a value <br> (e.g., picking a font)

Implemented in MenuExperimentView

Works in all Menus!
]

<div class="mermaid">
graph TD
S((.)) --> A(Start)
A -- "Press?startSelection()" --> I(Selecting)
I -- "Release:endSelection()" --> E[End]
I -- "Drag:updateModel()" --> I

class S invisible
class A start
class E finish
class I normal
</div>

---
# Aside: Enums

.left-column-half[
Group of named constants

- Used for PPS in colorPicker (PPS States; Essential Geometry)

- Used for PPS in Menu assignment *and* for experimental conditions

- Easy to inspect if you want to (can use to determine possible states, number items, etc)

[Documentation](https://docs.oracle.com/javase/tutorial/java/javaOO/enum.html)
]
.right-column-half[
<div class="mermaid">
classDiagram

class TaskType {
	LINEAR
	RELATIVE
	UNCLASS
}

class State {
	START
	SELECTING
}

class MenuType {
	NORMAL
	PIE
	CUSTOM
}

</div>

]

---
# How do we draw a menu?

.left-column-half[
Coordinates depend on which class you're in

MenuExperimentView set to  `MATCH_PARENT`

Makes drawing tricky

Translate (like a parent view does for kids)
]

.right-column-half[

<div class="mermaid">
classDiagram

MenuExperimentView <|.. PieMenuView :  onDraw translates  and calls drawMenu

class PieMenuView {
  Canvas: 0,0 at first touch point. 
  Clipping rect: Whole screen
  drawMenu()
}

class MenuExperimentView {
   Canvas : 0,0 at top left of screen;
   Clipping rect: Whole screen
   onDraw() translate
}
</div>

]

???
Do we have to translate back?

--
We don't have to rotate or translate back because this is 
an inheritance drawing trick, not an interactor hierarchy drawing implementation

---
# Aside: Custom Listeners

- Used in ColorPicker and Menus. You'll use them lots
- Why create custom listeners?

--
 - Let you execute code when your view's model has changed
 - No other way to know that has happened

---
# How to implement

Custom View needs
- To define the custom interface 
- To keep track of listeners

--
Anything using the view needs
- To implement the interface (specifically, the method that will be called)
- To register itself as a listener

---
# Example Custom View -- ColorPicker: We setup the Custom View side for you

```java
    // Currently registered ColorListener instance or null. 
    private List<ColorChangeListener> mColorChangeListeners;

// Class which defines a listener to be called when a new color is selected.
    public interface ColorChangeListener {
        void onColorSelected(@ColorInt int color);
    }

// Registers a new listener
    public final void addColorChangeListener(@NonNull ColorChangeListener colorChangeListener) {
        mColorChangeListeners.add(colorChangeListener);
    }
```

---
# Example Custom Listener -- ColorPicker: We implemented this

.left-column-half[
`// TODO: Register callback to update {color,label} View when color changed.`

What method do we call to register the callback?  `addColorChangeListener()`

What do we usually do in a callback? update application (`MainActivity`) model
]
.right-column-half[
<div class="mermaid">
classDiagram
MainActivity ..> ColorPickerView : addColorChangeListener()
MainActivity --> ColorChangeListener : Contains
class ColorChangeListener {
  onColorChanged()
}
class ColorPickerView {
  addColorChangeListener()
}
class MainActivity {
  ColorPickerView view
}

</div>
]
---
# Example Custom Listener -- ColorPicker: We implemented this

.left-column-half[
`// TODO: Register callback to update {color,label} View when color changed.`

What method do we call to register the callback?  `addColorChangeListener()`

What do we usually do in a callback? update application (`MainActivity`) model
]
.right-column-half[
<div class="mermaid">
classDiagram
ColorPickerView ..> ColorChangeListener : onColorChanged()
ColorChangeListener ..> MainActivity : update model

class ColorChangeListener {
  onColorChanged()
}

class MainActivity {
  ColorPickerView view
}
</div>
]

---
# You need to do this yourself in Menus

```java
// TODO: register a new listener with the menu so that the application knows when a selection is made

// TODO: implement the listener. When the user completes a trial, the menu listener should store 
// the results of the trial, and setup for the next trial
```

---
class: center, middle, inverse

# Experimenting with Interfaces

Important part of buliding interfaces is experimenting

Need structured ways to decide what's better

---
# Experiments should be tied to *hypotheses* based on *theories* what will go better

- Fitts Law (compare distance and size; check for infinite size)
- Steering Law (distance and size over a path)
- Cognitive modeling (complex multi-step model of expert behavior)
 - Does someone have to 'check' something? More than once?
 - Do they have to move? More than once
- Gestalt Psychology (will they see it at all?)
- Errors (will they be reduced)

???
why theory and not intuition?

---
# Which is better and which laws explain it?
.left-column50[
## A Pie Menu
![:img Pie Menu, 80%](img/menus/pie.jpg)
]
.right-column50[
## B Pull down Menu
![:img Traditional Menu inside a web browser, 80%](img/menus/menu.png)
]
???

What analysis methods can we use to predict?

- *Fitts Law* (compare distance and size; check for infinite size)
- Steering Law (distance and size over a path)
- Cognitive modeling (complex multi-step model of expert behavior)
 - Does someone have to 'check' something? More than once?
 - Do they have to move? More than once
- Gestalt Psychology (will they see it at all?)
- Errors (will they be reduced)

---
# Steering Law (based on Fitts' law)
.left-column50[
## A Pie Menu
![:img Pie Menu, 80%](img/menus/pie.jpg)
]
.right-column50[
## B Pull down Menu
![:img Traditional Menu inside a web browser, 80%](img/menus/menu.png)
]

---
# Which is better and which law explains it?
.left-column50[
## A Pie Menu
![:img Pie Menu, 80%](img/menus/pie.jpg)
]
.right-column50[
## B Marking Menu
![:youtube Video assigned before class, 8c58bN6ajJ4?t=30]
]
???

- Fitts Law (compare distance and size; check for infinite size)
- Steering Law (distance and size over a path)
- Cognitive modeling (complex multi-step model of expert behavior)
 - **Does someone have to 'check' something? More than once?**
 - **Do they have to move? More than once**
- Gestalt Psychology (will they see it at all?)
- Errors (will they be reduced)
---
# Cognitive modeling (less double checking)

.left-column50[
## A Pie Menu
![:img Pie Menu, 80%](img/menus/pie.jpg)
]
.right-column50[
## B Marking Menu
![:youtube Video assigned before class, 8c58bN6ajJ4?t=30]
]

---
# Which is better and which laws explain it?

A: Tapping B: Crossing
![:youtube Video of using crossing for selection, kp2Zl4ONuik]

???

- **Fitts Law (compare distance and size; check for infinite size)**
- Steering Law (distance and size over a path)
- Cognitive modeling (complex multi-step model of expert behavior)
 - Does someone have to 'check' something? More than once?
 - Do they have to move? More than once
- Gestalt Psychology (will they see it at all?)
- Errors (will they be reduced)

---
# Fitts' Law (infinite width)

A: Tapping B: Crossing
![:youtube Video of using crossing for selection, kp2Zl4ONuik]

---
# Which is better and which laws explain it?
.left-column50[
## A

![:img Picture of the Graffiti gesture recognition alphabet from the Palm Pilot, 100%](img/menus/Grafitti.png)
]
.right-column50[
## B

![:img Picture of the Edgewrite gesture recognition alphabet, 50%](img/menus/Edgewrite.png)

]
???

.left-column50[
## A

![:img Picture of the Graffiti gesture recognition alphabet from the Palm Pilot, 100%](img/menus/Grafitti.png)
]
.right-column50[
## B

![:img Picture of the Edgewrite gesture recognition alphabet, 50%](img/menus/Edgewrite.png)

]

---
# Which is better and which laws explain it?

.left-column50[
## A
![:img Picture of old facebook security page with icons and text mixed
up,100%](img/menus/facebook.png)
]
.right-column50[
## B

![:img Picture of March 2019 facebook security page with icons and
text clearly aligned, 100%](img/menus/facebook2.png)
]
???

- Fitts Law (compare distance and size; check for infinite size)
- Steering Law (distance and size over a path)
- Cognitive modeling (complex multi-step model of expert behavior)
 - Does someone have to 'check' something? More than once?
 - Do they have to move? More than once
- **Gestalt Psychology (group?)**
- **Errors (will they be reduced)**
---
# Gestalt Psychology (better grouping strategies)

.left-column50[
## A
![:img Picture of old facebook security page with icons and text mixed
up,100%](img/menus/facebook.png)
]
.right-column50[
## B

![:img Picture of March 2019 facebook security page with icons and
text clearly aligned, 100%](img/menus/facebook2.png)
]

---
# Back to comparing menus

Kurtenbach:
![:youtube Illustration of advantages of marking menus,dtH9GdFSQaw]

---
# How do we prove our theories? Hypothesis

<div class="mermaid">
graph LR
S((.)) --> Hypothesis(Hypothesis)

linkStyle 0 stroke-width:4px;

class S invisible
class Hypothesis start
</div>

Discuss

Marking Menus are better than Pie Menus

--
Marking Menus are *faster* than Pie Menus

--
Pie Menus are faster than Linear Menus for less than 8 items that have no obvious order

---
.left-column-half[
# What conditions does this experiment have?

`MenuType`

`TaskType` (could be a condition)
]
.right-column-half[
# What might we measure? (from Friday)
]
--
.right-column-half[
- Time on Task 
- Accuracy 
- How strenuous 
- Recall 
- Emotional Response

]

---
# How do we prove our theories? Method

<div class="mermaid">
graph LR
S((.)) --> Hypothesis(Hypothesis)
Hypothesis -- "Study Design" --> Method(Method)

linkStyle 0 stroke-width:4px;
linkStyle 1 stroke-width:4px;

class S invisible
class Hypothesis start
class Method normal
</div>

Compare specific conditions

| MenuType | 8 items | 12 items |
|----------|---------|----------|
| Pie      |         |          |
| Marking  |         |          |
| Linear   |         |          |

What if we wanted to do length AND ordered vs not?

Want to consider every combination, or run twice (menuType x length and menuType x ordering)

---
# How do we prove our theories? Data

<div class="mermaid">
graph LR
S((.)) --> Hypothesis(Hypothesis)
Hypothesis -- "Study Design" --> Method(Method)
Method -- "Run Study" --> Data(Data)

linkStyle 0 stroke-width:4px;
linkStyle 1 stroke-width:4px;
linkStyle 2 stroke-width:4px;

class S invisible
class Hypothesis start
class Method,Data normal
</div>

What might we want to measure? (discussed friday)

--
- Time on task -- how long to complete basic tasks? (For example, find something to buy, create a new account, and order the item.)

--
- Accuracy -- How many mistakes did people make? (And were they fatal or recoverable with the right information?)

--
- How strenuous (e.g. gaze has lower throughput but is less strenuous than head pointing (Mackenzie 2018)

--
- Recall -- How much does the person remember afterwards or after periods of non-use?

--
- Emotional Response -- How does the person feel about the tasks completed? (Confident? Stressed? Would the user recommend this system to a friend?)

???
Build up a list of good UI design principals from these basics

Undo
Predictability
... What is missing? (e.g. fun)
---
# How to get such data?

- Video
- Timestamps
- Notes
- Can transcribe and analyze interviews with users
- Can look for patterns across users

---
# How do we prove our theories? Analysis

<div class="mermaid">
graph LR
S((.)) --> Hypothesis(Hypothesis)
Hypothesis -- "Study Design" --> Method(Method)
Method -- "Run Study" --> Data(Data)
Data -- "Clean and Prep" --> Analysis(Analysis)

linkStyle 0 stroke-width:4px;
linkStyle 1 stroke-width:4px;
linkStyle 2 stroke-width:4px;
linkStyle 3 stroke-width:4px;

class S invisible
class Hypothesis start
class Method,Data,Analysis normal
</div>

---
# How do we prove our theories? Conclusions

<div class="mermaid">
graph LR
S((.)) --> Hypothesis(Hypothesis)
Hypothesis -- "Study Design" --> Method(Method)
Method -- "Run Study" --> Data(Data)
Data -- "Clean and Prep" --> Analysis(Analysis)
Analysis --> Conclusions(Conclusions)

linkStyle 0 stroke-width:4px;
linkStyle 1 stroke-width:4px;
linkStyle 2 stroke-width:4px;
linkStyle 3 stroke-width:4px;
linkStyle 4 stroke-width:4px;

class S invisible
class Hypothesis,Conclusions start
class Method,Data,Analysis normal
</div>

![:img Pie Menu, 80%](img/menus/exp1.png)

1 = no; 5 = yes

40 responses

---
# How do we prove our theories? Conclusions

linkStyle 0 stroke-width:4px;
linkStyle 1 stroke-width:4px;
linkStyle 2 stroke-width:4px;
linkStyle 3 stroke-width:4px;
linkStyle 4 stroke-width:4px;

class S invisible
class Hypothesis,Conclusions start
class Method,Data,Analysis normal
</div>

![:img Pie Menu, 80%](img/menus/exp2.png)

---
# How do we prove our theories? Conclusions

linkStyle 0 stroke-width:4px;
linkStyle 1 stroke-width:4px;
linkStyle 2 stroke-width:4px;
linkStyle 3 stroke-width:4px;
linkStyle 4 stroke-width:4px;

class S invisible
class Hypothesis,Conclusions start
class Method,Data,Analysis normal
</div>

![:img did you like using them (middle of the road), 80%](img/menus/exp3.png)

1 = no; 5 = very much

---
# Further results of class experiment

- 86.7% of students in the class participated in the experiment
- There were no big differences between the two groups
- The first day (Thu 1/9), very few no-dot students brought their own cards. However, starting with the next class, students were pretty good about bringing their own cards
- Students do not seem to bring their cards on Thursday (quiz sections)
- Most students have no trouble in remembering name tags
- Most students used name tents at least 4 times
- Students generally feel okay about using name tents (no strong preference)
- There was no evidence that name tents helped the TA remember students' names

Some factors that may have impacted the data:

- Some students did not attend classes/quiz sections
- Some students with dots attended classes but did not find/pick up their cards

Charts: https://tinyurl.com/wwaw5la

---