+ - 0:00:00
Notes for current slide
Notes for next slide

Properties of People II: Motor Control

Lauren Bricker

CSE 340 Spring 23

Slide 1 of 62

Agenda

  • Administrivia
  • Menus video checkpoint due (Tues 2-May)
  • Menus code part 1-4 due (Fri 5-May)
    • Code is due Saturday
      • Resubmissions
    • Layout will be accepted until 05/14
    • Accessibility will be accepted until 05/21
  • Today
    • Quick check on Ed about Menus codebase
    • Fitts' Law and implications for interface design
Slide 2 of 62

Let's try an experiment

Image of an interface with a red dot and many grey dots on the left
and a dialog on right to control randomization and highlight distance
and width in a Fitts' law experiment.

  • Fitts' Law Experiment (simonwallner.at/ext/fitts) Scroll down to just below 'A word of warning'
  • Click the red circles as fast as possible
Slide 3 of 62

Today's goals

  • Experience Fitts' Law
  • Discuss its implications for design
  • Introduce Guiard's model of Bi-Manual Control
  • Discuss its implications for design
Slide 4 of 62

Throughput

Picture of graph of fitts law data collected when I did the
experiment. Y axis is time and x is index of difficulty (ID). It's fairly
noisy because fitts law is designed to measure multiple
people. However it does fit a line.

  • Note: We will use D for Dist and W for Width throughout the rest of this lecture
MT=a+blog2(DistSize+1)

where

Slide 5 of 62

Throughput

Picture of graph of fitts law data collected when I did the
experiment. Y axis is time and x is index of difficulty (ID). It's fairly
noisy because fitts law is designed to measure multiple
people. However it does fit a line.

  • Note: We will use D for Dist and W for Width throughout the rest of this lecture

MT=a+blog2(DistSize+1)

where

  • MT is movement time (in seconds)
Slide 5 of 62

Throughput

Picture of graph of fitts law data collected when I did the
experiment. Y axis is time and x is index of difficulty (ID). It's fairly
noisy because fitts law is designed to measure multiple
people. However it does fit a line.

  • Note: We will use D for Dist and W for Width throughout the rest of this lecture

MT=a+blog2(DistSize+1)

where

  • MT is movement time (in seconds)
  • ID is the Index of Difficulty of a movement (in bits) log2(DistSize+1)
Slide 5 of 62

Throughput

Picture of graph of fitts law data collected when I did the
experiment. Y axis is time and x is index of difficulty (ID). It's fairly
noisy because fitts law is designed to measure multiple
people. However it does fit a line.

  • Note: We will use D for Dist and W for Width throughout the rest of this lecture

MT=a+blog2(DistSize+1)

where

  • MT is movement time (in seconds)
  • ID is the Index of Difficulty of a movement (in bits) log2(DistSize+1)
  • ID/MT is the Throughput of a device (in bits/second)
Slide 5 of 62

Throughput

Picture of graph of fitts law data collected when I did the
experiment. Y axis is time and x is index of difficulty (ID). It's fairly
noisy because fitts law is designed to measure multiple
people. However it does fit a line.

  • Note: We will use D for Dist and W for Width throughout the rest of this lecture
MT=a+blog2(DistSize+1)

where

  • MT is movement time (in seconds)
  • ID is the Index of Difficulty of a movement (in bits) log2(DistSize+1)
  • ID/MT is the Throughput of a device (in bits/second)
  • a and b are empirically derived constants
Slide 5 of 62

This is just a line

Fitts’ law tells us about difficulty for pointing and selection tasks

  • Time to move the hand depends only on relative precision required
  • MT increases as distance from target increases
  • MT decreases as size of target increases
  • Diagram this

Original Fitts' experiment

Original experiment from Paul Morris Fitts involved tapping on plates as quickly as possible.

Slide 6 of 62

Compare Device Performance & Human Performance

Graph of Fitts law lines for various body parts (Neck Arm Wrist
Finger) with time on the Y axis and bits on the X axis

Device/Part Study Throughput (bits/s) Relative to Optimal
Finger Langolf (1976) 38
Wrist Langolf (1976) 23
EyeTracker Ware & Mikaelian (1987) 13.7
Hand Fitts (1954) 10.6
Arm Langolf (1976) 9.5
Xbox 360 Zaranek (2014) 5 .47 (arm)
Mouse Mackenzie (2001) 4.9 .46 (hand)
Neck Pfaff (1985) 4.2
Trackball Mackenzie (2001) 3.0 .28 (hand)
Touchpad Mackenzie (2001) 2.9 .27 (hand)
Head Hansen (2018) 2.5 .51 (neck)
Gaze Hansen (2018) 2.1 .15 (eyetracker)
Joystick Mackenzie (2001) 1.8 .17 (hand)
Playstation Move Zaranek (2014) 1.5 .16 (arm)
Kinect Zaranek (2014) 1 .1 (arm)

Card, S. K., Mackinlay, J. D., & Robertson, G. G. (1991).
A morphological analysis of the design space of input devices.
ACM Transactions on Information Systems (TOIS), 9(2), 99-122.

Slide 7 of 62

Useful to know if you're designing something like VR

Doesn't have much impact on interface design

Doesn't capture everything; e.g. error rate is higher for eye tracker

Warnings

Applies to expert, errorless use

bar chart of Average WPM for error rates of 1% to 20% in a
study that falsely inserted typing errors is much higher for 1% than
20%

Error rate was manipulated by faking errors. Ahmed Sabbir Arif and Wolfgang Stuerzlinger. CHI 2010. Predicting the cost of error correction in character-based text entry technologies. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, April 2010

Slide 8 of 62

Axis: X is faked error rates, Y is the words per minute.

Shows how WPM did change in the study where error rate is being changed. If you know you are going to make a lot of errors you slow down your motion The underlying way we're designed to move is with small motions, to eliminate jerk We accelerate and decelerate smoothly Trace of motion for different body parts - after the initial motion, which is a rapid aiming motion, then corrective motions if you misss the target. Lots of those corrections, that is where you are goign to see people slow down. Subtle effects here – like the time to fix errors goes up as the error rate goes up… Act of correcting an error also takes time. Not worry about in the assignment, but interesting to know.

Example: Arm based input

Student project using a Kinect from 2012. Watch the movement of his arm...

Slide 9 of 62

Discussion: Implications of Fitts' law

  • Why are some things harder than others to click on?

Picture of mac desktop and windows desktop. Of note is the main menu bar which is at the very top of the mac desktop but at the top of *each* window on the windows desktop

Slide 10 of 62

Constantly relevant

Intuitively, things that are closer and/or bigger are faster and easier to hit (and vice versa)

What is different about the file location in Mac and Windows?

What is the width at the top of the screen.

  • Add your thoughts on the bullet line with your group number in this Google Doc

Thinking about Design Tips

Follow along on our Exit Ticket Ed Lesson to participate.

Recall Fitts law is:

MT=a+blog2(DistSize+1)

Slide 11 of 62

Design Tip #1: Make small targets larger

Pet Gallery buttons at 64dp Pet Gallery buttons at 80dp

Answer on Ed

What does making targets larger do to Fitts Law Predictions?

Slide 12 of 62

Design Tip #1: Make small targets larger

Pet Gallery buttons at 64dp Pet Gallery buttons at 80dp

Answer on Ed

What does making targets larger do to Fitts Law Predictions?

Why should you not maximize all of the targets?

Slide 12 of 62

Design Tip #1: Make small targets larger

Pet Gallery buttons at 64dp Pet Gallery buttons at 80dp

Answer on Ed

What does making targets larger do to Fitts Law Predictions?

Why should you not maximize all of the targets?

Fitts' law involves a logarithmic calculation -- exponentially bigger gains for small targets, not so much of a gain for larger targets

Slide 12 of 62

Design Tip #1: Make small targets larger

CNN website marketing using a small x to close the pop up marketing that obscures the screen

  • How is the CNN marketing group using Fitts' law in this situation?
Slide 13 of 62

Design Tip #1: Make small targets larger

CNN website marketing using a small x to close the pop up marketing that obscures the screen

  • How is the CNN marketing group using Fitts' law in this situation?
  • intentionally making it HARDER to remove this window
Slide 14 of 62

Design Tip #2:
Put commonly used things close together

Screenshot of the powerpoint toolbar with grouped things

Answer on Ed

Is this minimizing distance, maximizing size, or both?

Slide 15 of 62

Design Tip #2:
Put commonly used things close together

Image of the drag'n'pop interface which moves icons to the cursor when they are valid targets

Can also bring items closer to the cursor

(to also minimize the distance)

Baudisch, P., Cutrell, E., Robbins, D., Czerwinski, M., Tandler, P. Bederson, B., and Zierlinger, A. Drag-and-Pop and Drag-and-Pick: Techniques for Accessing Remote Screen Content on Touch- and Pen-operated Systems. In Proceedings of Interact 2003, Zurich Switzerland, August 2003, pp. 57-64.

Slide 16 of 62

Reduces Distance (D)

Not good design to put everything close together

No space to move all targets (contextual filtering helps)

Could do with a fancy container...

Design Tip #2:
Put commonly used things close together

Image of the drag'n'pop interface which moves icons to the cursor when they are valid targets

Why not do this to all targets?

Slide 17 of 62

Design Tip #2:
Put commonly used things close together

Image of the drag'n'pop interface which moves icons to the cursor when they are valid targets

Why not do this to all targets?

  • Other usability goals matter too -- e.g. grouping related things
  • It is not good design to put everything close together
  • There is not enough space to move all targets (contextual filtering helps)

How would you implement this?

Slide 18 of 62

Minimizing distance and maximizing size!

We can make things Closer and Larger at the same time

Image of mac with doc icons magnified

Slide 19 of 62

We're beating Beating Fitts' Law!

How else can we do this?

Slide 20 of 62

We're beating Beating Fitts' Law!

How else can we do this?

Just don't use a mouse! Shortcut buttons; scroll wheel

Or CHEAT... manipulate the interface

  • Minimize distance
  • Increase size
Slide 20 of 62

Menu Selection

Answer on Ed

Do these menus minimize distance or maximize size?

Select from a menu bar at top of screen

Picture of the apple menu opened on a mac

Slide 21 of 62

Problems?

  • Can be tiring to reach edges of very large screens
  • Not all interfaces can do this (e.g. web designers)

Design Tip #3:
Make use of Edges: They are Infinite

Picture of the Windows menu in Windows 95 Windows XP and Windows 8 showing progressive improvement in the relationship between look (namely a border around the icon) and feel (can click anywhere making it an infinitely large button which is easier to hit from a Fitts law perspective)

Slide 22 of 62
  • Windows 95 bug: bottom-left corner is offset by a few pixels, can’t click on it
  • Windows XP fixes it by making Start button go all the way to the corner
  • Windows 8 uses styling to make start menu circular, but still makes entire corner clickable

Recent Example (Zoom)

Which of these do you think is a better button for maximizing size using the infinite edge?

Old zoom leave button which looks only like text but really is an infinite edge


New zoom leave button which looks bigger but loses the infinite edge

Slide 23 of 62

Recent Example (Zoom)

Which of these do you think is a better button for maximizing size using the infinite edge?

Old zoom leave button which looks only like text but really is an infinite edge    Old zoom leave button which displays its infinite edge/larger button when hovered over

Although it looks like only the text is clickable, it turns out the whole edge was. Now we only have a "smaller" but brighter button to click.


New zoom leave button which looks bigger but loses the infinite edge

Slide 24 of 62

Design Tip #3:
Make use of Edges: They are Infinite

Problems with infinite edges?

  • Can be tiring to reach edges of very large screens
  • Not all interfaces can do this (e.g. web designers)
  • Does this work for mobile devices or other touch screens?
Slide 25 of 62

Menu Selection

Answer on Ed

Do these menus minimize distance, maximize size, or both?

Left -- A pie menu consisting of a circle broken into eight
wedges containing menu items like copy and undo; right-- A picture of
a standard linear menu

Left example: OneNote 2013, Right example: Firefox. Taken from Fitts law and user experience by https://www.smashingmagazine.com/author/anastasios-karafillis/

Slide 26 of 62

Why are marking menus rare & unfamiliar?

Right click in place and then select (starting at :19)

Kittenish, G., & Buxton, W. (1994, April). User learning and performance with marking menus. In CHI (Vol. 94, pp. 258-264).

Slide 27 of 62

Minimizes D AND increases W (only angle matters)

Design Tip #4: Use marking menus instead of context menus for expert tasks

Under some circumstances only...

  • Less than 8 options (Small target areas when too many menu entries are added)
  • Expert user (willing to memorize and gain advantage of marking menus)
  • Grouping not important (hard to group radially)

Pie menus are often not implemented in production code

  • Needs to be seen as a high priority feature
  • Managers need to be able to assign staff to the project
    • Sadly, project constraints define an interface. (Shouldn't be a thing! Whorfian effects)
Slide 28 of 62

Example in Maya

Example of how Marking Menus are used in practice (Maya)

Slide 29 of 62

Menus Assignment

Goal: to compare pie menus, linear menus, and a custom menu you make!

We provide support for running the experiment in AbstractMainActivity (and a testing harness in ExperimentActivity)

You implement a variety of menus

Slide 30 of 62

Snapping

Answer on Ed

Does snapping to a target minimize distance, maximize size, or both?

video of someone moving emails into folders with snapping

Slide 31 of 62

Design Tip #5:
Use snapping to minimize distance
when likely targets are known

Picture showing drag snapping in gmail trello and chrome

Gmail Trello Chrome
checkDrag Handle xNo Handle x No Handle
cursorGrab cursor cursorPointer cursor Default cursor
checkDrop Shadow checkDrop Shadow x No Drop Shadow
checkDrop Target checkDrop Target x No Drop Target
x No Natural Movement checkNatural Movement check Natural Movement

Examples courtesy of Drag and Drop for Design Systems

Slide 32 of 62

Also common in drawing programs

Most sophisticated approach: dynamic semantics: Check legality and consequences of each result at every move don’t catch errors, prevent them

Other Examples

Fan Cursor, Area Cursor and Bubble Cursor

Su, X., Au, O. K. C., & Lau, R. W. (2014, April). The implicit fan cursor: a velocity dependent area cursor. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 753-762). ACM.

Slide 33 of 62

Design Tip #6: Separate Motor Size from Visible Size

Picture of two versions of a dialog box; scrollbar; and
menu

  • (a) Visual space appearance of buttons in a dialogue box.
  • (b) Motor space version of button design in (a) with much larger targets for certain buttons.
  • (c) Standard scroll-bar design.
  • (d) Visual space appearance of scroll-bar redesigned to occupy smaller screen space.
  • (e) Motor space version of scroll-bar design in (d) with larger targets for active areas.
  • (f) Visual space appearance of menu.
  • (g) Motor space version of the menu design in (f) with the distance to more important items reduced by compressing the size of less important items)
Semantic pointing widget designs from Blanch et al. 2004
Slide 34 of 62

How would you implement this?

  • Manipulate picking
  • Whorfian effects

Small on purpose?

What might you make too small and why?

Slide 35 of 62

Small on purpose?

What might you make too small and why?

How about a cancel button?

Slide 35 of 62

How long will this take?

Select 'Open New Window in Process'

Cascading menus. User wants to use

Slide 36 of 62

How long will this take?

Select 'Open New Window in Process'

Cascading menus. User wants to use

Steering law ... Fitts law integrated over the path you have to follow

Slide 36 of 62

How long will this take?

Select 'Open New Window in Process'

Cascading menus. User wants to use

Steering law ... Fitts law integrated over the path you have to follow

But it's not predictive at all. Why?

Slide 36 of 62

How long will this take?

Select 'Open New Window in Process'

Cascading menus. User wants to use

Steering law ... Fitts law integrated over the path you have to follow

But it's not predictive at all. Why?

It doesn't account for actual user behavior

Slide 36 of 62

Actual User Behavior

Cascading menus. User wants to use

Slide 37 of 62

Actual User Behavior

Cascading menus. User wants to use

Cascading menus. User wants to use

Slide 38 of 62

Overly eager cascading menus, wrong submenu

  • Common problem w/ web menus (but not Mac or Win)
  • could brainstorm solutions (e.g. snapping? set focus somehow?)
  • whorfian effects: this is hard to implement

Fitts law has a huge limitation

It is only predictive of ERROR-FREE, EXPERT behavior

Related to our discussion of Interaction Design

Slide 39 of 62

What else might we want to measure?

Slide 40 of 62

What else might we want to measure?

  • Time on Task -- How long does it take people to complete basic tasks? (For example, find something to buy, create a new account, and order the item.)
Slide 41 of 62

What else might we want to measure?

  • Time on Task -- How long does it take people to complete basic tasks? (For example, find something to buy, create a new account, and order the item.)

  • Accuracy -- How many mistakes did people make? (And were they fatal or recoverable with the right information?)

Slide 42 of 62

What else might we want to measure?

  • Time on Task -- How long does it take people to complete basic tasks? (For example, find something to buy, create a new account, and order the item.)

  • Accuracy -- How many mistakes did people make? (And were they fatal or recoverable with the right information?)

  • How strenuous (e.g. gaze has lower throughput but is less strenuous than head pointing Mackenzie 2018)

Slide 43 of 62

What else might we want to measure?

  • Time on Task -- How long does it take people to complete basic tasks? (For example, find something to buy, create a new account, and order the item.)

  • Accuracy -- How many mistakes did people make? (And were they fatal or recoverable with the right information?)

  • How strenuous (e.g. gaze has lower throughput but is less strenuous than head pointing Mackenzie 2018)

  • Recall -- How much does the person remember afterwards or after periods of non-use?

Slide 44 of 62

What else might we want to measure?

  • Time on Task -- How long does it take people to complete basic tasks? (For example, find something to buy, create a new account, and order the item.)

  • Accuracy -- How many mistakes did people make? (And were they fatal or recoverable with the right information?)

  • How strenuous (e.g. gaze has lower throughput but is less strenuous than head pointing Mackenzie 2018)

  • Recall -- How much does the person remember afterwards or after periods of non-use?

  • Emotional Response -- How does the person feel about the tasks completed? (Confident? Stressed? Would the user recommend this system to a friend?)

Slide 45 of 62

Build up a list of good UI design Principles from these basics

  • Undo
  • Predictability
  • ... What is missing? (e.g. fun)

What else might we want to measure?

  • Time on Task
  • Accuracy
  • How strenuous
  • Recall
  • Emotional Response

What is missing?

Slide 46 of 62

(e.g. fun)

  • Undo
  • Predictability
  • ...

What else might we want to measure?

  • Time on Task
  • Accuracy
  • How strenuous
  • Recall
  • Emotional Response

What is missing?

  • Fun
  • Predictability
  • Error Likelihood & Recovery
  • ...
Slide 47 of 62

Summary of Design Tips

  • Design Tip #1: Make small targets larger
  • Design Tip #2: Put commonly used things close together
  • Design Tip #3: Use edges. They are infinite in size (W)
  • Design Tip #4: Use pie menus for expert tasks (better yet, marking menus)
  • Design Tip #5: Use snapping to reduce distance (D) when targets are known
  • Design Tip #6: Separate motor and visible size
Slide 48 of 62

End of Lecture

Extra information is on the slides that follow

Slide 49 of 62

Two Handed Interaction

Picture by Escher of two hands each drawing the other

Slide 50 of 62

Used to be a norm

Seems we forgotten to include it in our interfaces

Guiard's model of bimanual control

Hand Role and Action
Non-preferred Leads the preferred hand (in time)
Sets the spatial frame of reference for the preferred hand
Performs coarse movements
Preferred Follows the non-preferred hand
Works within established frame of reference set by the non-preferred hand
Performs fine movement

Aside: Guiard is assuming right-handed. Tendencies somewhat less clear for left-handers, but we typically assume this still applies ]

Seminal work Yves Guiard, “Asymmetric Division of Labor in Human Skilled Bimanual Action: The Kinematic Chain as a Model”, Journal of Motor Behavior, Vol. 19, No. 4, 1987, pp. 486-517. http://cogprints.org/625/

Slide 51 of 62

Applications?

  • Mouse+keyboard
  • Lenses

What does theory say about keyboard layout?

Picture of right-handed user typing on keyboard with leftside
and rightside power keys and mouse on right side (top). Picture of
user  mousing with same set ups (bottom). Of note is the bad angle for
the mouse in the keyboard with the right side power keys

From Evoluent keyboard advertisement

Slide 52 of 62

Analysis of alternative keyboard layout

Task Leading Movement Trailing/Overlapping Movement
Delete Right hand — manipulate pointer with mouse; select by double clicking/dragging Left hand — press DELETE (probably with little finger)
Select an option in a window (see Figure 15) Right hand — manipulate pointer with mouse; click option Left hand — press ENTER (Assumes OK button is default)
Click on a link in a browser Left hand — navigate to link via PAGE UP and/or PAGE DOWN keys Right hand — manipulate pointer with mouse; select link by clicking
Open file; open folder; launch program Right hand — manipulate pointer with mouse; single click on icon Right hand — press ENTER (avoids error prone double-click operation)
Slide 53 of 62

Other Examples: Tool Stone for Magic Lenses

Inspired by 1994 CHI paper Published in UIST 2000

Slide 54 of 62

Magnification

Picture of a mandelbrot fractal with a lens over part of it magnifying that part

Slide 55 of 62

Font Selection

Picture of a paragraph of text with a lens containing a menu of options for bold italic regular and bold italic

Slide 56 of 62

Fitts' law implications?

Debugging Lenses

Picture of a GUI application with a lens showing information about two lines including their coordinates and the option to turn on and off additional features

Hudson, S. E., Rodenstein, R., and Smith, I. 1997. Debugging lenses: a new class of transparent tools for user interface debugging. In Proceedings UIST '97, 179-187. http://doi.acm.org/10.1145/263407.263542

Slide 57 of 62

Advantages of Lenses

In context interaction

  • Little or no shift in focus of attention and/or movement
  • Alternate views in context and on demand
Slide 58 of 62
  • tool is at/near action point
  • can compare in context
  • useful for “detail + context” visualization techniques

Advantages of Lenses

In context interaction

  • Little or no shift in focus of attention and/or movement
  • Alternate views in context and on demand

Structured well for 2 handed input

  • non-dominant hand does course positioning (of the lens)
  • dominant hand does fine work

"Spatial modes"

  • Use “where you click through” to establish meaning
  • Typically has a clear affordance for the meaning
Slide 59 of 62

Implementing Lenses

Root
Lens Parent
Lens
Interactor 1
Interactor 2
Interactor 3
Edwards, Hudson, et al., “Systematic output modification in a 2D user interface toolkit”, UIST ’97. http://doi.acm.org/10.1145/263407.263537
Slide 60 of 62

Can be implemented with special "lens parent" container & lens interactors

  • Lens may need to change results of picking (only positional is affected) in collusion with lens parent
  • Lens parent forwards all damage to all lenses
  • Lenses typically change any damage that overlaps them into damage of whole lens area
  • Can pass a subclass of Drawable to "ambush and modify" drawing for output effects

Discussion: Should scrolling be dominant or non-dominant?

What tasks is scrolling used for?

Which precedes/follows?

Slide 61 of 62

Task | Characteristics Scrolling | precedes/overlaps other tasks | sets the frame of reference | minimal precision needed (coarse) Selecting, editing, reading, drawing, etc. | follows/overlaps scrolling | works within frame of reference set by scrolling | demands precision (fine)

Summary Slide

Design tips for motor

  • Design Tip #1: Make small targets larger
  • Design Tip #2: Put commonly used things close together
  • Design Tip #3: Use edges. They are infinite in size (W)
  • Design Tip #4: Use pie menus for expert tasks (better yet, marking menus)
  • Design Tip #5: Use snapping to reduce distance (D) when targets are known
  • Design Tip #6: Separate motor and visible size

Two-Handed input principles

  • Non preferred leads
  • Sets frame of reference
  • Preferred does fine movement
Slide 62 of 62

Agenda

  • Administrivia
  • Menus video checkpoint due (Tues 2-May)
  • Menus code part 1-4 due (Fri 5-May)
    • Code is due Saturday
      • Resubmissions
    • Layout will be accepted until 05/14
    • Accessibility will be accepted until 05/21
  • Today
    • Quick check on Ed about Menus codebase
    • Fitts' Law and implications for interface design
Slide 2 of 62
Paused

Help

Keyboard shortcuts

, , Pg Up, k Go to previous slide
, , Pg Dn, Space, j Go to next slide
Home Go to first slide
End Go to last slide
Number + Return Go to specific slide
b / m / f Toggle blackout / mirrored / fullscreen mode
c Clone slideshow
p Toggle presenter mode
s Start & Stop the presentation timer
t Reset the presentation timer
?, h Toggle this help
Esc Back to slideshow