name: inverse layout: true class: center, middle, inverse --- # Properties of People II: Motor Control & Mental Models Jennifer Mankoff CSE 340 Spring 2019 --- layout: false .title[ Let's try an experiment ] .right-column[ - [Fitts' Law Experiment](http://simonwallner.at/ext/fitts/) (simonwallner.at/ext/fitts) Scroll down to just below 'A word of warning' - Click the red circles as fast as possible  ] --- [//]: # (Outline Slide) .title[ Today's goals ] .body[ Experience Fitts' Law Discuss its implications for design Introduce Guiard's model of Bi-Manual Control Discuss its implications for design ] --- .left-column-half[ # Throughput  ] .right-column-half[ .jax[$$MT = a + b*log_2({Dist \over Size} + 1)$$ ] where - *MT* is movement time - *ID/MT* is the *Throughput* of a device in bits/second - *a* and *b* are empirically derived constants - ID is the *Index of Difficulty* (ID, in bits) of a movement .jax[$$log_2({Dist \over Size} + 1)$$] ] ??? This is just a line Fitts’ law tells us about difficulty for pointing and selection tasks - Time to move the hand depends only on relative precision required - MT increases as __distance__ from target increases - MT decreases as __size__ of target increases - Diagram this --- .title[ Compare device performance & Human Performance ] .left-column-half[  ] .right-column-half[ .font-small[ Device/Part | Study | Throughput (bits/s) | Relative to Optimal -------|-------|------------ | ----------- Finger | Langolf (1976) | 38 Wrist | Langolf (1976) | 23 EyeTracker| Ware & Mikaelian (1987) | 13.7 | Hand | Fitts (1954) | 10.6 Arm | Langolf (1976) | 9.5 Xbox 360 | Zaranek (2014) | 5 | **.47** (arm) Mouse | Mackenzie (2001) | 4.9 | .46 (hand) Neck | Pfaff (1985) | 4.2 | Trackball | Mackenzie (2001)| 3.0 | .28 (hand) Touchpad | Mackenzie (2001) | 2.9 | .27 (hand) Head | Hansen (2018) | 2.5 | **.51** (neck) Gaze | Hansen (2018) | 2.1 | **.15** (eyetracker) Joystick | Mackenzie (2001)| 1.8 | .17 (hand) Playstation Move | Zaranek (2014) | 1.5 | .16 (arm) Kinect | Zaranek (2014) | 1 | .1 (arm) ] ] .footnote[Card, S. K., Mackinlay, J. D., & Robertson, G. G. (1991). <br> A morphological analysis of the design space of input devices. <br> ACM Transactions on Information Systems (TOIS), 9(2), 99-122. ] ??? Useful to know if you're designing something like VR Doesn't have much impact on interface design Doesn't capture everything; e.g. error rate is higher for eye tracker --- .title[Warnings] .body[ Applies to *expert, errorless* use  ] .footnote[Error rate was manipulated by faking errors. Ahmed Sabbir Arif and Wolfgang Stuerzlinger. CHI 2010. Predicting the cost of error correction in character-based text entry technologies. In Proceedings of the ] ??? Subtle effects here – like the time to fix errors goes up as the error rate goes up… --- .title[That said, let's talk through implications of Fitts' law] .body[ Why are some things harder than others to click on?  ] ??? Constantly relevant Intuitively, things that are closer and/or bigger are faster and easier to hit (and vice versa) --- .title[ Design Tip #1: Make small targets larger ] .body[ What does this do to Fitts Law Predictions? ] -- .body[ - Maximizes W ] -- .body[ Why not all targets? ] -- .body[ - Fitts' law involves log -- exponentially bigger gains for small targets ] --- .title[Design Tip #2:<br> Put commonly used things close together  ] --- .title[Design Tip #2:<br> Put commonly used things close together] .left-column50[  ] .right-column50[ # ... Or bring them closer to cursor What does this do to Fitts Law predictions? Why not all targets? How to implement? ] .footnote[ Baudisch, P., Cutrell, E., Robbins, D., Czerwinski, M., Tandler, P. Bederson, B., and Zierlinger, A. Drag-and-Pop and Drag-and-Pick: Techniques for Accessing Remote Screen Content on Touch- and Pen-operated Systems. In Proceedings of Interact 2003, Zurich Switzerland, August 2003, pp. 57-64. ] ??? Reduces D Not good design to put everything close together No space to move all targets (contextual filtering helps) Could do with a fancy container... --- .title[What does this do to Fitts Law Predictions? ] ??? - Minimizes D - Other usability goals matter too -- e.g. grouping related things --- .title[ Can we make things *Closer* and *Larger* <br> at the same time? ] .body[  ] --- .title[ What we're doing is Beating Fitts' Law! ] .body[ How else can we do this? ] -- .body[ Just don't use a mouse! Shortcut buttons; scroll wheel Or CHEAT... manipulate the interface - Minimize D - Increase W ] --- .title[Does this minimize (A) D or increase (B) W?] .body[ Select from a menu bar at top of screen  ] ??? Problems? - Can be tiring to reach edges of *very* large screens - Not all interfaces can do this (e.g. web designers) --- .title[Design Tip #3: Make use of Edges: They are Infinite] .right-column[  ] ??? - Windows 95 bug: bottom-left corner is offset by a few pixels, can’t click on it - Windows XP fixes it by making Start button go all the way to the corner - Windows 8 uses styling to make start menu circular, but still makes entire corner clickable --- .title[Does this minimize (A) D or increase (B) W?] .body[ Right click in place and then select (starting at :19) ![:youtube Illustration of advantages of marking menus,dtH9GdFSQaw?start=19] ] .footnote[ Kittenish, G., & Buxton, W. (1994, April). User learning and performance with marking menus. In CHI (Vol. 94, pp. 258-264). ] ??? Minimizes D AND increases W (only angle matters) --- .title[Does this minimize (A) D or increase (B) W?] .body[ Why is it so rare & unfamiliar?  ] .footnote[ Left example: OneNote 2013, Right example: Firefox. Taken from [Fitts law and user experience](https://www.smashingmagazine.com/2012/12/fittss-law-and-user-experience/) by [https://www.smashingmagazine.com/author/anastasios-karafillis/](Anastasios Karafillis) ] --- .title[Design Tip #4: Use pie menus instead of context menus for expert tasks] -- .body[ Under some circumstances only... - Less than 8 options (Small target areas when too many menu entries are added) - Expert user (willing to memorize and gain advantage of marking menus) - Grouping not important (hard to group radially) - Have the implementation capacity on staff (**Shouldn't be a thing! Whorfian effects**) ] --- .title[ Does this minimize (A) D or increase (B) W? ] .body[  ] ??? Snapping to a target - [A] Minimize D? - [B] Maximize W? minimizes D --- .title[Design Tip #5: Use snapping to minimize distance<br> when likely targets are known ] .body[  .font-small[ Gmail | Trello | Chrome ----- | ------ | ----- Drag Handle | No Handle |  No Handle Grab cursor | Pointer cursor | Default cursor Drop Shadow | Drop Shadow |  No Drop Shadow Drop Target | Drop Target |  No Drop Target  No Natural Movement | Natural Movement |  Natural Movement ] .footnote[Examples courtesy of [Drag and Drop for Design Systems](https://uxdesign.cc/drag-and-drop-for-design-systems-8d40502eb26d)] ] ??? Also common in drawing programs Most sophisticated approach: dynamic semantics: Check legality and consequences of each result at every move don’t catch errors, prevent them --- .title[Does this (A) minimize D or (B) maximize W?] .body[ Fan Cursor, Area Cursor and Bubble Cursor ![:youtube Video showing a variety of schemes for making the cursor bigger including fan cursor; area cursor; and bubble cursor,bq1x5cRqgUc] ] .footnote[Su, X., Au, O. K. C., & Lau, R. W. (2014, April). The implicit fan cursor: a velocity dependent area cursor. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 753-762). ACM.] --- .title[Design Tip #6: Separate Motor Size from Visible Size] .left-column[  ] .right-column[ - (a) Visual space appearance of buttons in a dialogue box. - (b) Motor space version of button design in (a) with much larger targets for certain buttons. - (c) Standard scroll-bar design. (d) Visual space appearance of scroll-bar redesigned to occupy smaller screen space. - (e) Motor space version of scroll-bar design in - (d) with larger targets for active areas. - (f) Visual space appearance of menu. - (g). Motor space version of the menu design in (f) with the distance m to more important items reduced by compressing the size Wm of less important items) ] .footnote[Semantic pointing widget designs from Blanch et al. 2004] ??? How would you implement this? - Manipulate picking - Whorfian effects --- .title[Would you ever make something too small virtually?] ??? How about a cancel button? --- .title[How long will this take?] .right-column-half[ Select 'Open New Window in Process' ] .left-column[  ] -- .right-column-half[ Steering law ... Fitts law integrated over the path you have to folow ] -- .right-column-half[ But it's not predictive at all. Why? ] ??? Doesn't account for actual user behavior --- .title[Actual User Behavior] .right-column[  ] --- .title[Actual User Behavior] .right-column[  ] ??? Overly eager cascading menus, wrong submenu - Common problem w/ web menus (but not Mac or Win) - could brainstorm solutions (e.g. snapping? set focus somehow?) - whorfian effects: this is hard to implement --- .title[Fitts law has a huge limitation] .body[ It is only predictive of **ERROR-FREE, EXPERT** behavior We will talk about this more in our discussion of [Interaction Design](../wk04/interfaces.html) ] --- .title[ What else might we want to measure? ] ??? - Time on Task -- How long does it take people to complete basic tasks? (For example, find something to buy, create a new account, and order the item.) - Accuracy -- How many mistakes did people make? (And were they fatal or recoverable with the right information?) - How strenuous (e.g. gaze has lower throughput but is less strenuous than head pointing Mackenzie 2018) ... - Recall -- How much does the person remember afterwards or after periods of non-use? - Emotional Response -- How does the person feel about the tasks completed? (Confident? Stressed? Would the user recommend this system to a friend?) Build up a list of good UI design principals from these basics - Undo - Predictability - ... What is missing? (e.g. fun) ] --- .title[Case Study: Text Entry Speed on Mobile Devices] .left-column[  ] .right-column[ Multi-tap ``` 77 88 444 222 55 0 22 777 666 9 66 0 333 666 99 q u i c k _ b r o w n _ f o x ``` Slow, lot's of overhead ] --- .title[Case Study: Text Entry Speed on Mobile Devices] .left-column[  ] .right-column[ Multi-tap (~ 20 WPM) ``` 77 88 444 222 55 0 22 777 666 9 66 0 333 666 99 q u i c k _ b r o w n _ f o x ``` T9 - dictionary based disambiguation (~ 20 WPM) ``` 7 8 4 2 5 0 2 7 6 9 6 0 3 6 9 q u i c k _ b r o w n _ f o x 843 78425 27696 369 58677 6837 843 5299 364 the quick brown fox jumps over the jazz dog tie stick crown lumps muds tie lazy fog vie vie ``` ] ??? Most common word (top) not always right --- .title[Case Study: Text Entry Speed on Mobile Devices] .left-column[  ] .right-column[ Method | Speed ----|---- Multi-tap | ~ 20WPM T9 - dictionary based disambiguation | ~ 20 WPM Swype | ~ 55 WPM Keyboard | ~ 55 WPM Flesky | Up to 80 WPM ([Guinness World Record!](https://www.cnn.com/2014/05/15/tech/mobile/guiness-record-fastest-text/) ) Speech | ~ 150 WPM Skeptical? Need to know % errors and time cost of error correction to truly evaluate ] ??? Flesky is combination of gestures and automatic error correction --- .title[Two Handed Interaction] .body[  ] ??? Used to be a norm Seems we forgotten to include it in our interfaces --- .title[Guiard's model of bimanual control] .body[ Hand | Role and Action ---- | ---- Non-preferred | Leads the preferred hand (in time) | Sets the spatial frame of reference for the preferred hand | Performs coarse movements Preferred | Follows the non-preferred hand | Works within established frame of reference set by the non-preferred hand | Performs fine movement Aside: Guiard is assuming right-handed. Tendencies somewhat less clear for left-handers, but we typically assume this still applies ] .footnote[ Seminal work Yves Guiard, “Asymmetric Division of Labor in Human Skilled Bimanual Action: The Kinematic Chain as a Model”, Journal of Motor Behavior, Vol. 19, No. 4, 1987, pp. 486-517. http://cogprints.org/625/ ] ??? non-preferred hand sets context for preferred coarser, slower movement tends to move first ??? Applications? - Mouse+keyboard - Lenses ] --- .title[What does theory say about keyboard layout?] .body[  ] .footnote[From [Evoluent keyboard advertisement](https://turningpointtechnology.com/KB/EV/KB1.asp)] --- .title[Analysis of alternative keyboard layout] .body[ Task | Leading Movement | Trailing/Overlapping Movement ----|----|---- Delete | Right hand — manipulate pointer with mouse; select by double clicking/dragging | Left hand — press DELETE (probably with little finger) Select an option in a window (see Figure 15) | Right hand — manipulate pointer with mouse; click option | Left hand — press ENTER (Assumes OK button is default) Click on a link in a browser | Left hand — navigate to link via PAGE UP and/or PAGE DOWN keys | Right hand — manipulate pointer with mouse; select link by clicking Open file; open folder; launch program | Right hand — manipulate pointer with mouse; single click on icon | Right hand — press ENTER (avoids error prone double-click operation) ] --- .title[Other Examples: Tool Stone for Magic Lenses] .body[ ![:youtube By manipulating a mouse and a cube (one in each hand) the user controls a lens and uses it on screen, V32SnUnAe2E] Inspired by [1994 CHI paper](https://dl.acm.org/citation.cfm?id=166126) Published in [UIST 2000](https://lab.rekimoto.org/projects/toolstone/) ] --- .title[Magnification] .body[  ] --- .title[Font Selection] .body[  ] ??? Fitts' law implications? --- .title[Debugging Lenses] .body[  ] .footnote[ Hudson, S. E., Rodenstein, R., and Smith, I. 1997. Debugging lenses: a new class of transparent tools for user interface debugging. In Proceedings UIST '97, 179-187. http://doi.acm.org/10.1145/263407.263542 ] --- .title[Advantages of Lenses] .body[ In context interaction Little or no shift in focus of attention and/or movement Alternate views in context and on demand ] ??? - tool is at/near action point - can compare in context - useful for “detail + context” visualization techniques -- .body[ Structured well for 2 handed input - non-dominant hand does course positioning (of the lens) - dominant hand does fine work "Spatial modes" - Use “where you click through” to establish meaning - Typically has a clear affordance for the meaning ] --- .title[Implementing Lenses] .small[ <div class="mermaid"> graph TD Root((Root)) --> LP((Lens Parent)) LP --> L[Lens] LP --> I((Interactor 1)) LP --> II((Interactor 2)) LP --> III((Interactor 3)) </div> ] .footnote[Edwards, Hudson, et al., “Systematic output modification in a 2D user interface toolkit”, UIST ’97. http://doi.acm.org/10.1145/263407.263537 ] ??? Can be implemented with special "lens parent" container & lens interactors - Lens may need to change results of picking (only positional is affected) in collusion with lens parent - Lens parent forwards all damage to all lenses - Lenses typically change any damage that overlaps them into damage of whole lens area - Can pass a subclass of Drawable to "ambush and modify" drawing for output effects --- .title[Discussion: Should scrolling be dominant or non-dominant?] .body[ What tasks is scrolling used for? Which precedes/follows? ] ??? Task | Characteristics Scrolling | precedes/overlaps other tasks | sets the frame of reference | minimal precision needed (coarse) Selecting, editing, reading, drawing, etc. | follows/overlaps scrolling | works within frame of reference set by scrolling | demands precision (fine) --- .title[# Summary Slide] .body[ - Make small targets larger - Put commonly used things close together - Use edges. They are infinite in W - Use pie menus for expert tasks (better yet, marking menus) - Use snapping to reduce D when targets are known - Separate motor and visible size 2 Handed input principles - Non preferred leads - Sets frame of reference - Preferred does fine movement ]
layout: true
