name: inverse layout: true class: center, middle, inverse --- # The whole toolkit Jennifer Mankoff CSE 340 Winter 2021 --- layout:false # The Whole Toolkit Architecture Input - **Input models (events)** - Event dispatch - Event handling (state machine) - Callbacks to application Output - Interactor Hierarchy design & use - Drawing models (`onDraw()`) - Layout (`onLayout()` or `XML`) - Damage and redraw process --- # Events: Review  -- CTRL does not generate an event (it is a modifier) Everything else does --- layout:false # The Whole Toolkit Architecture Input - Input models (events) - **Event dispatch** - Event handling (state machine) - Callbacks to application Output - Interactor Hierarchy design & use - Drawing models (`onDraw()`) - Layout (`onLayout()` or `XML`) - Damage and redraw process --- # Event Dispatch: Picking  An interactor hierarchy and the corresponding interface are shown above. If the user clicks on the image at (4,4) what will be the order of views in the pick list? Assume that the drawing canvas does not accept touch input. Assign the correct number at right to the item at left. - BAD QUESTION! you should be confused (sorry!). Why? ??? - 1: LineA (Black) - 2: CircleA (Green) - Nothing else is positionally associated --- # Event Dispatch: Picking  - 1: LineA (Black) - 2: CircleA (Green) - Nothing else is positionally associated --- # Event Dispatch: Delivering Review - Top-down, Bottom-Up, Bubble-out, Focused-based: these are theoretical Event dispatch approaches. - *Capture* is rarely used by component developers - Capture is top-down: start on the in the biggest interactor that contains the event, then narrow in on which window actually will use the event - *Bubbling* is far more common - Bubbling is bottom-up - start with the window at the front (the last drawn, lowest in the interactor tree) - see if the event is consumed by that interactor. If not, go up the tree. - Android does it a bit differently. - Hacky version of capture before picking - Most stuff is in bubble order, but some callbacks assume you consume the event (no option to return `true` vs `false`) --- # Event Dispatch: Bubble  For the same image, what interactor will get the event first in Bubble? -- Why LineA, Black? -- It is the last thing (under the locator event) drawn (reflected in its position in the interactor hierarchy) --- # Focus versus Positional dispatch In which of the following situations is focus dispatch used during event handling, and in which is positional dispatch used? - When the mouse moves off a scrollbar Most of you got this right, quiz was auto-marked wrong.  ??? Right answer: "Even when the mouse isn't positionally above the scrollbar it is important to still update the thumb position and thus input is delivered to it positionally. When you click, the position of the mouse/finger determines position. The same is true for select" --- # Focus versus Positional dispatch In which of the following situations is focus dispatch used during event handling, and in which is positional dispatch used? - When the mouse moves off a scrollbar - When clicking on a button (answer is positional) - When selecting a word in a text area (same) - When typing in a text area (answer is focus) --- layout:false # The Whole Toolkit Architecture Input - Input models (events) - Event dispatch - **Event handling (state machine)** - Callbacks to application Output - Interactor Hierarchy design & use - Drawing models (`onDraw()`) - Layout (`onLayout()` or `XML`) - Damage and redraw process --- # Essential Geometry represents places --- layout:false # Essential Geometry represents places .left-column-half[  ] .right-column-half[ - What is the essence (or nature) of this scrollbar? Where can you interact with it? ] --- # Essential Geometry represents places .left-column-half[  ] .right-column-half[ - What is the essence (or nature) of this scrollbar? Where can you interact with it? - on the thumb - inside the scrollbar above the thumb - inside the scrollbar below the thumb - inside up arrow - inside the down arrow - How do we implement a scroll bar like this? ] --- # Essential Geometry represents places Implementatin using Enums Enums are a group of named constants - Often used for developing interactions for defining PPS States and Essential Geometry - Will be used in ColorPicker and in Menu assignment for interaction *and* for experimental conditions - Good code quality: limits choices of what a variable can be. [Documentation](https://docs.oracle.com/javase/tutorial/java/javaOO/enum.html) --- # Essential Geometry Implementation - An `essentialGeometry(MotionEvent event)` method converts position to geometry - It returns a enum that tells you what part of the geometry you are in for that point. - such as `Inside` and `Outside` --- # State Machine uses Essential Geometry <div class="mermaid"> stateDiagram-v2 [*] --> Scrolling: MouseDown / Inside [*] --> Ready: MouseMove / Inside Ready --> [*]: MouseClick / <BR> AboveThumb?<BR>Scrollup() Ready --> [*]: MouseClick / <BR> BelowThumb?<BR>Scrolldown() Ready --> [*]: MouseMove / <BR> Outside Scrolling --> Scrolling: MouseMove / <BR> UpdateThumbDocument() Scrolling --> [*]: MouseUp / <BR> KeepLocation() </div> --- # Let's try it for a button .left-column-half[ Essential geometry is: - InsideButton - OutsideButton and methods for - `indentButton()` (when button is pressed) - `normalButton()` (when button is not pressed) - `invokeAction()` (when the user releases in the button) - `cancelAction()` (when the user releases outside the button) ] --- # You should have something like this <BR><BR> .left-column-half[ Essential geometry is: - Inside - Outside and methods for - `indentButton()` - `normalButton()` - `invokeAction()` - `cancelAction()` ] .right-column-half[ <div class="mermaid" > stateDiagram-v2 [*] --> Inside: DOWN / <BR> Inside? <BR>indentButton() Inside --> Outside: MOVE / <BR> Outside? <BR>normalButton() Inside --> [*]: UP / <BR> invokeAction() Outside --> [*]: UP / <BR> cancelAction() Outside --> Inside: MOVE / <BR> Inside? <BR> indentButton() Inside --> Outside: MOVE / <BR> Outside? <BR> normalButton() </div> ] --- # How do we implement this? - Implement in `onTouch()` using a switch statement - Assume implementations of all the methods - Current state is an `Enum` - Current geometric location is an `Enum` --- # How do we implement this? ```java @Override public boolean onTouch(MotionEvent e) { EssentialGeometry geometry = essentialGeometry(event); switch (state) { case State.START: // arrow from start to INSIDE break; case State.INSIDE: // need both arrows case State.OUTSIDE: // need both arrows default: break; // ignore everything else } return false; } ``` --- # How do we implement this? ```java @Override public boolean onTouch(MotionEvent e) { EssentialGeometry geometry = essentialGeometry(event); switch (state) { case State.START: if (e.getAction() == MotionEvent.ACTION_DOWN && geometry == Geometry.INSIDE) { indentButton(); state = State.INSIDE; return true; } break; case State.INSIDE: ... ``` --- # How do we implement this? ```java case State.INSIDE: case State.OUTSIDE: ... ``` --- # How do we implement this? ```java case State.INSIDE: if (e.getAction() == MotionEvent.ACTION_UP) { invokeAction(); state = State.START; return true; } else if (e.getAction() == MotionEvent.ACTION_MOVE && geometry == Geometry.OUTSIDE) { normalButton() state = State.OUTSIDE return true; } break; case State.OUTSIDE: if (e.getAction() == MotionEvent.ACTION_UP) { cancelAction(); state = State.START; return true; } else if (e.getAction() == MotionEvent.ACTION_MOVE && geometry == Geometry.INSIDE) { indentButton() state = State.INSIDE return true; } break; default: break; ``` --- # Notes PPSs are very good (for this job) but do have limits Don't handle independent actions very well (state explosion) Mostly useful for smaller things - Great for individual components - Not so great for whole dialogs Path of least resistance is rigid sequencing Ask: is this good for what I am doing? ??? --- # When to use PPSs You're probably already using them, just not intentionally (and maybe less well as a result) PPSs are a good way to do control flow in event driven systems Can do (formal or informal) analysis - are all possible inputs (e.g. errors) handled from each state - what are next legal inputs: can use to enable / disable Can be automated based on higher level specification --- # Harder Option: Checkbox .left-column50[  ] .right-column40[ - Determine the Events (triggers) - Determine the States - Determine the Queries (essential geometry, context) - Determine the Actions ] ??? What constitutes an “event” varies - may be just low level events, or - higher level (synthesized) events - e.g. region-enter, press-inside What is missing? Query fields --- # Harder Option: Checkbox .left-column40[  ] .right-column60[ - **Events**: Mouse MOVE, UP, DOWN - **States**: PRESSED, NOT_PRESSED - **Queries**: INSIDE, OUTSIDE - **Actions**: toggleCheckbox(); highlight()/unhighlight() ] --- # Harder Option: Checkbox .left-column40[  ] .right-column60[ - **Events**: Mouse MOVE, UP, DOWN - **States**: PRESSED, NOT_PRESSED - **Queries**: INSIDE, OUTSIDE - **Actions**: toggleCheckbox(); highlight()/unhighlight() ] --- # Harder Option: Checkbox .left-column40[  ] .right-column60[ - **Events**: Mouse MOVE, UP, DOWN - **States**: PRESSED, NOT_PRESSED - **Queries**: INSIDE, OUTSIDE - **Actions**: toggleCheckbox(); highlight()/unhighlight() ] --- # END OF DECK --- # Another Harder Button Example .left-column[  ] .right-column[ <!-- <div class="mermaid"> --> <!-- stateDiagram-v2 --> <!-- [*] -\->Active: Press/inside?<BR>highlight(),start_animation() --> <!-- Active -\-> Active: AnimateStep/<BR>update() --> <!-- Active -\-> Active: AnimateFinish/<BR>!small --> <!-- Active -\-> [*]: Release,inside/<BR>small, unhighlight --> <!-- Active -\-> [*]: Release,inside/<BR>!small,add_to_chat(),unhighlight() --> <!-- </div> --> <div class="mermaid"> graph LR S((.)) --> A((Start)) A -- "Press/inside?highlight(),start_animation()" --> B((Active)) B -- "AnimateStep/ <BR> update()" --> B B -- "AnimateFinish/ <BR> !small"--> B B -- "Release,inside/ <BR> small, unhighlight" -->D(End) B -- "Release,inside/<BR> !small,add_to_chat(),unhighlight()" --> D classDef finish outline-style:double,fill:#d1e0e0,stroke:#333,stroke-width:2px; classDef normal fill:#e6f3ff,stroke:#333,stroke-width:2px; classDef start fill:#d1e0e0,stroke:#333,stroke-width:4px; classDef invisible fill:#FFFFFF,stroke:#FFFFFF,color:#FFFFFF linkStyle 0 stroke-width:2px; linkStyle 1 stroke-width:2px; linkStyle 2 stroke-width:2px; linkStyle 3 stroke-width:2px; linkStyle 4 stroke-width:2px; linkStyle 5 stroke-width:2px; class S invisible class A start class D finish class B normal </div> ]
