name: inverse layout: true class: center, middle, inverse --- # The whole toolkit Jennifer Mankoff CSE 340 Winter 2020 --- layout:false [//]: # (Outline Slide) # Today's goals - Discuss plan for midterm - Errata and finishing Monday - Let's look at colorpicker - Review and summarize core toolkit architecture --- # Plan for Midterm Will cover material from weeks 1-5 Will be in-class (50 minute exam) Contents - Long answer questions - Short answer questions - Coding questions 'CheatSheet' allowed (2 sided, hand written) --- # ERRATA from Monday .left-column[ <div class="mermaid"> graph TD V0((V0)) --> V1((V1)) V0 --> V4((V4)) V1 --> V2((V2)) V1 --> V3((V3)) V4 --> V5((V5)) V4 --> V6((V6)) class V0,V2,V5 blue class V1,V4,V3,V6 bluegreen </div> ] .right-column[ - I said that the pick list is done in-order. It's actually *post-order* - Picked Views = {`V2`, `V3`, `V1`, `V5`, `V6`, `V4`, `V0`} ] --- # ERRATA from Monday .left-column[ <div class="mermaid"> graph TD V0((V0)) --> V1((V1*)) V0 --> V4((V4*)) V1 --> V2((V2)) V1 --> V3((V3*)) V4 --> V5((V5)) V4 --> V6((V6*)) class V0,V2,V5 blue class V1,V4,V3,V6 bluegreen </div> ] .right-column[ - I said that the pick list is done in-order. It's actually *post-order* - Picked Views = {`V3`, `V1` `V6`, `V4`} ] --- # Other comments - layout varies depending on the specific layout container - Constraints don't need multiple passes/top down structure. - LinearLayout is post order for measure (measure the chilrden, who first measure their children) - in general, have to look at implementation - `onDraw` is usually pre-order (why? intuitively leaf nodes should be 'on top' of parents) - *Capture* is rarely used by component developers - *Bubble* is far more common - App developrs just need to use callbacks, not worry about this stuff --- # Colorpicker assignment (out Friday) --- .left-column[ ## Have now seen *almost* all the parts of an interface ] .right-column[ Input - Input models (events) - Event dispatch - Event handling (state machine) - Callbacks to application *likely coding problem* Output - Interactor Hierarchy design & use - Drawing models (`onDraw()`) *likely coding problem* - Layout (`onLayout()` or `XML`) *likely coding problem* - Damage and redraw process ] --- .left-column-half[ # Core Toolkit Architecture Wait for *Event* *Dispatch* may cause change to: - interactor state - interactor hierarchy - application model ] ??? What do you need to know though? Mostly only if you are a component developer -- .right-column-half[ # Component Developer Implement event handler (e.g. `onTouch()`) Handle *Dispatch* - Update interactor state & model if needed - Notify application if application model should change (with a callback) - Call `invalidate()` ] ??? What about if you are an app developer? Basically, just use callbacks --- .left-column-half[ # Core Toolkit Architecture If damage do - *layout* (may change) - position - size - If damage do *redraw* ] -- .right-column-half[ # Component Developer maybe Implement `onMeasure()` and `onLayout()` (if a container) always Implement `onDraw()` but *never call it* (call `invalidate()` instead) ] --- ## View Update: .red[Damage/Redraw] How does the toolkit know what to redraw? What causes damage? ??? concrete example on next slide --- .left-column[ ## View Update: .red[Damage/Redraw] ] .right-column[ What should be redrawn?  ] --- .left-column[ ## View Update: .red[Damage/Redraw] ] .right-column[ What should be redrawn?  ] --- .left-column[ ## View Update: .red[Damage/Redraw] ] .right-column[ What should be redrawn?  ] --- ## View Update: .red[Damage/Redraw] How does the toolkit know what to redraw? What causes damage? ??? Can you think about other things? - Window hidden and re-exposed - Resizing - redrawing -- <br><br>Naive approach to redraw --- ## View Update: .red[Damage/Redraw]  ??? XXX TODO ADD pic like this using divs? - Can be slow (redrawing everything unecessary) - Can cause flickering - double buffering is better, - hence the 'Canvas' abstraction or equivalent - can then switch which FB is displayed (very fast) --- ## View Update: .red[Damage/Redraw]   --- ## View Update: .red[Damage/Redraw]   **Never** just draw: Why not? ??? - Update *state* - Report *damage* (by calling 'repaint()) - Wait for *toolkit to request redraw* (also works if damage comes from elsewhere) - How does it generalize to any cause of damage (always need state!!) --- ## View Update: .red[Damage/Redraw] How does the toolkit know what to redraw? - Let the component report: Damage/Redraw invoked by `invalidate()` or equivalent --- ## View Update: .red[Damage/Redraw] How does the toolkit know what to redraw? - Let the component report (`invalidate()`) **NOTE** we are *not* calling *onDraw()* directly (important for your assignment) - Aggregate - Usually calculate bounding box --- ## View Update: .red[Draw/Redraw] Virtual device abstraction provided by windowing system Component abstraction provided by toolkit - Enforced using clipping - Supported using coordinate system transformations Drawing is recursive - Makes it possible for parent to *decorate* kids - Parent responsible for making kids think they are the center of the universe (translate) - Clipping: intersect parent and child, also handled by parent ??? Allows each program to (mostly) pretend that it has the screen (frame buffer) to itself Allows each component to (mostly) pretend that it has the screen to itself --- # Core Toolkit Architecture If damage do - *layout* (may change) - position - size - If damage do *redraw* - Union of damage (any of those can cause it) used to trigger redraw of anything inside that union - Drawing + clipping – standard drawing order, but only for things damaged; clipped to damage region - Clear damage --- name: inverse layout: true class: center, middle, inverse --- # Using Essential Geometry as the basis for state --- layout:false # Using Essential Geometry as the basis for state What is the essence of this scrollbar?  How do we capture/implement that? --- # Scrollbar State machine with Essential Geometry .left-column-half[ Sometimes also use “guards” --> **Propositional Production System** - predicate (Boolean expr) before event - adds extra conditions required to fire - typical notation: event : pred ? action - e.g. MouseDown : InsideAboveThumb? Scrollup() Note: FSM augmented with guards is Turing complete ] .right-column-half[ <div class="mermaid"> graph LR S((.)) --> START((START)) START -- "MouseDown:InThumb?" --> SCROLLING((SCROLLING)) START -- "MouseClick:InsideAboveThumb?Scrollup()" --> DONE((DONE)) START -- "MouseClick:InsideBelowThumb?Scrolldown()" --> DONE((DONE)) SCROLLING -- "MouseMove:updateThumbDocument()" --> SCROLLING SCROLLING -- "MouseUp:Keep()" --> DONE linkStyle 0 stroke-width:4px; linkStyle 1 stroke-width:4px; linkStyle 2 stroke-width:4px; linkStyle 3 stroke-width:4px; linkStyle 4 stroke-width:4px; linkStyle 5 stroke-width:4px; class S invisible class START start class SCROLLING normal class DONE finish </div> ] --- # Let's try it for a button .left-column-half[ Essential geometry is: - InsideButton - OutsideButton and methods for - `indentButton()` (when button is pressed) - `normalButton()` (when button is not pressed) - `invokeAction()` (when the user releases in the button) - `cancelAction()` (when the user releases outside the button) ] --- # You should have something like this .left-column-half[ Essential geometry is: - InsideButton - OutsideButton and methods for - `indentButton()` (when button is pressed) - `normalButton()` (when button is not pressed) - `invokeAction()` (when the user releases in the button) - `cancelAction()` (when the user releases outside the button) ] .right-column-half[ <div class="mermaid"> graph TD S((.)) --> START((START)) START -- "DOWN:Inside?indentButton()" --> PRESSED((PRESSED)) PRESSED -- "MOVE:Outside?normalButton()" --> PRESSED PRESSED -- "UP:Outside?cancelAction()" --> DONE[DONE] PRESSED -- "UP:Inside?invokeAction()" --> DONE PRESSED -- "MOVE:Inside?indentButton()" --> PRESSED linkStyle 0 stroke-width:4px; linkStyle 1 stroke-width:4px; linkStyle 2 stroke-width:4px; linkStyle 3 stroke-width:4px; linkStyle 4 stroke-width:4px; linkStyle 5 stroke-width:4px; class S invisible class START start class PRESSED normal class DONE finish </div> ] --- # How do we implement this? .left-column-half[ - Implement in `onTouch()` using a switch statement - Assume there is an `essentialGeometry(Point p)` method. It returns a struct that tells you what part of the geomery you are in for that point. - Assume implementations of all the methods - Assume a field, `state` which is the current state of the state machine - enums `Geometry` and `State` for comparing against ] .right-column-half[ ```java public boolean onTouch(MotionEvent e) { geometry = essentialGeometry(new Point(e.getX(), e.getY()); switch(state) { case State.START: if (geometry == Geometry.INSIDE && e.getAction() == MotionEvent.ACTION_DOWN) { indentButton(); state = State.PRESSED; } case PRESSED if (e.getAction() == MotionEvent.ACTION_MOVE) { if (geometry == Geometry.INSIDE) { indentButton() } else { normalButton() } } else if (e.getAction() == MotionEvent.ACTION_UP) { state = State.START; // note we don't actually use the DONE state if (geometry == Geometry.INSIDE) { invokeAction() } else { cancelAction() } } ... ``` ] --- .left-column[ ## Harder Button  ] .right-column[ - Determine the Events (triggers) - Determine the States - Determine the Queries (essential geometry, context) - Determine the Actions ] ??? What constitutes an “event” varies - may be just low level events, or - higher level (synthesized) events - e.g. region-enter, press-inside What is missing? Query fields --- .left-column[ ## Facebook Button Solution  ] -- .right-column[ Press:?inside => highlight(), start_animation(), small, active<br> AnimateStep ==> update(), active<br> AnimateFinish ==> !small, active<br> Release:inside,small => unhighlight(), exit()<br> Release:inside,!small => add_to_chat(), small, unhighlight(), exit()<br> __rest is unknowable from this animation__ <div class="mermaid"> graph LR S((.)) --> A((Start)) A -- "Press:?inside/highlight(), start_animation()" --> B((Active)) B -- "AnimateStep,update()" --> B B -- "AnimateFinish,!small"--> B B -- "Release,inside:small, unhighlight" -->D(End) B -- "Release,inside:!small,add_to_chat(),unhighlight()" --> D classDef finish outline-style:double,fill:#d1e0e0,stroke:#333,stroke-width:2px; classDef normal fill:#e6f3ff,stroke:#333,stroke-width:2px; classDef start fill:#d1e0e0,stroke:#333,stroke-width:4px; classDef invisible fill:#FFFFFF,stroke:#FFFFFF,color:#FFFFFF linkStyle 0 stroke-width:4px; linkStyle 1 stroke-width:4px; linkStyle 2 stroke-width:4px; linkStyle 3 stroke-width:4px; linkStyle 4 stroke-width:4px; linkStyle 5 stroke-width:4px; class S invisible class A start class D finish class B normal </div> ] --- # When to use PPSs You're probably already using them, just not intentionally (and maybe less well as a result) PPSs are a good way to do control flow in event driven systems Can do (formal or informal) analysis - are all possible inputs (e.g. errors) handled from each state - what are next legal inputs: can use to enable / disable Can be automated based on higher level specification --- # How do we implement this? .left-column-half[ - Implement in `onTouch()` using a switch statement - Assume there is an `essentialGeometry(Point p)` method. It returns a struct that tells you what part of the geomery you are in for that point. - Assume implementations of all the methods - Assume a field, `state` which is the current state of the state machine - enums `Geometry` and `State` for comparing against ] .right-column-half[ ```java public boolean onTouch(MotionEvent e) { geometry = essentialGeometry(new Point(e.getX(), e.getY()); switch(state) { case State.START: if (geometry == Geometry.INSIDE && e.getAction() == MotionEvent.ACTION_DOWN) { indentButton(); state = State.PRESSED; } case PRESSED if (e.getAction() == MotionEvent.ACTION_MOVE) { if (geometry == Geometry.INSIDE) { indentButton() } else { normalButton() } } else if (e.getAction() == MotionEvent.ACTION_UP) { state = State.START; // note we don't actually use the DONE state if (geometry == Geometry.INSIDE) { invokeAction() } else { cancelAction() } } ... ``` ] --- .left-column[ ## Harder Button  ] .right-column[ - Determine the Events (triggers) - Determine the States - Determine the Queries (essential geometry, context) - Determine the Actions ] ??? What constitutes an “event” varies - may be just low level events, or - higher level (synthesized) events - e.g. region-enter, press-inside What is missing? Query fields --- .left-column[ ## Facebook Button Solution  ] -- .right-column[ Press:?inside => highlight(), start_animation(), small, active<br> AnimateStep ==> update(), active<br> AnimateFinish ==> !small, active<br> Release:inside,small => unhighlight(), exit()<br> Release:inside,!small => add_to_chat(), small, unhighlight(), exit()<br> __rest is unknowable from this animation__ <div class="mermaid"> graph LR S((.)) --> A((Start)) A -- "Press:?inside/highlight(), start_animation()" --> B((Active)) B -- "AnimateStep,update()" --> B B -- "AnimateFinish,!small"--> B B -- "Release,inside:small, unhighlight" -->D(End) B -- "Release,inside:!small,add_to_chat(),unhighlight()" --> D classDef finish outline-style:double,fill:#d1e0e0,stroke:#333,stroke-width:2px; classDef normal fill:#e6f3ff,stroke:#333,stroke-width:2px; classDef start fill:#d1e0e0,stroke:#333,stroke-width:4px; classDef invisible fill:#FFFFFF,stroke:#FFFFFF,color:#FFFFFF linkStyle 0 stroke-width:4px; linkStyle 1 stroke-width:4px; linkStyle 2 stroke-width:4px; linkStyle 3 stroke-width:4px; linkStyle 4 stroke-width:4px; linkStyle 5 stroke-width:4px; class S invisible class A start class D finish class B normal </div> ] --- name: inverse layout: true class: center, middle, inverse --- # END OF DECK
