name: inverse layout: true class: center, middle, inverse --- # Event Handling II - Event Delivery using the Interactor Hierarchy Lauren Bricker CSE 340 Winter 23 --- name: normal layout: true class: --- # Accessibility warmup Do the problem in our [Ed Lesson](https://edstem.org/us/courses/32031/lessons/51105/slides/286378) for today. This is related to our [Spot the Heron](https://gitlab.cs.washington.edu/cse340/exercises/cse340-spot-the-heron) case study. Possibly helpful: Talkback will "tell" what an item on the screen is if - it has a content description (`android:contentDescription="..."`) OR - is focusable (`android:focusable="true"`) OR - it is clickable (`android:clickable="true"`) OR - You have attached a `onClickListener` to the interactor. --- [//]: # (Outline Slide) # Today's goals - Last class: - What is a Model View Controller? - What is an event? ([finish](/courses/cse340/23wi/slides/wk04/events.html#27)) - What is a callback? - Today: how events are related to App Development... how input events are delivered - The event hierarchy - Input dispatch process - Review & New information about Callbacks Administrivia - Accessibility (code and report/reflection) due Thur 2-Feb --- # Review: Logical (Physical) Device Approach - Valuator → returns a scalar value (like a slider) - Button → returns integer value - Locator → returns position on a logical view surface - Keyboard → returns character string - Stroke → obtain sequence of points --- # Review: Contents of Event Record Think about one of your real world events from last class again. What do we need to know? **What**: Event Type **Where**: Event Target **When**: Timestamp **Value**: Event-specific variable **Context**: What was going on? ??? Discuss each with examples --- # Review: Contents of Event Record .left-column50[ Example: The cat meowed: What do we need to know? **What**: A cat's meow **Where**: My ears (as the input target) **When**: 9:00pm **Value**: From the kitchen, near the food bowl **Context**: Whining, pacing, and generally being a pest. ] .right-column50[ ![:youtube Jack the Cat begs for food every day at 9pm and makes pouty sounds when he doesn't get it,lZfw6nqg7ko] ] ??? Jack wants food. --- # Review: Contents of Event Record Example: The cat meowed: What do we need to know? **What**: A cat's meow **Where**: My ears (as the input target) **When**: 10:00am **Value**: From the hallway or following us around **Context**: Purring and rubbing up against us ??? Jack wants attention. --- # Your turn Imagine you are writing a program to respond to a user saying "Ok Google" followed by some statement in the vicinity of an Android phone. Fill in the fields of the Event Record for this event on the [Ed Lesson](https://edstem.org/us/courses/32031/lessons/51105/slides/286383). **What**: **Where**: **When**: **Value**: **Context**: Any possible modifiers ??? # Answer Imagine you are writing a program to listen for "Ok Google" on an Android phone. **What**: Speaking starts (or speaking ends) **Where**: Normally position on screen. In this case, ill defined because this event is not dispatched positionally. Can be blank, or may hold the current cursor location. **When**: Timestamp of the audio is heard or stops **Value**: The content of the audio recorded **Context**: Any possible modifiers --- # Review Device Independence - We need device independence to handle a variety of ways to get input - We need a uniform and higher level abstraction for input (events) Component Independence - Given a model for representing input, how do we get inputs delivered to the right component? - Valuator → returns a scalar value - Button → returns integer value - Locator → returns position on a logical view surface - Keyboard → returns character string - Stroke → obtain sequence of points --- ## Android Events Each kind of event in Android its own class but they all implement the [InputEvent](https://developer.android.com/reference/android/view/InputEvent) abstract class. - A little hard to find all the parts defined in one place - Harder to deal with uniformly - But easily extensible for new event types .red[Artificial] events are a thing - Window Events - Search Event - others. --- # Android [InputEvent](https://developer.android.com/reference/android/view/InputEvent) ```java java.lang.Object ↳ android.view.InputEvent public abstract class InputEvent extends Object implements Parcelable { InputDevice getDevice() // The device this event came from abstract long getEventTime() // When the event happened abstract int getSource() // The source of the event } ``` --- # Android [MotionEvent](https://developer.android.com/reference/android/view/MotionEvent) ```java java.lang.Object ↳ android.view.InputEvent ↳ android.view.MotionEvent public final class MotionEvent extends InputEvent implements Parcelable { int getAction() // up, down etc int getActionIndex() // multi touch support -- which pointer float getX() // position float getY() // position int getButtonState() // pressed or not, for example long getDownTime() float getOrientation(int pointerIndex) float getPressure() float getSize() // fingers aren't pixel sized // and many more... } ``` --- # Aside: Multiple Hierarchies discussed so far Can you think of them? -- count: false - Inheritance hierarchy for interactors (View) - Inheritance hierarchy for layout - Component hierarchy (Interactors and View Groups) - Animation hierarchy - Also remember how the component hierarchy is used in drawing (Z-order) -- count: false And now... - Inheritance hierarchy for events --- # Events can represent abstract concepts... Can you think of any? -- ...Think of how the state of the app changes as you use it. -- - __Create__: Android is done creating the object -- - __Active__: in foreground of screen, and receives input -- - __Paused__: activity lost focus (is no longer displayed on screen) -- - __Stopped__: - Another activity has focus - Original activity remains in memory -- perfect place to save data & state - Often can be *killed* by Android OS if not resumed relatively quickly -- - __Inactive__: after an activity has been killed; or before it is launched -- - __Timer__: Animation --- # Activity State Change Events .left-column-half[  ] .right-columns-half[ Look familiar? ] --- template: inverse ## Event Delivery (Theory) ### How do events get to where they need to be? --- # Application’s main UI thread How the Toolkit Architecture deliver events... ```java while (true) { // Gets event from the application message queue. // If the queue is empty, we just wait here. Event e = GetEventFromMessageQueue(); // Dispatches the event to the appropriate application window // (the destination address is in the event data). DispatchEventToMessageDestination(e); } ``` -- count: false This is setup and managed by the UI framework (e.g., .NET, WPF, Android, Swift). You don't program this directly but you should know it’s there and how it functions! --- # Input __Dispatch__ Process Input thread: - When a user interacts, __events__ are created - Events go into a queue ??? What do you think of when you hear the word "thread"? How does it relate to CS? Remember a queue is a data structure that is first in first out. --- # Input __Dispatch__ Process Input thread Dispatch thread: - Front event comes off queue - How does a toolkit decide where to send events? -- count: false - Depends: Is it Focus or Positional input? - Focus list (in order based on interest) - Positional list (z-order under cursor based on interactor hierarchy) --- ## Event Dispatch (theory) .left-column-half[ ] .right-column-half[ Dispatch Strategies (theory) - Positional - Bottom-up - Top-down - Bubble out - Focus-based ] --- ## Event Dispatch (theory) .left-column-half[  ] .right-column-half[ - Bottom up dispatch - The event is directed to the "lowest", frontmost interactor in the tree that contains the mouse position - That interactor might not want the input, so it goes to window at the next level up the tree ] --- ## Event Dispatch (theory) .left-column-half[  ] .right-column-half[ - Bottom up dispatch - Top down dispatch - The event is directed to topmost window that contains the mouse location - That window decides how to dispatch it further (recursively) - Common in OO toolkits - Useful in situations where the interactor is "view only" (parent wants to disable input to child) ] --- ## Event Dispatch (theory) .left-column-half[  ] .right-column-half[ - Bottom up dispatch - Top down dispatch - Bubble-out dispatch - Used when there is no clear nesting of windows and groups of interactive objects - Tree is traversed as in top down approach, but bounding rectangles are hints, not guarantees - Objects checked in opposite order from drawing: frontmost items are checked first - Object that was hit is attached to the event, ancestors know what was selected ] --- ## Event Dispatch .left-column-half[  ] .right-column-half[ - Bottom up dispatch - Top down dispatch - Bubble-out dispatch - Focus dispatch - No straighforward location of where the event happened - Windowing system determines which window/interactor should get the input. - Common example: where should key press/release events go? ] --- ## Event Dispatch .left-column-half[ ] .right-column-half[ What happens after an event is consumed by an *interactor* - Possibly nothing (I might open a menu but not select anything, for example) - The *application* may need to be notified ] --- ## Event Dispatch .left-column-half[ ] .right-column-half[ Callbacks handle *application* response to events - Allows application to update model - Can be implemented using toolkit provided listeners - Can implment custom listeners for best flexibility ] --- # Summary: Relating this back to the phone .left-column[ <div class="mermaid"> graph LR ap[Application Program] hlt[High Level Tools] t[Toolkit] w[Window System] o[OS] h[Hardware] class ap,w,o,h,hlt,t yellow </div> ] .right-column[ Application Program: - Views - **(M) Stores local model (am I pressed, etc)** - **(V) Draws itself on screen** - (C) Implements callbacks for user events (such as onTouch) - Whole Application - **(M) Stores application model (data/state of app)** - **(V) Sets up the interactor hierarchy** - (C) Implements callbacks for interface events (such as buttonPressed) ] --- # End of Deck
