name: inverse layout: true class: center, middle, inverse --- # Event Delivery using the Interactor Hierarchy Jennifer Mankoff CSE 340 Winter 2021 --- name: normal layout: true class: --- [//]: # (Outline Slide) # Today's goals Discuss how Input Events are delivered - Review: Events - Event hierarchy - Input dispatch process - Review & New information about Callbacks --- # Review: Logical Device Approach - Valuator → returns a scalar value (like a slider) - Button → returns integer value - Locator → returns position on a logical view surface - Keyboard → returns character string - Stroke → obtain sequence of points --- # Review: Contents of Event Record Think about one of your real world events from last class again. What do we need to know? **What**: Event Type **Where**: Event Target **When**: Timestamp **Value**: Event-specific variable **Context**: What was going on? ??? Discuss each with examples --- # Review: Contents of Event Record Example: The cat meowed: What do we need to know? **What**: A cat's meow **Where**: My ears (this is the input target) **When**: 9:00pm **Value**: From the kitchen, near the food bowl **Context**: Whining, pacing, and generally being a pest. .footnote[It really does happen see [this video](img/event-delivery/jackhungry.mov)] ??? Jack wants food. --- # Contents of Event Record Example: The cat meowed: What do we need to know? **What**: A cat's meow **Where**: My ears (this is the input target) **When**: 10:00am **Value**: From the hallway or following us around **Context**: Purring and rubbing up against us ??? Jack wants attention. --- # Your turn Imagine you are writing a program to listen for "Hey Siri" on an Android phone. Fill in the fields of the Event Record for this event in the chat. Don't press send! **What**: **Where**: **When**: **Value**: **Context**: Any possible modifiers --- # Your turn Imagine you are writing a program to listen for "Hey Siri" on an Android phone. Fill in the fields of the Event Record for this event in the chat. Don't press send! **What**: Speaking starts (or speaking ends) **Where**: Normally position on screen. In this case, ill defined because this event is not dispatched positionally. Can be blank, or may hold the current cursor location. **When**: Timestamp of the audio heard **Value**: The content of the audio recorded **Context**: Any possible modifiers --- # Review Device Independence - We need device independence to handle a variety of ways to get input - We need a uniform and higher level abstraction for input (events) Component Independence - Given a model for representing input, how do we get inputs delivered to the right component? - Valuator → returns a scalar value - Button → returns integer value - Locator → returns position on a logical view surface - Keyboard → returns character string - Stroke → obtain sequence of points --- ## Android Events Each kind of event in Android its own class but they all implement the [InputEvent](https://developer.android.com/reference/android/view/InputEvent) abstract class. - A little hard to find all the parts defined in one place - Harder to deal with uniformly - But easily extensible for new event types .red[Artificial] events are a thing - Window Events - Search Event - others. --- # Android [MotionEvent](https://developer.android.com/reference/android/view/MotionEvent) ```java java.lang.Object ↳ android.view.InputEvent ↳ android.view.MotionEvent public final class MotionEvent extends InputEvent implements Parcelable { int getAction() // up, down etc int getActionIndex() // multi touch support -- which pointer float getX() // position float getY() // position int getButtonState() // pressed or not, for example long getDownTime() float getOrientation(int pointerIndex) float getPressure() float getSize() // fingers aren't pixel sized // and many more... } ``` --- # Aside: Multiple Hierarchies discussed so far in class Can you think of them? -- - Inheritance hierarchy for interactors (View) - Inheritance hierarchy for layout - Inheritance hierarchy for events - Component hierarchy (Interactors and View Groups) - Animation hierarchy -- - Also remember how the component hierarchy is used in drawing (Z-order) --- # Events can represent abstract concepts... Can you think of any? -- ...Think of how the state of the app changes as you use it. -- - __Create__: Android is done creating the object -- - __Active__: in foreground of screen, and receives input -- - __Paused__: activity lost focus (is no longer displayed on screen) -- - __Stopped__: - Another activity has focus - Original activity remains in memory -- perfect place to save data & state - Often can be *killed* by Android OS if not resumed relatively quickly -- - __Inactive__: after an activity has been killed; or before it is launched -- - __Timer__: Animation --- # Activity State Change Events .left-column-half[  ] .right-columns-half[ Look familiar? ] --- # How does the Toolkit Architecture deliver events? Application’s UI Thread (This is the main thread of your program) ```java while (true) { // Gets event from the application message queue. // If the queue is empty, we just wait here. Event e = GetEventFromMessageQueue(); // Dispatches the event to the appropriate application window // (the destination address is in the event data). DispatchEventToMessageDestination(e); } ``` -- This code is automatically setup and managed by the UI framework you are using (e.g., .NET, WPF, Android, Swift). You don’t really need to worry about it (but you should know it’s there and how it functions!) --- # Input __Dispatch__ Process Input thread: - When a user interacts, __events__ are created - Events go into a queue ??? What do you think of when you hear the word "thread"? How does it relate to CS? Remember a queue is a data structure that is first in first out. --- # Input __Dispatch__ Process Input thread Dispatch thread: - Front event comes off queue - How does a toolkit decide where to send events? -- - Depends: Is it Focus or Positional input? - Focus list (in order based on interest) - Positional list (z-order under cursor based on interactor hierarchy) --- .left-column-half[ ## Event Dispatch ] .right-column-half[ Dispatch Strategies (theory) - Positional - Bottom-up - Top-down - Bubble out - Focus-based ] --- .left-column-half[ ## Event Dispatch (theory)  ] .right-column-half[ - Bottom up dispatch - The event is directed to the "lowest", frontmost interactor in the tree that contains the mouse position - That interactor might not want the input, so it goes to window at the next level up the tree ] --- .left-column-half[ ## Event Dispatch (theory)  ] .right-column-half[ - Bottom up dispatch - Top down dispatch - The event is directed to topmost window that contains the mouse location - That window decides how to dispatch it further (recursively) - Common in OO toolkits - Useful in situations where the interactor is "view only" (parent wants to disable input to child) ] --- .left-column-half[ ## Event Dispatch (theory)  ] .right-column-half[ - Bottom up dispatch - Top down dispatch - Bubble-out dispatch - Used when there is no clear nesting of windows and groups of interactive objects - Tree is traversed as in top down approach, but bounding rectangles are hints, not guarantees - Objects checked in opposite order from drawing: frontmost items are checked first - Object that was hit is attached to the event, ancestors know what was selected ] --- .left-column-half[ ## Event Dispatch  ] .right-column-half[ - Bottom up dispatch - Top down dispatch - Bubble-out dispatch - Focus dispatch - No straighforward location of where the event happened - Windowing system determines which window/interactor should get the input. - Common example: where should key press/release events go? ] --- .left-column-half[ ## Event Dispatch ] .right-column-half[ What happens after an event is consumed by an *interactor* - Possibly nothing (I might open a menu but not select anything, for example) - The *application* may need to be notified ] --- .left-column-half[ ## Event Dispatch ] .right-column-half[ Callbacks handle *application* response to events - Update Application Model - Best implemented using custom listeners ] --- template: inverse ## Event Listeners ### Putting theory into practice ??? - Implementation strategy - Basic goal: notify something of a change --- # Callbacks in Android - At the time Android was created the toolkit developers have *no* idea how every app may want to respond to events - Listeners are an interface that acts as a _callback_ method -- - Recall interfaces specify behavior (but no data) that can be inherited. -- - Must be *registered with* a particular `View` - The toolkit architecture (implemented in `View`) then delivers events that arrive at *that View* to *those listeners* - i.e. once the `View` is interacted with by the user -- - A `View` can have listen for different types of interactions ??? What is an interface? --- # Standard Listener Interfaces The toolkit has pre-defined interfaces so apps or components can respond to events such as clicks or touches. .left-column30[ - These interfaces are implemented inside `View` as *inner classes* - More listed in the [API Documentation](https://developer.android.com/guide/topics/ui/ui-events.html) ] .right-column70[ ```java // User tapped a view public static interface View.OnClickListener { ... } // User long pressed on a View public static interface View.OnLongClickListener { ... } // State of view has changed // (e.g. user clicked outside a EditText input box) public static interface View.OnFocusChangeListener { ... } // user typed something public static interface View.OnKeyListener { ... } ``` ] --- # Implementing clicking Create your button Register an event listener with the button --- # Registering an Event Listener ~~Three~~ five ways to register an event listener: - Creating a separate class/file or an inner class - Creating an anonymous inner class - Implementing an Interface in your custom view/application - Creating an anonymous class as a parameter - Lambdas Examples of all of these in the [Counter](https://gitlab.cs.washington.edu/cse340/exercises/counter) exercise --- Code for the [Counter](https://gitlab.cs.washington.edu/cse340/exercises/counter) exercise uses a helper function ```java /** Method to increment the count field */ private void incrementCount(View v) { TextView counter = findViewById(R.id.count); String textCount = counter.getText().toString(); int count = Integer.parseInt(textCount); counter.setText("" + ++count); ((TextView)findViewById(R.id.whichButton)).setText(((Button)v).getText()); } ``` --- # Simplest approach: Implement the method .left-column[ Create the Listener subclass ] .right-column[ ```java // Approach 3 implementation: implement the listener in *this* class @Override public void onClick(View v) { incrementCount(v); } ``` ] --- # Then register the listener .left-column[ Typically done in onCreate() ] .right-column[ ```java ```java // Approach 3 implementation: implement the listener in *this* class @Override public void onClick(View v) { incrementCount(v); } // in onCreate... // Registering the listener in Approach 3 (using this class) Button b3 = findViewById(R.id.button_three); if (b3 != null) { // Should always check for null b3.setOnClickListener(this); } ``` ] --- # Preferred Approach: Create an Anonymous Inner Class ```java public class EventExampleActivity extends AppCompatActivity { // An anonymous inner class as a member variable in an Activity View.OnClickListener mButtonClickListener = new View.OnClickListener() { @Override public void onClick(View v) { // call the private method in EventExampleActivity incrementCount(v); } }; protected void onCreate(Bundle savedState) { Button b1 = findViewById(R.id.button_one); if (b1 != null) { b1.setOnClickListener(mButtonClickListener); } } } ``` ] ??? incrementCount is a private method Mention that mXxxx variables are private fields and this is a quick way to find all private fields when searching variables in code --- # Digging deeper: Creating an Anonymous Inner Class Let's take some time to parse this... ```java View.OnClickListener mButtonClickListener = new View.OnClickListener() { ``` - `View.OnClickListener` is the variable type ([Documentation](https://developer.android.com/reference/android/view/View.OnClickListener.html)) - a nested class in `View` - `new View.OnClickListener()` is the the anonymous object - The on method that you MUST implement (in order to create a new object) is `OnClick` which overrides the abstract method ```java public void OnClick(View v) { /* stuff in here does the work when a click is received! */ } ``` --- # For very simple listeners you can use a lambda .left-column[ Create the and register the listener all at once with lamba syntax ] .right-column[ ```java Button b5 = findViewById(R.id.button_five); if (b5 != null) { // Should always check for null b5.setOnClickListener((View v) -> incrementCount(v)); } ``` ] .right-column[ To use Lambdas you have to upgrade to Java 8. See these [instructions](https://developer.android.com/studio/write/java8-support) to do this. ] --- # Registering Multiple Listeners Can register more than one listener For example: ```java View v = new View(); v.setOnClickListener(...); v.setOnLongClickListener(...); ``` --- # Many Views, Same Listener (1/3)] Event callbacks are passed the `View` as a parameter -- We can reuse a listener for views that handle the same action (e.g. all 5 buttons could use the same class/method for the action) --- # Many Views, Same Listener (2/3) And we can handle different actions by checking the `View` or its `id`: ```java protected void onCreate(Bundle savedState) { Button b1 = (Button) findViewById(R.id.button_one); if (b1 != null) { b1.setOnClickListener(mButtonClickListener); } Button b2 = (Button) findViewById(R.id.button_two); if (b2 != null) { b2.setOnClickListener(mButtonClickListener); } } ``` --- # Many Views, Same Listener (3/3) You can use the ID of the view to determine where the event originated from ```java View.OnClickListener mButtonClickListener = new View.OnClickListener({ public void onClick(View v) { if (v == null) { return; } int viewId = v.getId(); switch (viewId) { case R.id.button_one: // First Button Log.d("Button Press", "First Button"); break; case R.id.button_two: // Second Button Log.d("Button Press", "Second Button"); break; default: // Someone else is using the listener! Log.d("Button Press", "Invalid Button!"); } } }); ``` --- # Marking an event as handled This ensures only one view gets it (we talk about the algorithm behind this later) Return `boolean` value that indicate the event has been handled (`true`) ```java /** * onLongClick - triggered when a view is long clicked * @param v - the view long pressed * @return - true if the callback consumed the long click, false otherwise. **/ boolean onLongClick (View v) { ... }; ``` .footnote[[API Documentation for Long Click](https://developer.android.com/reference/android/view/View.OnLongClickListener.html) ] --- # Summary: Relating this back to the phone .left-column30[ <div class="mermaid"> graph LR ap[Application Program] t[Toolkit] w[Window System] o[OS] h[Hardware] classDef yellow font-size:14pt,text-align:center class ap,w,o,h,hlt,t yellow </div> ] .right-column60[ Hardware level: electronics to sense circuits closing or movement - Difference between hardware (Event vs sampled) - Sensor based input OS: "Interrupts" that tell the Window system something happened - Logical device abstraction and types of devices ] --- # Summary: Relating this back to the phone .left-column30[ <div class="mermaid"> graph LR ap[Application Program] t[Toolkit] w[Window System] o[OS] h[Hardware] classDef yellow font-size:14pt,text-align:center class ap,w,o,h,hlt,t yellow </div> ] .right-column60[ Windows system: Tells which window received the input Toolkit Architecture: - Receives events from windowing system - Packages up Events and determines which `Views` should get them - Calls Callbacks - Handles other aspects of MVC (like redraw) ] --- # Summary: Relating this back to the phone .left-column30[ <div class="mermaid"> graph LR ap[Application Program] t[Toolkit] w[Window System] o[OS] h[Hardware] classDef yellow font-size:14pt,text-align:center class ap,w,o,h,hlt,t yellow </div> ] Application Program: Views - (M) Stores local model (am I pressed, etc) - (V) Draws itself on screen - (C) Implements callbacks for user events (such as onTouch) Whole Application - (M) Stores application model (data/state of app) - (V) Sets up the interactor hierarchy - (C) Implements callbacks for interface events (such as buttonPressed) ] --- # End of Deck Thing to think about... How would you handle input in a circular component?
