name: inverse layout: true class: center, middle, inverse --- # Event Delivery using the Interactor Hierarchy Lauren Bricker CSE 340 Spring 2020 --- name: normal layout: true class: --- # Accessibilty warmup Try the problem on the front of the worksheet -- - TIL: Talkback will "tell" what an item on the screen is if - it has a content description (`android:contentDescription="..."`) OR - is focusable (`android:focusable="true"`) OR - it is clickable (`android:clickable="true"`) OR - You have attached a `onClickListener` to the interactor. --- [//]: # (Outline Slide) # Today's goals Discuss how Input Events are delivered - Event hierarchy - Input dispatch process - Listeners <- Prework for Wednesday - ~~Picking, capture and bubble~~ <- Moved to Wednesday --- # Administrivia For Friday's examlet, you need to enable your UW GSuite account. - During class I will give out a link that will allow you to copy the Layout Examlet. - The Google Doc will copy to your drive. - You'll answer the questions in the doc. - You'll save the doc as a PDF - You'll upload the PDF to Gradescope. See this [page](https://itconnect.uw.edu/connect/email/google-apps/getting-started/) for information on how ensure your GSuite account is activated. --- # Review: Logical Device Approach - Valuator → returns a scalar value (like a slider) - Button → returns integer value - Locator → returns position on a logical view surface - Keyboard → returns character string - Stroke → obtain sequence of points --- # Meta notes We are working on grading the Layout Assignment. - It's mostly going well. - A common issues we're seeing is not reading the starter code and understanding what it is doing - Recommedation: - START EARLY: this gives you time to go to OH, read our discussion board, ask questions, and get responses. (Remember responses may not be instantaneous, so you may have to work on something else while you wait. ) - Skim the spec. Write down a list of questions the spec raises (i.e what don't you know or understand at a quick glance) - Accept and open the code. - Understand the class hierarchy that is given to you. A visual of the hierarchy should be in the spec. - Understand what is happening in the onCreate of whatever extends from `AppCompatActivity` - Start implementing Part1 and look for those TODOs. - REREAD the spec. - Remember to read the spec one more time before turning in the assignment to make sure you got all the pieces. --- # Meta notes - Remember: this class is about Interaction Programming. Programming Android is the substrate for the discussion. - We try to explicitly state when are discussing theory vs practice. - Subgoal of this class: a transition or bridge to 300 level courses. - Important that you learn how to read less structured specifications - Important that you learn how to search for __usable__ information - Is StackOverflow friend or foe? --- .left-column[ # Review <div class="mermaid" > graph LR ip[Interface Programmer] w[Component Developer] l[Library Extender] a[Architecture Extender] t[Toolkit Builder] classDef green font-size:14pt,text-align:center class ip,t,w,l,a green </div> ] .right-column[ <p> </p> <p> </p> - Which developer role were you when implementing Layout Part 1, 2, 3? - Did you take on any other developer roles for part 4? ] --- # Interacting with the User .left-column-half[ Command line interfaces  ] -- .right-column-half[ - Think about the from 142/143 or in prior classes did they create that took user interaction - e.g. Budgeter, Guessing game, Hangman - How would you write code to do interactivity on a command line or in the console in JGrasp? - What control flow structure did you use to get input repeatedly from the user? - Did the system continue to work or did it wait for the user input? ] --- # Interacting with the User .left-column-half[ Command line interfaces  ] .right-column-half[ Mac Desktop interface | ] ??? - Event-Driven Interfaces (GUIs) - Interaction driven by user - UI constantly waiting for input events - Pointing & text input, graphical output --- # Review .left-column[ <div class="mermaid"> graph LR ap[Application Program] hlt[High Level Tools] t[Toolkit] w[Window System] o[OS] h[Hardware] classDef yellow font-size:14pt,text-align:center classDef green font-size:14pt,text-align:center class ap,o,h,hlt,t yellow class w green </div> ] .right-column[ The Window System layer has several responsibilities. It - provides each view (app) with an independent drawing interface. - ensures that the display gets updates to reflect changes - recieves input from the OS and dispatches it to the appropriate widget ] --- # Review Device Independence - We need device independence to handle a variety of ways to get input - We need a uniform and higher level abstraction for input (events) --- # Contents of Event Record Think about one of your real world events from last class again. What do we need to know? **What**: Event Type **Where**: Event Target **When**: Timestamp **Value**: Event-specific variable **Context**: What was going on? ??? Discuss each with examples --- # Contents of Event Record Example: The cat meowed: What do we need to know? **What**: A cat's meow **Where**: My ears (this is the input target) **When**: 9:00pm **Value**: From the kitchen, near the food bowl **Context**: Whining, pacing, and generally being a pest. .footnote[It really does happen see [this video](img/event-delivery/jackhungry.mov)] ??? Jack wants food. --- # Contents of Event Record Example: The cat meowed: What do we need to know? **What**: A cat's meow **Where**: My ears (this is the input target) **When**: 10:00am **Value**: From the hallway or following us around **Context**: Purring and rubbing up against us ??? Jack wants attention. --- --- # Another example: : Contents of Event Record Now imagine a user clicks on the previous button in our PetGallery. What would the event button look like? What do we need to know about that UI event? -- **What**: Click (Could also think about the mouse down and mouse up separately) -- **Where**: X,Y coordinate of the finger when the click ended -- **When**: The timestamp when the click occured -- **Value**: 1 Finger (or maybe None) -- **Context**: Any possible modifiers (Ctrl, Shift, Alt, etc); Number of clicks; etc. --- # Your turn Imagine you are writing a program to listen for "Hey Siri" on an Android phone. Work with your neighbor to fill in the fields of the Event Record for this event on your worksheet. **What**: **Where**: **When**: **Value**: **Context**: Any possible modifiers --- # Your turn Imagine you are writing a program to ironically listen for "Hey Siri" on an Android phone. Work with your neighbor to fill in the fields of the Event Record for this event on your worksheet. **What**: Speaking starts (or speaking ends) **Where**: Normally position on screen. In this case, ill defined because this event is not dispatched positionally. Can be blank, or may hold the current cursor location. **When**: Timestamp of the audio heard **Value**: The content of the audio recorded **Context**: Any possible modifiers --- # Review Device Independence - We need device independence to handle a variety of ways to get input - We need a uniform and higher level abstraction for input (events) Component Independence - Given a model for representing input, how do we get inputs delivered to the right component? - Valuator → returns a scalar value - Button → returns integer value - Locator → returns position on a logical view surface - Keyboard → returns character string - Stroke → obtain sequence of points --- ## Android Events Each kind of event in Android its own class but they all implement the [InputEvent](https://developer.android.com/reference/android/view/InputEvent) abstract class. - A little hard to find all the parts defined in one place - Harder to deal with uniformly - But easily extensible for new event types .red[Artificial] events are a thing - Window Events - Search Event - others. --- # Android [MotionEvent](https://developer.android.com/reference/android/view/MotionEvent) ```java java.lang.Object ↳ android.view.InputEvent ↳ android.view.MotionEvent public final class MotionEvent extends InputEvent implements Parcelable { int getAction() // up, down etc int getActionIndex() // multi touch support -- which pointer float getX() // position float getY() // position int getButtonState() // pressed or not, for example long getDownTime() float getOrientation(int pointerIndex) float getPressure() float getSize() // fingers aren't pixel sized // and many more... } ``` --- # Aside: Multiple Hierarchies discussed so far in class Can you think of them? -- - Inheritance hierarchy for interactors (Viewp) - Inheritance hierarchy for layout - Inheritance hierarchy for events - Component hierarchy (Interactors and View Groups) -- - Also remember how the component hierarchy is used in drawing (Z-order) --- # Events can represent abstract concepts... Can you think of any? -- ...Think of how the state of the app changes as you use it. -- - __Create__: Android is done creating the object -- - __Active__: in foreground of screen, and receives input -- - __Paused__: activity lost focus (is no longer displayed on screen) -- - __Stopped__: - Another activity has focus - Original activity remains in memory -- perfect place to save data & state - Often can be *killed* by Android OS if not resumed relatively quickly -- - __Inactive__: after an activity has been killed; or before it is launched -- - __Timer__: Animation --- # Activity State Change Events .left-column-half[  ] .right-columns-half[ Look familiar? ] --- # Thinking as a Toolkit Developer .left-column-half[ <br> <div class="mermaid"> graph LR ip[Interface Programmer] w[Component Developer] l[Library Extender] a[Architecture Extender] t[Toolkit Builder] classDef yellow font-size:14pt,text-align:center classDef green font-size:14pt,text-align:center class ip,w,l,a green class t yellow </div> ] .right-column-half.font-smaller[ <div class="mermaid"> graph LR ap[Application Program] hlt[High Level Tools] t[Toolkit] w[Window System] o[OS] h[Hardware] classDef yellow font-size:14pt,text-align:center classDef green font-size:14pt,text-align:center class ap,o,h,hlt,w green class t yellow </div> ] --- # How does the Toolkit Architecture deliver events? Application’s UI Thread (This is the main thread of your program) ```java while (true) { // Gets event from the application message queue. // If the queue is empty, we just wait here. Event e = GetEventFromMessageQueue(); // Dispatches the event to the appropriate application window // (the destination address is in the event data). DispatchEventToMessageDestination(e); } ``` -- This code is automatically setup and managed by the UI framework you are using (e.g., .NET, WPF, Android, Swift). You don’t really need to worry about it (but you should know it’s there and how it functions!) --- # Input __Dispatch__ Process Input thread: - When a user interacts, __events__ are created - Events go into a queue ??? What do you think of when you hear the word "thread"? How does it relate to CS? Remember a queue is a data structure that is first in first out. --- # Input __Dispatch__ Process Input thread Dispatch thread: - Front event comes off queue - How does a toolkit decide where to send events? -- - Depends: Is it Focus or Positional input? - Focus list (in order based on interest) - Positional list (z-order under cursor based on interactor hierarchy) --- .left-column-half[ ## Event Dispatch ] .right-column-half[ Dispatch Strategies (theory) - Positional - Bottom-up - Top-down - Bubble out - Focus-based ] --- .left-column-half[ ## Event Dispatch (theory)  ] .right-column-half[ - Bottom up dispatch - The event is directed to the "lowest", frontmost interactor in the tree that contains the mouse position - That interactor might not want the input, so it goes to window at the next level up the tree ] --- .left-column-half[ ## Event Dispatch (theory)  ] .right-column-half[ - Bottom up dispatch - Top down dispatch - The event is directed to topmost window that contains the mouse location - That window decides how to dispatch it further (recursively) - Common in OO toolkits - Useful in situations where the interactor is "view only" (parent wants to disable input to child) ] --- .left-column-half[ ## Event Dispatch (theory)  ] .right-column-half[ - Bottom up dispatch - Top down dispatch - Bubble-out dispatch - Used when there is no clear nesting of windows and groups of interactive objects - Tree is traversed as in top down approach, but bounding rectangles are hints, not guarantees - Objects checked in opposite order from drawing: frontmost items are checked first - Object that was hit is attached to the event, ancestors know what was selected ] --- .left-column-half[ ## Event Dispatch  ] .right-column-half[ - Bottom up dispatch - Top down dispatch - Bubble-out dispatch - Focus dispatch - No straighforward location of where the event happened - Windowing system determines which window/interactor should get the input. - Common example: where should key press/release events go? ] --- .left-column-half[ ## Event Dispatch ] .right-column-half[ Callbacks handle *application* response to events - Update Application Model ] --- .left-column-half[ ## Event Dispatch ] .right-column-half[ Callbacks handle *application* response to events - Update Application Model - Best implemented using custom listeners ] --- template: inverse ## Event Listeners ### Putting theory into practice ??? - Implementation strategy - Basic goal: notify something of a change --- # Basics of Event Listeners (1/3) - Interface on the `View` class that acts as a _callback_ method -- - Recall interfaces specify behavior (but no data) that can be inherited. -- - Must be attached to a particular `View` -- - Android framework triggers the method once the `View` is interacted with by the user -- - A `View` can have listen for different types of interactions - But: the `View` must implement and registering the appropriate listeners ??? What is an interface? --- layout: false .left-column[ ## Standard Event Listeners provided by Android More listed in the [API Documentation](https://developer.android.com/guide/topics/ui/ui-events.html) ] .right-column[ The set of Android's provided interfaces include: ```java // User tapped a view public static interface View.OnClickListener { ... } // User long pressed on a View public static interface View.OnLongClickListener { ... } // State of view has changed // (e.g. user clicked outside a EditText input box) public static interface View.OnFocusChangeListener { ... } ``` ] --- # Implementing clicking Create your button Register an event listener with the button --- # Registering an Event Listener ~~Three~~ five ways to register an event listener: - Creating a separate class/file or an inner class - Creating an anonymous inner class - Implementing an Interface - Creating an anonymous class as a parameter - Lambdas Code for the following demo is [here](Counter.zip). This code uses a helper function ```java /** Method to increment the count field */ private void incrementCount(View v) { TextView counter = findViewById(R.id.count); String textCount = counter.getText().toString(); int count = Integer.parseInt(textCount); counter.setText("" + ++count); ((TextView)findViewById(R.id.whichButton)).setText(((Button)v).getText()); } ``` --- .left-column[ ## Registering a Listener - Inner class (1/2) Create the Listener subclass ] .right-column[ ```java private class MyButtonListener implements View.OnClickListener { @Override public void onClick(View v) { incrementCount(v); } } ``` ] --- .left-column[ ## Registering a Listener - Inner class (2/2) Create the Listener subclass Register it (by instantiating an object of that type) ] .right-column[ ```java // in onCreate... Button b1 = findViewById(R.id.button_three); if (b1 != null) { // Should always check for null b1.setOnClickListener(new MyButtonListener()); } ``` ] --- .left-column[ ## Create an Anonymous Inner Class ] .right-column[ ```java public class EventExampleActivity extends AppCompatActivity { // An anonymous inner class as a member variable in an Activity View.OnClickListener mButtonClickListener = new View.OnClickListener() { @Override public void onClick(View v) { // call the private method in EventExampleActivity incrementCount(v); } }; protected void onCreate(Bundle savedState) { Button b1 = findViewById(R.id.button_one); if (b1 != null) { b1.setOnClickListener(mButtonClickListener); } } } ``` ] ??? incrementCount is a private method Mention that mXxxx variables are private fields and this is a quick way to find all private fields when searching variables in code --- # Digging deeper: Creating an Anonymous Inner Class Let's take some time to parse this... ```java private View.OnClickListener mButtonClickListener = new View.OnClickListener() { ``` - `private` - it's only available inside the class that contains it (i.e. `EventExampleActivity`) - `View.OnClickListener` is the variable type ([Documentation](https://developer.android.com/reference/android/view/View.OnClickListener.html)) - a nested class in `View` - `mButtonClickListener` is the variable name which is being set to... - a `new View.OnClickListener()` which is an anonymous object from an abstract class - For those of you who have not taken 331, that means that there are methods that have not been implemented in the class. - The on method that you MUST implement (in order to create a new object) is `OnClick` which overrides the abstract method ```java public void OnClick(View v) { /* stuff in here does the work when a click is received! */ } ``` --- .left-column[ ## Registering a Listener - Implement the Interface ] .right-column[ ```java // Start by implementing the interface public class EventExampleActivity extends AppCompatActivity implements View.OnClickListener { // Necessary to implement protected void onCreate(Bundle savedState) { Button b3 = findViewById(R.id.button_three); if (b3 != null) { // Should always check for null b3.setOnClickListener(this); } } // Here's where you do the implementation of your listener in your class @Override public void onClick(View v) { incrementCount(v); } ``` ] --- .left-column[ ## Registering a Listener - Anonymous class as a parameter Create and register the listener all at once ] .right-column[ ```java Button b4 = findViewById(R.id.button_four); if (b4 != null) { // Should always check for null b4.setOnClickListener(new View.OnClickListener() { public void onClick(View v) { incrementCount(v); } }); } ``` ] --- .left-column[ ## Registering a Listener - lambda Create the and register the listener all at once with lamba syntax ] .right-column[ ```java Button b5 = findViewById(R.id.button_five); if (b5 != null) { // Should always check for null b5.setOnClickListener((View v) -> incrementCount(v)); } ``` ] .right-column[ To use Lambdas you have to upgrade to Java 8. See these [instructions](https://developer.android.com/studio/write/java8-support) to do this. ] --- .left-column[ ## Registering Multiple Listeners ] .right-column[ Can register more than one listener For example: ```java View v = new View(); v.setOnClickListener(...); v.setOnLongClickListener(...); ``` ] --- .left-column[ ## Many Views, Same Listener (1/3)] .right-column[ Event callbacks are passed the `View` as a parameter ] -- .right-column[ We can reuse a listener for views that handle the same action (e.g. all 5 buttons could use the same class/method for the action) ] --- ## Many Views, Same Listener (2/3) - And we can handle different actions by checking the `View` or its `id`: ```java protected void onCreate(Bundle savedState) { Button b1 = (Button) findViewById(R.id.button_one); if (b1 != null) { b1.setOnClickListener(mButtonClickListener); } Button b2 = (Button) findViewById(R.id.button_two); if (b2 != null) { b2.setOnClickListener(mButtonClickListener); } } ``` --- ## Many Views, Same Listener (3/3) You can use the ID of the view to determine where the event originated from ```java View.OnClickListener mButtonClickListener = new View.OnClickListener({ public void onClick(View v) { if (v == null) { return; } int viewId = v.getId(); switch (viewId) { case R.id.button_one: // First Button Log.d("Button Press", "First Button"); break; case R.id.button_two: // Second Button Log.d("Button Press", "Second Button"); break; default: // Someone else is using the listener! Log.d("Button Press", "Invalid Button!"); } } }); ``` --- .left-column[ ## Marking an event as handled ] .right-column[ This ensures only one view gets it (we talk about the algorithm behind this later) Return `boolean` value that indicate the event has been handled (`true`) ```java /** * onLongClick - triggered when a view is long clicked * @param v - the view long pressed * @return - true if the callback consumed the long click, false otherwise. **/ boolean onLongClick (View v) { ... }; ``` ] .footnote[[API Documentation for Long Click](https://developer.android.com/reference/android/view/View.OnLongClickListener.html) ] --- # End of Deck Thing to think about... How would you handle input in a circular component?
