name: inverse layout: true class: center, middle, inverse --- # Event Delivery using the Interactor Hierarchy Lauren Bricker CSE 340 Winter 2020 --- name: normal layout: true class: --- [//]: # (Outline Slide) .left-column[ # Today's goals ] .right-column[ Reminders - Layout Part 3-4 due tonight at 10pm - Remember git add/commit/push __then go to [GitGrade to turn in](https://gitgrade.cs.washington.edu/student/assignment/103/turnin)__ - Accessiblity is now open! - Remember to [accept](https://gitgrade.cs.washington.edu/student/assignment/105) the assignment through GitGrade Feedback redux Discuss how Input Events are delivered - Event hierarchy - Input dispatch process - Listeners - Picking, capture and bubble ] --- # Feedback redux - Thank you for participating in the course evaluation! Overall we're making progress but we have work to do. -- - Response rates on Piazza: we're trying balance between responding too slow but getting you to think and help each other -- - Honing our active learning activities: Feedback always welcome! -- - More examples of midterm questions -- - [Post Layout Quiz](https://canvas.uw.edu/courses/1369399/quizzes/1201379) due Monday. - Post Accessibility quiz next week -- - Paraphrasing: More concrete connection between lecture and assignments. -- - Remember: this class is about Interaction Programming. Programming Android is the substrate for the discussion. - We will try to be more explicit in letting you know when we are discussing theory vs practice. - Subgoal of this class: a transition or bridge to 300 level courses. - Important that you learn how to read less structured specifications - Important that you learn how to search for __usable__ information --- # Interacting with the User | Command Line | |---| || ??? - What programs in 142/143 or in prior classes did they create that took user interaction (Budgeter, Guessing game, Hangman) - How would you write code to do interactivity on a command line or in the console in JGrasp? - What control flow structure did you use to get input from the user. (loop) --- # Interacting with the User | Command Line | Windows | |---|---| ||| ??? - Event-Driven Interfaces (GUIs) - Interaction driven by user - UI constantly waiting for input events - Pointing & text input, graphical output --- .left-column[ # Review <div class="mermaid" style="font-size: 14pt"> graph LR ip[Interface Programmer] w[Component Developer] l[Library Extender] a[Architecture Extender] t[Toolkit Builder] class ip,t,w,l,a green </div> ] .right-column[ <p> </p> <p> </p> - Which developer role were you when implementing Layout Part 1, 2, 3? - Did you take on any other developer roles for part 4? ] --- .left-column[ # Review ] .right-column[ Device Independence - We need device independence to handle a variety of ways to get input - Need a uniform and higher level abstraction for input (Events) ] --- # Contents of Event Record Think about your real world event again. What do we need to know? **What**: Event Type **Where**: Event Target **When**: Timestamp **Value**: Event-specific variable **Context**: What was going on? ??? Discuss each with examples --- # Contents of Event Record What do we need to know about each UI event? **What**: Event Type (mouse moved, key down, etc) **Where**: Event Target (the input component) **When**: Timestamp (when did event occur) **Value**: Mouse coordinates; which key; etc. **Context**: Modifiers (Ctrl, Shift, Alt, etc); Number of clicks; etc. ??? Discuss each with examples --- .left-column[ # Review ] .right-column[ Device Independence - We need device independence to handle a variety of ways to get input - Need a uniform and higher level abstraction for input (Events) Component Independence - Given a model for representing input, how do we get inputs delivered to the right component? - Valuator → returns a scalar value - Button → returns integer value - Locator → returns position on a logical view surface - Keyboard → returns character string - Stroke → obtain sequence of points ] --- ## Android Events Each kind of event has its own class but they all implement the event interface - A little hard to find all the parts defined in one place - Harder to deal with uniformly - But easily extensible for new event types .red[Artificial] events are a thing (e.g. window events) --- # Android [MotionEvent](https://developer.android.com/reference/android/view/MotionEvent) ```java java.lang.Object ↳ android.view.InputEvent ↳ android.view.MotionEvent public final class MotionEvent extends InputEvent implements Parcelable { int getAction() // up, down etc int getActionIndex() // multi touch support -- which pointer float getX() // position float getY() // position int getButtonState() // pressed or not, for example long getDownTime() float getOrientation(int pointerIndex) float getPressure() float getSize() // fingers aren't pixel sized // and many more... } ``` --- # Aside: Multiple Hierarchies discussed so far in class Can you think of them? ??? Inheritance hierarchy for interactors Inheritance hierarchy for layout Inheritance hierarchy for events Component hierarchy -- - Inheritance hierarchy for interactors - Inheritance hierarchy for layout - Inheritance hierarchy for events - Component hierarchy --- # Events can represent abstract concepts... Can you think of any? -- ...Think of how the state of the app changes as you use it. -- - __Create__: Android is done creating the object -- - __Active__: in foreground of screen, and receives input -- - __Paused__: activity lost focus (is no longer displayed on screen) -- - __Stopped__: - Another activity has focus - Original activity remains in memory -- perfect place to save data & state - Often can be *killed* by Android OS if not resumed relatively quickly -- - __Inactive__: after an activity has been killed; or before it is launched -- - __Timer__: Animation --- # Activity State Change Events .left-column-half[  ] .right-columns-half[ Look familiar? ] --- # Thinking as a Toolkit Developer .left-column-half[ <div class="mermaid" style="font-size: 14pt"> graph LR ip[Interface Programmer] w[Component Developer] l[Library Extender] a[Architecture Extender] t[Toolkit Builder] class ip,w,l,a green class t yellow </div> ] .right-column-half.font-smaller[ <div class="mermaid" style="font-size: 14pt"> graph LR ap[Application Program] hlt[High Level Tools] t[Toolkit] w[Window System] o[OS] h[Hardware] class ap,o,h,hlt,w green class t yellow </div> ] --- # How does the Toolkit Architecture deliver events? Application’s UI Thread (This is the main thread of your program) ```java while (true) { // Gets event from the application message queue. // If the queue is empty, we just wait here. Event e = GetEventFromMessageQueue(); // Dispatches the event to the appropriate application window // (the destination address is in the event data). DispatchEventToMessageDestination(e); } ``` -- Don’t worry, this code is automatically setup and managed by the UI framework you are using (e.g., .NET, WPF, Android, Swift). You don’t really need to worry about it (but you should know it’s there and how it functions!) --- # Input __Dispatch__ Process Input thread: - When a user interacts, __events__ are created - Events go into a queue ??? What do you think of when you hear the word "thread"? How does it relate to CS? Remember a queue is a data structure that is first in first out. --- # Input __Dispatch__ Process Input thread Dispatch thread: - Front event comes off queue - How does a toolkit decide where to send events? -- - Depends: Is it Focus or Positional input? - Focus list (in order based on interest) - Positional list (z-order under cursor based on interactor hierarchy) --- .left-column-half[ ## Event Dispatch ] .right-column-half[ Dispatch Strategies - Positional (Bottom-first and Top-down) - Focus-based ] --- .left-column-half[ ## Event Dispatch ] .right-column-half[ Callbacks handle *application* response to events - Update Application Model ] --- .left-column-half[ ## Event Dispatch ] .right-column-half[ Callbacks handle *application* response to events - Update Application Model - Best implemented using custom listeners ] --- template: inverse ## Event Listeners ### Putting theory into practice ??? - Implementation strategy - Basic goal: notify something of a change --- # Basics of Event Listeners (1/3) - Interface on the `View` class that acts as a _callback_ method -- - Recall interfaces specify behavior (but no data) that can be inherited. -- - Must be attached to a particular `View` -- - Android framework triggers the method once the `View` is interacted with by the user -- - A `View` can have listen for different types of interactions - But: the `View` must implement and registering the appropriate listeners ??? What is an interface? --- layout: false .left-column[ ## Standard Event Listeners provided by Android More listed in the [API Documentation](https://developer.android.com/guide/topics/ui/ui-events.html) ] .right-column[ The set of Android's provided interfaces include: ```java // User tapped a view public static interface View.OnClickListener { ... } // User long pressed on a View public static interface View.OnLongClickListener { ... } // State of view has changed // (e.g. user clicked outside a EditText input box) public static interface View.OnFocusChangeListener { ... } ``` ] --- # Implementing clicking Create your button Register an event listener with the button --- # Registering an Event Listener Two ways to register an event listener: - Create an Anonymous Class - Implement an Interface --- .left-column[ ## Create an Anonymous Class ] .right-column[ ```java public class ExampleActivity extends AppCompatActivity { private Button mButton; private int mButtonClicks = 0; // As a member variable in an Activity private View.OnClickListener mClickListener = new View.OnClickListener() { public void onClick(View v) { if (mButton != v) { return; // Do nothing if its not the right view } mButtonClicks += 1; // Increment the click count } }; protected void onCreate(Bundle savedState) { Button mButton = (Button) findViewById(R.id.button_test); if (mButton != null) { mButton.setOnClickListener(mClickListener); } } } ``` ] ??? Mention that mXxxx variables are private fields and this is a quick way to find all private fields when searching variables in code --- # Digging deeper: Create an Anonymous Class Let's take some time to parse this... ```java private View.OnClickListener mClickListener = new View.OnClickListener() { ``` - `private` - it's only available inside the class that contains it (i.e. `ExampleActivity`) - `View.OnClickListener` is the variable type ([Documentation](https://developer.android.com/reference/android/view/View.OnClickListener.html)) - a nested class in `View` - `mClickListener` is the variable name which is being set to... - a `new View.OnClickListener()` which is an anonymous object from an abstract class - For those of you who have not taken 331, that means that there are methods that have not been implemented in the class. - The on method that you MUST implement (in order to create a new object) is `OnClick` which overrides the abstract method ```java public void OnClick(View v) { /* stuff in here does the work when a click is received! */ } ``` --- .left-column[ ## Registering a Listener (1/2) Create the Listener ] .right-column[ ```java View.OnClickListener buttonListener = new View.OnClickListener({ public void onClick(View v) { if (mButton != v) { return; // Do nothing if its not the right view } mButtonClicks += 1; // Increment the click count } }); ``` ] --- .left-column[ ## Registering a Listener (2/2) Create the Listener Register it ] .right-column[ ```java Button mButton = (Button) findViewById(R.id.button_test); if (mButton != null) { mButton.setOnClickListener(buttonListener); } ``` ] --- .left-column[ ## Implement the Interface ] .right-column[ ```java // Start by implementing the interface public class ExampleActivity extends AppCompatActivity implements View.OnClickListener { // Necessary to implement private Button mButton; private int mButtonClicks = 0; protected void onCreate(Bundle savedState) { Button mButton = (Button) findViewById(R.id.button_test); if (mButton != null) { // 'this' refers to the activity itself here // When the button clicked, Android triggers Activity's OnClick method mButton.setOnClickListener(this); } } // Here's where you do the implementation of your listener public void onClick(View v) { if (mButton != v) { return; // Do nothing if its not the right view } mButtonClicks += 1; // Increment the click count } } ``` ] --- .left-column[ ## Registering Multiple Listeners ] .right-column[ Can register more than one listener For example: ```java View v = new View(); v.setOnClickListener(...); v.setLongClickListener(...); ``` ] --- .left-column[ ## Many Views, Same Listener (1/3)] .right-column[ Event callbacks are passed the `View` as a parameter ] -- .right-column[ We can reuse a listener for views that handle the same action (e.g. 3 buttons increment the same click count) ] --- ## Many Views, Same Listener (2/3) - And we can handle different actions by checking the `View` or its `id`: ```java protected void onCreate(Bundle savedState) { Button mButtonOne = (Button) findViewById(R.id.button_one); if (mButtonOne != null) { mButtonOne.setOnClickListener(mButtonClickListener); } Button mButtonTwo = (Button) findViewById(R.id.button_two); if (mButtonTwo != null) { mButton.setOnClickListener(mButtonClickListener); } } ``` --- ## Many Views, Same Listener (3/3) ```java View.OnClickListener mButtonClickListener = new View.OnClickListener({ public void onClick(View v) { if (v == null) { return; } int viewId = v.getId(); switch (viewId) { case R.id.button_one: // First Button Log.d("Button Press", "First Button"); break; case R.id.button_two: // Second Button Log.d("Button Press", "Second Button"); break; default: // Someone else is using the listener! Log.d("Button Press", "Invalid Button!"); } } }); ``` --- .left-column[ ## Marking an event as handled ] .right-column[ This ensures only one view gets it (we talk about the algorithm behind this later) Return `boolean` value that indicate the event has been handled (`true`) ```java /** * onLongClick - triggered when a view is long clicked * @param v - the view long pressed * @return - true if the callback consumed the long click, false otherwise. **/ boolean onLongClick (View v) { ... }; ``` ] .footnote[[API Documentation for Long Click](https://developer.android.com/reference/android/view/View.OnLongClickListener.html) ] --- # End of Deck --- # How would you handle input in a circular component? ??? Consumption is based on the bounding box... -- Delivery is based on bounding box Return `false` if input is outside circle even if it's in the bounding box --- # Event listeners Android sets up by default - `onCreate` - called when the activity is first created - we've seen this in our `HelloAndroidWorld` App: ```java @Override protected void onCreate(Bundle savedInstanceState) { // We want to do any view initialization work here super.onCreate(savedInstanceState); // Load the XML representation of our layout into the view setContentView(R.layout.activity_main); // Remember findViewById let's us find the xml views we declared in our // program (this is a link between the xml world and java) mMainTextView = (TextView) findViewById(R.id.text_main_title); mSumTextView = (TextView) findViewById(R.id.text_sum); } ``` --- # Event listeners Android sets up by default - `onStart` - Called when activity is about to be visible to the user - Always gets called after `onCreate` or `onRestart` -- - `onResume` - Called when the activity will start interacting with a use -- - Always gets called after `onStart` -- - `onPause` - Called when the system is about to start resuming another previous activity -- - __Commit changes__ to persistent data, __stop animations & intense operations__, __kill network__ requests -- - .red[Fast implementation required!] - the next activity will not resume until this finishes in your activity -- - `onStop`: Called when the activity is no longer visible to the user --- ## Responding to State Changes with Callbacks (3/3) - `onRestart` -- - Called when the activity was previously stopped but is coming back to the user - Followed by a call to `onStart` -- - `onDestroy`: the final notice before your activity is .red[__destroyed__] -- .center.half-width-img[  ] --- # How are Event Listeners called? - Android traverses the view hierarchy (starting at root) - "Picks" `Views` that respond to an input event - Loops through the list of `View` objects, checks if they will "capture" the event - If none capture the event, the event "bubbles" back up the `View` object list --- # Summary - MVC: Separation of concerns for user interaction -- - Events: logical input device abstraction -- - We model everything as events - Sampled devices - Handled as “incremental change” events - Each measurable change: a new event with new value - Device differences - Handled implicitly by only generating events they can generate - Recognition Based Input? - Yes, can generate events for this too --- layout: false .left-column[ #Input Event Goals ] .right-column[ Device Independence - Want / need device independence - Need a uniform and higher level abstraction for input Component Independence - Given a model for representing input, how do we get inputs delivered to the right component? ] ??? --- # Review: Logical Device Approach - Valuator → returns a scalar value - Button → returns integer value - Locator → returns position on a logical view surface - Keyboard → returns character string - Stroke → obtain sequence of points --- # Summary - Events: logical input device abstraction - Event driven code: very different from procedural - Listeners are notified about events - Application callbacks can be implemented as custom listeners --- # End of Deck --- # Code you could use for a live demo ```java public class Counter extends ImageView { private int count = 0; // model public Counter(Context context, AttributeSet attrs) { super(context, attrs); } protected void onDraw(Canvas canvas) { Paint paint = new Paint(); paint.setColor(Color.WHITE); paint.setTextSize(60); canvas.drawText("Click count" + count, 100,100, paint); } @Override public boolean onTouchEvent(MotionEvent event) { if (event.getAction() == MotionEvent.ACTION_DOWN) { Log.i("debug", "onTouch: down"); count++; invalidate(); return true; } return false; } } ``` --- # XML for live demo ```xml <com.example.myapplicationeventdemo.Counter android:layout_width="358dp" android:layout_height="231dp" android:background="@android:color/black" android:text="Hello World!" app:layout_constraintBottom_toBottomOf="parent" app:layout_constraintLeft_toLeftOf="parent" app:layout_constraintRight_toRightOf="parent" app:layout_constraintTop_toTopOf="parent" /> ``` --- # How do we decide where to send events? Input Thread Dispatch thread: - Uses *callbacks* to deliver events to application - View (or Activity) implements these callbacks ```java @Override public boolean onTouchEvent(MotionEvent event) { ... } ``` --- # Input __Dispatch__ Process Input Thread Dispatch thread Components: - Components have to listen for events (callbacks) - How do components respond? ??? - remind them that component/view/interactor are interchangeable in this class - Update application state if appropriate - Request repaint if needed
