[//]: # (Outline Slide) # Today's goals - **Finish [Accessibility in Practice](/courses/cse340/23sp/slides/wk04/android-accessibility.html#19)** - Start Events - What is a Model View Controller? - What is an event? - What is a callback? Administrivia: - Accessibility (code and report/reflection) due Fri 28-Apr - Lecture and Section Exercises not only help you, they count for participation - As do practice quizzes - As does connecting with the Discussion board! - Suggestion: Use CSE 006 as a study room! - Slight change coming for week 5 resubmissions (to make it easier) --- layout: true class: center, middle --- # Warm Up (On [Ed](https://edstem.org/us/courses/38124/lessons/59414/slides/331470)) Write down three examples of things you reacted to this week on slide 2 --- # Events Your examples are event-driven interaction -- Next up: the theory and practice of event driven interactions in Android --- class: center, middle, inverse # Event Handling I - Callbacks, Model View Controller, and Events Lauren Bricker CSE 340 Spring 23 --- layout: false [//]: # (Outline Slide) # Today's goals - Finish [Accessibility in Practice](/courses/cse340/23sp/slides/wk04/android-accessibility.html#19) - Start Events - **What is a Model View Controller?** - What is an event? - What is a callback? --- # What happens with user input? .left-column[ <div class="mermaid"> graph LR ap[Application Program] hlt[High Level Tools] t[Toolkit] w[Window System] o[OS] h[Hardware] class ap,w,o,h,hlt,t yellow </div> ] .right-column[ What happens at each level of the hardware stack? ] --- count: false # What happens with user input? .left-column[ <div class="mermaid"> graph LR ap[Application Program] hlt[High Level Tools] t[Toolkit] w[Window System] o[OS] h[Hardware] class ap,w,o,hlt,t yellow class h green </div> ] .right-column[ What happens at each level of the hardware stack? - Hardware level: electronics to sense circuits closing or movement ] --- count: false # What happens with user input? .left-column[ <div class="mermaid"> graph LR ap[Application Program] hlt[High Level Tools] t[Toolkit] w[Window System] o[OS] h[Hardware] class ap,h,hlt,t yellow class o,w green </div> ] .right-column[ What happens at each level of the hardware stack? - Hardware level: electronics to sense circuits closing or movement - OS: "Interrupts" that tell the Window system something happened - Windows system: Tells which window received the input ] --- count: false # What happens with user input? .left-column[ <div class="mermaid"> graph LR ap[Application Program] hlt[High Level Tools] t[Toolkit] w[Window System] o[OS] h[Hardware] class ap,w,o,hlt,h yellow class t green </div> ] .right-column[ What happens at each level of the hardware stack? - Hardware level: electronics to sense circuits closing or movement - OS: "Interrupts" that tell the Window system something happened - Windows system: Tells which window received the input - Toolkit: defines how the app developer can use these events ] --- count: false # What happens with user input? .left-column[ <div class="mermaid"> graph LR ap[Application Program] hlt[High Level Tools] t[Toolkit] w[Window System] o[OS] h[Hardware] class w,o,hlt,h,t yellow class ap green </div> ] .right-column[ What happens at each level of the hardware stack? - Hardware level: electronics to sense circuits closing or movement - OS: "Interrupts" that tell the Window system something happened - Windows system: Tells which window received the input - Toolkit: defines how the app developer can use these events - App Developer: decides how to use the events (if at all) ] --- # Interacting with the User .left-column50[ Command line interfaces  ] -- .right-column50[ <br> - Think about the from 142/143 or in prior classes did they create that took user interaction - What control flow structure did you use to get input repeatedly from the user? - Did the system continue to work or did it wait for the user input? ] --- # Interacting with the User .left-column50[ Command line interfaces  ] .right-column50[ Mac Desktop interface  ] ??? - Event-Driven Interfaces (GUIs) - Interaction driven by user - UI constantly waiting for input events - Pointing & text input, graphical output --- # Windowing System is the glue... .left-column[ <div class="mermaid"> graph LR ap[Application Program] hlt[High Level Tools] t[Toolkit] w[Window System] o[OS] h[Hardware] class ap,o,h,hlt,t yellow class w green </div> ] .right-column[ The Window System layer allows modern UIs to happens - provides each view (app) with an independent drawing interface. - ensures that the display gets updates to reflect changes ] -- .right-column[ **But what happens after the app gets the user's input?** ] --- # Model View Controller (MVC) .left-column60[ View for a slider? (New definition of View: what does it show the user) ] .right-column40[  ] --- count: false # Model View Controller (MVC) .left-column60[ View for a slider? (What does it show the user) Controller for a slider? (What is the underlying logic of the interactor?) ] .right-column40[  ] --- count: false # Model View Controller (MVC) .left-column60[ View for a slider? (What does it show the user) Controller for a slider? (What is the underlying logic of the interactor?) Model for a slider? (what is its state?) ] .right-column40[  ] --- # Model View Controller (MVC) .left-column60[ <div class="mermaid"> sequenceDiagram loop Every time user provides input Note right of View: User provides input View->>Controller: Input Controller->>Model: Change state Model->>Controller: Update state of View(s) Controller->>View: Triggers redraw Note right of View: User sees response end </div> ] --- ## Example: Fingerprint for unlocking computer .left-column60[ <div class="mermaid"> sequenceDiagram Note right of View: User provides input View->Controller: User Fingerprint ID Controller->Model: Update Lock to open Model->Controller: Lock is open Controller->View: Redraw/open computer Note right of View: User sees response </div> ] .right-column40[ <br> <br>  ] --- # Model View Controller (MVC) .footnote[From [Wikipedia](https://en.wikipedia.org/wiki/Model-view-controller)] >MVC is a software design pattern commonly used for developing user interfaces which divides the related program logic into three interconnected elements. - *Model* - a representation of the state of your application (or a single view) - *View* - the visual representation presented to the user - *Controller* - communicates between the model and the view - Handles changing the model based on user input - Retrieves information from the model to display on the view -- count: false MVC exists within each `View` as well as for overall interface --- # Where is the MVC in your code? - Model - May be stored in the application - Persistent state must be stored *outside* the application (filesystem, database, etc) --- # Where is the MVC in your code? - Model - View (Output) -- we need to show people feedback. - What did we learn about how to do this? - What causes the screen to update? - How are things laid out on screen? --- # Where is the MVC in your code? - Model - View (Output) - Controller (Input) - We need to know when people are doing things. This needs to be event driven. - Responding to Users: Event Handling - When a user interacts with our apps, Android creates **events** - As app developers, we react "listening" for events and responding appropriately --- layout: false [//]: # (Outline Slide) # Today's goals - What is a Model View Controller? - **What is an event?** - What is a callback? --- |Procedural | Event Driven | | :--: | :--: | ||<br><br><br><br>| |Code is executed in sequential order | Code is executed based upon events| --- # But what is an Event? A representation of user input Generally, input is harder than output - More affected by human properties - More diversity, less uniformity - We need device independence - To handle a variety of ways to get input (finger, mouse, etc) - We need a uniform and higher level abstraction for input (events) --- # More on events - Device differences - Handled implicitly by only creating events they can generate - Discrete events (e.g. mouse or key click) - Sampled devices - Handled as “incremental change” events - Each measurable change: a new event with new value - Recognition Based Input? - Yes, can generate events for this too --- # Where does and Event come from? Consider the "location" of an event... What is different about a joystick, a touch screen, and a mouse? ??? - Mouse was originally just a 1:1 mapping in 2 dimensions == absolute location; bounded - Joystick is relative (maps movement into rate of change in location); unbounded - Touch screen is absolute; bounded - What about today's mouse? Lifting and moving? -- count: false - Mouse was originally just a 1:1 mapping in 2 dimensions == absolute location; bounded - Joystick is relative (maps movement into rate of change in location); unbounded - Touch screen is absolute; bounded -- count: false What about the modern mouse? Lifting and moving? -- count: false How about a wii controller? --- # Interaction techniques .left-column[ <br>  ] .right-column[ Now consider text entry: - 60-80 (physical keyboards; twiddler) - ~20 (soft keyboards) - ~50? Swype ] -- count: false .right-column[ (but are soft keyboards and swype input devices or an interaction facilitated by an input device?) ] --- # Higher level abstraction Logical (Physical) Device Approach: - Valuator → returns a scalar value (like a slider) - Button → returns integer value - Locator → returns position on a logical view surface - Keyboard → returns character string - Stroke → obtain sequence of points ??? - Can obscure important differences -- hence use inheritance - Discussion of mouse vs pen -- what are some differences? - Helps us deal with a diversity of devices - Make sure everyone understands types of events - Make sure everyone has a basic concept of how one registers listeners --- # Not really satisfactory... This high level abstraction doesn't capture device diversity fully | Event based devices | | Sampled devices | | -- | -- | -- | | Time of input determined by user | | Time of input determined by program | | Value changes only when activated | | Value is continuously changing | | e.g.: button | | e.g.: mouse | ??? Capability differences - Discussion of mouse vs pen - what are some differences? --- # Contents of Event Record What do we need to know about each UI event? **What**: Event Type (mouse moved, key down, etc) **Where**: Event Target (the input component) **When**: Timestamp (when did event occur) **Value**: Mouse coordinates; which key; etc. **Context**: Modifiers (Ctrl, Shift, Alt, etc); Number of clicks; etc. ??? Discuss each with examples --- # Review: Contents of Event Record .left-column50[ Example: The cat meowed: What do we need to know? **What**: A cat's meow **Where**: My ears (as the input target) **When**: 9:00pm **Value**: From the kitchen, near the food bowl **Context**: Whining, pacing, and generally being a pest. ] .right-column50[ ![:youtube Jack the Cat begs for food every day at 9pm and makes pouty sounds when he doesn't get it,lZfw6nqg7ko] ] ??? Jack wants food. --- # Review: Contents of Event Record Example: The cat meowed: What do we need to know? **What**: A cat's meow **Where**: My ears (as the input target) **When**: 10:00am **Value**: From the hallway or following us around **Context**: Purring and rubbing up against us ??? Jack wants attention. --- # Your turn Imagine you are writing a program to respond to a user saying "Ok Google" followed by some statement in the vicinity of an Android phone. Fill in the fields of the Event Record for this event on the [Ed Lesson](https://edstem.org/us/courses/38124/lessons/59417/slides/331487). **What**: **Where**: **When**: **Value**: **Context**: Any possible modifiers ??? # Answer Imagine you are writing a program to listen for "Ok Google" on an Android phone. **What**: Speaking starts (or speaking ends) **Where**: Normally position on screen. In this case, ill defined because this event is not dispatched positionally. Can be blank, or may hold the current cursor location. **When**: Timestamp of the audio is heard or stops **Value**: The content of the audio recorded **Context**: Any possible modifiers --- # Input Event Goals Device Independence - Want / need device independence - Need a uniform and higher level abstraction for input Component Independence - Given a model for representing input, how do we get inputs delivered to the right component? ??? --- layout: false [//]: # (Outline Slide) # Today's goals - What is a Model View Controller? - What is an event? - **What is a callback?** --- # Java 102: `Comparator<E>` Recall that if you have an array of items that have a "natural sorting order" such as: ```java String[] names = {"Emma", "Ncuti", "Tina", "Idris", "Awkwafina"}; ``` We can sort those things using `Arrays.sort` ```java Arrays.sort(names); System.out.println(Arrays.toString(names)); ``` Which will output: ``` [Awkwafina, Emma, Idris, Ncuti, Tina] ``` --- # Java 102: `Comparator<E>` What if you want to create a new class: `Person` ([sample code](person.zip), or on [Ed](https://edstem.org/us/courses/38124/lessons/51102/slides/286369)) ```java public class Person { private String mFirstName; private String mLastName; private int mAge; public Person(String firstName, String lastName, int age) { mFirstName = firstName; mLastName = lastName; mAge = age; } public String toString() { return mFirstName + " " + mLastName + " - age " + mAge; } ``` --- # Java 102: `Comparator<E>` If we had an array of these `Person` objects ```java Person[] people = {new Person("Emma", "Watson", 30), new Person("Ncuti", "Gatwa", 27), ... } ``` How would we sort that array using `Arrays.sort`? -- count: false ```java public class Person implements Comparable<Person> { ... public int compareTo(Person other) {// what would go in here?!?!?!?!?} } ``` Should we choose based on the first name? Last name? age? --- # Java 102: `Comparator<E>` There's another way to set up sorting: by creating an object that extends from [`Comparator<E>`](https://docs.oracle.com/en/java/javase/13/docs/api/java.base/java/util/Comparator.html) A subclass of `Comparator<E>` must implement one method: ```java public int compare(E object1, E object2) // Compares its two arguments for order. ``` `compare` compare returns * -1 if object1 is "less than" object2 * 0 if object1 and object2 are equivalent * 1 if object1 is "greater than" object 2 --- # Java 102: `Comparator<E>` Example: ```java // this class is defined in PersonLastNameComparator.java public class PersonLastNameComparator implements Comparator<Person> { @Override public int compare(Person p1, Person p2) { return p1.getLastName().compareTo(p2.getLastName()); } } ``` Then we could do the folowing to sort the array by the last name. ```java Arrays.sort(people, new PersonLastNameComparator()); ``` --- # What is a callback? It's a method that gets called when the system decides to (like when an event is delivered) Not unique to Android > A callback ... is any executable code that is passed as an argument to other code; > that other code is expected to call back (execute) the argument at a given time. .footnote[ From [Wikipedia](https://en.wikipedia.org/wiki/Callback_(computer_programming) ] --- # Callback Exercise 1. Group exercise 2. One person in your group open up the slide. 3. Fill out the [Callback Exercise](https://edstem.org/us/courses/38124/lessons/59417/slides/331484) on Ed 4. Wait until I call your enclosure name/time, then everyone in the group make the sound for the requested number of seconds. .footnote[Please participate! This is really fun with a lot of participation. ] --- # Callback Exercise .font-medium[As code, imagine each enclosure implemented the following interface to define the behavior as a Listeners (an event handling callback in Java/Android)] ```java public interface EnclosureListener { public void makeNoise(int numSeconds); } ``` .font-medium[For example:] ```java public class LionCage implements EnclosureListener { ... public void makeNoise(int numSeconds) { // make the animal noise for the given number of seconds } } ``` --- # Callback Exercise Further, by telling me the name of your enclosure you're registering your listener WITH me, the "zookeeper" ```java public class LionCage implements EnclosureListener { ... public void makeNoise(int numSeconds) { // make the animal noise for the given number of seconds } public void onCreate(Bundle savedInstanceState) { ... zookeeper.setEnclosureListener(this); } } ``` --- # Callback Exercise ```java public class Zookeeper { // The zookeeper keeps a list of all EnclosureListeners that were registered, private List<EnclosureListeners> mEnclosureListeners; public final void setEnclosureListener(EnclosureListener listener) { if (listener == null) { throw new IllegalArgumentException("enclosureListener should never be null"); } mEnclosureListeners.add(listener); } // And can call on them whenever they need with the right amount of time. public void cacophony(int seconds) for (EnclosureListener listener : mEnclosureListeners) { listener->makeNoise(seconds); } } } ``` --- # Callback Exercise - The Zookeeper has NO idea what each enclosure will do until the listener is registered and called. - Each enclosure separately defines how it will react to a `makeNoise` event. - The "contract" between the zookeeper and the enclosures is the defined `EnclosureListener` interface. --- # Implementing a callback There are three ways to implement callbacks in Java 1. Creating a class in a separate file (like `PersonLastNameComparator`<sup>*</sup>) 2. Creating an inner class 3. Creating an anonymous class (like the following) ```java Arrays.sort(people, new Comparator<Person>() { @Override public int compare(Person p1, Person p2) { return p1.getAge() - p2.getAge(); } }); ``` .footnote[<sup>*</sup>Also `MyReceiver.java` in AskForHelp] --- # Implementing a callback There are <del>three</del> four ways to implement callbacks in Java 1. Creating a class in a separate file (like `PersonLastNameComparator`) 2. Creating an inner class 3. Creating an anonymous class 4. Using a lambda ```java Arrays.sort(people, (Person p1, Person p2)->{return p1.getAge() - p2.getAge();}); ``` --- # Callbacks in Android - At the time Android was created the toolkit developers have *no* idea how every app may want to respond to events - The toolkit has pre-defined interfaces so apps or components can respond to events such as clicks or touches. For example: - The `View` class defines the following Listeners as inner classes: - `View.OnClickListener` - `View.OnLongClickListener` - `View.OnFocusChangeListener` - `View.OnKeyListener` - `View.OnTouchListener` (this is used in Menus by subclassing AppCompatImageView) - `View.OnCreateContextMenuListener` We will come back to these in our next class. --- # Summary - MVC: Separation of concerns for user interaction -- count: false - Events: logical input device abstraction -- count: false - Callbacks: a programatic way to get or send information from/to our program from the system --- # Summary: Relating this back to the phone .left-column[ <div class="mermaid"> graph LR ap[Application Program] hlt[High Level Tools] t[Toolkit] w[Window System] o[OS] h[Hardware] class ap,w,o,h,hlt,t yellow </div> ] .right-column[ - Hardware level: electronics to sense circuits closing or movement - Difference between hardware (Event vs sampled) - Sensor based input - OS: "Interrupts" that tell the Window system something happened - Logical device abstraction and types of devices - Windows system: Tells which window received the input - Toolkit: Packages up Events & calls Callbacks & Redraw - Application Program: - Implements callbacks, stores data ] --- # End of Deck
