name: inverse layout: true class: center, middle, inverse --- # Introduction to Drawing Interfaces Lauren Bricker CSE 340 Spring 2020 --- layout: false [//]: # (Outline Slide) # Today's goals - Reviewing Transformations - Breakout room test 2 - Reviewing Rotation - Role of Interactor Hierarchy in drawing the screen - How to use Animation to move Interactors - Summary of what we've learned so far --- # Administrivia Important [Ed Post about two changes to the spec](https://us.edstem.org/courses/381/discussion/24921) 1. You do not need to handle lines that have a "positive slope" 2. We are asking that you turn in a video of your running custom doodle at the same time as your code. (demo) --- # Linear ("affine") Transformations Translate, Scale, Rotate, Shear (and any combination thereof) -- - Translate: Move origin (and everything else) in x and y  ??? used extensively in GUIS because child objects just draw themselves at *their* origin, so a component doesn't have to calculate how to draw itself based on its position -- - Scale: change size (negative == flip)  -- - Rotate and Shear  --- # Coordinate Transformations - Can modify any shape, including text. - In practice, complex transformations are done with matrices, and matrices are using `concat(Matrix)` - But Android helps with this by providing methods in the [Canvas](https://developer.android.com/reference/android/graphics/Canvas) object to transform a canvas such as ```java translate(float dx, float dy) rotate(float degrees) // the whole canvas around the canvas origin rotate (float degrees, float px, float py) // around a particular point scale(float, float) scale (float sx, float sy, float px, float py) // around a pivot point skew(float sx, float sy) // skew by sx and sy save() // save the current transform restore() // restore the previous transform ``` ??? - important thing to point out here: This is a value proposition for a toolkit again – Affine transformations are based on two-dimensional matrices of the following form: P' = T*P where P is 1x3 and T is the transform matrix (3x3) with the bottom row 0 0 1 Thus, x' = ax + cy + t_x and y' = bx + dy + t_y *Note* Any sequence of transform, rotate and shear can be represented in a single matrix of this form (just multiple the matrices together) --- # Breakout question: How can you rotate around the center of an object: <iframe src="https://embed.polleverywhere.com/multiple_choice_polls/DfxDLPdpd2zTpLlUJt9h9?controls=none&short_poll=true" width="800" height="500" frameBorder="0"></iframe> ??? [raise your hands] - A: Translate, rotate, translate - B: Rotate, Translate, Rotate - C: Scale, Rotate, Scale - D: Rotate XX define exercise maybe put this after android stuff? I usually draw this out on a piece of paper using the document camera to help --- # Breakout rooms test 2 - You've been randomly assigned you to breakout rooms with one TA each. - In the future, we will experiment with a system where you can choose your breakout partners - Goal: Meet other students, test out the technology - Please turn your cameras on in the breakouts - Introduce yourself, and tell your group what color your favorite shirt is. - Discuss the poll everywhere question and vote We'll do this for 5 minutes, then I will close the breakouts --- # Explanation .left-column[  ] .right-column[ New view created called `RotatedRectangleView` that has `mEndPoint` and `mBrush` defined in a similar way to your `LineView`. It also is given the number of degrees (stored in the variable mDegrees) by which to rotate the rectangle. ```java public void onDraw(Canvas canvas) { canvas.drawRect(0, 0, mEndPoint.x , mEndPoint.y, mBrush); int saveColor = mBrush.getColor(); mBrush.setColor(Color.GREEN); canvas.rotate(mDegrees); canvas.drawRect(0, 0, mEndPoint.x, mEndPoint.y, mBrush); } ``` ] --- # Moving the canvas .left-column[  ] .right-column[ Move origin of the `canvas` to the center of the object, THEN rotate it, then move the origin back to it's original location (in this new rotated orientation) ```java public void onDraw(Canvas canvas) { canvas.drawRect(0, 0, mEndPoint.x , mEndPoint.y, mBrush); float px = mEndPoint.x / 2; float py = mEndPoint.y / 2; int saveColor = mBrush.getColor(); mBrush.setColor(Color.GREEN); canvas.translate(px, py); canvas.rotate(mDegrees); canvas.translate(-px, -py); canvas.drawRect(0, 0, mEndPoint.x, mEndPoint.y, mBrush); } ``` ] --- name: inverse layout: true class: center, middle, inverse --- # Toolkits --- layout: false ## Review: Developer roles .left-column[ <div class="mermaid"> graph LR ip[Interface Programmer] w[Component Developer] l[Library Extender] a[Architecture Extender] t[Toolkit Builder] classDef yellow font-size:14pt,text-align:center classDef green font-size:14pt,text-align:center class t,l,a,w yellow class ip green </div> ] .right-column[ Recall the role Interface Programmer a developer can play in app development. ] --- # Draw the interactor hierarchy Think about why we use a `View` for **each** thing on the screen...   --- # Simpler Example as an Interface Programmer .left-column[  ] .right-column[ ```java LineView line = new LineView(this, startX, startY, endX, endY, width, color); canvas.addView(line); TextView text = new TextView(this); text.setX(x); text.setY(y); text.setText("placeholder"); canvas.addView(text); ``` ] --- ## Review: Developer roles .left-column[ <div class="mermaid"> graph LR ip[Interface Programmer] w[Component Developer] l[Library Extender] a[Architecture Extender] t[Toolkit Builder] classDef yellow font-size:14pt,text-align:center classDef green font-size:14pt,text-align:center class t green class ip,l,a,w yellow </div> ] .right-column[ Today we're going to discuss the role Toolkit builder plays in app development ] --- # What is the toolkit architecture? ??? Applies to the flow of information within the toolkit Example: How the interface is drawn What could you add? Support for animation, machine learning, etc Supported by very few toolkits --- # What is the toolkit architecture? Applies to the flow of information within the toolkit Examples: - How the interface is drawn - How the interface is layed out - How the interface responds to the user What could you add? Support for animation, machine learning, etc Adding new features at the toolkit level is supported by very few toolkits --- # How is a View drawn on the screen? **Toolkit Architecture Builder** - Depth-first tree traversal of the interactor hierarchy - Each View draws itself in its location - Siblings drawn in order they appear Naive Pseudocode: ```java protected void drawAll() { onDraw(); // Draw yourself. (lower in the Z-order) foreach child c { // for all of the children in this node if (child.isVisible()) { // if the child is visible child.drawAll(); // tell the child to draw itself } } } ``` ??? What is missing here? How can children misbehave? What are they expecting that we don't do? Draw out what will happen on paper --- # How is a View drawn on the screen? Naive Pseudocode: ```java protected void drawAll() { onDraw(); foreach child c { if (child.isVisible()) { child.drawAll(); } } } ``` - How could children misbehave? - Work arounds could we do? ??? Imagine if all the children were the same dimensions. --- # Work arounds? Make the **Interface Programmer** fix this Could make every view the size of the whole screen; Then just draw items in those views in the correct position Could tell every view where it is so it draws properly --- # For this, view needs a position, width and height .left-column-half[  ] .right-column-half[ What could go wrong with this approach? ] ??? - What if we want to place a different view, automatically, to the right of the text? - How do we guarrantee that a view doesn't draw over it's neighbor? --- # For this, view needs a position, width and height .left-column-half[  ] .right-column-half[ What could go wrong with this approach? - What if we want to place a different view, automatically, to the right of the text? - How do we guarrantee that a view doesn't draw over it's neighbor? ] --- # Specifying View position, width and height? .left-column-half[ <br>  ] .right-column-half[ **Interface Programmer** does this ```java View view = new View(); view.setX(); view.setY(); ViewGroup.LayoutParams params = view.getLayoutParams(); param.width = width; param.height = height; view.setLayoutParams(param); ``` ] ??? We'll go into much more depth on this in your next assignment --- # How does a toolkit use this? **Toolkit Architecture Builder** **Uses coordinate transformations to make each child the center of its universe!** - Parent (usually) specifies position of child when adding to the interactor hierarchy - Child counts on parent to set things up so child's internal coordinate system starts at (0,0) -- We just saw this with rotating the canvas! --- # How is a View drawn on the screen **Toolkit Architecture Builder** ```java protected void drawAll() { onDraw(); foreach child c { if (child.isVisible()) { child.drawAll(); } } } ``` Is this part of the _Toolkit Architecture_ or _Toolkit Library_? --- # How is a View drawn on the screen **Toolkit Architecture Builder** ```java protected void drawAll() { onDraw(); foreach child c { if (child.isVisible()) { * Rectangle r = child.getLayoutParams(); // Find child coordinates * canvas.save(); // Capture the current state of canvas * canvas.translate(r.x, r.y); // Move origin to child's top-left corner * canvas.clip(0,0,r.width,r.height); // Clip to child child.drawAll(); // Draw child (and all it's kids) * canvas.restore(); // Restore } } } ``` Is this part of the _Toolkit Architecture_ or _Toolkit Library_? ??? Draw out on paper again Note that this is has absolute layout! --- # Why is a bounding box important here?   --- name: inverse layout: true class: center, middle, inverse --- # Animation --- layout: false .left-column[  ] .right-column[ # Animation becoming core pillar of UX design Greater awareness of role for communicating UI behavior Guide, provide context, delight, engage .quote[‘Animation is increasingly becoming an important part of the UI design experience. Google’s material design guidelines are a good illustration of this. Expect to see even more tools and optimizations made to improve the production workflow and performance in browsers and on devices.’ ([Weareathlon](https://www.weareathlon.com/ideas/ten-ux-design-trends-for-2015))] ] --- # Old implementation approach Frame-based - Redraw scene at regular intervals - Developer defines redraw function What toolkit principals does this violate? ??? it's bad because it doesn't provide any useful abstractions Fails to provide separation of concerns -- - Doesn't provide supportive abstractions -- - Fails to provide separation of concerns --- # Main implementation approach Transition-based (Hudson & Stasko '93) - Specify property values of animation transition speed and process - Uses 'pacing' functions to stylize animation, e.g. slow-in slow-out (see [easings.net](http://easings.net)) - Typically computed via __interpolation__ ```java step(fraction){x_now = x_start + fraction*(x_end - x_start);} ``` - Timing and redraw managed by toolkit --- # Toolkit Architecture for Animation .left-column[ <br> <br> <br> <br> <br>  ] .right-column[ Steamlined process for animation of each frame - An update function is called - `invalidate()` is called - View is redrawn ] --- # How an animation is set up Define an animation that changes on object's property (a field on a object) over a length of time.  .footnote[[image source: Android animation documentation](https://developer.android.com/guide/topics/graphics/prop-animation)] --- # How an animation is set up Need the start and end value of the properties to be modified. Typically use a [Path](https://developer.android.com/reference/android/graphics/Path) for this. Need a *duration* (total time in ms for the animation) Need the *pacing function* for animation using an [Interpolator](https://developer.android.com/reference/android/view/animation/Interpolator) - Default is [AccelerateDecelerateInterpolator](https://developer.android.com/reference/android/view/animation/AccelerateDecelerateInterpolator) - Other subclasses include AccelerateInterpolator, AnticipateInterpolator, AnticipateOvershootInterpolator, BounceInterpolator, CycleInterpolator, DecelerateInterpolator, LinearInterpolator, OvershootInterpolator, PathInterpolator - Or make your own! --- # [ObjectAnimator](https://developer.android.com/reference/android/animation/ObjectAnimator) Directly animate a properties on an object **However**: the property that you are animating must have a setter function (in camel case) in the form of `set<propertyName>()` for this to work Can use a string (like `"alpha"`) or a property (like `View.X` in [View](https://developer.android.com/reference/android/view/View)) --- # [ObjectAnimator](https://developer.android.com/reference/android/animation/ObjectAnimator) .left-column[ <br> <br>  ] .right-column[ Example for color ```java // takes a target view, a property name, and values ObjectAnimator anim = ObjectAnimator.ofFloat(boundlessView, "alpha", 0f, 1f); anim.setDuration(1000); anim.start(); ``` ] --- # [ObjectAnimator](https://developer.android.com/reference/android/animation/ObjectAnimator) .left-column[  ] .right-column[ Example for position (using a [Path](https://developer.android.com/reference/android/graphics/Path)) ```java ImageView mouse = addImage(doodleView, "mouse", 500f, 50f, size); Path path = new Path(); path.moveTo(400f, 50f); path.arcTo(200f, 50f, 600f, 450, 270, -180, true); path.arcTo(200f, 450f, 600f, 850, 270f, 180f, true); ObjectAnimator anim = ObjectAnimator.ofFloat(mouse, View.X, View.Y, path); anim.setDuration(5000); anim.start(); ``` ] --- # [ObjectAnimator](https://developer.android.com/reference/android/animation/ObjectAnimator) .left-column[  ] .right-column[ - The static functions (like ofFloat) in ObjectAnimator are "factories" - the method creates the ObjectAnimator object for you! - You can have multiple ObjectAnimators working on a single view at the same time! ```java ImageView mouse = addImage(doodleView, "mouse", 500f, 50f, size); Path path = new Path(); // code to set up the path.... ObjectAnimator anim = ObjectAnimator.ofFloat(mouse, View.X, View.Y, path); anim.setDuration(5000); anim.start(); ObjectAnimator anim = ObjectAnimator.ofFloat(mouse,"rotation", 720); anim.setDuration(5000); anim.start(); ``` ] --- # Other useful properties for animation - `translationX` /`translationY` - view location as a delta from its top/left coordinates relative to the parent - `rotation` / `rotationX`/`rotationY` - control 2D rotation and 3D rotation around a pivot point - `scaleX` / `scaleY` - 2D scaliong of a `View` around a pivot point - `pivotX` / `pivotY` - changes location of thej pivot point (default is object's center) - `x` / `y` - utility property to describe the final location of a `View` in its container as a sum of (left, top) + `translationX`, `translationY`) - `alpha` .footnote[All found in the [View](https://developer.android.com/reference/android/view/View) class] --- # For more more flexibility... You can specify `Keyframe` objects to control the animation ```java // Key for start at 0 Keyframe kf0 = Keyframe.ofFloat(0f, 0f); // Key for half way finished animation Keyframe kf1 = Keyframe.ofFloat(.5f, 360f); // Key for end state Keyframe kf2 = Keyframe.ofFloat(1f, 0f); // ValueName-to-keyframes PropertyValuesHolder pvhRotation = PropertyValuesHolder.ofKeyframe("rotation", kf0, kf1, kf2); // Create the animation ObjectAnimator rotationAnim = ObjectAnimator.ofPropertyValuesHolder(target, pvhRotation) rotationAnim.setDuration(5000ms); ``` --- # Taking Animation to the next level .quote[ Despite the differences between user interfaces and cartoons -- cartoons are frivolous, passive entertainment and user interfaces are serious, interactive tools -- cartoon animation has much to lend user interfaces to realize both affective and cognitive benefits] .small[Principals of animation borrowed from Johnston & Thomas ‘81, Lasseter ‘87] --- # Chang & Ungar, UIST '93 What strategies do you see here? ![:youtube Opening cartoon from who framed roger rabbit showing him trying to keep a baby safe,oQe0OWZvwWo] ??? - Squash-and-Stretch - Staging, Overlapping Action - Anticipation - Follow-Through - Slow In Slow Out --- ## Example pacing functions -- derived from Disney style animation! .left-column[  ] .right-column[ Slow in slow out (Accelerate/decelerate) <br> <br> Slow in (Accelerate) <br> <br> Anticipate (back up slightly, then accelerate) <br> <br> Anticipate Overshoot (same, then go too far and slow down) ] --- # Why is pacing so important? Need to mimic real world - Observing motion tells us about size, weight, rigidity - No abrupt changes in velocity! Gives a feeling of reality and liveness - “animation” = “bring to life” - make inanimate object animate With this can come appeal and desirability --- # Example pacing functions Watch this and note the pacing you see! ![:youtube Video showing a mother and child lamp playing with a ball illustrating a range of techniques,6G3O60o5U7w] ??? Normally I show this twice and ask them what they see: - No teleportation! - Squash and Stretch (preserve volume; can approximate inertia (ball)) - Follow through (i.e. cord lags behind lamp) - Anticipation (small amount of counter movement (lampshade motion)) - Exaggeration (cord up and down) - (not shown) Motion blur (doesn't need to be realistic --- # How implement pacing in animation? How would `t` and `x` change for slow in slow out? --  .footnote[[image source: Android animation documentation](https://developer.android.com/guide/topics/graphics/prop-animation)] --- # We implement these with `Interpolators` in Android .left-column-half[     ] .right-column-half[ The durations are the same for each of these animations - Left side is _Linear_ - Second one is _Accelerate_ - Third one is ?? - Right side is ?? ] ??? overshoot bounce --- # Using an Interpolator ```java ObjectAnimator anim = ObjectAnimator.ofFloat(mouse, View.X, View.Y, path); // Create the ObjectAnimator OvershootInterpolator interpolator = new OvershootInterpolator(); // Create an Interpolator anim.setInterpolator(interpolator); // Tell the animator to use the interpolator anim.setDuration(3000); // set the duration anim.start(); // Tell the animation to start ``` --- # Using Animation Well - Accessbility .quote['The impact of animation on people with vestibular disorders can be quite severe. Triggered reactions include nausea, migraine headaches, and potentially needing bed rest to recover.' [W3C Accessibility Guidelines](https://www.w3.org/WAI/WCAG21/Understanding/animation-from-interactions.html)] Best option: provide control, be minimalistic ??? --- # Summary: Animation Design Tips 1. Used sparingly and understandingly, animation can enhance the interface … otherwise can distract! 2. Need to mimic real world 3. Observing motion tells us about size, weight, rigidity 4. No abrupt changes in velocity! 5. Think about accessibility. --- # What to do for Doodle Part2Activity? Your chance to make something creative. Peers will reviewed your custom doodle to ensure it uses some combination of lines, text, images, and animation. Peers will also "evaluate" (give you feedback on how much they liked it). --- # Summary & revisiting learning goals for this week and last .left-column-half[ - What is HCI? - What is a toolkit? How do an architecture and library differ? - How do we draw on the screen? - What is a pixel? - What is a raster vs a vector model? - What abstractions help with drawing? - How do we use affine transformations? - What is the interactor hierarchy? - How is it used for drawing? - What role do affine transformations play in this? - What are the key abstractions for animation? ] .right-column-half[ - Android basics - What is a `View`? - `onCreate()` and `onDraw()` - How does `Canvas` work? - How do we construct the interactor hierarchy? - Implementing Animation ]
