name: inverse layout: true class: center, middle, inverse --- # Introduction to Drawing Lauren Bricker CSE 340 Winter 2020 --- layout: false [//]: # (Outline Slide) # Today's goals - Warmup - Abstractions for drawing on the screen - Drawing in Android - Clipping and other transformations - Animating --- Warmup <iframe src="https://embed.polleverywhere.com/multiple_choice_polls/V7VeJFUXSakHa1pswiLoi?controls=none&short_poll=true" width="800" height="500" frameBorder="0"></iframe> -- What was missing from this list? ??? - High Level Tools - Window System] --- # The Application Stack <div class="mermaid"> graph LR ap[Application Program] hlt[High Level Tools] t[Toolkit] w[Window System] o[OS] h[Hardware] class ap,o,w,hlt,h,t yellow </div> --- ## Developer roles .left-column[ <div class="mermaid"> graph LR ip[Interface Programmer] w[Component Developer] l[Library Extender] a[Architecture Extender] t[Toolkit Builder] class t,l,a yellow class ip,w green </div> ] .right-column[ From [last class](toolkits.html) we briefly discussed of the roles (the Interface Programmer) a developer can play in app development. - **Important**: Go back and read those slides - What do you think each one of these mean? - Today we are going to discuss how to be a Component Developer ] --- # Drawing on the Screen - Take out your phone. Open an App. What do you see on your screen? --- # Drawing on the Screen - Take out your phone. Open an App. What do you see on your screen? .left-column[ ] .right-column[ (this is what I currently see) What is an app made up of? ] --- # What is on your screen? .left-column[  ] .right-column[ App is made up of? Interactors Last class we discussed the "tree" of components. Discuss with your neighbor what this hierarchy might look like. ] --- # What is on your screen? .left-column[ ] .right-column[ <div class="mermaid" style="width: 75%"> graph LR W(Window) --> T[Title Bar] W --> L[Label: My Folders] W --> F[Folders] W --> B[Toolbar] T --> TA[Icon] T --> TB[Label: Activities] T --> TC[Button: Search] T --> TD[Button: Folder] T --> TE[Button: Add] F --> F1[Folder 1] F --> F2[Folder 2] F --> F3[Folder 3] F1 --> I[Icon] F1 --> N[Folder Name] F1 --> C[Activity Count] B --> H[Button:Home] B --> A[Button: Activities] B --> R[Button: Reports] B --> P[Button: Profile] class W start class T,L,E,F,B,TA,TB,TC,TD,TE,F1,F2,F3,H,A,R,P,I,N,C blue </div> ] --- # What is on your screen? .left-column[  ] .right-column[ App is made up of? Interactors What are Interactors made up of? ] ??? Should push them down to the lowest level on the screen App is made up of? Interactors Interactors are made up of? lines, circles, text Lines, circles and text are made up of? Pixels --- # What is on your screen? .left-column[ ] .right-column[ App is made up of? Interactors Interactors are made up of? lines, circles, text Lines, circles and text are made up of? Pixels Aside: Do you know how pixels are lit up on your screen? What about in a projector? How does your screen show color? ] ??? Get them to at least realize that the actual pixels on their screen are LEDs --- # Drawing on the Screen - What would you want control over if you were building an application? -- Interactors ??? Toolkit provides pre-drawn interactors for you -- In Android, we call these `Views` *Aside: Widgets, Interactors, and `Views` are all variations on the same term* -- - What if you were building an interactor? ??? Basic drawing... Toolkit allows you to draw using lines, circles, and so on --- # Toolkit Support for Basic Drawing .left-column[ ] .right-column[ All toolkits have some sort of `Canvas` object or abstraction - `Canvas` is an object level abstraction for performing drawing operations - Each `View` has a `Canvas` in Android - You can access it in the `onDraw(Canvas canvas)` method When is `onDraw()` called? ] ??? Connect to DrawingPanel in 143? -- - Handled automatically for you by toolkit - Triggered whenever the pixels are *dirty* - You can trigger this with `invalidate()` (we'll use this in the future) ??? Lots to discuss here. Mainly focus on introducing the concept of a library (a bunch of views you can pick from, Canvas and its methods, etc) vs an architecture (which controls when and in what order things happen). We'll learn about both in this class. You need to understand both to use a toolkit. --- # How does Canvas work? - Drawing is done by calling methods for different kinds of objects -- - For example: `drawRect`, `drawOval`, `drawLine`, `drawPath` ... [Full List in the API Documentation](https://developer.android.com/reference/android/graphics/Canvas.html) --- # How does Canvas work? - Drawing is done by calling methods for different kinds of objects - Lots happens under the covers... let's look at an example... -- "I want to draw a line on the screen" -- So I would likely call `drawLine` on the `Canvas`. -- So... how do I tell the computer where the start and end of the line will be? --- # How is location decided? - (0,0) is at the top left of the screen - The screen is drawn pixel by pixel from top left to bottom right in "raster" lines   [Source: Wikipedia](https://en.wikipedia.org/wiki/Analog_television#Displaying_an_image) --- # How is a drawing object rendered onscreen? - An object (like a line) would be converted into one or more **strokes** - A stroke has properties such as width or color - The stroke is converted to **pixels** (correspond to dots on your screen) - These pixels are put in a `Bitmap` (frame buffer) first, then updated on the screen all at once ??? top left is from old-style raster graphics, where a beam of light literally moved down the screen from top left to bottom right, causing chemical excitation when on and thus lighting the screen the last is left over from when both graphics and drawing were slower and more linear Still most reliable approach --- # How is a stroke converted to pixels? - Strokes (can be) size independent. Pixels are not - Strokes are continuous. Pixels are not - Anti-aliasing Essentially converting from **vector** to **raster** graphics ([Bresenham's algorithm](https://en.wikipedia.org/wiki/Bresenham%27s_line_algorithm)) --- # Quick understanding test What internal data structure (abstraction) represents an entire picture to display on a color screen? -- pixels in a 2D array  Pixel Calculator (Tanimoto 2010) ??? **A frame buffer (2D array) (picture)** The following are wrong, they represent a single pixel (but we haven't studied this yet) - A pixel (R,G,B) - A number from 0 to 255 --- # What is the [Paint](https://developer.android.com/reference/android/graphics/Paint.html) Object? .left-column[  ] .right-column[A Paint object is an abstraction that determines how drawing operations should look ```java Paint drawPaint = new Paint(); drawPaint.setColor(Color.BLUE); drawPaint.setAntiAlias(true); drawPaint.setStrokeWidth(5); drawPaint.setStyle(Paint.Style.STROKE); canvas.drawCircle(500, 500, 400, drawPaint); drawPaint.setStrokeJoin(Paint.Join.ROUND); drawPaint.setStrokeCap(Paint.Cap.ROUND); drawPaint.setColor(Color.GREEN); drawPaint.setStrokeWidth(15); canvas.drawLine(350, 300, 500,600, drawPaint); ```] --- # What is the [Paint](https://developer.android.com/reference/android/graphics/Paint.html) Object? - Abstraction that determines how drawing operations should look - Drawing style: stroked, filled - Line Characteristics: color, line width, join, cap styles - Fill Characteristics (color)    --- # Other Paint properties - Text Characteristics: font family, style, size, alignment settings, etc. -- - font: shapes for chars (called *glyphs*) plus layout - family or typeface  --- # Other Paint properties - Text Characteristics: font family, style, size, alignment settings, etc. - font: shapes for chars (called *glyphs*) plus layout - family or typeface - style: plain, *italic*, **bold**, **_both_**, ~~strikethrough~~, *etc.* -- - points: 72.27 points per inch ??? because french inches were different (72 per inch) Really just a guideline these days It is the *width* of a capital M (no height specified) --- # The Path Object - Canvas supports drawing primitive shapes _and_ arbitrary paths: `void drawPath(Path p, Paint paint);` -- - Declaring a `Path` can be done by combining basic primitive shapes (e.g rectangles, circles, ovals, curves like arc, cubic, etc.) -- - What drawing model does creating a path in this way correspond to? - Raster model - Vector model --- # The Path Object - Canvas supports drawing primitive shapes _and_ arbitrary paths: `void drawPath(Path p, Paint paint);` - Declaring a `Path` can be done by combining basic primitive shapes (e.g rectangles, circles, ovals, curves like arc, cubic, etc.) - Which is more scalable? - Raster model - Vector model ??? XXXX could do in-class [drawing exercise](https://github.com/mriveralee/ssui-mobile-exercises-2016/tree/master/exercises/lab05/CustomDrawing) IF time XXXX needs updating --- # Doodle assignment: practice with drawing Inspired by Google's Doodles, and will include animation (to be discussed) First we will create holders for things you might draw - Android already provides an `ImageView` and a `TextView` - You will add a `LineView` - You will produce some things we specify - Then you will have a chance to create any animation you want! --- # [Example from last year's class](img/drawing/doodlevid.mp4) .left-column[ Note that we use a `View` for **each** thing on the screen ] .right-column[ ![:youtube Animation showing images of food moving in a line down the page around a U finally forming a W, Sx8oiJGjaIM] ] --- # Why so many separate `Views`? Useful later when we want to do animation Also provides **separation of concerns** and good high level control - Can have a different paint object for each view - Can easily change z-order (what is on top of what) - Can store information used to draw that thing (whatever it is) - Can either make the size of the whole screen or resize around what is drawn Views can also be re-used easily ??? This could be a good place to ask them to think about what that means and demonstrating two things with different z order. --- # Advanced Canvas: Coordinate Transformations .left-column[ ## How does the android toolkit move the various objects on this screen? ] .right-column[ ![:youtube A phone screen with patterned circles spinning in different directions on it, w7DEWWtIlrs] ] ??? Be sure to orally narrate what is happening for low-vision people in the room --- # Linear ("affine") Transformations available Translate, Scale, Rotate, Shear (and any combination thereof) -- - Translate: Move origin (and everything else) in x and y  ??? used extensively in GUIS because child objects just draw themselves at *their* origin, so a component doesn't have to calculate how to draw itself based on its position -- - Scale: change size (negative == flip)  -- - Rotate and Shear  --- # Coordinate Transformations - Can modify any shape, including text. - In practice, complex transformations are done with matrices, and matrices are using `concat(Matrix)` - But Android helps with this by providing methods in the [Canvas](https://developer.android.com/reference/android/graphics/Canvas) object to transform a canvas such as ```java translate(float dx, float dy) rotate(float degrees) // the whole canvas around the canvas origin rotate (float degrees, float px, float py) // around a particular point scale(float, float) scale (float sx, float sy, float px, float py) // around a pivot point skew(float sx, float sy) // skew by sx and sy save() // save the current transform restore() // restore the previous transform ``` ??? - important thing to point out here: This is a value proposition for a toolkit again – Affine transformations are based on two-dimensional matrices of the following form: P' = T*P where P is 1x3 and T is the transform matrix (3x3) with the bottom row 0 0 1 Thus, x' = ax + cy + t_x and y' = bx + dy + t_y *Note* Any sequence of transform, rotate and shear can be represented in a single matrix of this form (just multiple the matrices together) --- # Question: How to rotate about the center of an object <iframe src="https://embed.polleverywhere.com/multiple_choice_polls/DfxDLPdpd2zTpLlUJt9h9?controls=none&short_poll=true" width="800" height="500" frameBorder="0"></iframe> ??? [raise your hands] - A: Translate, rotate, translate - B: Rotate, Translate, Rotate - C: Scale, Rotate, Scale - D: Rotate XX define exercise maybe put this after android stuff? I usually draw this out on a piece of paper using the document camera to help --- # Solution: How to rotate about the center of an object .left-column[ ] .right-column[ ```java int rectHeight = 300; int rectWidth = 300; int rectY = 100; int rectX = 100; float px = (rectWidth - rectX)/2 + rectX; float py = (rectHeight - rectY)/2 + rectY; canvas.drawRect(rectX, rectY, rectWidth, rectHeight, paint); for (int i = 15; i < 360; i += 15) { int color = (i % 2 == 0) ? Color.GREEN : Color.BLUE; paint.setColor(color); canvas.drawRect(rectX, rectY, rectWidth, rectHeight, paint); canvas.translate(px, py); canvas.rotate(i); canvas.translate(-px, -py); } ``` ] --- .left-column[  ] .right-column[ # Animation becoming core pillar of UX design Greater awareness of role for communicating UI behavior Guide, provide context, delight, engage .quote[‘Animation is increasingly becoming an important part of the UI design experience. Google’s material design guidelines are a good illustration of this. Expect to see even more tools and optimizations made to improve the production workflow and performance in browsers and on devices.’ ([Weareathlon](https://www.weareathlon.com/ideas/ten-ux-design-trends-for-2015))] ] --- # Using Animation Well - Used sparingly and understandingly, animation can enhance the interface … otherwise can distract! - Need to mimic real world - Observing motion tells us about size, weight, rigidity - No abrupt changes in velocity! --- # Using Animation Well - Think about accessibility .quote['The impact of animation on people with vestibular disorders can be quite severe. Triggered reactions include nausea, migraine headaches, and potentially needing bed rest to recover.' [W3C Accessibility Guidelines](https://www.w3.org/WAI/WCAG21/Understanding/animation-from-interactions.html)] Best option: provide control, be minimalistic ??? --- # Old implementation approach Frame-based - Redraw scene at regular intervals - Developer defines redraw function What toolkit principals does this violate? ??? it's bad because it doesn't provide any useful abstractions Fails to provide separation of concerns -- Doesn't provide supportive abstractions Fails to provide separation of concerns --- # Main implementation approach Transition-based (Hudson & Stasko '93) - Specify property values of animation transition speed and process - Uses 'pacing' functions to stylize animation, e.g. slow-in slow-out (see [easings.net](http://easings.net)) - Typically computed via __interpolation__ ```java step(fraction){x_now = x_start + fraction*(x_end - x_start);} ``` - Timing and redraw managed by toolkit --- .left-column[ <br> <br> <br> <br> <br>  ] .right-column[ ##Toolkit Architecture for Animation Steamlined process for animation of each frame - An update function is called - `invalidate()` is called - View is redrawn ] --- .left-column[ ## What to do for Doodle Part2Activity? ] .right-column[ Your chance to make something creative Peers will "evaluate" (give you feedback on how much they liked it) Will also be peer graded to ensure it uses some combination of lines, text, and images ] --- # How an animation is set up Define an animate that changes on object's property (a field on a object) over a length of time  .footnote[[image source: Android animation documentation](https://developer.android.com/guide/topics/graphics/prop-animation)] --- # How an animation is set up Need the start and end value of the properties to be modified. Typically use a [Path](https://developer.android.com/reference/android/graphics/Path) for this. Need a *duration* (total time in ms for the animation) Need the *pacing function* for animation. Can explore subclasses of [Interpolator](https://developer.android.com/reference/android/view/animation/Interpolator) (or make your own!) for this --- ## Example pacing functions -- derived from Disney style animation! .left-column[  ] .right-column[ Slow in slow out (Accelerate/decelerate) <br> <br> Slow in (Accelerate) <br> <br> Anticipate (back up slightly, then accelerate) <br> <br> Anticipate Overshoot (same, then go too far and slow down) ] --- # Why is pacing so important? Need to mimic real world - Observing motion tells us about size, weight, rigidity - No abrupt changes in velocity! Gives a feeling of reality and liveness - “animation” = “bring to life” - make inanimate object animate With this can come appeal and desirability --- # Example pacing functions ![:youtube Video showing a mother and child lamp playing with a ball illustrating a range of techniques,6G3O60o5U7w] ??? Lauren you may need/want to cut this rather than try to analyze this video given time constraints Normally I show this twice and ask them what they see: - No teleportation! - Squash and Stretch (preserve volume; can approximate inertia (ball)) - Follow through (i.e. cord lags behind lamp) - Anticipation (small amount of counter movement (lampshade motion)) - Exaggeration (cord up and down) - (not shown) Motion blur (doesn't need to be realistic --- # [ObjectAnimator](https://developer.android.com/reference/android/animation/ObjectAnimator) Directly animate a properties on an object **However**: the property that you are animating must have a setter function (in camel case) in the form of set<propertyName>() for this to work Can use a string (like `"alpha"`) or a property (like `View.X` in [View](https://developer.android.com/reference/android/view/View)) --- # Useful properties for animation - `translationX` /`translationY` - view location as a delta from its top/left coordinates relative to the parent - `rotation` / `rotationX`/`rotationY` - control 2D rotation and 3D rotation around a pivot point - `scaleX` / `scaleY` - 2D scaliong of a `View` around a pivot point - `pivotX` / `pivotY` - changes location of thej pivot point (default is object's center) - `x` / `y` - utility property to describe the final location of a `View` in its container as a sum of (left, top) + `translationX`, `translationY`) - `alpha` .footnote[All found in the [View](https://developer.android.com/reference/android/view/View) class] --- # [ObjectAnimator](https://developer.android.com/reference/android/animation/ObjectAnimator) .left-column[ <br> <br>  ] .right-column[ Example for color ```java // takes a target view, a property name, and values ObjectAnimator anim = ObjectAnimator.ofFloat(foo, "alpha", 0f, 1f); anim.setDuration(1000); anim.start(); ``` ] --- # [ObjectAnimator](https://developer.android.com/reference/android/animation/ObjectAnimator) .left-column[  ] .right-column[ Example for position (using a [Path](https://developer.android.com/reference/android/graphics/Path)) ```java ImageView mouse = addImage(mainCanvas, "mouse", 500f, 50f, size); Path path = new Path(); path.moveTo(400f, 50f); path.arcTo(200f, 50f, 600f, 450, 270, -180, true); path.arcTo(200f, 450f, 600f, 850, 270f, 180f, true); ObjectAnimator anim = ObjectAnimator.ofFloat(mouse, View.X, View.Y, path); anim.setDuration(10000); anim.start(); ``` ] --- # For more more flexibility... You can specify `Keyframe` objects to control the animation ```java // Key for start at 0 Keyframe kf0 = Keyframe.ofFloat(0f, 0f); // Key for half way finished animation Keyframe kf1 = Keyframe.ofFloat(.5f, 360f); // Key for end state Keyframe kf2 = Keyframe.ofFloat(1f, 0f); // ValueName-to-keyframes PropertyValuesHolder pvhRotation = PropertyValuesHolder.ofKeyframe("rotation", kf0, kf1, kf2); // Create the animation ObjectAnimator rotationAnim = ObjectAnimator.ofPropertyValuesHolder(target, pvhRotation) rotationAnim.setDuration(5000ms); ``` --- # Summary & revisiting learning goals for this week - What are the layers of the application stack - What are the roles of developers at each layer - How to add interactors to the app - How interactors are drawn - Pixel & stroke models - Rotating around an object - Canvas & Paint & Paths in Android - Animation Implementation ??? needs updating
