name: inverse layout: true class: center, middle, inverse --- layout: false # What went wrong? .left-column50[] .right-column50[] ??? - Changing available space e.g., window resized by user - Changing sizes, fonts, etc. - Adding and removing components - Layout mechanism has to deal with all cases --- # What went wrong?  ??? - No scroll bar for text boxes that are too narrow - No way to redistribute space between directory & file list - Important controls (e.g., Open) get hidden - Min size is much too small - No way to send the dialog away (buttons gone) --- name: inverse layout: true class: center, middle, inverse --- # Layout Algorithms Jennifer Mankoff CSE 340, Winter 2020 --- name: normal layout: true class: --- layout: false [//]: # (Outline Slide) # Goals for today - __Review__ - ConstraintLayout in Android - How layout is implemented in the toolkit --- # Review: User Interfaces on Android .left-column-half[ - Views - Base class for __all__ UI elements - Controls (switches, sliders, etc) - UI Components - typical display elements mentioned in SSUI lecture (e.g buttons, labels, image views, etc) - ViewGroups - Encapsulates one or more views (e.g. Android Components, Layouts) - Can define specific **layout** strategies ] .right-column-half[ <div class="mermaid"> graph TD W(ViewGroup) --> V[ViewGroup] W --> V1[View] W --> V2[View] V --> V3[View] V --> V4[View] V --> V5[View] class W,V darkblue class V1,V2,V3,V4,V5 blue </div> ] --- # Review: XML vs programmatic layout ```java LayoutInflater inflater = LayoutInflater.from(context); addView(inflater.inflate(R.layout.part3_grid, null), MainActivity.PARAMS); ``` vs ```java addView(new Part2View(this, getImageStrings(), vMargin), PARAMS); ``` ??? --- # Layout Types in Android - [FrameLayout](https://developer.android.com/reference/android/component/FrameLayout.html) - good for position views on top of each other, or encapsulating a bunch of views. Used in [Doodle](/assignments/doodle). - [__LinearLayout__](https://developer.android.com/reference/android/component/LinearLayout.html) - places views one after the other in order according to the orientation (Horizontal or Vertical). Used in [Layout](/assignments/layout). - [__TableLayout__](https://developer.android.com/reference/android/component/TableLayout.html) - Rows and columns style way of declaring a layout - [GridLayout](https://developer.android.com/reference/android/component/GridLayout.html) - Uses an [*adapter*](https://developer.android.com/reference/android/interactor/Adapter.html) that provides items to display in a grid - [ConstraintLayout](https://developer.android.com/reference/android/component/ConstraintLayout.html) Let's you use constraints to specify how things should lay out. Used in [Layout](/assignments/layout). - More on https://developer.android.com/guide/topics/ui/declaring-layout.html --- layout: false # Layouts in Play .left-column-half[ Which layouts are used in these apps? ] .right-column-half[   ] -- .left-column-half[ Probably: - **LinearLayout** - **ConstraintLayout** ] --- layout: false ## ConstraintLayout in Android .left-column[  ] .right-column[ - [ConstraintLayout](https://developer.android.com/reference/android/support/constraint/ConstraintLayout) - Allows you to position widgets in a flexible way - Useful for building **responsive** interfaces in Android. - Will be using ConstraintLayout for the Layout exercise - Review Friday's slides for more details ] --- layout: false [//]: # (Outline Slide) # Goals for today - Review - ConstraintLayout in Android - **How layout is implemented in the toolkit** --- # Time to talk about position again .left-column-half[ .quote[You have been asked to create a new `CircleView` object that can draw a circle. The user wants to add a circle with its center at (100,100) and a radius of (10). In `CircleView.onDraw()` you will need to call `canvas.drawCircle(cx, cy, radius, paint)` (note that `drawCircle` takes the `x,y` location of the center of the circle as input). What values should you use for cx and cy? ] ] .right-column-half[  ] --- # From the quiz: .left-column-half[ .quote[Consider the same circle (at `100,100` with a radius of `10` in its parent). How should we set up the bounding box for the `CircleView` so that its `Canvas` will be correctly clipped? What are `x`, `y`, `width` and `height` for the bounding box (in parent coordinates?) ] **NOTE: This quiz question has an error. The correct answer is .jax[(90,90,20,20)]. At .jax[(100,100)] the circle will be clipped so only .jax[1/4] of it shows.** ] .right-column-half[  ] --- # Time to talk about position again .left-column-half[ .quote[You have been asked to create a new `SquareView` object that can draw a square. The user wants to add a square with its top left corner at (150,50) and a width of (20). In `SquareView.onDraw()` you will need to call `canvas.drawRect(left, top, right, bottom, paint)` What values should you use for `left` and `top`? ] ] ??? (0,0) -- .right-column-half[ `(0,0)` ] --- # Time to talk about position again .left-column-half[ .quote[Consider the same square (at `150,50` with a width of `20` in its parent). How should we set up the bounding box for the `SquareView` so that its `Canvas` will be correctly clipped? What are `x`, `y`, `width` and `height` for the bounding box (in parent coordinates?) ] ] ??? (150,50,20,20) -- .right-column-half[ `(150,50,20,20)` ] --- # So: what have we learned? -- When an interactor draws itself, its drawing area always starts at (0,0) The *View* that interactor is in (it's *parent*) determines the location of its bounding box. This is what correctly positions it! --- # How does layout come into this? In Doodle you hard-coded the position of everything (or used animation to set it) But in most interfaces, we use *layout containers* to accomplish this. - Layout containers make decisions like "stack these views vertically" or "make them fit into this size area" and set their position and bounding box on this basis - The *toolkit architecture* then helps to enforce this --- .left-column[ # How is position calculated by the toolkit architecture? <div class="mermaid"> graph TD W(ViewGroup) --> V[ViewGroup] W --> V1[View] W --> V2[View] V --> V3[View] V --> V4[View] V --> V5[View] class W,V darkblue class V1,V2,V3,V4,V5 blue </div> ] .right-column[ Toolkit architecture invokes layout Containers do the work - Manage size and position of children - Enforce component abstraction: Simple loop through child components --> Recursive tree traversal - Translate child to correct location - Clip child after drawing is complete Know how to handle resizing (reactive) ] ??? - Parent knows how to setup for drawing of children, invoke their drawing code, and add additional output (before and/or after) based on what the parent is and its internal state - Whorfian effects (things not in the library are hard) - Nesting not well defined --- .left-column[ # How do Container Components actually do layout? ] .right-column[ Key Issues: Where do components get placed? How much space should they occupy? How to react to changes - number of components - size of window ] ??? Who should get to decide kids' size? parents or kids? - top down vs bottom up approach - Depth first tree traversal --- # Simplest possible layout implementation (similar to what is in Frame layout) Depth first tree traversal No consideration of child size or position in doing layout Simplified from FrameLayout [source code](https://android.googlesource.com/platform/frameworks/base/+/master/core/java/android/widget/FrameLayout.java), [View](https://android.googlesource.com/platform/frameworks/base/+/master/core/java/android/view/View.java) and [ViewGroup](https://android.googlesource.com/platform/frameworks/base/+/master/core/java/android/view/ViewGroup.java) ```java protected void onLayout(boolean changed, int l, int t, int r, int b) { foreach child c { if (child.isVisible()) { Rectangle r = child.getLayoutParams(); childLeft = padding + r.x; childTop = padding + r.y; child.layout(childLeft, childTop, r.w, r.h); } } } ``` ??? we're ignoring the implementation of gravity here remind them what protected means? --- # Generalized implementation of layout Details of most else, and two-pass process, handled in [View](https://android.googlesource.com/platform/frameworks/base/+/master/core/java/android/view/View.java) and [ViewGroup](https://android.googlesource.com/platform/frameworks/base/+/master/core/java/android/view/ViewGroup.java) ```java // Second: do the actual updating public void layout(Rectangle newBounds) { onMeasure() // measure myself; this is recursive onLayout() // again; recursive. Makes use of calculated measurements } ``` --- .left-column[ # Measuring ] .right-column[ `onMeasure()` is a callback invoked by `View.measure()` ] -- .right-column[ Guidance about the size is given from parent view in the form of `MeasureSpec` parameters ] -- .right-column[ Ensures every view has a width and height value set prior to a layout pass ] -- .right-column[ Note: You must manually take padding into account (subtract padding from the width/height dimensions) ] --- .left-column[ # Best toolkits let child specify ] .right-column[ Size (based on nature of component (think texbox vs button vs LinearLayout) - preferred size - minimum size - maximum size Parent gets to ignore all of that, but usually tries not to. Someone has to be in charge; it's the parents (my kids hate that! :) - Usually tries to give each child as much as possible & divide up the remainder proportionally - However, child component has to deal if not enough (i.e. truncate text) ] --- # Reference Implementation of onMeasure ```java // First: Measure everything protected void onMeasure(int widthspec, int heightspec) { // loop through children in order they were added foreach child { // get child's preferred layout size onMeasure() // story relevant information & do calculations about // the container size } // based on all that, update our dimensions setMeasuredDimension() } ``` Why does a container need to know about its kids' sizes to measure itself? ??? Example: it might need to be big enough to fit them all (LinearLayout) --- # Aside `ImageView` height: - What is it's height before layout is done? Unknown! - But you need it in Part3 to decide where to put it - You haven't added it to the interactor hierarchy yet - How do you get height? -- Call `image.measure()` (you'll have to think about what the right parameters are) Then call `image.getMeasuredHeight()` (why not `image.getHeight()`?) -- Because it's 0! Since this image isn't laid out. --- # Frame layout -- actual implementation (slightly simplified) ```java protected void onMeasure(int widthMeasureSpec, int heightMeasureSpec { // recursively calculate size of children foreach child { // ... calculate max child width and max child height maxWidth = Math.max(maxWidth, child.getMeasuredWidth() // + margins); maxHeight = ... } // ... add in padding setMeasuredDimension(... based on what was found) // now recursively set dimensions for all children foreach child { final View child = getChildAt(i); child.measure(childWidthMeasureSpec, childHeightMeasureSpec); } } ``` ??? How would we do this for linearLayout? --- # Generalized implementation of layout Details of most else, and two-pass process, handled in [View](https://android.googlesource.com/platform/frameworks/base/+/master/core/java/android/view/View.java) and [ViewGroup](https://android.googlesource.com/platform/frameworks/base/+/master/core/java/android/view/ViewGroup.java) ```java // Second: do the actual updating public void layout(Rectangle newBounds) { onMeasure() // measure myself; this is recursive onLayout() // again; recursive. Makes use of calculated measurements } ``` --- .left-column[ # Layout ] .right-column[ `onLayout()` is a callback triggered when a view's position is changed ] -- .right-column[ Gives the new position __relative__ to the parent ] -- .right-column[ Causes a layout call on all children (!) ] --- # Reference implementation of onLayout ```java // a layout class should override onLayout. protected void onLayout(boolean changed, int l, int t, int r, int b) { // recursively calls layout on each child foreach child { LayoutParams lp = child.getLayoutParams(); // calculate the width and height of the kids int width = calculated width int height = calculated height // calculate the position of kids int childLeft = calculated position int childTop = calculated position child.layout(childLeft, childTop, childLeft + width, childTop + height); } } ``` ??? What might we use for left/top/width/height - in FrameLayout? - in LinearLayout? --- # How would LinearLayout calculate these? This is just setting the bounding box! What is child *n*'s position? Width? Height? - need to know we are vertical - can decide to set our width entirely based on kids - we get to decide position --- # Generalized implementation of onLayout When is this called? ??? - Whenever the interface is *damaged* (call `invalidate(true)` to force this) - Before the recursive `drawAll()` function we talked about last week. **Why?** -- - Whenever the interface is *damaged* (call `invalidate(true)` to force this) - Before the recursive `drawAll()` function we talked about last week. **Why?** ??? Because `drawAll()` needs to know where interactors are to properly *translate* and *clip* before calling `onDraw()` on each of them -- - Because `drawAll()` needs to know where interactors are to properly *translate* and *clip* before calling `onDraw()` on each of them --- .left-column[ # How does this relate to drawing? <div class="mermaid"> graph TD W(ViewGroup) --> V[ViewGroup] W --> V1[View] W --> V2[View] V --> V3[View] V --> V4[View] V --> V5[View] class W,V darkblue class V1,V2,V3,V4,V5 blue </div> ] .right-column[ *After* measuring & layout: __Tree traversal__ may be done on the layout hierarchy. Order depends on the specific layout algorithm (constraints don't require any!) ] -- .right-column[ Each view is responsible for drawing itself ] -- .right-column[ Siblings are drawn in the order that they appear ] --- .left-column[ # When to (re)Draw? ] .right-column[ Drawing is costly- (e.g. frame rates dropping for scrolling interactions) ] -- .right-column[ Often the screen is static ] -- .right-column[ We should *draw when we need to* ] -- .right-column[ Re-drawing "damaged" areas: call `invalidate()` ] --- .left-column[ # Summary of the Drawing Process ] .right-column[ 1) Android calls measure on root node (view) ```java // View specifies size of self protected void onMeasure(int widthMeasureSpec, int heightMeasureSpec); ``` ] --- .left-column[ # Summary of the Drawing Process ] .right-column[ 1) Android calls measure on root node (view) ```java // View specifies size of self protected void onMeasure(int widthMeasureSpec, int heightMeasureSpec); ``` 2) Android calls layout on view ```java // View specifies size and position of children protected void onLayout(boolean changed, int left, int top, int right, int bottom); ``` ] --- .left-column[ # Summary of the Drawing Process ] .right-column[ 1) Android calls measure on root node (view) ```java // View specifies size of self protected void onMeasure(int widthMeasureSpec, int heightMeasureSpec); ``` 2) Android calls layout on view ```java // View specifies size and position of children protected void onLayout(boolean changed, int left, int top, int right, int bottom); ``` 3) Android calls draw on view ```java // Add the drawing happens protected void onDraw(Canvas canvas); ``` ] --- # What of this do *you* need to do - If you are creating an interface, none of it - Just use the layouts - If you are creating a new type of layout class - It depends - If you can manage layout entirely by specifying an interactor hierarchy with layout params that do what you want, you are golden (our assignment) - If you want to do something entirely new (lay out images in a spiral for example) you have to override `onLayout()` or `onMeasure()` or both (not assigned) --- # End of deck --- #Conceptual form of UI constraints ``` java // this is 5 pixels to the right of that this.x = that.x + that.w + 5 // this is centered this.x = that.x + that.w/2 - this.w/2 // this is 10 larger than children this.w = 10 + max_child().x + max_child().w ``` --- .left-column[ ## Example: Linear Layout of Email [Linear Layout Tutorial](https://developer.android.com/guide/topics/ui/layout/linear) ] .right-column[ ```xml <LinearLayout xmlns:android="http://schemas.android.com/apk/res/android" android:layout_width="match_parent" android:layout_height="match_parent" android:paddingLeft="16dp" android:paddingRight="16dp" android:orientation="vertical" > <EditText android:layout_width="match_parent" android:layout_height="wrap_content" android:hint="@string/to" /> <EditText android:layout_width="match_parent" android:layout_height="wrap_content" android:hint="@string/subject" /> <EditText android:layout_width="match_parent" android:layout_height="0dp" android:layout_weight="1" android:gravity="top" android:hint="@string/message" /> <Button android:layout_width="100dp" android:layout_height="wrap_content" android:layout_gravity="right" android:text="@string/send" /> </LinearLayout> ``` ] ??? Can implement all of the things we discussed earlier using constraints note that they can be hard to debug? Discussion of specific inheritance hierarchy for constraints - Only have to write once when we use classes properly - can mix and match things --- # Frame layout -- actual implementation (slightly simplified) Simplified from FrameLayout [source code](https://android.googlesource.com/platform/frameworks/base/+/master/core/java/android/widget/FrameLayout.java), [View](https://android.googlesource.com/platform/frameworks/base/+/master/core/java/android/view/View.java) and [ViewGroup](https://android.googlesource.com/platform/frameworks/base/+/master/core/java/android/view/ViewGroup.java) ```java // this is implemented in view public void layout(int l, int t, int r, int b) { // first make sure everything has a size (recursively) onMeasure() // then do layout onLayout(changed, left, top, right, bottom) } // a layout class should override onLayout. In FrameLayout protected void onLayout(boolean changed, int l, int t, int r, int b) { // recursively calls layout on each child foreach child { LayoutParams lp = child.getLayoutParams(); int width = child.getMeasuredWidth(); int height = child.getMeasuredHeight(); int childLeft = parentLeft + lp.leftMargin; // add padding int childTop = parentTop + lp.topMargin; // add padding child.layout(childLeft, childTop, childLeft + width, childTop + height); } } ``` ??? Traverses the hierarchy many times over as implemented Can try to be more efficient or give more control... tradeoffs --- # Try it yourself  --- A More Flexible Model of Interactor Size - Natural size (preferred size) - Min size - Max size --- # General & Powerful Approach: Constraints General mechanism for establishing and maintaining relationships between things - Layout is one use - Several other uses in UI - Connection of application to UI, e.g. deriving appearance from data - Multiple views of same data - Automated semantic feedback - Automatic arrangement of lines (powerpoint snapping!) --- layout: false [//]: # (Outline Slide) # Goals for today - Review - __ConstraintLayout in Android__ - How layout is implemented in the toolkit --- # What are Constraints? .left-column[ ] .right-column[ - Very general - Can reproduce most other things - Can operate on multiple axes - Can enhance other layout options What do you think this does? `app:layout_constraintBottom_toTopOf="@+id/linearLayout"` And this? `app:layout_constraintEnd_toEndOf="parent"` We also use `layout_constraintStart_toStartOf` and `layout_constraintTop_toTopOf` to create this (what else could we use?) (note: full xml spec if you scroll down) ```java <TextView android:id="@+id/textView" android:layout_width="0dp" android:layout_height="0dp" android:layout_marginStart="8dp" android:layout_marginTop="8dp" android:layout_marginEnd="8dp" android:layout_marginBottom="8dp" android:text="@string/sample_text" app:layout_constraintBottom_toTopOf="@+id/linearLayout" app:layout_constraintEnd_toEndOf="parent" app:layout_constraintStart_toStartOf="parent" app:layout_constraintTop_toTopOf="parent" /> lp.setMargins((int)(1*density),0,0,0); ``` ] ??? Righttorightof Righttoleftof --- .left-column[ ## Constraints in Android ] .right-column[ [Docs](https://developer.android.com/training/constraint-layout/): limited set of constraints - 1 Size ratio - 2 Delete constraint (not a constraint, just removes them) - 3 Height/width mode (3 main types): - Wrap constraint  - Fixed size  - Match Constraint  - 4 Margins - 5 Constraint bias (essentially weights on competing constraints) Range of attachment options (e.g. button sides, corners) Worth getting to know additional abstractions (groups; guidelines; barriers; chains) ] ??? -bias lets us create something that is 2-3 of the way over rather than centered go back to android to demo again --- # Powerful option Can do everything we can do with springs, struts, and linear layouts Declare relationships (.red[what] should be true) System automatically maintains relationships under change (.red[how]) --- # Powerful option Can do everything we can do with springs, struts, and linear layouts Declare relationships (what should be true) - This should be centered in that - This should be 12 pixels to the right of that - Parent should be 5 pixels larger than child System automatically maintains relationships under change (how) --- # Powerful option Can do everything we can do with springs, struts, and linear layouts Declare relationships (what should be true) System automatically maintains relationships under change (how) Too good to be true? - Unsolvable for arbitrary things - Works really well for a limited set ??? a good set for ui programming xx need to make sure I know why it is hard for arbitrary things --- # Note that these are one-way constraints You can change the right side, and it will update the left side (not the reverse) Can be very inefficient .jax[$$O(2^n)$$] But highly efficient incremental update algorithms exist ??? Only have to update things that might change Hudson's work on this was seminal can have a DAG but not a cycle (thus, a tree) hard to guard against cycles --- # Fame or Shame? 
