# What went wrong? .left-column50[] .right-column50[] ??? - Changing available space e.g., window resized by user - Changing sizes, fonts, etc. - Adding and removing components - Layout mechanism has to deal with all cases --- # What went wrong?  ??? - No scroll bar for text boxes that are too narrow - No way to redistribute space between directory & file list - Important controls (e.g., Open) get hidden - Min size is much too small - No way to send the dialog away (buttons gone) --- name: inverse layout: true class: center, middle, inverse --- # Layout Algorithms Jennifer Mankoff CSE 340 Winter 2021 --- name: normal layout: true class: --- layout: false [//]: # (Outline Slide) # Goals for today - **Constraints** - How layout is implemented in the toolkit --- # Reminder: Layout Types in Android - [FrameLayout](https://developer.android.com/reference/android/component/FrameLayout.html) - good for position views on top of each other, or encapsulating a bunch of views. Used in [Doodle](/assignments/doodle). - [__LinearLayout__](https://developer.android.com/reference/android/component/LinearLayout.html) - places views one after the other in order according to the orientation (Horizontal or Vertical). Used in [Layout](/assignments/layout). - [__RelativeLayout__](https://developer.android.com/reference/android/widget/RelativeLayout) - Positions of the children are desribed in relation to one another - [__TableLayout__](https://developer.android.com/reference/android/component/TableLayout.html) - Rows and columns style way of declaring a layout - [GridLayout](https://developer.android.com/reference/android/component/GridLayout.html) - Uses an [*adapter*](https://developer.android.com/reference/android/interactor/Adapter.html) that provides items to display in a grid - [ConstraintLayout](https://developer.android.com/reference/android/component/ConstraintLayout.html) Let's you use constraints to specify how things should lay out. Used in [Layout](/assignments/layout). - More on https://developer.android.com/guide/topics/ui/declaring-layout.html ??? Talk about different Toolkits may have different layouts. --- name: constraint-template # Constraints: A Powerful Option .left-column[ <br> ] --- template: constraint-template .right-column[ Can do everything we can do with springs, struts, and linear layouts Declare relationships (.red[what] should be true) System automatically maintains relationships under change (.red[how]) ] --- template: constraint-template .right-column[ Can do everything we can do with springs, struts, and linear layouts Declare relationships (what should be true) - This should be centered in that - This should be 12 pixels to the right of that - Parent should be 5 pixels larger than child System automatically maintains relationships under change (how) ] --- # Constraint Layout in Android .row[ You can see little lines connecting the `textView` to its container and its sibling (the `linearLayout`). ] .row[ .left-column30[  ] .right-column60[ - This specifies how it's attached (can change type by clicking on right) - If you were to change the interface (e.g. a different sized screen), it would stay attached and keep filling the space - All ends up in XML you can explore too ] ] --- # Constraint Layout in Android .left-column50[  ] .right-column50[ You can test this in the emulator: modify the aspect ratio of your display, and flip it to horizontal to test things ] ??? Demonstration: consider demoing this interface live PRINT THIS LIST - Show dragging things to create layout - Show different size and orientation phones - Show editing attributes - Show clicking on constraints in box to change type - Show simple interface hierarchy - Show simplifying xml - Show changing button names/ids --- template: constraint-template .right-column[ - What do you think this does? `app:layout_constraintBottom_toTopOf="@+id/linearLayout"` ] --- template: constraint-template count: false .right-column[ - What do you think this does? `app:layout_constraintBottom_toTopOf="@+id/linearLayout"` - And this? `app:layout_constraintEnd_toEndOf="parent"` ] --- template: constraint-template count:false .right-column[ - What do you think this does? `app:layout_constraintBottom_toTopOf="@+id/linearLayout"` - And this? `app:layout_constraintEnd_toEndOf="parent"` - We also use `Start_tStartOf` and `Top_toTopOf` to create this layout ] --- # All of this can be specified in XML ```xml <TextView android:id="@+id/textView" android:layout_width="0dp" android:layout_height="0dp" android:text="@string/sample_text" app:layout_constraintBottom_toTopOf="@+id/linearLayout" app:layout_constraintEnd_toEndOf="parent" app:layout_constraintStart_toStartOf="parent" app:layout_constraintTop_toTopOf="parent" android:layout_marginStart="8dp" android:layout_marginTop="8dp" android:layout_marginEnd="8dp" android:layout_marginBottom="8dp"/> ``` --- .left-column[ ## Android's Limits ] .right-column[ Android provides a limited set of constraints ([Docs](https://developer.android.com/training/constraint-layout/)): - 1 Size ratio - 2 Delete constraint (not a constraint, just removes them) - 3 Height/width mode (3 main types): - Wrap constraint  - Fixed size  - Match Constraint  - 4 Margins - 5 Constraint bias (essentially weights on competing constraints) Range of attachment options (e.g. button sides, corners) Worth getting to know additional abstractions (groups; guidelines; barriers; chains) ] ??? -bias lets us create something that is 2-3 of the way over rather than centered go back to android to demo again --- exclude: true # Note that these are one-way constraints You can change the right side, and it will update the left side (not the reverse) Can be very inefficient .jax[$$O(2^n)$$] But highly efficient incremental update algorithms exist ??? Only have to update things that might change Hudson's work on this was seminal can have a Directed Acyclic Graph (DAG) but not a cycle (thus, a tree) hard to guard against cycles --- # Constraints: Very General General mechanism for establishing and maintaining relationships between things - Layout is one use - Several other uses in UI - Connection of application to UI, e.g. deriving appearance from data - Multiple views of same data - Automated semantic feedback - Automatic arrangement of lines (Snapping in drawing --- # Layouts in Play .left-column50[ Which layouts are used in these apps? ] .right-column50[   ] -- .left-column50[ Probably: - **LinearLayout** - **ConstraintLayout** Thinking about what layouts are in an app is part of your Part 4 of the Layout assignment. ] --- layout: false [//]: # (Outline Slide) # Goals for today - Constraints - **How layout is implemented in the toolkit** --- # Time to talk about position again .left-column50[ .quote[You have been asked to create a new `SquareView` object that can draw a square. The user wants to add a square with its top left corner at (150,50) and a width of (20). In `SquareView.onDraw()` you will need to call `canvas.drawRect(left, top, right, bottom, paint)` What values should you use for `left` and `top`? ] ] ??? (0,0) -- count: false .right-column50[ `(0,0)` ] --- # Time to talk about position again .left-column50[ .quote[Consider the same square (at `150,50` with a width of `20` in its parent). How should we set up the bounding box for the `SquareView` so that its `Canvas` will be correctly clipped? What are `x`, `y`, `width` and `height` for the bounding box (in parent coordinates?) ] ] ??? (150,50,20,20) -- count: false .right-column50[ `(150,50,20,20)` ] --- # So: what have we learned? -- count: false When an interactor draws itself, its drawing area always starts at (0,0) The *View* that interactor is in (its *parent*) determines the location of its bounding box. This is what correctly positions it! --- # How does layout come into this? In Doodle you hard-coded the position of everything (or used animation to set it) But in most interfaces, we use *layout containers* to accomplish this. - Layout containers make decisions like "stack these views vertically" or "make them fit into this size area" and set their position and bounding box on this basis - The *toolkit architecture* then helps to enforce this --- # Key Issues Containers have to Consider - Where do components get placed? - How much space should they occupy? - How should components to react to changes in layout (such as screen rotation) - number of components - size of window - How to enforce this during drawing? --- name: measure-layout # Measure, then Layout, then Draw .left-column30[ <div class="mermaid"> graph TD W(ViewGroup 1) --> V[ViewGroup 2] W --> V1[View 1.1] W --> V2[View 1.2] V --> V3[View 2.1] V --> V4[View 2.2] V --> V5[View 2.3] classDef blue font-size:14pt,text-align:center classDef darkblue font-size:14pt,text-align:center class W,V darkblue class V1,V2,V3,V4,V5 blue </div> ] ??? let's talk about how the size and position of each view is determined --- template: measure-layout .right-column60[ ## Measure - Parent calculates some constraints (based on its measured size) - Passes them to child and calls `onMeasure()` - Child calculates and stores its measured width and height ] .bottom[ We did this for you in Doodle ```java protected void onMeasure(int widthMeasureSpec, int heightMeasureSpec) { setMeasuredDimension(getMaxWidth(), getMaxHeight()); } ``` ] ??? Note that we ignore the specs here and always return a fixed width and height --- template: measure-layout .right-column60[ ## Measure - Typically this is recursive, [depth-first post-order](https://www.tutorialspoint.com/data_structures_algorithms/tree_traversal.htm) (i.e. based on its childrens' responses to `onMeasure()`) - Why does a container need to know about its kids' sizes to measure itself? ] ??? Example: it might need to be big enough to fit them all (LinearLayout) -- .right-column60[ What if it needs to `WRAP_CHILDREN`? ] --- # More about parent-child communication Child can define `layoutParams` in XML or java (`setLayoutParams()`) - These are type-specific (e.g. in a `LinearLayout` child should define [`LinearLayout.LayoutParams`](https://developer.android.com/reference/android/widget/LinearLayout.LayoutParams)) - They give the parent hints (such as "center me" or "give me priority") and can also specific preferred width and height: ```java ViewGroup.LayoutParams params = view.getLayoutParams(); param.width = width; param.height = height; view.setLayoutParams(param); ``` --- # More about parent-child communication Parent passes a "spec" ('AT_MOST' or 'EXACTLY' or 'UNSPECIFIED') plus a dimension (width/height) Child *must* respond with `setMeasuredDimension()` (its desired width and height) - Lots of helpers for this (such as `resolveSize(size, measureSpec)`) - Handle things you would otherwise deal with manually such as padding --- # Reference Implementation of onMeasure ```java // First: Measure everything protected void onMeasure(int widthspec, int heightspec) { // loop through children in order they were added foreach child { // get child's preferred layout size onMeasure() // story relevant information & do calculations about // the container size } // based on all that, update our dimensions setMeasuredDimension(...) } ``` Of course in a subclass you can call `super.onMeasure()` instead to make sure the recursion is handled Is this part of the _Toolkit Architecture_ or _Toolkit Library_? --- template: measure-layout .right-column60[ ## Layout - Parent *decides* size allocated to children - Can ignore child preferences (children will be clipped) - Parent *also decides* position of children - This is typically done recursively, [depth first post-order](https://www.tutorialspoint.com/data_structures_algorithms/tree_traversal.htm) (i.e. kids layout their kids, then parents layout them) - These are stored in `getWidth()`, `getHeight()`, `getX()` and `getY()` (i.e. they are *relative to the parent*) ] --- # Reference implementation of onLayout ```java // a layout class should override onLayout. protected void onLayout(boolean changed, int l, int t, int r, int b) { // recursively calls layout on each child foreach child { LayoutParams lp = child.getLayoutParams(); // calculate the width and height of the kids int width = calculated width int height = calculated height // calculate the position of kids int childLeft = calculated position int childTop = calculated position child.layout(childLeft, childTop, childLeft + width, childTop + height); } } ``` ??? What might we use for left/top/width/height - in FrameLayout? - in LinearLayout? --- template: measure-layout .right-column60[ ## Draw - This is when `child.onDraw()` is called - `onDraw()` is actually recursive too!!! <br> Why wasn't this an issue in Doodle? ] ??? because the things you implemented had no children!! -- .right-column60[ Because the things you implemented had no children!! ] --- template: measure-layout .right-column60[ ## Draw - This is when `child.onDraw()` is called - `onDraw()` is actually called by a recursive method too (`drawAll()`)!!! - The tree determines *z-order* in `drawAll()` - This is also recursive, but this time it is a [depth-first **pre-order**](https://www.tutorialspoint.com/data_structures_algorithms/tree_traversal.htm) traversal (parents first) | z-height | order | id | |----------|-------|----| | 3 | 5,6 | 1.1, 1.2 | | 2 | 4,5,6 | 2.1, 2.2, 2.3 | | 1 | 2 | 2 | | 0 | 1 | 1 | ] --- # Note: Parents enforce position and size During Drawing - Translate child's `Canvas` to correct location before calling `onDraw(canvas)` - Clip child to width and height after drawing is complete - This is childs last change to *obey* parent (it can draw bigger but will be clipped. Or it can try to fit in what was allocated to it) - Translate `Canvas` back - **Why?** -- - Ensures `(0,0)` is what you would expect --- # Reference Implementation of drawAll ```java protected void drawAll() { onDraw(); foreach child c { if (child.isVisible()) { child.drawAll(); } } } ``` --- # Reference Implementation of drawAll ```java protected void drawAll() { onDraw(); foreach child c { if (child.isVisible()) { * Rectangle r = child.getLayoutParams(); // Find child coordinates * canvas.save(); // Capture the current state of canvas * canvas.translate(r.x, r.y); // Move origin to child's top-left corner * canvas.clip(0,0,r.width,r.height); // Clip to child child.drawAll(); // Draw child (and all its kids) * canvas.restore(); // Restore } } } ``` ??? Draw out on paper again Note that this is has absolute layout! --- # Simplest possible layout implementation (similar to what is in Frame layout) No consideration of child size or position in doing layout Simplified from FrameLayout [source code](https://android.googlesource.com/platform/frameworks/base/+/master/core/java/android/widget/FrameLayout.java), [View](https://android.googlesource.com/platform/frameworks/base/+/master/core/java/android/view/View.java) and [ViewGroup](https://android.googlesource.com/platform/frameworks/base/+/master/core/java/android/view/ViewGroup.java) ```java protected void onLayout(boolean changed, int l, int t, int r, int b) { foreach child c { if (child.isVisible()) { Rectangle r = child.getLayoutParams(); childLeft = padding + r.x; childTop = padding + r.y; child.layout(childLeft, childTop, r.w, r.h); } } } ``` ??? we're ignoring the implementation of gravity here remind them what protected means? --- # Generalized implementation of layout Details of most else, and two-pass process, handled in [View](https://android.googlesource.com/platform/frameworks/base/+/master/core/java/android/view/View.java) and [ViewGroup](https://android.googlesource.com/platform/frameworks/base/+/master/core/java/android/view/ViewGroup.java) ```java // Second: do the actual updating public void layout(Rectangle newBounds) { onMeasure() // measure myself; this is recursive onLayout() // again; recursive. Makes use of calculated measurements } ``` --- # Frame layout -- actual implementation (slightly simplified) ```java protected void onMeasure(int widthMeasureSpec, int heightMeasureSpec { foreach child { // recursively calculate size of children // ... calculate max child width and max child height maxWidth = Math.max(maxWidth, child.getMeasuredWidth() // + margins); maxHeight = ... } // ... add in padding setMeasuredDimension(... based on what was found) // now recursively set dimensions for all children foreach child { final View child = getChildAt(i); child.measure(childWidthMeasureSpec, childHeightMeasureSpec); // recursion through callback } } ``` ??? How would we do this for linearLayout? --- # How would LinearLayout calculate these? This is just setting the bounding box! What is child *n*'s position? Width? Height? - need to know we are vertical - can decide to set our width entirely based on kids - decide position based on order and height of preceding kids --- # You can build anything you want!!! If you build your own container you could - Change where things are placed - Change the z-order (traverse the tree differently) - Decorate your kids (after they draw) - anything else you can imagine! --- # What of this do *you* need to do - If you are creating an interface ("Interface Developer"), none of it - Just use the layouts - If you are creating a new type of layout class ("Component Developer") - It depends - If you can manage layout entirely by specifying an interactor hierarchy with layout params that do what you want, you are golden (our assignment part 2) - If you want to do something entirely new (lay out images in a spiral for example) you have to override `onLayout()` or `onMeasure()` or both (our assignment part 3) --- class: center, middle, inverse --- # When does all of this happen? --- # When to (re)Draw? Drawing is costly- (e.g. frame rates dropping for scrolling interactions) -- Often the screen is static -- We should *draw when we need to* -- Re-drawing "damaged" areas: call `invalidate()` on those areas. Rest is asynchronous --- # When is Layout used ??? - Whenever the interface is *damaged* (call `invalidate(true)` to force this) - Before the recursive `drawAll()` function we talked about last week. **Why?** -- - Whenever the interface is *damaged* (call `invalidate(true)` to force this) - Before any recursive `drawAll()` call **Why?** ??? Because `drawAll()` needs to know where interactors are to properly *translate* and *clip* before calling `onDraw()` on each of them -- - Because `drawAll()` needs to know where interactors are to properly *translate* and *clip* before calling `onDraw()` on each of them --- # Summary of the Drawing Process 1) Android calls measure on root node (view) --- # Summary of the Drawing Process 1) Android calls measure on root node (view) 2) Android calls layout --- # Summary of the Drawing Process 1) Android calls measure on root node (view) 2) Android calls layout 3) Android draws everything --- # End of deck --- exclude: true #Conceptual form of UI constraints ``` java // this is 5 pixels to the right of that this.x = that.x + that.w + 5 // this is centered this.x = that.x + that.w/2 - this.w/2 // this is 10 larger than children this.w = 10 + max_child().x + max_child().w ``` --- exclude: true # Frame layout -- actual implementation (slightly simplified) Simplified from FrameLayout [source code](https://android.googlesource.com/platform/frameworks/base/+/master/core/java/android/widget/FrameLayout.java), [View](https://android.googlesource.com/platform/frameworks/base/+/master/core/java/android/view/View.java) and [ViewGroup](https://android.googlesource.com/platform/frameworks/base/+/master/core/java/android/view/ViewGroup.java) ```java // this is implemented in view public void layout(int l, int t, int r, int b) { // first make sure everything has a size (recursively) onMeasure() // then do layout onLayout(changed, left, top, right, bottom) } // a layout class should override onLayout. In FrameLayout protected void onLayout(boolean changed, int l, int t, int r, int b) { // recursively calls layout on each child foreach child { LayoutParams lp = child.getLayoutParams(); int width = child.getMeasuredWidth(); int height = child.getMeasuredHeight(); int childLeft = parentLeft + lp.leftPadding; // add padding int childTop = parentTop + lp.topPadding; // add padding child.layout(childLeft, childTop, childLeft + width, childTop + height); // recursion through callback } } ``` ??? Traverses the hierarchy many times over as implemented Can try to be more efficient or give more control... tradeoffs
