1	Announcements Project 1 Grading session this afternoon Artifacts due Friday (voting TBA) Project 2 out (online) Signup for panorama kits ASAP (weekend slots go quickly…) help session at end of class
2	Mosaics Today’s Readings Szeliski and Shum paper (sections 1 and 2, skim the rest) http://www.cs.washington.edu/education/courses/455/08wi/readings/szeliskiShum97.pdf
3	Image Mosaics + + … + =
4	How to do it? Basic Procedure Take a sequence of images from the same position Rotate the camera about its optical center Compute transformation between second image and first Shift the second image to overlap with the first Blend the two together to create a mosaic If there are more images, repeat
5	Aligning images How to account for warping? Translations are not enough to align the images Photoshop demo
6	Image reprojection The mosaic has a natural interpretation in 3D The images are reprojected onto a common plane The mosaic is formed on this plane
7	Image reprojection Basic question How to relate two images from the same camera center? how to map a pixel from PP1 to PP2
8	Image reprojection Observation Rather than thinking of this as a 3D reprojection, think of it as a 2D image warp from one image to another
9	Homographies Perspective projection of a plane Lots of names for this: homography, texture-map, colineation, planar projective map Modeled as a 2D warp using homogeneous coordinates
10	Image warping with homographies
11	Panoramas What if you want a 360° field of view?
12	Spherical projection Map 3D point (X,Y,Z) onto sphere
13	Spherical reprojection
14	Spherical reprojection
15	Spherical reprojection Map image to cylindrical coordinates need to know the focal length
16	Spherical image stitching What if you don’t know the camera rotation? Solve for the camera rotations Note that a pan (rotation) of the camera is a translation of the sphere! Use feature matching to solve for translations of spherical-warped images
17	Computing image translations
18	RAndom SAmple Consensus
19	RAndom SAmple Consensus
20	Least squares fit
21	RANSAC
22	Assembling the panorama Stitch pairs together, blend, then crop
23	Problem: Drift Error accumulation small errors accumulate over time
24	Problem: Drift Solution add another copy of first image at the end this gives a constraint: y_n = y₁ there are a bunch of ways to solve this problem add displacement of (y₁ – y_n)/(n -1) to each image after the first compute a global warp: y’ = y + ax run a big optimization problem, incorporating this constraint best solution, but more complicated known as “bundle adjustment”
25	Full-view Panorama
26	Different projections are possible
27	Project 2 (out today) Take pictures on a tripod (or handheld) Warp to spherical coordinates Extract features Align neighboring pairs using RANSAC Write out list of neighboring translations Correct for drift Read in warped images and blend them Crop the result and import into a viewer Roughly based on Autostitch By Matthew Brown and David Lowe http://www.cs.ubc.ca/~mbrown/autostitch/autostitch.html
28	Image Blending
29	Feathering
30	Effect of window size
31	Effect of window size
32	Good window size
33	Pyramid blending
34	Alpha Blending
35	Poisson Image Editing For more info: Perez et al, SIGGRAPH 2003 http://research.microsoft.com/vision/cambridge/papers/perez_siggraph03.pdf
36	Image warping Given a coordinate transform (x’,y’) = h(x,y) and a source image f(x,y), how do we compute a transformed image g(x’,y’) = f(h(x,y))?
37	Forward warping Send each pixel f(x,y) to its corresponding location (x’,y’) = h(x,y) in the second image
38	Forward warping Send each pixel f(x,y) to its corresponding location (x’,y’) = h(x,y) in the second image
39	Inverse warping Get each pixel g(x’,y’) from its corresponding location (x,y) = h^-1(x’,y’) in the first image
40	Inverse warping Get each pixel g(x’,y’) from its corresponding location (x,y) = h^-1(x’,y’) in the first image
41	Forward vs. inverse warping Q: which is better? A: usually inverse—eliminates holes however, it requires an invertible warp function—not always possible...
42	Other types of mosaics Can mosaic onto any surface if you know the geometry See NASA’s Visible Earth project for some stunning earth mosaics http://earthobservatory.nasa.gov/Newsroom/BlueMarble/