1	Announcements Project 2 out today panorama signup help session at end of class Today mosaic recap blending
2	Mosaics
3	Image Mosaics + + … + =
4	How to do it? Basic Procedure Take a sequence of images from the same position Rotate the camera about its optical center Compute transformation between second image and first Shift the second image to overlap with the first Blend the two together to create a mosaic If there are more images, repeat
5	Image reprojection The mosaic has a natural interpretation in 3D The images are reprojected onto a common plane The mosaic is formed on this plane
6	Image reprojection Basic question How to relate two images from the same camera center? how to map a pixel from PP1 to PP2
7	Image reprojection Observation Rather than thinking of this as a 3D reprojection, think of it as a 2D image warp from one image to another
8	Homographies Perspective projection of a plane Lots of names for this: homography, texture-map, colineation, planar projective map Modeled as a 2D warp using homogeneous coordinates
9	Image warping with homographies
10	Panoramas What if you want a 360° field of view? Idea is the same as before, but the warp is a little more complicated (see Rick’s slides) key property: once you warp two images onto a cylinder or sphere, you can align them with a simple translation assuming camera pan
11	Assembling the panorama Stitch pairs together, blend, then crop
12	Problem: Drift Error accumulation small errors accumulate over time
13	Problem: Drift Solution add another copy of first image at the end this gives a constraint: y_n = y₁ there are a bunch of ways to solve this problem add displacement of (y₁ – y_n)/(n -1) to each image after the first compute a global warp: y’ = y + ax run a big optimization problem, incorporating this constraint best solution, but more complicated known as “bundle adjustment” (we’ll cover next week, proj 3)
14	Image warping Given a coordinate transform (x’,y’) = h(x,y) and a source image f(x,y), how do we compute a transformed image g(x’,y’) = f(h(x,y))?
15	Forward warping Send each pixel f(x,y) to its corresponding location (x’,y’) = h(x,y) in the second image
16	Forward warping Send each pixel f(x,y) to its corresponding location (x’,y’) = h(x,y) in the second image
17	Inverse warping Get each pixel g(x’,y’) from its corresponding location (x,y) = h^-1(x’,y’) in the first image
18	Inverse warping Get each pixel g(x’,y’) from its corresponding location (x,y) = h^-1(x’,y’) in the first image
19	Image resampling How can we estimate image colors off the grid?
20	Resampling filters What does the 2D version of this hat function look like?
21	Bilinear interpolation A simple method for resampling images
22	Forward vs. inverse warping Q: which is better? A: usually inverse—eliminates holes however, it requires an invertible warp function—not always possible...
23	Image Blending
24	Feathering
25	Effect of window size
26	Effect of window size
27	Good window size
28	Alpha Blending
29	Pyramid blending
30	Poisson Image Editing Blend the gradients of the two images, then integrate For more info: Perez et al, SIGGRAPH 2003 http://research.microsoft.com/vision/cambridge/papers/perez_siggraph03.pdf
31	Graph cut blending
32	Deghosting using optical flow Compensate for misalignments, parallax, lens distortion compute residual pixel-wise warp needed for perfect alignment (optical flow—will cover this later in the course) apply that warp to register the images (deghosting) covered in Szesliski & Shum reading
33	Project 2 (out today) Take pictures on a tripod (or handheld) Warp to spherical coordinates Automatically compute pair-wise alignments Correct for drift Blend the images together Crop the result and import into a viewer
34	AutoStitch Method so far is not completely automatic need to know which pairs fit together Newer methods are fully automatic AutoStitch, by Matthew Brown and David Lowe: http://www.cs.ubc.ca/~mbrown/autostitch/autostitch.html Microsoft Picture It! Digital Image Pro 10.0
35	Other types of mosaics Can mosaic onto any surface if you know the geometry See NASA’s Visible Earth project for some stunning earth mosaics http://earthobservatory.nasa.gov/Newsroom/BlueMarble/
36	Slit images y-t slices of the video volume are known as slit images take a single column of pixels from each input image
37	Slit images: cyclographs
38	Slit images: photofinish