| Project 1 artifact voting | ||
| Project 2 out today | ||
| panorama signup | ||
| help session at end of class | ||
| Guest lectures next week: Li Zhang, Jiwon Kim | ||
| Today’s Readings | |||
| Szeliski and Shum paper (sections 1 and 2, skim the rest) | |||
| http://www.acm.org/pubs/citations/proceedings/graph/258734/p251-szeliski/ | |||
+ + … + = |
| Basic Procedure | |||
| Take a sequence of images from the same position | |||
| Rotate the camera about its optical center | |||
| Compute transformation between second image and first | |||
| Lucas & Kanade registration | |||
| Shift the second image to overlap with the first | |||
| Blend the two together to create a mosaic | |||
| If there are more images, repeat | |||
| How to account for warping? | ||
| Translations are not enough to align the images | ||
| Photoshop demo | ||
| The mosaic has a natural interpretation in 3D | ||
| The images are reprojected onto a common plane | ||
| The mosaic is formed on this plane | ||
| Basic question | |||
| How to relate two images from the same camera center? | |||
| how to map a pixel from PP1 to PP2 | |||
| Observation | ||
| Rather than thinking of this as a 3D reprojection, think of it as a 2D image warp from one image to another | ||
| Perspective projection of a plane | |||
| Lots of names for this: | |||
| homography, texture-map, colineation, planar projective map | |||
| Modeled as a 2D warp using homogeneous coordinates | |||
Image warping with homographies
| What if you want a 360° field of view? |
| Map 3D point (X,Y,Z) onto cylinder | ||
| How to map from a cylinder to a planar image? |
| Map image to cylindrical coordinates | ||
| need to know the focal length | ||
| Steps | ||
| Reproject each image onto a cylinder | ||
| Blend | ||
| Output the resulting mosaic | ||
| What if you don’t know the camera rotation? | |||
| Solve for the camera rotations | |||
| Note that a pan (rotation) of the camera is a translation of the cylinder! | |||
| Use Lukas-Kanade to solve for translations of cylindrically-warped images | |||
| Stitch pairs together, blend, then crop |
| Error accumulation | ||
| small errors accumulate over time | ||
| Solution | ||||
| add another copy of first image at the end | ||||
| this gives a constraint: yn = y1 | ||||
| there are a bunch of ways to solve this problem | ||||
| add displacement of (y1 – yn)/(n -1) to each image after the first | ||||
| compute a global warp: y’ = y + ax | ||||
| run a big optimization problem, incorporating this constraint | ||||
| best solution, but more complicated | ||||
| known as “bundle adjustment” | ||||
Different projections are possible
| Take pictures on a tripod (or handheld) | |
| Warp to cylindrical coordinates | |
| Automatically compute pair-wise alignments | |
| Correct for drift | |
| Blend the images together | |
| Crop the result and import into a viewer |
| For more info: Perez et al, SIGGRAPH 2003 | ||
| http://research.microsoft.com/vision/cambridge/papers/perez_siggraph03.pdf | ||
| Given a coordinate transform (x’,y’) = h(x,y) and a source image f(x,y), how do we compute a transformed image g(x’,y’) = f(h(x,y))? |
| Send each pixel f(x,y) to its corresponding location | |
| (x’,y’) = h(x,y) in the second image |
| Send each pixel f(x,y) to its corresponding location | |
| (x’,y’) = h(x,y) in the second image |
| Get each pixel g(x’,y’) from its corresponding location | |
| (x,y) = h-1(x’,y’) in the first image |
| Get each pixel g(x’,y’) from its corresponding location | |
| (x,y) = h-1(x’,y’) in the first image |
| Q: which is better? | ||
| A: usually inverse—eliminates holes | ||
| however, it requires an invertible warp function—not always possible... | ||
| Can mosaic onto any surface if you know the geometry | |||
| See NASA’s Visible Earth project for some stunning earth mosaics | |||
| http://earthobservatory.nasa.gov/Newsroom/BlueMarble/ | |||
| Method so far is not completely automatic | |||
| need to know which pairs fit together | |||
| need to initialize Lukas-Kanade to get good results | |||
| Newer methods are fully automatic | |||
| AutoStitch, by Matthew Brown and David Lowe: | |||
| http://www.cs.ubc.ca/~mbrown/autostitch/autostitch.html | |||
| Based on feature matching techniques (next lecture) | |||