| Project 1 | ||
| Grading session this afternoon | ||
| Artifacts due Friday (voting TBA) | ||
| Project 2 out (online) | ||
| Signup for panorama kits ASAP (weekend slots go quickly…) | ||
| help session at end of class | ||
| Today’s Readings | |||
| Szeliski and Shum paper (sections 1 and 2, skim the rest) | |||
| http://www.cs.washington.edu/education/courses/455/08wi/readings/szeliskiShum97.pdf | |||
+ + … + = |
| Basic Procedure | |||
| Take a sequence of images from the same position | |||
| Rotate the camera about its optical center | |||
| Compute transformation between second image and first | |||
| Shift the second image to overlap with the first | |||
| Blend the two together to create a mosaic | |||
| If there are more images, repeat | |||
| How to account for warping? | ||
| Translations are not enough to align the images | ||
| Photoshop demo | ||
| The mosaic has a natural interpretation in 3D | ||
| The images are reprojected onto a common plane | ||
| The mosaic is formed on this plane | ||
| Basic question | |||
| How to relate two images from the same camera center? | |||
| how to map a pixel from PP1 to PP2 | |||
| Observation | ||
| Rather than thinking of this as a 3D reprojection, think of it as a 2D image warp from one image to another | ||
| Perspective projection of a plane | |||
| Lots of names for this: | |||
| homography, texture-map, colineation, planar projective map | |||
| Modeled as a 2D warp using homogeneous coordinates | |||
Image warping with homographies
| What if you want a 360° field of view? |
| Map 3D point (X,Y,Z) onto sphere | ||
| Map image to cylindrical coordinates | ||
| need to know the focal length | ||
| What if you don’t know the camera rotation? | |||
| Solve for the camera rotations | |||
| Note that a pan (rotation) of the camera is a translation of the sphere! | |||
| Use feature matching to solve for translations of spherical-warped images | |||
| Stitch pairs together, blend, then crop |
| Error accumulation | ||
| small errors accumulate over time | ||
| Solution | ||||
| add another copy of first image at the end | ||||
| this gives a constraint: yn = y1 | ||||
| there are a bunch of ways to solve this problem | ||||
| add displacement of (y1 – yn)/(n -1) to each image after the first | ||||
| compute a global warp: y’ = y + ax | ||||
| run a big optimization problem, incorporating this constraint | ||||
| best solution, but more complicated | ||||
| known as “bundle adjustment” | ||||
Different projections are possible
| Take pictures on a tripod (or handheld) | ||
| Warp to spherical coordinates | ||
| Extract features | ||
| Align neighboring pairs using RANSAC | ||
| Write out list of neighboring translations | ||
| Correct for drift | ||
| Read in warped images and blend them | ||
| Crop the result and import into a viewer | ||
| Roughly based on Autostitch | ||
| By Matthew Brown and David Lowe | ||
| http://www.cs.ubc.ca/~mbrown/autostitch/autostitch.html | ||
| For more info: Perez et al, SIGGRAPH 2003 | ||
| http://research.microsoft.com/vision/cambridge/papers/perez_siggraph03.pdf | ||
| Given a coordinate transform (x’,y’) = h(x,y) and a source image f(x,y), how do we compute a transformed image g(x’,y’) = f(h(x,y))? |
| Send each pixel f(x,y) to its corresponding location | |
| (x’,y’) = h(x,y) in the second image |
| Send each pixel f(x,y) to its corresponding location | |
| (x’,y’) = h(x,y) in the second image |
| Get each pixel g(x’,y’) from its corresponding location | |
| (x,y) = h-1(x’,y’) in the first image |
| Get each pixel g(x’,y’) from its corresponding location | |
| (x,y) = h-1(x’,y’) in the first image |
| Q: which is better? | ||
| A: usually inverse—eliminates holes | ||
| however, it requires an invertible warp function—not always possible... | ||
| Can mosaic onto any surface if you know the geometry | |||
| See NASA’s Visible Earth project for some stunning earth mosaics | |||
| http://earthobservatory.nasa.gov/Newsroom/BlueMarble/ | |||