| Project 2 out today | ||
| panorama signup | ||
| help session at end of class | ||
| Today | ||
| mosaic recap | ||
| blending | ||
+ + … + = |
| Basic Procedure | |||
| Take a sequence of images from the same position | |||
| Rotate the camera about its optical center | |||
| Compute transformation between second image and first | |||
| Shift the second image to overlap with the first | |||
| Blend the two together to create a mosaic | |||
| If there are more images, repeat | |||
| The mosaic has a natural interpretation in 3D | ||
| The images are reprojected onto a common plane | ||
| The mosaic is formed on this plane | ||
| Basic question | |||
| How to relate two images from the same camera center? | |||
| how to map a pixel from PP1 to PP2 | |||
| Observation | ||
| Rather than thinking of this as a 3D reprojection, think of it as a 2D image warp from one image to another | ||
| Perspective projection of a plane | |||
| Lots of names for this: | |||
| homography, texture-map, colineation, planar projective map | |||
| Modeled as a 2D warp using homogeneous coordinates | |||
Image warping with homographies
| What if you want a 360° field of view? | |||
| Idea is the same as before, but the warp is a little more complicated (see Rick’s slides) | |||
| key property: once you warp two images onto a cylinder or sphere, you can align them with a simple translation | |||
| assuming camera pan | |||
| Stitch pairs together, blend, then crop |
| Error accumulation | ||
| small errors accumulate over time | ||
| Solution | ||||
| add another copy of first image at the end | ||||
| this gives a constraint: yn = y1 | ||||
| there are a bunch of ways to solve this problem | ||||
| add displacement of (y1 – yn)/(n -1) to each image after the first | ||||
| compute a global warp: y’ = y + ax | ||||
| run a big optimization problem, incorporating this constraint | ||||
| best solution, but more complicated | ||||
| known as “bundle adjustment” (we’ll cover next week, proj 3) | ||||
| Given a coordinate transform (x’,y’) = h(x,y) and a source image f(x,y), how do we compute a transformed image g(x’,y’) = f(h(x,y))? |
| Send each pixel f(x,y) to its corresponding location | |
| (x’,y’) = h(x,y) in the second image |
| Send each pixel f(x,y) to its corresponding location | |
| (x’,y’) = h(x,y) in the second image |
| Get each pixel g(x’,y’) from its corresponding location | |
| (x,y) = h-1(x’,y’) in the first image |
| Get each pixel g(x’,y’) from its corresponding location | |
| (x,y) = h-1(x’,y’) in the first image |
| How can we estimate image colors off the grid? |
| What does the 2D version of this hat function look like? |
| A simple method for resampling images |
| Q: which is better? | ||
| A: usually inverse—eliminates holes | ||
| however, it requires an invertible warp function—not always possible... | ||
| Blend the gradients of the two images, then integrate | ||
| For more info: Perez et al, SIGGRAPH 2003 | ||
| http://research.microsoft.com/vision/cambridge/papers/perez_siggraph03.pdf | ||
| Compensate for misalignments, parallax, lens distortion | ||
| compute residual pixel-wise warp needed for perfect alignment | ||
| (optical flow—will cover this later in the course) | ||
| apply that warp to register the images (deghosting) | ||
| covered in Szesliski & Shum reading | ||
| Take pictures on a tripod (or handheld) | |
| Warp to spherical coordinates | |
| Automatically compute pair-wise alignments | |
| Correct for drift | |
| Blend the images together | |
| Crop the result and import into a viewer |
| Method so far is not completely automatic | |||
| need to know which pairs fit together | |||
| Newer methods are fully automatic | |||
| AutoStitch, by Matthew Brown and David Lowe: | |||
| http://www.cs.ubc.ca/~mbrown/autostitch/autostitch.html | |||
| Microsoft Picture It! Digital Image Pro 10.0 | |||
| Can mosaic onto any surface if you know the geometry | |||
| See NASA’s Visible Earth project for some stunning earth mosaics | |||
| http://earthobservatory.nasa.gov/Newsroom/BlueMarble/ | |||
| y-t slices of the video volume are known as slit images | ||
| take a single column of pixels from each input image | ||