|
|
|
Midterm due Friday, beginning of lecture |
|
Guest lecture on Friday: Antonio Criminisi, Microsoft Research |
|
|
|
|
|
|
Today’s Readings |
|
Szeliski and Shum paper (sections 1 and 2, skim
the rest) |
|
http://www.acm.org/pubs/citations/proceedings/graph/258734/p251-szeliski/ |
|
|
|
|
|
|
|
|
|
Basic Procedure |
|
Take a sequence of images from the same position |
|
Rotate the camera about its optical center |
|
Compute transformation between second image and
first |
|
Lucas & Kanade registration |
|
Shift the second image to overlap with the first |
|
Blend the two together to create a mosaic |
|
If there are more images, repeat |
|
|
|
|
|
|
|
How to account for warping? |
|
Translations are not enough to align the images |
|
Photoshop demo |
|
|
|
|
|
The mosaic has a natural interpretation in 3D |
|
The images are reprojected onto a common plane |
|
The mosaic is formed on this plane |
|
|
|
|
|
|
Basic question |
|
How to relate two images from the same camera
center? |
|
how to map a pixel from PP1 to PP2 |
|
|
|
|
|
Observation |
|
Rather than thinking of this as a 3D
reprojection, think of it as a 2D image warp from one image to another |
|
|
|
|
|
|
Perspective projection of a plane |
|
Lots of names for this: |
|
homography, texture-map, colineation, planar
projective map |
|
Modeled as a 2D warp using homogeneous
coordinates |
|
|
|
|
|
What if you want a 360° field of view? |
|
|
|
|
|
Map 3D point (X,Y,Z) onto cylinder |
|
|
|
|
How to map from a cylinder to a planar image? |
|
|
|
|
|
Radial distortion of the image |
|
Caused by imperfect lenses |
|
Deviations are most noticeable for rays that
pass through the edge of the lens |
|
|
|
|
|
|
To model lens distortion |
|
Use above projection operation instead of
standard projection matrix multiplication |
|
|
|
|
|
Map image to cylindrical coordinates |
|
need to know the focal length |
|
|
|
|
|
Steps |
|
Reproject each image onto a cylinder |
|
Blend |
|
Output the resulting mosaic |
|
|
|
|
|
|
What if you don’t know the camera rotation? |
|
Solve for the camera rotations |
|
Note that a rotation of the camera is a translation
of the cylinder! |
|
Use Lukas-Kanade to solve for translations of
cylindrically-warped images |
|
|
|
|
|
|
Take pictures on a tripod (or handheld) |
|
Warp to cylindrical coordinates |
|
Automatically compute pair-wise alignments |
|
Correct for drift |
|
Blend the images together |
|
Crop the result and import into a viewer |
|
|
|
|
|
|
|
|
|
|
Given a coordinate transform (x’,y’) = h(x,y)
and a source image f(x,y), how do we compute a transformed image g(x’,y’) =
f(h(x,y))? |
|
|
|
|
Send each pixel f(x,y) to its corresponding
location |
|
(x’,y’) = h(x,y) in the second image |
|
|
|
|
Send each pixel f(x,y) to its corresponding
location |
|
(x’,y’) = h(x,y) in the second image |
|
|
|
|
Get each pixel g(x’,y’) from its corresponding
location |
|
(x,y) = h-1(x’,y’) in the first image |
|
|
|
|
Get each pixel g(x’,y’) from its corresponding
location |
|
(x,y) = h-1(x’,y’) in the first image |
|
|
|
|
A common method for resampling images |
|
|
|
|
|
Q: which
is better? |
|
|
|
A:
usually inverse—eliminates holes |
|
however, it requires an invertible warp
function—not always possible... |
|
|
|
|
|
|
Can mosaic onto any surface if you know the
geometry |
|
See NASA’s Visible Earth project for some
stunning earth mosaics |
|
http://earthobservatory.nasa.gov/Newsroom/BlueMarble/ |
|
|
|
|
|
|
Things to take home from this lecture |
|
Image alignment |
|
Image reprojection |
|
homographies |
|
cylindrical projection |
|
Radial distortion |
|
Creating cylindrical panoramas |
|
Image blending |
|
Image warping |
|
forward warping |
|
inverse warping |
|
bilinear interpolation |
|