Announcements
|
|
|
Midterm due Friday, beginning of
lecture |
|
Guest lecture on Friday: Antonio Criminisi, Microsoft Research |
Mosaics
|
|
|
|
|
Today’s Readings |
|
Szeliski and Shum paper (sections 1 and
2, skim the rest) |
|
http://www.acm.org/pubs/citations/proceedings/graph/258734/p251-szeliski/ |
|
|
Image Mosaics
How to do it?
|
|
|
|
|
Basic Procedure |
|
Take a sequence of images from the same
position |
|
Rotate the camera about its optical
center |
|
Compute transformation between second
image and first |
|
Lucas & Kanade registration |
|
Shift the second image to overlap with
the first |
|
Blend the two together to create a
mosaic |
|
If there are more images, repeat |
|
|
Aligning images
|
|
|
|
How to account for warping? |
|
Translations are not enough to align
the images |
|
Photoshop demo |
Image reprojection
|
|
|
|
The mosaic has a natural interpretation
in 3D |
|
The images are reprojected onto a
common plane |
|
The mosaic is formed on this plane |
Image reprojection
|
|
|
|
|
Basic question |
|
How to relate two images from the same
camera center? |
|
how to map a pixel from PP1 to PP2 |
Image reprojection
|
|
|
|
Observation |
|
Rather than thinking of this as a 3D
reprojection, think of it as a 2D image warp from one image to another |
Homographies
|
|
|
|
|
Perspective projection of a plane |
|
Lots of names for this: |
|
homography, texture-map, colineation,
planar projective map |
|
Modeled as a 2D warp using homogeneous
coordinates |
Image warping with
homographies
Panoramas
|
|
|
What if you want a 360° field of view? |
Cylindrical projection
|
|
|
|
Map 3D point (X,Y,Z) onto cylinder |
Cylindrical reprojection
|
|
|
How to map from a cylinder to a planar
image? |
Distortion
|
|
|
|
Radial distortion of the image |
|
Caused by imperfect lenses |
|
Deviations are most noticeable for rays
that pass through the edge of the lens |
Distortion
Modeling distortion
|
|
|
|
To model lens distortion |
|
Use above projection operation instead
of standard projection matrix multiplication |
Cylindrical reprojection
|
|
|
|
Map image to cylindrical coordinates |
|
need to know the focal length |
Cylindrical panoramas
|
|
|
|
Steps |
|
Reproject each image onto a cylinder |
|
Blend |
|
Output the resulting mosaic |
Cylindrical image
stitching
|
|
|
|
|
What if you don’t know the camera
rotation? |
|
Solve for the camera rotations |
|
Note that a rotation of the camera is a
translation of the cylinder! |
|
Use Lukas-Kanade to solve for
translations of cylindrically-warped images |
Full-view Panorama
Different projections are
possible
Project 2 (out on Friday)
|
|
|
Take pictures on a tripod (or handheld) |
|
Warp to cylindrical coordinates |
|
Automatically compute pair-wise
alignments |
|
Correct for drift |
|
Blend the images together |
|
Crop the result and import into a
viewer |
Image Blending
Feathering
Effect of window size
Effect of window size
Good window size
Pyramid blending
Image warping
|
|
|
Given a coordinate transform (x’,y’) = h(x,y)
and a source image f(x,y), how do we compute a transformed image g(x’,y’) = f(h(x,y))? |
Forward warping
|
|
|
Send each pixel f(x,y) to its
corresponding location |
|
(x’,y’) = h(x,y) in the second image |
Forward warping
|
|
|
Send each pixel f(x,y) to its
corresponding location |
|
(x’,y’) = h(x,y) in the second image |
Inverse warping
|
|
|
Get each pixel g(x’,y’) from its
corresponding location |
|
(x,y) = h-1(x’,y’) in the first image |
Inverse warping
|
|
|
Get each pixel g(x’,y’) from its
corresponding location |
|
(x,y) = h-1(x’,y’) in the first image |
Bilinear interpolation
|
|
|
A common method for resampling images |
Forward vs. inverse
warping
|
|
|
|
Q:
which is better? |
|
|
|
A:
usually inverse—eliminates holes |
|
however, it requires an invertible warp
function—not always possible... |
Other types of mosaics
|
|
|
|
|
Can mosaic onto any surface if you know
the geometry |
|
See NASA’s Visible Earth project for
some stunning earth mosaics |
|
http://earthobservatory.nasa.gov/Newsroom/BlueMarble/ |
Summary
|
|
|
|
|
Things to take home from this lecture |
|
Image alignment |
|
Image reprojection |
|
homographies |
|
cylindrical projection |
|
Radial distortion |
|
Creating cylindrical panoramas |
|
Image blending |
|
Image warping |
|
forward warping |
|
inverse warping |
|
bilinear interpolation |