Notes
Slide Show
Outline
1
Announcements
  • Project 1 artifact voting
  • Project 2 out today (help session at end of class)
2
Mosaics
  • Today’s Readings
    • Szeliski and Shum paper (sections 1 and 2, skim the rest)
      • http://www.acm.org/pubs/citations/proceedings/graph/258734/p251-szeliski/


3
Image Mosaics



  •    +        +   …   + =
4
How to do it?
  • Basic Procedure
    • Take a sequence of images from the same position
      • Rotate the camera about its optical center
    • Compute transformation between second image and first
      • Lucas & Kanade registration
    • Shift the second image to overlap with the first
    • Blend the two together to create a mosaic
    • If there are more images, repeat


5
Aligning images
  • How to account for warping?
    • Translations are not enough to align the images
    • Photoshop demo
6
Image reprojection
  • The mosaic has a natural interpretation in 3D
    • The images are reprojected onto a common plane
    • The mosaic is formed on this plane
7
Image reprojection
  • Basic question
    • How to relate two images from the same camera center?
      • how to map a pixel from PP1 to PP2
8
Image reprojection
  • Observation
    • Rather than thinking of this as a 3D reprojection, think of it as a 2D image warp from one image to another
9
Homographies
  • Perspective projection of a plane
    • Lots of names for this:
      • homography, texture-map, colineation, planar projective map
    • Modeled as a 2D warp using homogeneous coordinates
10
Image warping with homographies


11
Panoramas
  • What if you want a 360° field of view?
12
Cylindrical projection
    • Map 3D point (X,Y,Z) onto cylinder
13
Cylindrical reprojection
  • How to map from a cylinder to a planar image?
14
Cylindrical reprojection
  • Map image to cylindrical coordinates
    • need to know the focal length
15
Cylindrical panoramas
  • Steps
    • Reproject each image onto a cylinder
    • Blend
    • Output the resulting mosaic
16
Cylindrical image stitching
  • What if you don’t know the camera rotation?
    • Solve for the camera rotations
      • Note that a rotation of the camera is a translation of the cylinder!
      • Use Lukas-Kanade to solve for translations of cylindrically-warped images
17
Full-view Panorama
18
Different projections are possible
19
Project 2 (out today)
  • Take pictures on a tripod (or handheld)
  • Warp to cylindrical coordinates
  • Automatically compute pair-wise alignments
  • Correct for drift
  • Blend the images together
  • Crop the result and import into a viewer
20
Image Blending
21
Feathering
22
Effect of window size
23
Effect of window size
24
Good window size
25
Pyramid blending
26
Image warping
  • Given a coordinate transform (x’,y’) = h(x,y) and a source image f(x,y), how do we compute a transformed image g(x’,y’) = f(h(x,y))?
27
Forward warping
  • Send each pixel f(x,y) to its corresponding location
  •            (x’,y’) = h(x,y) in the second image
28
Forward warping
  • Send each pixel f(x,y) to its corresponding location
  •            (x’,y’) = h(x,y) in the second image
29
Inverse warping
  • Get each pixel g(x’,y’) from its corresponding location
  •            (x,y) = h-1(x’,y’) in the first image
30
Inverse warping
  • Get each pixel g(x’,y’) from its corresponding location
  •            (x,y) = h-1(x’,y’) in the first image
31
Forward vs. inverse warping
  • Q:  which is better?


  • A:  usually inverse—eliminates holes
    • however, it requires an invertible warp function—not always possible...
32
Other types of mosaics
  • Can mosaic onto any surface if you know the geometry
    • See NASA’s Visible Earth project for some stunning earth mosaics
      • http://earthobservatory.nasa.gov/Newsroom/BlueMarble/
33
Summary
  • Things to take home from this lecture
    • Image alignment
    • Image reprojection
      • homographies
      • cylindrical projection
    • Radial distortion
    • Creating cylindrical panoramas
    • Image blending
    • Image warping
      • forward warping
      • inverse warping
      • bilinear interpolation