Computer Vision (CSE 490CV/EE 400B), Winter 2002
Project 2: Panoramic
Mosaic Stitching
Assigned: Friday, Feb 8, 2002
Due: Thursday, Feb 26, 2002 (by 11:59pm)
In this project, you will implement a system to combine a
series of photographs into a 360°
panorama. Your software will
automatically align the photographs (determine their overlap and relative
positions) and then blend the resulting photos into a single seamless
panorama. You will then be able to
view the resulting panorama inside an interactive Web viewer. To start your project, you will be
supplied with some test
images, skeleton
code you can use as the basis of your project, and instructions on how to
use the viewer.
Here are the suggested steps you should follow (skip steps 1
- 3 for test data):
- Take a series of photos
with a digital camera mounted on a tripod. You should sign
up to borrow the Kaidan head that lets you make precise rotations and the
Canon PowerShot A10 Camera for this purpose. For best results, overlap each image by 50%
with the previous one, and keep the camera level using the levelers on the
Kaidan head.
- Also take a series of images with a handheld camera. You can use
your own or sign up to borrow the Canon PowerShot A10 camera from Cris Mesling (mesling@cs.washington.edu). If
you are using the Canon camera, it has a “stitch assist” mode you can use to
overlap your images correctly, which only works in regular landscape mode.
- Make sure the images are right side up (rotate the images by 90°
if you took them in landscape mode), and downsample them to a more workable
size (around 400x500--50% size for the course cameras). You can use external
software such as PhotoShop or the Microsoft Photo Editor to do this. If you are using the skeleton
software, save your images in (TrueVision) Targa format (.tga), since this is
the only format the skeleton software can currently read.
- Warp each image into cylindrical coordinates. The easiest way to do this is to flesh
out the skeleton code in WarpCylindrical.cpp by adding the necessary
coordinate transformation equations (from the lecture notes). Warp each image using the
warpCylindrical routine, using the following focal length f
estimates for the half-resolution images (you can either take pictures and
same them in small files or save them in large files and downsample them
afterwards) . If you use other image size, do remember to scale f with
image size):
Camera |
resolution |
focal length |
k1 |
k2 |
Canon Powershot A1, tag CS30012716 |
480x640 |
??? pixels |
??? |
??? |
|
|
|
|
|
|
|
|
|
|
- Compute the alignment of the images in pairs. To do this, you will have to implement
a hierarchical (coarse-to-fine) Lucas-Kanade style translational motion
estimation. The skeleton for this
code is provided in LucasKanade.cpp. You will have to fill in the missing
code in LucasKanade to:
- compute the
per-pixel error and intensity gradients
- accumulate the 2x2
matrix and 2x1 vector
- solve the 2x2
system and update the translation estimate
The control routine
that uses the image pyramid to search for the best displacement at a coarse
level, and then refines these estimates using Lucas-Kanade has already been
written for you. Once you have
computed each pair-wise translation (including the translation between the
last image and the first one), write these numbers out to a file.
- Stitch the resulting aligned images.
Read in a descriptor file containing the warped image names and their
relative displacements. Figure
out how large the final stitched image will be. Then, resample each image to its final
location and blend it with its neighbors. Try a simple horizontal “hat” function
as your weighting function (this is a simple 1-D version of the distance map
described in [Szeliski &
Shum]).
For extra credit, you can try other blending functions or figure out
some way to compensate for exposure differences.
- Crop the resulting image to make the left and right edges seam
perfectly. The horizontal extent
can be computed in the previous blending routine since the first image occurs
at both the left and right end of the stitched sequence (draw the “cut” line
halfway through this image). Use
an affine warp to remove any vertical “drift” between the first and last
image. Optionally, remove any black (a
= 0) pixels at the top and bottom edges.
- Convert your resulting image to a JPEG and paste it on a Web page along
with code to run the interactive viewer. For
the instruction on how to do this,
click here.
Turn in the code that you wrote (just the .cpp files you
modified and any new files you needed), and a web page containing the
following:
-
At least three panoramas: (1) the test sequence,
(2), one from the Kaidan head, and (3) one from a hand-held sequence.
Each panorama should be shown as (1) a low-res inlined image on the web
page, (2) a link that you can click on to show the full-resolution .jpg
file, AND (3) embedded in a viewer as described above.
-
a short description of what worked well and what
didn’t. If you tried several
variants or did something non-standard, please describe this as well.
Extra credit:
You can try some of the following for extra credit:
- Try
shooting a sequence with some people moving in it. What did you do to remove “ghosted”
versions of the people?
- Try
stitching something other than a cylindrical panorama, e.g., seam together two
scans from a flatbed scanner.
Warning: this is a fair
amount of work, since your alignment code now needs to estimate rotations or
even more.
- Detect and remove
moving objects from your panorama, filling in the background behind them.
To download the skeleton code and start working on it,
click here.