Computer Vision (CSE 490CV/EE 400B), Winter 2002

Project 2:  Panoramic Mosaic Stitching

Notes on using the skeleton software

To help you with your first project, we provide a simple C++ imaging library, and much of the code needed to perform the warping, alignment, and stitching. All those codes are written by Richard Szeliski. Click here to download the latest skeleton code as a zip file and click here to download the sample images. 

The code provides the following basic functionality, inside the library called ImageLib:

Image.h            main CByteImage, CFloatImage, … classes

ImageLib.h            main include file

FileIO.h            read / write Targa (TGA) files

Convert.h            convert between image types and select bands

Convolve.h            perform 2D and separable convolutions

Pyramid.h            image pyramids

Transform.h            3x3 coordinate transformation matrices

WarpImage.h            per-pixel and global (parametric) image warping

Images are pretty simple:  they have a type (byte, int, or float), a width and a height, and a number of bands (RGBA color images are 4-banded, and alpha is stored in band 3).

You can allocate images on the stack and pass them around as parameters and return values.  The storage in an image is reference counted and shared during construction and assignment.  You should never need to use new and delete with the image classes.

The code is now stand-alone. However, if you are still having trouble compiling, please send email to the class mailing list (cse590ss@cs.washington.edu). While we expect most students will find the supplied code helpful, you are welcome to write it from scratch yourself and use other packages if you clear it by us first.

How to extend the sample code for your project

In the v4gP1 project, you will find 3 files to which you will have to add code:

WarpCylindrical.cpp Compute the inverse map coordinate to warp your images. Your job is, given a cylindrical image coordinate, solve the correponding planar image coordinate. 

You will find 1 "TODO" in this file. 
LucasKanade.cpp Iterative coarse-to-fine alignment with Lucas-Kanade. Your job is, 
  1. compute the per-pixel error and intensity gradients
  2. accumulate the 2x2 matrix and 2x1 vector
  3. solve the 2x2 system and update the translation estimate

You will find 2 "TODO" in this file.
BlendImages.cpp Blend the aligned images and crop/trim the result. Your job is,
  1. Find the size of the bounding box of the mosaic image
  2. Paste each images to the mosaic image at the right position
  3. Trim the mosaic result

Yuu will find 5 "TODO" in this file.

The places where you need to add code are marked by “TODO”.  If you figure out the right things to add, it shouldn’t be more than a few dozen lines (at most) to each file.

How to debug and use your extension

There is also one command-line dispatch file

Project1.cpp

that lets you call all of the necessary routines from a shell, passing images and parameter values as inputs/outputs.  Each processing function (warping, alignment, etc) is invoked by specifying a different “command” (cylWarp, blendPairs) as the second argument of Project1 (see the .cpp file for documentation, click here for an example of command line format). When debugging, you can run cylWarp, LucasKanade, and blendPairs seperately, and watch intermediate results, e.g., cylindrical images, motion vectors and final mosaic. You can easily locate your bugs because each step involves only a couple of lines of codes. You can also write a script with a series of commands in it (so you don’t have to remember the parameters) and run with with the script command.  A sample script to do a two-image alignment is provided in stitch2.txt.

There are also some TGA formatted images (the first 4 from the sequence) in the images subdirectory of the sample code project, as well as the result of stitching these 4 images together (stitch4x.tga).

Tips

  1. When you convert the test sequence (.jpg) files to .tga files, you want to use 24bit tga format instead of 32bit. This is very important for Lucas-Kanade to work.
  2. You need to rotate and shrink each image in test sequence to the size of 384x512 (the same as sample images) to build the panorama. You DON'T want to use other image resolution, because the focal length in the stitch.txt files are for this resolution only! If you really want to use another image resolution, you need to keep the two resolution the same aspect ratio and scale the focal length accordingly. For example, if you use resolution 768*1024, you want to double the focal length.
  3. For Lucas-Kanade part, you want to try different parameter settings in command line to get a best alignment. One set of parameters I recommend is:

LucasKanade warp08.tga warp09.tga -200 0 3