Computer Vision (CSE 455), Winter 2008

Project 2:  Autostitch

Assigned:  Thursday, January 24, 2008
Due:  Friday, Februrary 8, 2008 (by 11:59pm)
Artifact Due:  Friday, Februrary 8, 2008 (by 11:59pm)

Project Head TA: Ryan Kaminsky (send your questions here first!)
Project Secondary TA: Noah Snavely

***Reminder:  Sign-up for Panorama kit here.  

Overall Synopsis

In this project, you will implement a system to combine a series of photographs into a 360° panorama (see panorama below). Your software will detect discriminating features in the images, find the best matching features in the other images, automatically align the photographs (determine their overlap and relative positions) and then blend the resulting photos into a single seamless panorama. You will then be able to view the resulting panorama inside an interactive Web viewer. To start your project, you will be supplied with some test images and skeleton code you can use as the basis of your project and instructions on how to use the viewer.

Because this project is more extensive you can work in groups of two.  Use this link to register your group on the grouper tool.

Panorama by Loren Meritt

This project can be thought of as two major components.  Each has its own web page with detailed information:

1. Feature Detection and Matching
2. Panorama Mosaic Stitching

The project will consist of a pipeline of command line EXE programs (Feature.exe and Panorama.exe) that will operate on images or intermediate results to produce the final panorama output.  A complete description of each component can be found on the dedicated component web page.

The steps required to complete the panorama are listed below:

	Step	EXE
1.	Take pictures on a tripod (or handheld)
2.	Warp to spherical coordinates	(Panorama.exe)
3.	Extract features	(Features.exe)
4.	Match features	(Features.exe)
5.	Align neighboring pairs using RANSAC	(Panorama.exe)
6.	Write out list of neighboring translations	(Panorama.exe)
7.	Correct for drift	(Panorama.exe)
8.	Read in warped images and blend them	(Panorama.exe)
9.	Crop the result and import into a viewer

Although this is the required order, you can complete Features.exe and Panorama.exe in parallel and test on other images.

Getting Things to Run

Running the sample solution

Features.exe and Panorama.exe are command line programs that requires arguments to work properly. Thus you need to run it from the command line, or from a shortcut to the executable that has the arguments specified in the "Target" field of the shortcut properties.  The one exception to this is that running Features.exe without command line arguments opens up the UI.

Running from the command line

To run from the command line, click the Windows Start button and select "Run". Then enter "cmd" in the "Run" dialog and click "OK". A command window will pop up where you can type DOS commands. Use the DOS "cd" (change directory) command to navigate to the directory where Features.exe or Panorama.exe is located. Then type "Features" or "Panorama" followed by your arguments. If you do not supply any arguments, the program will print out information on what arguments it expects or open the UI in the case of Features.exe.

Running from a shortcut

Another way to pass arguments to a program is to create a shortcut to it. To create a shortcut, right-click on the executable and drag to the location where you wish to place the shortcut. A menu will pop up when you let go of the mouse button. From the menu, select "Create Shortcut Here". Now right-click on the short-cut you've created and select "Properties". In the properties dialog select the "Shortcut" tab and add your arguments after the text in the "Target" field. Your arguments must be outside of the quotation marks and separated with spaces.

Running the skeleton program

You can run the skeleton program from inside Visual Studio. However, you will need to tell Visual Studio what arguments to pass. Here's how:

1. Select the "ImageLib" project in the Solution Explorer (do NOT select the "Panorama" project, for some reason this won't work).
2. From the "Project" menu choose "Properties" to bring up the "Property Pages" dialog.
3. Select the "Debugging" Property page.
4. Enter your arguments in the "Command Arguments" field.
5. Click "Ok".
6. Now when you execute your program from within Visual Studio the arguments you entered will be passed to it automatically.
7. Repeat the above steps for the solution for the Features component.
   

What to Turn In

In addition to your source code and executables, turn in a web page describing your approach and results.  In particular:

       Feature Detection and Matching

• You will need to compute two sets of ROC curves and post them on your web page.  For both pairs of test images provided on the Features project webpage, create a plot with two curves, one using the SSD distance and the other using the ratio test distance.  Also, please include the SIFT curves we provide for you on your graphs (these can be found on the Features project webpage).  You can learn more about ROC curves from the class slides and on the web here and here. Instructions for generating the plots are provided on the Features project webpage under "To do."
• For one image each in both the Yosemite and graf test pairs, please include an image of the Harris operator on your webpage.  This image is produced by the Features.exe executable every time it is run in computeFeatures mode (it is saved to the file harris.tga). If these images seem too dark, you can multiply the Harris score by a constant, such as 2, to make the scores more visible.
• Describe any extra credit items that you did (if applicable)

      Panorama Mosaic Stitching

      This portion of the web page should containin the following:

• At least three panoramas:  (1) the test sequence, (2), one from the Kaidan head, and (3) one from a hand-held sequence.  Each panorama should be shown as (1) a low-res inlined image on the web page, (2) a link that you can click on to show the full-resolution .jpg file, AND (3) embedded in a viewer as described above.
• A short description of what worked well and what didn’t.  If you tried several variants or did something non-standard, please describe this as well.
• Describe any extra credit.
  

The web-page should be placed in the project2/artifact directory along with all the images in JPEG format. If you are unfamiliar with HTML you can use any web-page editor such as FrontPage, Word, or Visual Studio 7.0 to make your web-page.  The KompoZer HTML editor is easy to use and highly recommended.  Here are some webpage design tips.

 

---

Last modified on Jan 21, 2008