Computer Vision (CSE 455), Winter 2009

Project 2:  Feature Detection and Panoramic Mosaic Stitching

Assigned:  Thursday, January 29, 2009


********* NOTE: NEW DUE DATE/TIME:  Monday, February 16, 2009 (by 11:59pm)


Follow the instructions in Project Turnin Procedure to turn in all of your files (source code, binaries,…) as well as a Web Page showing your results (details below)

***Reminder:  Sign-up for Panorama kit here.  

Overall Synopsis

In this project, you will implement a system to combine a series of photographs into a 360° panorama (see example below). Your software will detect discriminating features in the images, find the best matching features in the other images, automatically align the photographs (determine their overlap and relative positions) and then blend the resulting photos into a single seamless panorama. You will then be able to view the resulting panorama inside an interactive Web viewer. To start your project, you will be supplied with some test images and skeleton code you can use as the basis of your project along with instructions on how to use the viewer.

Groups

Because this project is more extensive, you are encouraged to work in groups of two.  Use this link to form/register your group on the grouper tool. If you are unable to form your own group of two or wish to work alone, register as a group of one and let the instructor know if you would like to be paired up. For those who want to be paired up, we will use a heuristic such as nearest neighbor in last-name space to pair people up. Groups should ideally be finalized by Friday January 30.


Panorama by Loren Meritt

This project can be thought of as two major components.  Each has its own web page with detailed information:

  1. Feature Detection and Matching
  2. Panoramic Mosaic Stitching


The project will consist of a pipeline of command line EXE programs (Feature.exe and Panorama.exe) that will operate on images or intermediate results to produce the final panorama output.  A complete description of each component can be found on the dedicated component web page.

The steps required to complete the panorama are listed below:

 

Step

EXE

1.

Take pictures on a tripod (or handheld)

 

2.

Warp to spherical coordinates

(Panorama.exe)

3.

Extract features

(Features.exe)

4.

Match features

(Features.exe)

5.

Align neighboring pairs using RANSAC

(Panorama.exe)

6.

Write out list of neighboring translations

(Panorama.exe)

7.

Correct for drift

(Panorama.exe)

8.

Read in warped images and blend them

(Panorama.exe)

9.

Crop the result and import into a viewer

 


Although this is the required order, you can complete Features.exe and Panorama.exe in parallel and test on other images.

Getting Things to Run

Running the sample solution

Features.exe and Panorama.exe are command line programs that require arguments to work properly. Thus, as explained in more detail below, you need to run it from the command line, or from a shortcut to the executable that has the arguments specified in the “Target” field of the shortcut properties.  The one exception to this is that running Features.exe without command line arguments opens up the UI.

Running from the command line

To run from the command line, click the Windows Start button and select "Run". Then enter "cmd" in the "Run" dialog and click "OK". A command window will pop up where you can type DOS commands. Use the DOS "cd" (change directory) command to navigate to the directory where Features.exe or Panorama.exe is located. Then type "Features" or "Panorama" followed by your arguments. If you do not supply any arguments, the program will print out information on what arguments it expects or open the UI in the case of Features.exe.

Running from a shortcut

Another way to pass arguments to a program is to create a shortcut to it. To create a shortcut, right-click on the executable and drag to the location where you wish to place the shortcut. A menu will pop up when you let go of the mouse button. From the menu, select "Create Shortcut Here". Now right-click on the short-cut you've created and select "Properties". In the properties dialog select the "Shortcut" tab and add your arguments after the text in the "Target" field. Your arguments must be outside of the quotation marks and separated with spaces.

Running the skeleton program

You can run the skeleton program from inside Visual Studio. However, you will need to tell Visual Studio what arguments to pass. Here's how:

  1. Select the "ImageLib" project in the Solution Explorer (do NOT select the "Panorama" project).
  2. From the "Project" menu, choose "Properties" to bring up the "Property Pages" dialog.
  3. Select the "Debugging" Property page.
  4. Enter your arguments in the "Command Arguments" field.
  5. Click "Ok".
  6. Now when you execute your program from within Visual Studio, the arguments you entered will be passed to it automatically.
  7. Repeat the above steps for the solution to the Features component.

What to Turn In

In addition to your source code and executables, turn in a web page describing your approach and results.  In particular:

       Feature Detection and Matching

  • You will need to compute two sets of ROC curves and post them on your web page.  For both pairs of test images provided on the Feature Detection webpage, create a plot with two curves, one using the SSD distance and the other using the ratio test distance.  Also, please include the SIFT curves we provide for you on your graphs (these can be found on the Feature Detection webpage).  You can learn more about ROC curves from the class slides and on the web here. Instructions for generating the plots are provided on the Feature Detection webpage under "To do."
  • For one image each in both the Yosemite and graf test pairs, please include an image of the Harris operator on your webpage.  This image is produced by the Features.exe executable every time it is run in computeFeatures mode (it is saved to the file harris.tga). If these images seem too dark, you can multiply the Harris score by a constant, such as 2, to make the scores more visible.
  • Describe any extra credit items you did.

      Panoramic Mosaic Stitching

      This portion of the web page should contain the following:

  • At least three panoramas:  (1) the test sequence, (2), one from the Kaidan head, and (3) one from a hand-held sequence.  Each panorama should be shown as (1) a low-res inlined image on the web page, (2) a link that you can click on to show the full-resolution .jpg file, AND (3) embedded in a viewer as described in the instructions above.
  • A short description of what worked well and what didn’t.  If you tried several variants or did something non-standard, please describe this as well.
  • Describe any extra credit items you did.

The web-page should be placed in the project2/artifact directory along with all the images in JPEG format. If you are unfamiliar with HTML you can use any web-page editor such as FrontPage, Word, or Visual Studio 7.0 to make your web-page.  Here are some webpage design tips.

 

Last modified on 1/24/2009 6:35:53 PM