Assigned:
Friday, February 19
Due: Monday, March 1 (1:30 PM)
In this part of the project, you will use your feature detection and matching code
from project 3a to combine a series of photographs into a 360° panorama. Your software will automatically
align the photographs (determine their overlap and relative positions) and then
blend the resulting photos into a single seamless panorama. You will then be
able to view the resulting panorama inside an interactive Web viewer.
To start this component, you will be supplied with some test images and skeleton code
that will guide you.
Taking
the Pictures
Code to Write
Creating
the Panorama
Debugging
Extra
Credit
Each group will be checking out a panorama kit (camera, tripod, and Kaidan head). Sign up for a kit here.
Remember to bring extra batteries with you – these cameras drain batteries.
Camera |
resolution |
focal length |
k1 |
k2 |
Canon Powershot A10, tag CS30012716 |
480x640 |
678.21239 pixels |
-0.21001 |
0.26169 |
Canon Powershot A10, tag CS30012717 |
480x640 |
677.50487 pixels |
-0.20406 |
0.23276 |
Canon Powershot A10, tag CS30012718 |
480x640 |
676.48417 pixels |
-0.20845 |
0.25624 |
Canon Powershot A10, tag CS30012927 |
480x640 |
671.16649 pixels |
-0.19270 |
0.14168 |
Canon Powershot A10, tag CS30012928 |
480x640 |
674.82258 pixels |
-0.21528 |
0.30098 |
Canon Powershot A10, tag CS30012929 |
480x640 |
674.79106 pixels |
-0.21483 |
0.32286 |
test images |
384x512 |
595 pixels |
-0.15 |
0.0 |
(Note: If you are using the skeleton software, save your images in (TrueVision) JPEG format (.jpg), since this is the only format the skeleton software can read. Also make sure the aspect ratio of the image (width vs. height) is either 4:3 or 3:4 (480x640 will do) which is the only aspect ratio supported by the skeleton software.)
Warp each image into spherical coordinates.
Compute the inverse map to warp the
image by filling in the skeleton code in the warpSphericalField
routine to:
(Note: You will have to use the focal length f estimates for the
half-resolution images provided above (you can either take pictures and save
them in small files or save them in large files and reduce them afterwards) .
If you use a different image size, do remember to scale f according to
the image size.)
2. (15%) [M]=alignPair(locs1,locs2,match_idx,
nRANSAC, RANSACthresh) (file:
alignPair.m, routine: alignPair)
alignPair takes two feature sets, locs1 and locs2, and match_idx , the list of feature matches obtained from the feature detecting and matching component (described in the first part of the project), and estimates and inter-image transform matrix M.
alignPair uses
RANSAC (RAndom SAmpling
Consensus) to pull out a minimal set of feature matches (one match for this
project), estimates the corresponding motion (alignment) and then invokes countInliers to count how many of the feature
matches agree with the current motion estimate. After repeated trials,
the motion estimate with the largest number of inliers is used to compute a
least squares estimate for the motion leastSquaresFit, which is then returned in the
motion estimate M.
You will have to fill in the missing code in alignPair to:
3. (10%)[count,inliers]=countInliers(locs1,locs2,match_idx,M,
RANSACthresh) (file: countInliers.m,
routine: countInliers)
CountInliers computes the number of matches that have a distance below RANSACthresh is computed. It also returns a list of inlier match ids.
4. [M]=leastSquaresFit(locs1,locs2,match_idx,inliers) (file:
leastSquareFit.m, routine: leastSquareFit)
LeastSquaresFit computes a least squares estimate for the translation using all of the matches previously estimated as inliers. It returns the resulting translation estimate in the last column of M.
5. (15%) BlendImages(ipv, blendWidth,panoImgName) (file: BlendImages.m, routine: BlendImages)
Given the warped images and their relative displacements, figure out how large
the final stitched image will be and their absolute displacements in the
panorama (BlendImages).
It then calls AccumulateBlend and NormalizeBlend to resample each image to its final location and blend it with its neighbors.
Crop the resulting image to make the left and right edges seam perfectly (BlendImages). The horizontal extent can be computed in the previous blending routine since the first image occurs at both the left and right end of the stitched sequence (draw the “cut” line halfway through this image). Use a linear warp to the mosaic to remove any vertical “drift” between the first and last image. This warp, of the form y' = y + ax, should transform the y coordinates of the mosaic such that the first image has the same y-coordinate on both the left and right end. Calculate the value of 'a' needed to perform this transformation.
6. (10%) [acc]=AccumulateBlend(img, acc, blendWidth) (file: AccumulatedBlend.m, routine: AccumulatedBlend)
Add up the (α premultiplied) RGBα values at each pixel. Try a simple feathering function as your weighting function (see mosaics lecture slide on "feathering") (this is a simple 1-D version of the distance map described in [Szeliski & Shum]). For extra credit, you can try other blending functions or figure out some way to compensate for exposure differences.
7. (5%) [img]=NormalizeBlend(acc)
(file: NormalizeBlend.m, routine:
NormalizeBlend)
Divide each pixel’s accumulated RGB by its α value. Remember to set the alpha channel of the resultant panorama to opaque!
In addition to your source code, turn in a web page describing your approach and results. In particular:
The web-page should be placed in the artifact directory along with all the images in JPEG format. If you are unfamiliar with HTML you can use any web-page editor such as FrontPage, Word, or Visual Studio 7.0 to make your web-page. Here are some webpage design tips.
Sample SIFT features and matches have been provided to you.
A helper function ipvwrite.m that takes a cell array of the image names and a cell of the output Ms from alignPair as input and outputs ipv for BlendImages is provided.
You can use the test results included in the images/ folder to check whether your program is running correctly. Comparing your output to that of the sample solution is also a good way of debugging your program.
Here is a list of suggestions for extending the program for extra credit. You are encouraged to come up with your own extensions. We're always interested in seeing new, unanticipated ways to use this program!
To turn in your project, submit a single zip file to the dropbox at:
https://catalysttools.washington.edu/collectit/dropbox/iansimon/8903
The zip file should be named project3b.zip and contain all of your
M-files and your assignment writeup as an HTML, PDF,
or Word document. Make sure you include all the images you used for face finding
and cropping (other than the provided cropped images), and your plots. Also
remember the average face and eigenfaces in the first
experiment. Make sure any images that need it are normalized so they can be
seen clearly. Finally, make sure to fully document any extra credit on this web
page.