CSE576
Project 2
Aaron Severance
Project 2 involved creating a panorama out of multiple overlapping images. To do this, the images were first warped to spherical coordinates, then aligned using a feature matcher, and finally blended together and exposure balanced (sortof). A more detailed explanation is given after the results are presented.
Clicking on an image will take you to a panorama viewer, and an option to view a full sized image
UW Campus
The sky being messed up is the camera’s fault, I swear. (More in the exposure balancing section). But the grass looks good, as do the buildings. This was the default panorama given along with the project.
Pretentious Panorama Featuring A Sad Roommate
This panorama was taken using the Kaidan head. The images all aligned nicely and came together with little effort. The hardest part may have been getting my model to remain motionless through shots which he was in multiple times. And no, I don’t usually keep a plastic bag on my ceiling fan. Nor would doing that help much with blooming if I did. In any case, all these pictures were taken with auto-white balance and didn’t blend well before exposure balancing, so at least my exposure balance routine somewhat worked.
Beth’s Cafe circa 1:59AM
The final featured panorama was taken using a handheld camera. Unfortunately, it really was just before 2AM and the drunks were about to roll in to the back room, so I didn’t take the time I should have. There is a noticeable seam about 2/3 the way through the panorama, as two of the pictures I took only overlapped by about 10 pixels, and also weren’t quite taken from the same viewpoint. Two other images had to be aligned by hand as there was significant rotation between the shots, but they are much less apparent.
How it goes:
1) Spherical Coordinate Warping & Radial Distortion Correction
This portion was simply geometry. The inverse warp (planar with distortion to spherical without) was computed using the formulas given in class. For the formulas, see the CSE576 homepage and lecture notes.
2) Image Alignment (RANSAC & Least Squares Fit)
Image alignment was accomplished by feature matching two images known to be next to one another. The features used were from David Lowe’s SIFT, which generated extremely excellent feature sets. RANSAC was used to find the best translation, and a least squares fit (for the translation, simply an average) done to find the final translation value.
The implementation was straightforward. The only change from the project spec was to use the floating point coordinates from the SIFT matches instead of integer coordinates when dealing with features and matches. This amounted to modifying about 2 lines of code, and produced slightly more accurate results.
3) Blending and Exposure Balancing
Once the images were aligned, they needed to be blended together. The placement again was geometry and was simple enough. The alpha channel was used to weight pixels lower near the edge of an image; once they were added and normalized feathering was achieved. I found that a having the alpha weights roll off as distance^2 looked nicer than having a linear falloff, though this is entirely subjective and empirical.
Exposure balancing is much harder than it should be. By this I mean that there is are nasty things that happen, like nonlinear variations in pixel value with exposure length. The worst part of this is when a pixel gets saturated, like when taking pictures of light sources. For instance in two pictures the same blade of grass may be off by a factor of two in brightness while the sky is the same in both (bright white!). I decided to attempt exposure correction but to ignore these effects (really, it’s the camera’s fault for not having high enough dynamic range, right?).
To find the exposure difference between two images which already had known regions overlapping I did the following:
This gave a reasonable number for the exposure difference. By multiplying each consecutive correction factor by the previous images, an absolute correction factor for each image can be obtained. And in theory, this would lead to the last image (which is the same as the first image) having a correction factor of 1.0 (the correction factor of the starting image). Of course, since exposure differences are nonlinear, and there’s error in alignment, etc. this does not happen. To compensate for this, pixels were divided by yet another correction factor, (correction of last image)^(x/width). This will give the same exposure correction at the first and last column of pixels, and therefore the panorama will look seamless. The entire panorama is divided by the geometric mean of the exposure differences to give a good average brightness.
The results of this are as good as can be expected given the simplicity of the algorithm. Transitions between images are much smoother and color balanced well. The grass on the UW lawn looks to be of approximately correct brightness throughout the image, as do the buildings. The sky, on the other hand, suffers horribly. The indoor panoramas, while having smooth transitions, have some areas that became brighter than they should be. Again, a better exposure model should fix this.