Panoramic mosaic stitching

For the most part, this project was implemented according to specifications. The basic setup worked quite well when images were taken at similar exposure and color balance settings. However, when there were more drastic exposure differences (in the HUB lawn sequence, for example), I found that the Lucas-Kanade algorithm didn't align the images well (note the fuzzy crap in the middle of the image). However, if I manually corrected the exposure, then the images would align well.

To correct this issue, I decided to implement a sort of "color transform" operation to correct for exposure and white balance. In the Lucas-Kanade code, there is an initial guess at the overlapping region. The main idea is that, in this overlapping region, the colors should be roughly the same. Of course, there are going to be errors introduced since the images are not perfectly aligned. To resolve this issue, I blurred both images. Afterwards, I determine the least squares cubic polynomial (in RGB space) that transforms the first image to the second. Why cubic? Hmm, why not? Finally, I transform the first image to the color space of the second using the polynomial that we determined. Yes, it sounds hokey, but the results are surprisingly decent (see test sequence panorama).

I didn't end up keeping the color transformations in the final mosaic. It would have been possible to attempt to propagate the color transform through the entire image, but the problem with this approach is that it can potentially cause a nasty color drift problem. But just for kicks, I included a demo image anyway.

For the most part, everything else worked pretty well. There were two little issues that came up. One had to do with the radial distorion numbers, k1 and k2. When I tried using the equations from the lecture slides, the results were not at all like the sample warp08.tga. I just divided instead of multiplied by the square root term, and it seemed to work.

Another problem is the vertical drift. I coded my stuff up with the assumption that the images were taken in counterclockwise order. It messes up when the pictures are taken clockwise. Oh well.

These are manually aligned, by the way. Lucas-Kanade screws up when it's trying to align the images if there's no color transformation.


And now, the artifacts ...

Test image - This worked fairly well despite exposure differences. Compare to this image.

Saurav in some hardware lab - This is in some hardware lab on the fourth floor of Sieg. Probably the best alignment of all the mosaics that I made. Note that the cardboard tube by his foot is kind of blurred; that's just because he kicked it when he moved out of the way.

Nappy Twiggy (handheld) - Chris fell asleep in the lab while I was working on stitching together my handheld panorama. It was going to do this fairly boring shot of MGH 228 before class. Anyway, since I had a digital camera on me, I took a few shots of Chris sleeping. When I noticed that he was completely passed out (no doubt after long nights of working on CSE490CV and research) I decided that I would include him in a panorama. Unfortunately I was unable to wheel Chris around the lab on his rolly chair for a multiple Chris panorama. Due to the handheld camera, there's quite a bit of error in this image, but I tweaked the parameters in an attempt to minimize the nastiness.

Red square at 4 AM - My personal favorite. Daylight is boring, and I had a tripod.

Urinating Eugenes - Chris got upset that I had taken an embarrasing panorama of him, so I returned the favor by posing for a significantly more embarassing one. This is a joint panorama project with Chris for the whistle.