full resolution jpeg
This part of the project was fairly simple. The hardest part of it was figuring out what algorithm to use for the derivatives in LucasKanade; at first I used the Sobel operators, but found that they gave inferior results to a simple matrix with only two nonzero entries: -1/+1 on either side. This seemed to hold true for the other sequences as well.
full resolution jpeg image
panoramic viewer
This was a very difficult sequence to work with. The exposure on our photos changes wildly, the ground texture is often hard to make out in the black areas, and the parallel trees caused problems for LucasKanade. I implemented a color adjustment feature within LucasKanade itself; the color levels are not adjusted in the output picture, but while it's calculating, it brings the darker of the overlapping pictures up to the level of the higher overlapping picture. This improved LucasKanade significantly at first.
The next problems I ran into were with ghosts. The hill on the right side had trouble acquiring its vertical lock when I ran more than three levels of LucasKanade, while the parallel tres just to the left of the sun were ghosted with less than four or five levels--if I was lucky. There were many other ghosts present around the entire image. Eventually I discovered that running LucasKanade with a very high number of iterations--around 20 or so--was actually effective against these ghosts, and the result has no major ghosting visible. The final result came from 3 levels of LucasKanade and 20 iterations.
We shot some duplicate frames with people in the picture, but I rejected these, as the people occurred in the blending areas and came out transparent.
I added code to automatically fix the left/right end stitching errors by forcing the first and last picture to the same height and interpolating. It worked on this image, but doesn't seem to have functioned on the hand shots; not sure why that is, but it only showed itself after the code was turned in.
full resolution jpeg image
panoramic viewer
The hand-held shots had even more problems with exposure differences, as well as repeating backgrounds (such as the rows of houses along the middle). To make matters worse, the pictures were spaced at more varied intervals. This image originally came out much worse: The path on the far right was very badly acquired, and the houses in the middle also had nasty ghosts. Unlike the Kaidan shots, however, these images had no major vertical problems.
As the ghosts were much bigger than before, I had to resort to running more levels of LucasKanade; this output came from 5 levels at 20 iterations. It came out pretty well, but the path at the right was still not acquiring. It turned out that the spacing between those two images was simply too high: For every single other image I ran, a base of 200 images horizontal was sufficient, but for those two images, I ran LucasKanade with 300 pixel separation, and it worked perfectly.
One important note: While it would have been possible to correct for these exposure problems using Photoshop, I left them exactly as taken as a better demonstration of the algorithm's flexibility. If it can match pictures of widely differing shades, then it's all the more powerful.
