EE/CSE 576 Project 1 -- Feature Detection and Matching

Andrew Chekerylla
University of Washington
Department of Electrical Engineering

Submitted: April 14, 2005

This project aimed to implement methods for feature detection and matching in two-dimensional digital images. Material taken from the lectures and readings was used to develop the approaches and algorithms used for this project. Skeleton code and provided images were used to facilitate development. This project was developed in Microsoft Windows using the Microsoft Visual Studio 2003 .NET development environment.

Most of the effort put forth on this project regarded familarizing the programmer with the C++ language and the Visual Studio environment. This diminished the available time for the project completion and the focus therefore shifted away from implementing the interesting techniques of computer vision and instead towards simply completing the project requirements.

Given more time, many additional features would be implemented. For example, the wavelet indexing technique would be explored as would the use of image pyramids for feature description.

The feature descriptor used intensity values from neighboring pixels to describe each feature. Conceptually, for each pixel that was determined to be a feature, a 5x5 grid was overlaid and the intensity values at these points were saved for later comparison with other features.

The feature descriptor was implemented in the project code by saving information to the (1D) "data" vector of each feature object. Before value assignment, the vector was resized to the appropriate size. Then, the intensity values of the neighboring pixels were stored to the elements of this vector. The features are elsewhere compared by examining the sum of the difference between each pair of pixel intensity values. This technique is known to work best for image translation though not as well for other effects such as rotation or scaling.

The design choices made for this project were made due to the programmer's time constraints. After considering the amount of time required to learn the background C++ material it was determined that two design choices must be made.

(1) Coding effort must be time effective. Therefore, the provided skeleton code was used as much as possible and the developer attempted to use previously acquired coding techniques wherever possible. New or unfamiliar coding techniques, such as using templates, were used sparingly.

(2) No time is available for extra credit. Unfortunately, time was scarce and effort was focused on getting basic functionality working.

The feature matching code performed similarly across each image set. The testBench results are shown below.

Figures 1 and 2: Benchmark results for image sets "graf" and "leuven" respectively

For each set, over 300 pixels average error were observed. This reflects the results of a thresholding approach such as the following:



Figure 3: C++ code snippet for thresholding during matching

MATLAB results of the data set before and after thresholding are included below, as Figures 4 and 5.





Figures 4 and 5: MATLAB plots of relevant data sets before and after thresholding for matching, respectively.

The results of feature thresholding are predicted using these histograms. The score distribution clearly is higher after thresholding and so it would be expected that the feature matching metric (summing up the absolute intensity difference between each pixel in the 5x5 grid around each feature) would determine that the thresholded image would be better than the pre-thresholded image. It was unexpected that the results of this thresholding caused the average error values to increase, compared to those observed before addition of the thresholding module. This may be due to improper implementation and deserves further examination.

The results of feature detection are shown below, as Figures 6 through 11.

Figures 6 through 11: GUI feature detection display for image sets "graf" and "leuven" respectively, using Harris corner detection.

These results show how the Harris corner detection properly identifies many edges and areas of high contrast. It was not determined how to generate a database file for use with the GUI, so image matching was not visualizable during this project. However, feature detection was able to be visualized using the "*.ppm" and "*.f" files with the provided GUI. Again, given more time, this would be an area of additional effort and it would be very satisfying to observe the feature matching using the GUI.

It is noted that these detected corners are thresholded as well to adjust the sensitivity. A code snippet showing this thresholding is shown below, as Figure 12.



Figure 12: C++ code snippet for thresholding during detection

In the previous snippet, the value threshold had been set to 0.01 and set to zero any c value that did not meet or exceed this cutoff point.

The SIFT feature description was observed to be much more effective than the simple window descriptor and the feature descriptor implemented for this project.

The project, as provided, effectively identifies corners via the Harris method and matches many of these features across various images. However, much improvement is possible and given more time it may be possible to improve many of these components in both effectiveness and efficiency.

One specific area of weakness is in the corner detection scheme. The project uses the Harris corner detect approach to highlight corners. However, it uses the color intensity (grayscale) version of the image. This results in overlooking edges that are obvious when observing the color image but are hard to catch when just observing the grayscale image. Further work on this algorithm may yield a more effective result that properly discriminates edges even in areas of similar intensity (but varied color content).

The developer had no access to a camera during this project and so simply used the provided image sets.

Although it was planned to do so, no extra credit work was implemented for this project. Tools such as improved matching via wavelet indexing and improved description via image pyramids would be considered first, given additional time.

(c) Andrew Chekerylla
April 14, 2005
deft1462 (at) ee.washington.edu