CSE576 Project 1: Feature Matching

Feature Descriptor

Our descriptor tried to extend the SIFT feature descriptor, using both an image's gradient information and an image's gray histogram information. Different from the original SIFT descriptor, we extended the features of a point from 16*8 to 16*16, adding 16*8 features for image around each feature point to represent the eight major gray scale component of the image. The log-polar binning structure is used to divide a round image patch around a feature point into 16 sub image patches. It also used the way MOPS use to find a normalized direction for wrapping the image patch, so that the algorithm is rotation invariant.

The values of the eight new features are calculated in this way. For each sub image patch (one of the 16 component of the round area surrounding a feature point), we find the eight discontinuous highest peaks in gray histogram of the image patch. Each peak represents a major gray scale component of the image patch, and we will then calculate the amount of pixels around the peak (within gray scale variation of around 10) in the histogram graph. The eight numbers (the number of pixels of the 4 peaks) are used as 8 features of the image patch. We then use the total number of pixels in the image patch to normalize the feature values.

Design Choice

The algorithm is designed because histogram information is as important as gradient information for an image. By adding the new feature component, we can have a combination of edge information of an image patch, and gray scale distribution of the image patch. At the same time, the method will be invariant to rotation, scaling, and all other transformation that the original SIFT has.

Performance

1. ROC curves

(1) ROC curves for the Graf images

(2) The ROC curves for the Yosemite images

2. Image of Harris operator

(1) Feature Detection for the Graf images

(2) Feature Detection for the Yosemite images

3. AUC for the four classes

Bike

SSD: average AUC: 0.546432

RD: average AUC: 0.587175

Graf

SSD: average AUC: 0.537375

RD: average AUC: 0.564156

Leuven

SSD: average AUC: 0.499849

RD: average AUC: 0.537125

Wall

SSD: average AUC: 0.508768

RD: average AUC: 0.526598

Strength & Weakness

The strength of the method is that the descriptor describes information of an image patch from multiple angles, both its gradient and its histogram. Thus it can be invariant to a lot of factors, including those that SIFT is invariant to. Weakness of the method is that, because of the time limitation, the model is not refined and parameters are not carefully picked. So the test results are not well now. But there is large space for improvements.

Feature Detection Results from Two Personal Images