Announcements
Image matching
Harder case
Even harder case
Harder still?
Answer below (look for
tiny colored squares…)
Features
Image Matching
Image Matching
Invariant local features
|
|
|
|
Find features that are
invariant to transformations |
|
geometric invariance: translation, rotation, scale |
|
photometric invariance: brightness, exposure, … |
Advantages of local
features
|
|
|
|
Locality |
|
features are local, so robust
to occlusion and clutter |
|
Distinctiveness: |
|
can differentiate a large
database of objects |
|
Quantity |
|
hundreds or thousands in a
single image |
|
Efficiency |
|
real-time performance
achievable |
|
Generality |
|
exploit different types of
features in different situations |
More motivation…
|
|
|
|
Feature points are used for: |
|
Image alignment (e.g., mosaics) |
|
3D reconstruction |
|
Motion tracking |
|
Object recognition |
|
Indexing and database retrieval |
|
Robot navigation |
|
… other |
What makes a good
feature?
Want uniqueness
Local measures of
uniqueness
Feature detection
Feature detection: the math
Small motion assumption
Feature detection: the math
Feature detection: the math
Quick
eigenvalue/eigenvector review
Feature detection: the math
Feature detection: the math
Feature detection: the math
Feature detection summary
Feature detection summary
The Harris operator
The Harris operator
Harris detector example
f value (red high, blue
low)
Threshold (f > value)
Find local maxima of f
Harris features (in red)
Invariance
|
|
|
|
Suppose you rotate the image by
some angle |
|
Will you still pick up the same
features? |
|
|
|
What if you change the
brightness? |
|
|
|
Scale? |
Scale invariant detection
|
|
|
|
Suppose you’re looking for
corners |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Key idea: find scale that gives local maximum of f |
|
f is a local maximum in both
position and scale |
Feature descriptors
|
|
|
We know how to detect good
points |
|
Next question: How to match
them? |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Feature descriptors
|
|
|
|
|
We know how to detect good
points |
|
Next question: How to match
them? |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Lots of possibilities (this is
a popular research area) |
|
Simple option: match square windows around the point |
|
State of the art approach: SIFT |
|
David Lowe, UBC http://www.cs.ubc.ca/~lowe/keypoints/ |
Invariance
Invariance
How to achieve invariance
Rotation invariance for
feature descriptors
|
|
|
|
|
Find dominant orientation of
the image patch |
|
This is given by x+,
the eigenvector of H corresponding to l+ |
|
l+ is the larger eigenvalue |
|
Rotate the patch according to
this angle |
Multiscale Oriented PatcheS
descriptor
|
|
|
|
Take 40x40 square window around
detected feature |
|
Scale to 1/5 size (using
prefiltering) |
|
Rotate to horizontal |
|
Sample 8x8 square window
centered at feature |
|
Intensity normalize the window
by subtracting the mean, dividing by the standard deviation in the window |
Detections at multiple
scales
SIFT descriptor
Properties of SIFT
Feature matching
Feature distance
Feature distance
Evaluating the results
|
|
|
How can we measure the
performance of a feature matcher? |
True/false positives
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The distance threshold affects
performance |
|
True positives = # of detected
matches that are correct |
|
Suppose we want to maximize
these—how to choose threshold? |
|
False positives = # of detected
matches that are incorrect |
|
Suppose we want to minimize
these—how to choose threshold? |
Evaluating the results
|
|
|
How can we measure the
performance of a feature matcher? |
Evaluating the results
|
|
|
How can we measure the
performance of a feature matcher? |
Lots of applications
|
|
|
|
Features are used for: |
|
Image alignment (e.g., mosaics) |
|
3D reconstruction |
|
Motion tracking |
|
Object recognition |
|
Indexing and database retrieval |
|
Robot navigation |
|
… other |
Object recognition (David
Lowe)
Slide 55