1	Announcements
2	Image matching
3	Harder case
4	Even harder case
5	Harder still?
6	Answer below (look for tiny colored squares…)
7	Features
8	Image Matching
9	Image Matching
10	Invariant local features Find features that are invariant to transformations geometric invariance: translation, rotation, scale photometric invariance: brightness, exposure, …
11	Advantages of local features Locality features are local, so robust to occlusion and clutter Distinctiveness: can differentiate a large database of objects Quantity hundreds or thousands in a single image Efficiency real-time performance achievable Generality exploit different types of features in different situations
12	More motivation… Feature points are used for: Image alignment (e.g., mosaics) 3D reconstruction Motion tracking Object recognition Indexing and database retrieval Robot navigation … other
13	What makes a good feature?
14	Want uniqueness
15	Local measures of uniqueness
16	Feature detection
17	Feature detection: the math
18	Small motion assumption
19	Feature detection: the math
20	Feature detection: the math
21	Quick eigenvalue/eigenvector review
22	Feature detection: the math
23	Feature detection: the math
24	Feature detection: the math
25	Feature detection summary
26	Feature detection summary
27	The Harris operator
28	The Harris operator
29	Harris detector example
30	f value (red high, blue low)
31	Threshold (f > value)
32	Find local maxima of f
33	Harris features (in red)
34	Invariance Suppose you rotate the image by some angle Will you still pick up the same features? What if you change the brightness? Scale?
35	Scale invariant detection Suppose you’re looking for corners Key idea: find scale that gives local maximum of f f is a local maximum in both position and scale Common definition of f: Laplacian (or difference between two Gaussian filtered images with different sigmas)
36
37
38
39
40
41
42
43
44	Feature descriptors We know how to detect good points Next question: How to match them?
45	Feature descriptors We know how to detect good points Next question: How to match them? Lots of possibilities (this is a popular research area) Simple option: match square windows around the point State of the art approach: SIFT David Lowe, UBC http://www.cs.ubc.ca/~lowe/keypoints/
46	Invariance
47	Invariance
48	How to achieve invariance
49	Rotation invariance for feature descriptors Find dominant orientation of the image window This is given by x₊, the eigenvector of H corresponding to l₊ l₊is the larger eigenvalue Rotate the window according to this angle
50	Multiscale Oriented PatcheS descriptor Take 40x40 square window around detected feature Scale to 1/5 size (using prefiltering) Rotate to horizontal Sample 8x8 square window centered at feature Intensity normalize the window by subtracting the mean, dividing by the standard deviation in the window
51	Detections at multiple scales
52	Scale Invariant Feature Transform
53	SIFT descriptor
54	Properties of SIFT
55	Maximally Stable Extremal Regions Maximally Stable Extremal Regions Threshold image intensities: I > thresh for several increasing values of thresh Extract connected components (“Extremal Regions”) Find a threshold when region is “Maximally Stable”, i.e. local minimum of the relative growth Approximate each region with an ellipse
56	Feature matching
57	Feature distance
58	Feature distance
59	Eliminating bad matches Throw out features with distance > threshold How to choose the threshold?
60	True/false positives The distance threshold affects performance True positives = # of detected matches that are correct Suppose we want to maximize these—how to choose threshold? False positives = # of detected matches that are incorrect Suppose we want to minimize these—how to choose threshold?
61	Evaluating the results How can we measure the performance of a feature matcher?
62	Evaluating the results How can we measure the performance of a feature matcher?
63	More on feature detection/description
64	Lots of applications Features are used for: Image alignment (e.g., mosaics) 3D reconstruction Motion tracking Object recognition Indexing and database retrieval Robot navigation … other
65	Object recognition (David Lowe)
66