1	Announcements Project1 due Tuesday
2	Motion Estimation Today’s Readings Trucco & Verri, 8.3 – 8.4 (skip 8.3.3, read only top half of p. 199) Supplemental: R. Bergen, P. Anandan, K.J. Hanna, and R. Hingorani. Hierarchical model-based motion estimation. European Conf. on Computer Vision (ECCV), 1992 http://www.cs.washington.edu/education/courses/576/03sp/readings/bergen_eccv92.pdf Numerical Recipes (Newton-Raphson), 9.4 (first four pages) http://www.ulib.org/webRoot/Books/Numerical_Recipes/bookcpdf/c9-4.pdf
3	Why estimate motion? Lots of uses Track object behavior Correct for camera jitter (stabilization) Align images (mosaics) 3D shape reconstruction Special effects
4	Optical flow
5	Problem definition: optical flow How to estimate pixel motion from image H to image I?
6	Optical flow constraints (grayscale images) Let’s look at these constraints more closely
7	Optical flow equation Combining these two equations
8	Optical flow equation Q: how many unknowns and equations per pixel?
9	Aperture problem
10	Aperture problem
11	Solving the aperture problem Basic idea: assume motion field is smooth Horn & Schunk: add smoothness term Lukas & Kanade: assume locally constant motion pretend the pixel’s neighbors have the same (u,v) If we use a 5x5 window, that gives us 25 equations per pixel! works better in practice than Horn & Schunk Many other methods exist. Here’s an overview: Barron, J.L., Fleet, D.J., and Beauchemin, S, Performance of optical flow techniques, International Journal of Computer Vision, 12(1):43-77, 1994. http://www.cs.washington.edu/education/courses/576/03sp/readings/barron92performance.pdf
12	Lukas-Kanade flow How to get more equations for a pixel? Basic idea: impose additional constraints most common is to assume that the flow field is smooth locally one method: pretend the pixel’s neighbors have the same (u,v) If we use a 5x5 window, that gives us 25 equations per pixel!
13	RGB version How to get more equations for a pixel? Basic idea: impose additional constraints most common is to assume that the flow field is smooth locally one method: pretend the pixel’s neighbors have the same (u,v) If we use a 5x5 window, that gives us 25*3 equations per pixel!
14	Lukas-Kanade flow Prob: we have more equations than unknowns
15	Conditions for solvability Optimal (u, v) satisfies Lucas-Kanade equation
16	Eigenvectors of A^TA
17	Edge
18	Low texture region
19	High textured region
20	Observation This is a two image problem BUT Can measure sensitivity by just looking at one of the images! This tells us which pixels are easy to track, which are hard very useful later on when we do feature tracking...
21	Errors in Lukas-Kanade What are the potential causes of errors in this procedure? Suppose A^TA is easily invertible Suppose there is not much noise in the image
22	Improving accuracy Recall our small motion assumption
23	Iterative Refinement
24	Revisiting the small motion assumption Is this motion small enough? Probably not—it’s much larger than one pixel (2^nd order terms dominate) How might we solve this problem?
25	Reduce the resolution!
26	Coarse-to-fine optical flow estimation
27	Coarse-to-fine optical flow estimation
28	Optical flow result
29	Robust methods L-K minimizes a sum-of-squares error metric least squares techniques overly sensitive to outliers
30	Robust optical flow Robust Horn & Schunk Robust Lukas & Kanade
31	Motion tracking Suppose we have more than two images How to track a point through all of the images?
32	Feature Detection
33	Tracking features Feature tracking Compute optical flow for that feature for each consecutive H, I
34	Handling large motions L-K requires small motion If the motion is much more than a pixel, use discrete search instead Given feature window W in H, find best matching window in I Minimize sum squared difference (SSD) of pixels in window Solve by doing a search over a specified range of (u,v) values this (u,v) range defines the search window
35	Tracking Over Many Frames Feature tracking with m frames Select features in first frame Given feature in frame i, compute position in i+1 Select more features if needed i = i + 1 If i < m, go to step 2
36	Incorporating Dynamics Idea Can get better performance if we know something about the way points move Most approaches assume constant velocity or constant acceleration Use above to predict position in next frame, initialize search
37	Feature tracking demo http://www.toulouse.ca/?/CamTracker/?/CamTracker/FeatureTracking.html
38	Image alignment Goal: estimate single (u,v) translation for entire image Easier subcase: solvable by pyramid-based Lukas-Kanade