Motion Estimation
|
|
|
|
|
Today’s Readings |
|
Trucco & Verri, 8.3 – 8.4
(skip 8.3.3, read only top half of p. 199) |
|
Numerical Recipes
(Newton-Raphson), 9.4 (first four pages) |
|
http://www.library.cornell.edu/nr/bookcpdf/c9-4.pdf |
|
|
Why estimate motion?
|
|
|
|
Lots of uses |
|
Track object behavior |
|
Correct for camera jitter
(stabilization) |
|
Align images (mosaics) |
|
3D shape reconstruction |
|
Special effects |
Optical flow
Problem definition: optical flow
|
|
|
How to estimate pixel motion
from image H to image I? |
Optical flow constraints (grayscale
images)
|
|
|
Let’s look at these constraints
more closely |
Optical flow equation
|
|
|
Combining these two equations |
Optical flow equation
|
|
|
Q: how many unknowns and equations per pixel? |
Aperture problem
Aperture problem
Solving the aperture
problem
|
|
|
|
|
|
How to get more equations for a
pixel? |
|
Basic idea: impose additional constraints |
|
most common is to assume that
the flow field is smooth locally |
|
one method: pretend the pixel’s neighbors have the same
(u,v) |
|
If we use a 5x5 window, that
gives us 25 equations per pixel! |
Lucas-Kanade flow
|
|
|
Prob: we have more equations than unknowns |
Conditions for
solvability
|
|
|
|
Optimal (u, v) satisfies
Lucas-Kanade equation |
Errors in Lucas-Kanade
|
|
|
|
What are the potential causes
of errors in this procedure? |
|
Suppose ATA is
easily invertible |
|
Suppose there is not much noise
in the image |
|
|
Improving accuracy
|
|
|
Recall our small motion
assumption |
Iterative Refinement
Revisiting the small
motion assumption
|
|
|
|
Is this motion small enough? |
|
Probably not—it’s much larger
than one pixel (2nd order terms dominate) |
|
How might we solve this
problem? |
Reduce the resolution!
Coarse-to-fine optical
flow estimation
Coarse-to-fine optical
flow estimation
Optical flow result
Motion tracking
|
|
|
|
Suppose we have more than two
images |
|
How to track a point through
all of the images? |
Tracking features
|
|
|
|
Feature tracking |
|
Find feature correspondence
between consecutive H, I |
|
Chain these together to find
long-range correspondences |
|
|
Application: Rotoscoping (demo)