|
|
|
|
Project 3 out today (help session at end of
class) |
|
|
|
|
|
|
|
Readings (Optional) |
|
S. M. Seitz and C. R. Dyer, Photorealistic Scene
Reconstruction by Voxel Coloring, International Journal of Computer Vision,
35(2), 1999, pp. 151-173. |
|
|
|
|
|
What’s the optimal baseline? |
|
Too small:
large depth error |
|
Too large:
difficult search problem |
|
|
|
|
|
|
|
|
Basic Approach |
|
Choose a reference view |
|
Use your favorite stereo algorithm BUT |
|
replace two-view SSD with SSD over all baselines |
|
|
|
Limitations |
|
Must choose a reference view (bad) |
|
Visibility! |
|
|
|
CMU’s 3D Room Video |
|
|
|
|
|
|
|
|
|
Theoretical Questions |
|
Identify class of all photo-consistent scenes |
|
|
|
Practical Questions |
|
How do we compute photo-consistent models? |
|
|
|
|
|
|
|
|
Reconstruction Contains the True Scene |
|
But is generally not the same |
|
In the limit (all views) get visual hull |
|
Complement of all lines that don’t intersect S |
|
|
|
|
|
Color voxel black if on silhouette in every
image |
|
for M images, N3
voxels |
|
Don’t have to search 2N3
possible scenes! |
|
|
|
|
|
Pros |
|
Easy to implement, fast |
|
Accelerated via octrees [Szeliski 1993] |
|
|
|
Cons |
|
No concavities |
|
Reconstruction is not photo-consistent |
|
Requires identification of silhouettes |
|
|
|
|
|
|
|
|
Cameras oriented in many different directions |
|
Planar depth ordering does not apply |
|
|
|
|
|
|
|
|
Calibrated Turntable |
|
360° rotation (21 images) |
|
|
|
|
|
A view-independent depth order may not exist |
|
|
|
|
|
|
|
|
Consistency Property |
|
The resulting shape is photo-consistent |
|
all inconsistent points are removed |
|
Convergence Property |
|
Carving converges to a non-empty shape |
|
a point on the true scene is never removed |
|
|
|
|
|
The Photo Hull is the UNION of all
photo-consistent scenes in V |
|
It is a photo-consistent scene reconstruction |
|
Tightest possible bound on the true scene |
|
|
|
|
|
|
The Basic Algorithm is Unwieldy |
|
Complex update procedure |
|
|
|
Alternative:
Multi-Pass Plane Sweep |
|
Efficient, can use texture-mapping hardware |
|
Converges quickly in practice |
|
Easy to implement |
|
|
|
|
|
|
|
Sweep plane in each of 6 principle directions |
|
Consider cameras on only one side of plane |
|
Repeat until convergence |
|
|
|
|
|
Sweep plane in each of 6 principle directions |
|
Consider cameras on only one side of plane |
|
Repeat until convergence |
|
|
|
|
|
Sweep plane in each of 6 principle directions |
|
Consider cameras on only one side of plane |
|
Repeat until convergence |
|
|
|
|
|
Sweep plane in each of 6 principle directions |
|
Consider cameras on only one side of plane |
|
Repeat until convergence |
|
|
|
|
|
Sweep plane in each of 6 principle directions |
|
Consider cameras on only one side of plane |
|
Repeat until convergence |
|
|
|
|
|
Sweep plane in each of 6 principle directions |
|
Consider cameras on only one side of plane |
|
Repeat until convergence |
|
|
|
|
|
|
24 rendered input views from inside and outside |
|
|
|
|
|
|
|
|
|
Volume Intersection |
|
Martin & Aggarwal, “Volumetric description
of objects from multiple views”, Trans. Pattern Analysis and Machine
Intelligence, 5(2), 1991, pp.
150-158. |
|
Szeliski, “Rapid Octree Construction from Image
Sequences”, Computer Vision, Graphics, and Image Processing: Image
Understanding, 58(1), 1993, pp. 23-32. |
|
Voxel Coloring and Space Carving |
|
Seitz & Dyer, “Photorealistic Scene
Reconstruction by Voxel Coloring”, Proc. Computer Vision and Pattern
Recognition (CVPR), 1997, pp. 1067-1073. |
|
Seitz & Kutulakos, “Plenoptic Image
Editing”, Proc. Int. Conf. on
Computer Vision (ICCV), 1998, pp. 17-24. |
|
Kutulakos & Seitz, “A Theory of Shape by
Space Carving”, Proc. ICCV, 1998,
pp. 307-314. |
|
|
|
|
|
Related References |
|
Bolles, Baker, and Marimont, “Epipolar-Plane
Image Analysis: An Approach to Determining Structure from Motion”,
International Journal of Computer Vision, vol 1, no 1, 1987, pp. 7-55. |
|
DeBonet & Viola, “Poxels: Probabilistic
Voxelized Volume Reconstruction”, Proc. Int. Conf. on Computer Vision
(ICCV) 1999. |
|
Broadhurst, Drummond, and Cipolla, "A
Probabilistic Framework for Space Carving“, International Conference of
Computer Vision (ICCV), 2001, pp. 388-393. |
|
Faugeras & Keriven, “Variational principles,
surface evolution, PDE's, level set methods and the stereo problem",
IEEE Trans. on Image Processing, 7(3), 1998, pp. 336-344. |
|
Szeliski & Golland, “Stereo Matching with
Transparency and Matting”, Proc. Int. Conf. on Computer Vision (ICCV),
1998, 517-524. |
|
Roy & Cox, “A Maximum-Flow Formulation of
the N-camera Stereo Correspondence Problem”, Proc. ICCV, 1998, pp. 492-499. |
|
Fua & Leclerc, “Object-centered surface
reconstruction: Combining
multi-image stereo and shading", International Journal of Computer
Vision, 16, 1995, pp. 35-56. |
|
Narayanan, Rander, & Kanade, “Constructing
Virtual Worlds Using Dense Stereo”, Proc. ICCV, 1998, pp. 3-10. |
|
|
|
|
|
Things to take away from this lecture |
|
Baseline tradeoff |
|
Multibaseline stereo approach |
|
Voxel coloring problem |
|
Volume intersection algorithm |
|
Voxel coloring algorithm |
|
Space carving algorithm |
|