| Project 2 winners | ||
| Think about Project 3 | ||
| Guest lecture on Monday: Aseem | ||
| Readings | |||
| S. M. Seitz and C. R. Dyer, Photorealistic Scene Reconstruction by Voxel Coloring, International Journal of Computer Vision, 35(2), 1999, pp. 151-173. | |||
| http://www.cs.washington.edu/homes/seitz/papers/ijcv99.pdf | |||
| What’s the optimal baseline? | ||
| Too small: large depth error | ||
| Too large: difficult search problem | ||
The Effect of Baseline on Depth Estimation
| Basic Approach | |||
| Choose a reference view | |||
| Use your favorite stereo algorithm BUT | |||
| replace two-view SSD with SSD over all baselines | |||
| Limitations | |||
| Must choose a reference view (bad) | |||
| Visibility! | |||
| CMU’s 3D Room Video | |||
Discrete formulation: Voxel Coloring
| Theoretical Questions | ||
| Identify class of all photo-consistent scenes | ||
| Practical Questions | ||
| How do we compute photo-consistent models? | ||
Reconstruction from Silhouettes (C = 2)
| Reconstruction Contains the True Scene | |||
| But is generally not the same | |||
| In the limit (all views) get visual hull | |||
| Complement of all lines that don’t intersect S | |||
Voxel algorithm for volume intersection
| Color voxel black if on silhouette in every image | ||
| for M images, N3 voxels | ||
| Don’t have to search 2N3 possible scenes! | ||
Properties of Volume Intersection
| Pros | ||
| Easy to implement, fast | ||
| Accelerated via octrees [Szeliski 1993] or interval techniques [Matusik 2000] | ||
| Cons | ||
| No concavities | ||
| Reconstruction is not photo-consistent | ||
| Requires identification of silhouettes | ||
Depth Ordering: visit occluders first!
| Cameras oriented in many different directions | ||
| Planar depth ordering does not apply | ||
Compatible Camera Configurations
| Calibrated Turntable | |
| 360° rotation (21 images) |
Voxel Coloring Results (Video)
| A view-independent depth order may not exist |
| Space Carving Algorithm |
| Consistency Property | |||
| The resulting shape is photo-consistent | |||
| all inconsistent points are removed | |||
| Convergence Property | |||
| Carving converges to a non-empty shape | |||
| a point on the true scene is never removed | |||
| The Photo Hull is the UNION of all photo-consistent scenes in V | ||
| It is a photo-consistent scene reconstruction | ||
| Tightest possible bound on the true scene | ||
| The Basic Algorithm is Unwieldy | ||
| Complex update procedure | ||
| Alternative: Multi-Pass Plane Sweep | ||
| Efficient, can use texture-mapping hardware | ||
| Converges quickly in practice | ||
| Easy to implement | ||
| Sweep plane in each of 6 principle directions | ||
| Consider cameras on only one side of plane | ||
| Repeat until convergence | ||
| Sweep plane in each of 6 principle directions | ||
| Consider cameras on only one side of plane | ||
| Repeat until convergence | ||
| Sweep plane in each of 6 principle directions | ||
| Consider cameras on only one side of plane | ||
| Repeat until convergence | ||
| Sweep plane in each of 6 principle directions | ||
| Consider cameras on only one side of plane | ||
| Repeat until convergence | ||
| Sweep plane in each of 6 principle directions | ||
| Consider cameras on only one side of plane | ||
| Repeat until convergence | ||
| Sweep plane in each of 6 principle directions | ||
| Consider cameras on only one side of plane | ||
| Repeat until convergence | ||
Space Carving Results: African Violet
| 24 rendered input views from inside and outside |
| Volume Intersection | ||
| Martin & Aggarwal, “Volumetric description of objects from multiple views”, Trans. Pattern Analysis and Machine Intelligence, 5(2), 1991, pp. 150-158. | ||
| Szeliski, “Rapid Octree Construction from Image Sequences”, Computer Vision, Graphics, and Image Processing: Image Understanding, 58(1), 1993, pp. 23-32. | ||
| Voxel Coloring and Space Carving | ||
| Seitz & Dyer, “Photorealistic Scene Reconstruction by Voxel Coloring”, Proc. Computer Vision and Pattern Recognition (CVPR), 1997, pp. 1067-1073. | ||
| Seitz & Kutulakos, “Plenoptic Image Editing”, Proc. Int. Conf. on Computer Vision (ICCV), 1998, pp. 17-24. | ||
| Kutulakos & Seitz, “A Theory of Shape by Space Carving”, Proc. ICCV, 1998, pp. 307-314. | ||
| Related References | ||
| Bolles, Baker, and Marimont, “Epipolar-Plane Image Analysis: An Approach to Determining Structure from Motion”, International Journal of Computer Vision, vol 1, no 1, 1987, pp. 7-55. | ||
| DeBonet & Viola, “Poxels: Probabilistic Voxelized Volume Reconstruction”, Proc. Int. Conf. on Computer Vision (ICCV) 1999. | ||
| Broadhurst, Drummond, and Cipolla, "A Probabilistic Framework for Space Carving“, International Conference of Computer Vision (ICCV), 2001, pp. 388-393. | ||
| Faugeras & Keriven, “Variational principles, surface evolution, PDE's, level set methods and the stereo problem", IEEE Trans. on Image Processing, 7(3), 1998, pp. 336-344. | ||
| Szeliski & Golland, “Stereo Matching with Transparency and Matting”, Proc. Int. Conf. on Computer Vision (ICCV), 1998, 517-524. | ||
| Roy & Cox, “A Maximum-Flow Formulation of the N-camera Stereo Correspondence Problem”, Proc. ICCV, 1998, pp. 492-499. | ||
| Fua & Leclerc, “Object-centered surface reconstruction: Combining multi-image stereo and shading", International Journal of Computer Vision, 16, 1995, pp. 35-56. | ||
| Narayanan, Rander, & Kanade, “Constructing Virtual Worlds Using Dense Stereo”, Proc. ICCV, 1998, pp. 3-10. | ||