1	Announcements Midterm due now Project 3 due next Wednesday
2	Multiview stereo Readings (Optional) S. M. Seitz and C. R. Dyer, Photorealistic Scene Reconstruction by Voxel Coloring, International Journal of Computer Vision, 35(2), 1999, pp. 151-173.
3	Choosing the Baseline What’s the optimal baseline? Too small: large depth error Too large: difficult search problem
4	The Effect of Baseline on Depth Estimation
5
6
7	Multibaseline Stereo Basic Approach Choose a reference view Use your favorite stereo algorithm BUT replace two-view SSD with SSD over all baselines Limitations Must choose a reference view (bad) Visibility! CMU’s 3D Room Video
8	The visibility problem
9	Volumetric stereo
10	Discrete formulation: Voxel Coloring
11	Complexity and computability
12	Issues Theoretical Questions Identify class of all photo-consistent scenes Practical Questions How do we compute photo-consistent models?
13
14	Reconstruction from Silhouettes (C = 2)
15	Volume intersection Reconstruction Contains the True Scene But is generally not the same In the limit (all views) get visual hull Complement of all lines that don’t intersect S
16	Voxel algorithm for volume intersection Color voxel black if on silhouette in every image for M images, N³ voxels Don’t have to search 2^N³ possible scenes!
17	Properties of Volume Intersection Pros Easy to implement, fast Accelerated via octrees [Szeliski 1993] Cons No concavities Reconstruction is not photo-consistent Requires identification of silhouettes
18
19	Voxel Coloring Approach
20	Depth Ordering: visit occluders first!
21	Panoramic Depth Ordering Cameras oriented in many different directions Planar depth ordering does not apply
22	Panoramic Depth Ordering
23	Panoramic Layering
24	Panoramic Layering
25	Compatible Camera Configurations
26	Calibrated Image Acquisition Calibrated Turntable 360° rotation (21 images)
27	Voxel Coloring Results (Video)
28	Limitations of Depth Ordering A view-independent depth order may not exist
29
30	Space Carving Algorithm Space Carving Algorithm
31	Convergence Consistency Property The resulting shape is photo-consistent all inconsistent points are removed Convergence Property Carving converges to a non-empty shape a point on the true scene is never removed
32	Which shape do you get? The Photo Hull is the UNION of all photo-consistent scenes in V It is a photo-consistent scene reconstruction Tightest possible bound on the true scene
33
34	Space Carving Algorithm The Basic Algorithm is Unwieldy Complex update procedure Alternative: Multi-Pass Plane Sweep Efficient, can use texture-mapping hardware Converges quickly in practice Easy to implement
35	Multi-Pass Plane Sweep Sweep plane in each of 6 principle directions Consider cameras on only one side of plane Repeat until convergence
36	Multi-Pass Plane Sweep Sweep plane in each of 6 principle directions Consider cameras on only one side of plane Repeat until convergence
37	Multi-Pass Plane Sweep Sweep plane in each of 6 principle directions Consider cameras on only one side of plane Repeat until convergence
38	Multi-Pass Plane Sweep Sweep plane in each of 6 principle directions Consider cameras on only one side of plane Repeat until convergence
39	Multi-Pass Plane Sweep Sweep plane in each of 6 principle directions Consider cameras on only one side of plane Repeat until convergence
40	Multi-Pass Plane Sweep Sweep plane in each of 6 principle directions Consider cameras on only one side of plane Repeat until convergence
41	Space Carving Results: African Violet
42	Space Carving Results: Hand
43	House Walkthrough 24 rendered input views from inside and outside
44	Space Carving Results: House
45	Space Carving Results: House
46	Space Carving Results: House
47	Other Approaches
48	Bibliography Volume Intersection Martin & Aggarwal, “Volumetric description of objects from multiple views”, Trans. Pattern Analysis and Machine Intelligence, 5(2), 1991, pp. 150-158. Szeliski, “Rapid Octree Construction from Image Sequences”, Computer Vision, Graphics, and Image Processing: Image Understanding, 58(1), 1993, pp. 23-32. Voxel Coloring and Space Carving Seitz & Dyer, “Photorealistic Scene Reconstruction by Voxel Coloring”, Proc. Computer Vision and Pattern Recognition (CVPR), 1997, pp. 1067-1073. Seitz & Kutulakos, “Plenoptic Image Editing”, Proc. Int. Conf. on Computer Vision (ICCV), 1998, pp. 17-24. Kutulakos & Seitz, “A Theory of Shape by Space Carving”, Proc. ICCV, 1998, pp. 307-314.
49	Bibliography Related References Bolles, Baker, and Marimont, “Epipolar-Plane Image Analysis: An Approach to Determining Structure from Motion”, International Journal of Computer Vision, vol 1, no 1, 1987, pp. 7-55. DeBonet & Viola, “Poxels: Probabilistic Voxelized Volume Reconstruction”, Proc. Int. Conf. on Computer Vision (ICCV) 1999. Broadhurst, Drummond, and Cipolla, "A Probabilistic Framework for Space Carving“, International Conference of Computer Vision (ICCV), 2001, pp. 388-393. Faugeras & Keriven, “Variational principles, surface evolution, PDE's, level set methods and the stereo problem", IEEE Trans. on Image Processing, 7(3), 1998, pp. 336-344. Szeliski & Golland, “Stereo Matching with Transparency and Matting”, Proc. Int. Conf. on Computer Vision (ICCV), 1998, 517-524. Roy & Cox, “A Maximum-Flow Formulation of the N-camera Stereo Correspondence Problem”, Proc. ICCV, 1998, pp. 492-499. Fua & Leclerc, “Object-centered surface reconstruction: Combining multi-image stereo and shading", International Journal of Computer Vision, 16, 1995, pp. 35-56. Narayanan, Rander, & Kanade, “Constructing Virtual Worlds Using Dense Stereo”, Proc. ICCV, 1998, pp. 3-10.
50	Summary Things to take away from this lecture Baseline tradeoff Multibaseline stereo approach Voxel coloring problem Volume intersection algorithm Voxel coloring algorithm Space carving algorithm