Announcements

Panorama signups

Panorama project issues?

Nalwa handout

Projective geometry

Readings

Mundy, J.L. and Zisserman, A., Geometric Invariance in Computer Vision, Chapter 23: Appendix: Projective Geometry for Machine Vision, MIT Press, Cambridge, MA, 1992, pp. 463-534 (for this week, read 23.1 - 23.5, 23.10)

available online: http://www.cs.cmu.edu/~ph/869/papers/zisser-mundy.pdf

Projective geometry—what’s it good for?

Uses of projective geometry

Drawing

Measurements

Mathematics for projection

Undistorting images

Focus of expansion

Camera pose estimation, match move

Object recognition

Measurements on planes

Image rectification

Solving for homographies

The projective plane

Why do we need homogeneous coordinates?

represent points at infinity, homographies, perspective projection, multi-view relationships

What is the geometric intuition?

a point in the image is a ray in projective space

Projective lines

What is a line in projective space?

Point and line duality

A line l is a homogeneous 3-vector

It is ^ to every point (ray) p on the line: l p=0

Ideal points and lines

Ideal point (“point at infinity”)

p @ (x, y, 0) – parallel to image plane

It has infinite image coordinates

Homographies of points and lines

Computed by 3x3 matrix multiplication

To transform a point: p’ = Hp

To transform a line: lp=0 ® l’p’=0

3D projective geometry

These concepts generalize naturally to 3D

Homogeneous coordinates

Projective 3D points have four coords: P = (X,Y,Z,W)

Duality

A plane N is also represented by a 4-vector

Points and planes are dual in 3D: N P=0

Projective transformations

Represented by 4x4 matrices T: P’ = TP, N’ = N T^-1

3D to 2D: “perspective” projection

Matrix Projection:

Vanishing points

Vanishing point

projection of a point at infinity

Vanishing points (2D)

Vanishing points

Properties

Any two parallel lines have the same vanishing point

The ray from C through v point is parallel to the lines

An image may have more than one vanishing point

Vanishing lines

Multiple Vanishing Points

Any set of parallel lines on the plane define a vanishing point

The union of all of these vanishing points is the horizon line

also called vanishing line

Note that different planes define different vanishing lines

Computing vanishing points

Properties

P_¥ is a point at infinity, v is its projection

They depend only on line direction

Parallel lines P₀ + tD, P₁ + tD intersect at P_¥

Computing vanishing lines

Properties

l is intersection of horizontal plane through C with image plane

Compute l from two sets of parallel lines on ground plane

All points at same height as C project to l

Provides way of comparing height of objects in the scene

Fun with vanishing points

Perspective cues

Comparing heights

Measuring height

Computing vanishing points (from lines)

Intersect p₁q₁ with p₂q₂

Measuring height without a ruler

The cross ratio

A Projective Invariant

Something that does not change under projective transformations (including perspective projection)

Measuring height

Measurements within reference plane

Solve for homography H relating reference plane to image plane

H maps reference plane (X,Y) coords to image plane (x,y) coords

Fully determined from 4 known points on ground plane

Option A: physically measure 4 points on ground

Option B: find a square, guess the dimensions

Option C: Note H = columns 1,2,4 projection matrix

derive on board

Given (x, y), can find (X,Y) by H^-1

Criminisi et al., ICCV 99

Complete approach

Load in an image

Click on lines parallel to X axis

repeat for Y, Z axes

Compute vanishing points

Specify 3D and 2D positions of 4 points on reference plane

Compute homography H

Specify a reference height

Compute 3D positions of several points

Create a 3D model from these points

Extract texture maps

Output a VRML model

Vanishing points and projection matrix

3D Modeling from a photograph

Camera calibration

Goal: estimate the camera parameters

Version 1: solve for projection matrix

Calibration: Basic Idea

Place a known object in the scene

identify correspondence between image and scene

compute mapping from scene to image

Chromaglyphs

Estimating the Projection Matrix

Place a known object in the scene

identify correspondence between image and scene

compute mapping from scene to image

Direct Linear Calibration

Direct linear calibration

Advantages:

Very simple to formulate and solve

Once you know the projection matrix, can compute intrinsics and extrinsics using matrix factorizations

Alternative: Multi-plane calibration

Summary

Things to take home from this lecture

Homogeneous coordinates and their geometric intuition

Homographies

Points and lines in projective space

projective operations: line intersection, line containing two points

ideal points and lines (at infinity)

Vanishing points and lines and how to compute them

Single view measurement

within a reference plane

height

Cross ratio

Camera calibration

using vanishing points

direct linear method


Readings
	Mundy, J.L. and Zisserman, A., Geometric Invariance in Computer Vision, Chapter 23: Appendix: Projective Geometry for Machine Vision, MIT Press, Cambridge, MA, 1992, pp. 463-534 (for this week, read 23.1 - 23.5, 23.10)
		available online: http://www.cs.cmu.edu/~ph/869/papers/zisser-mundy.pdf


	Uses of projective geometry
		Drawing
		Measurements
		Mathematics for projection
		Undistorting images
		Focus of expansion
		Camera pose estimation, match move
		Object recognition


	Why do we need homogeneous coordinates?
		represent points at infinity, homographies, perspective projection, multi-view relationships
	What is the geometric intuition?
		a point in the image is a ray in projective space


		A line l is a homogeneous 3-vector
		It is ^ to every point (ray) p on the line: l p=0


	Ideal point (“point at infinity”)
		p @ (x, y, 0) – parallel to image plane
		It has infinite image coordinates


	Computed by 3x3 matrix multiplication
		To transform a point: p’ = Hp
		To transform a line: lp=0 ® l’p’=0


These concepts generalize naturally to 3D
	Homogeneous coordinates
		Projective 3D points have four coords: P = (X,Y,Z,W)
	Duality
		A plane N is also represented by a 4-vector
		Points and planes are dual in 3D: N P=0
	Projective transformations
		Represented by 4x4 matrices T: P’ = TP, N’ = N T^-1


	Properties
		Any two parallel lines have the same vanishing point
		The ray from C through v point is parallel to the lines
		An image may have more than one vanishing point


Multiple Vanishing Points
	Any set of parallel lines on the plane define a vanishing point
	The union of all of these vanishing points is the horizon line
		also called vanishing line
	Note that different planes define different vanishing lines


	Properties
		P_¥ is a point at infinity, v is its projection
		They depend only on line direction
		Parallel lines P₀ + tD, P₁ + tD intersect at P_¥


	Properties
		l is intersection of horizontal plane through C with image plane
		Compute l from two sets of parallel lines on ground plane
		All points at same height as C project to l
		Provides way of comparing height of objects in the scene


	A Projective Invariant
		Something that does not change under projective transformations (including perspective projection)


Solve for homography H relating reference plane to image plane
	H maps reference plane (X,Y) coords to image plane (x,y) coords
	Fully determined from 4 known points on ground plane
		Option A: physically measure 4 points on ground
		Option B: find a square, guess the dimensions
		Option C: Note H = columns 1,2,4 projection matrix
			derive on board
	Given (x, y), can find (X,Y) by H^-1


Complete approach
	Load in an image
	Click on lines parallel to X axis
		repeat for Y, Z axes
	Compute vanishing points
	Specify 3D and 2D positions of 4 points on reference plane
	Compute homography H
	Specify a reference height
	Compute 3D positions of several points
	Create a 3D model from these points
	Extract texture maps
	Output a VRML model


	Goal: estimate the camera parameters
		Version 1: solve for projection matrix


	Place a known object in the scene
		identify correspondence between image and scene
		compute mapping from scene to image


	Advantages:
		Very simple to formulate and solve
		Once you know the projection matrix, can compute intrinsics and extrinsics using matrix factorizations


Things to take home from this lecture
	Homogeneous coordinates and their geometric intuition
	Homographies
	Points and lines in projective space
		projective operations: line intersection, line containing two points
		ideal points and lines (at infinity)
	Vanishing points and lines and how to compute them
	Single view measurement
		within a reference plane
		height
	Cross ratio
	Camera calibration
		using vanishing points
		direct linear method