Projection

Readings

Szeliski 2.1

Image formation

Let’s design a camera

Idea 1: put a piece of film in front of an object

Do we get a reasonable image?

Pinhole camera

Add a barrier to block off most of the rays

This reduces blurring

The opening known as the aperture

How does this transform the image?

Camera Obscura

The first camera

Known to Aristotle

How does the aperture size affect the image?

Shrinking the aperture

Why not make the aperture as small as possible?

Shrinking the aperture

Adding a lens

A lens focuses light onto the film

There is a specific distance at which objects are “in focus”

other points project to a “circle of confusion” in the image

Changing the shape of the lens changes this distance

Lenses

A lens focuses parallel rays onto a single focal point

focal point at a distance f beyond the plane of the lens

f is a function of the shape and index of refraction of the lens

Aperture of diameter D restricts the range of rays

aperture may be on either side of the lens

Lenses are typically spherical (easier to produce)

Thin lenses

Thin lens equation:

Any object point satisfying this equation is in focus

What is the shape of the focus region?

How can we change the focus region?

Thin lens applet: http://www.phy.ntnu.edu.tw/java/Lens/lens_e.html (by Fu-Kwun Hwang )

Depth of field

Changing the aperture size affects depth of field

A smaller aperture increases the range in which the object is approximately in focus

The eye

The human eye is a camera

Iris - colored annulus with radial muscles

Pupil - the hole (aperture) whose size is controlled by the iris

What’s the “film”?

Digital camera

A digital camera replaces film with a sensor array

Each cell in the array is a Charge Coupled Device

light-sensitive diode that converts photons to electrons

other variants exist: CMOS is becoming more popular

http://electronics.howstuffworks.com/digital-camera.htm

Issues with digital cameras

Noise

big difference between consumer vs. SLR-style cameras

low light is where you most notice noise

Compression

creates artifacts except in uncompressed formats (tiff, raw)

Color

color fringing artifacts from Bayer patterns

Blooming

charge overflowing into neighboring pixels

In-camera processing

oversharpening can produce halos

Interlaced vs. progressive scan video

even/odd rows from different exposures

Are more megapixels better?

requires higher quality lens

noise issues

Stabilization

compensate for camera shake (mechanical vs. electronic)

Modeling projection

The coordinate system

We will use the pin-hole model as an approximation

Put the optical center (Center Of Projection) at the origin

Put the image plane (Projection Plane) in front of the COP

Why?

The camera looks down the negative z axis

we need this if we want right-handed-coordinates

Modeling projection

Projection equations

Compute intersection with PP of ray from (x,y,z) to COP

Derived using similar triangles (on board)

Homogeneous coordinates

Is this a linear transformation?

Perspective Projection

Projection is a matrix multiply using homogeneous coordinates:

Perspective Projection

How does scaling the projection matrix change the transformation?

Orthographic projection

Special case of perspective projection

Distance from the COP to the PP is infinite

Good approximation for telephoto optics

Also called “parallel projection”: (x, y, z) → (x, y)

What’s the projection matrix?

Orthographic (“telecentric”) lenses

Variants of orthographic projection

Scaled orthographic

Also called “weak perspective”

Affine projection

Also called “paraperspective”

Camera parameters

Distortion

Radial distortion of the image

Caused by imperfect lenses

Deviations are most noticeable for rays that pass through the edge of the lens

Correcting radial distortion

Distortion

Modeling distortion

To model lens distortion

Use above projection operation instead of standard projection matrix multiplication

Other types of projection

Lots of intriguing variants…

(I’ll just mention a few fun ones)

360 degree field of view…

Basic approach

Take a photo of a parabolic mirror with an orthographic lens (Nayar)

Or buy one a lens from a variety of omnicam manufacturers…

See http://www.cis.upenn.edu/~kostas/omni.html

Tilt-shift

Rotating sensor (or object)

Photofinish


	Let’s design a camera
		Idea 1: put a piece of film in front of an object
		Do we get a reasonable image?


	Add a barrier to block off most of the rays
		This reduces blurring
		The opening known as the aperture
		How does this transform the image?


	The first camera
		Known to Aristotle
		How does the aperture size affect the image?


A lens focuses light onto the film
	There is a specific distance at which objects are “in focus”
		other points project to a “circle of confusion” in the image
	Changing the shape of the lens changes this distance


A lens focuses parallel rays onto a single focal point
	focal point at a distance f beyond the plane of the lens
		f is a function of the shape and index of refraction of the lens
	Aperture of diameter D restricts the range of rays
		aperture may be on either side of the lens
	Lenses are typically spherical (easier to produce)


	Thin lens equation:


		Any object point satisfying this equation is in focus
		What is the shape of the focus region?
		How can we change the focus region?
		Thin lens applet: http://www.phy.ntnu.edu.tw/java/Lens/lens_e.html (by Fu-Kwun Hwang )


	Changing the aperture size affects depth of field
		A smaller aperture increases the range in which the object is approximately in focus


	The human eye is a camera
		Iris - colored annulus with radial muscles
		Pupil - the hole (aperture) whose size is controlled by the iris
		What’s the “film”?


A digital camera replaces film with a sensor array
	Each cell in the array is a Charge Coupled Device
		light-sensitive diode that converts photons to electrons
		other variants exist: CMOS is becoming more popular
		http://electronics.howstuffworks.com/digital-camera.htm


	Noise
			big difference between consumer vs. SLR-style cameras
			low light is where you most notice noise
	Compression
			creates artifacts except in uncompressed formats (tiff, raw)
	Color
			color fringing artifacts from Bayer patterns
	Blooming
			charge overflowing into neighboring pixels
	In-camera processing
			oversharpening can produce halos
	Interlaced vs. progressive scan video
			even/odd rows from different exposures
	Are more megapixels better?
			requires higher quality lens
			noise issues
	Stabilization
			compensate for camera shake (mechanical vs. electronic)


The coordinate system
	We will use the pin-hole model as an approximation
	Put the optical center (Center Of Projection) at the origin
	Put the image plane (Projection Plane) in front of the COP
		Why?
	The camera looks down the negative z axis
		we need this if we want right-handed-coordinates


	Projection equations
		Compute intersection with PP of ray from (x,y,z) to COP
		Derived using similar triangles (on board)


	Projection is a matrix multiply using homogeneous coordinates:


	How does scaling the projection matrix change the transformation?


	Special case of perspective projection
		Distance from the COP to the PP is infinite







		Good approximation for telephoto optics
		Also called “parallel projection”: (x, y, z) → (x, y)
		What’s the projection matrix?


	Scaled orthographic
		Also called “weak perspective”





	Affine projection
		Also called “paraperspective”


	Radial distortion of the image
		Caused by imperfect lenses
		Deviations are most noticeable for rays that pass through the edge of the lens


	To model lens distortion
		Use above projection operation instead of standard projection matrix multiplication


	Lots of intriguing variants…
	(I’ll just mention a few fun ones)


Basic approach
	Take a photo of a parabolic mirror with an orthographic lens (Nayar)
	Or buy one a lens from a variety of omnicam manufacturers…
		See http://www.cis.upenn.edu/~kostas/omni.html