Light

Readings

Forsyth, Chapters 4, 6 (through 6.2)

Properties of light

Today

What is light?

How do we measure it?

How does light propagate?

How does light interact with matter?

What is light?

The light field

Known as the plenoptic function

If you know R, you can predict how the scene would appear from any viewpoint. How?

Stanford light field gantry

More info on light fields

If you’re interested to read more:

The plenoptic function

Original reference: E. Adelson and J. Bergen, "The Plenoptic Function and the Elements of Early Vision," in M. Landy and J. A. Movshon, (eds) Computational Models of Visual Processing, MIT Press 1991.

L. McMillan and G. Bishop, “Plenoptic Modeling: An Image-Based Rendering System”, Proc. SIGGRAPH, 1995, pp. 39-46.

The light field

M. Levoy and P. Hanrahan, “Light Field Rendering”, Proc SIGGRAPH 96, pp. 31-42.

S. J. Gortler, R. Grzeszczuk, R. Szeliski, and M. F. Cohen, "The lumigraph," in Proc. SIGGRAPH, 1996, pp. 43-54.

What is light?

The visible light spectrum

We “see” electromagnetic radiation in a range of wavelengths

Light spectrum

The appearance of light depends on its power spectrum

How much power (or energy) at each wavelength

The human visual system

Color perception

Light hits the retina, which contains photosensitive cells

Density of rods and cones

Rods and cones are non-uniformly distributed on the retina

Rods responsible for intensity, cones responsible for color

Fovea - Small region (1 or 2°) at the center of the visual field containing the highest density of cones (and no rods).

Less visual acuity in the periphery—many rods wired to the same neuron

Demonstrations of visual acuity

Brightness contrast and constancy

The apparent brightness depends on the surrounding region

brightness contrast: a constant colored region seem lighter or darker depending on the surround:

http://www.sandlotscience.com/Contrast/CheckerBoard_illusion.htm

brightness constancy: a surface looks the same under widely varying lighting conditions.

Light response is nonlinear

Our visual system has a large dynamic range

We can resolve both light and dark things at the same time

One mechanism for achieving this is that we sense light intensity on a logarithmic scale

an exponential intensity ramp will be seen as a linear ramp

Another mechanism is adaptation

rods and cones adapt to be more sensitive in low light, less sensitive in bright light.

Visual dynamic range

After images

Tired photoreceptors

Send out negative response after a strong stimulus

Color perception

Three types of cones

Each is sensitive in a different region of the spectrum

but regions overlap

Short (S) corresponds to blue

Medium (M) corresponds to green

Long (L) corresponds to red

Different sensitivities: we are more sensitive to green than red

Colorblindness—deficiency in at least one type of cone

Color perception

Rods and cones act as filters on the spectrum

To get the output of a filter, multiply its response curve by the spectrum, integrate over all wavelengths

Each cone yields one number

Q: How can we represent an entire spectrum with 3 numbers?

Perception summary

The mapping from radiance to perceived color is quite complex!

We throw away most of the data

We apply a logarithm

Brightness affected by pupil size

Brightness contrast and constancy effects

Afterimages

Camera response function

Now how about the mapping from radiance to pixels?

It’s also complex, but better understood

This mapping known as the film or camera response function

Recovering the camera response

Method 1

Carefully model every step in the pipeline

measure aperture, model film, digitizer, etc.

this is *really* hard to get right

Method 2

Calibrate (estimate) the response function

Image several objects with known radiance

Measure the pixel values

Fit a function

Find the inverse: maps pixel intensity to radiance

Recovering the camera response

Method 3

Calibrate the response function from several images

Consider taking images with shutter speeds 1/1000, 1/100, 1/10, and 1

Q: What is the relationship between the radiance or pixel values in consecutive images?

A: 10 times as much radiance

Can use this to recover the camera response function

High dynamic range imaging

Techniques

Debevec: http://www.debevec.org/Research/HDR/

Columbia: http://www.cs.columbia.edu/CAVE/tomoo/RRHomePage/rrgallery.html

Light transport

Slide 26

Slide 27

The interaction of light and matter

What happens when a light ray hits a point on an object?

Some of the light gets absorbed

converted to other forms of energy (e.g., heat)

Some gets transmitted through the object

possibly bent, through “refraction”

Some gets reflected

as we saw before, it could be reflected in multiple directions at once

Let’s consider the case of reflection in detail

In the most general case, a single incoming ray could be reflected in all directions. How can we describe the amount of light reflected in each direction?

The BRDF

The Bidirectional Reflection Distribution Function

Given an incoming ray and outgoing ray
what proportion of the incoming light is reflected along outgoing ray?

Diffuse reflection

Diffuse reflection

Dull, matte surfaces like chalk or latex paint

Microfacets scatter incoming light randomly

Effect is that light is reflected equally in all directions

Diffuse reflection

Specular reflection

Phong illumination model

Phong approximation of surface reflectance

Assume reflectance is modeled by three components

Diffuse term

Specular term

Ambient term (to compensate for inter-reflected light)

Measuring the BRDF

Gonioreflectometer

Device for capturing the BRDF by moving a camera + light source

Need careful control of illumination, environment

Columbia-Utrecht Database

Captured BRDF models for a variety of materials

http://www.cs.columbia.edu/CAVE/curet/.index.html

Advanced topics

Ongoing research in BRDF’s seeks to:

Recover BRDF’s from “just a few” images, model global light transport

Yu, Debevec, Malik and Hawkins, “Inverse Global Illumination”, SIGGRAPH 1999.

Model semi-transparent, refractive surfaces

Zongker, Werner, Curless, and Salesin, “Environment Matting and Compositing”, SIGGRAPH 99, pp. 205-214.

Model sub-surface scattering

Jensen, Marschner, Levoy and Hanrahan: “A Practical Model for Subsurface Light Transport”, SIGGRAPH'2001.


	Today
		What is light?
		How do we measure it?
		How does light propagate?
		How does light interact with matter?



		Known as the plenoptic function
		If you know R, you can predict how the scene would appear from any viewpoint. How?


	If you’re interested to read more:

	The plenoptic function
		Original reference: E. Adelson and J. Bergen, "The Plenoptic Function and the Elements of Early Vision," in M. Landy and J. A. Movshon, (eds) Computational Models of Visual Processing, MIT Press 1991.
		L. McMillan and G. Bishop, “Plenoptic Modeling: An Image-Based Rendering System”, Proc. SIGGRAPH, 1995, pp. 39-46.

	The light field
		M. Levoy and P. Hanrahan, “Light Field Rendering”, Proc SIGGRAPH 96, pp. 31-42.
		S. J. Gortler, R. Grzeszczuk, R. Szeliski, and M. F. Cohen, "The lumigraph," in Proc. SIGGRAPH, 1996, pp. 43-54.


	We “see” electromagnetic radiation in a range of wavelengths


	The appearance of light depends on its power spectrum
		How much power (or energy) at each wavelength


	Color perception
		Light hits the retina, which contains photosensitive cells


	Rods and cones are non-uniformly distributed on the retina
		Rods responsible for intensity, cones responsible for color
		Fovea - Small region (1 or 2°) at the center of the visual field containing the highest density of cones (and no rods).
		Less visual acuity in the periphery—many rods wired to the same neuron


The apparent brightness depends on the surrounding region
	brightness contrast: a constant colored region seem lighter or darker depending on the surround:






		http://www.sandlotscience.com/Contrast/CheckerBoard_illusion.htm

	brightness constancy: a surface looks the same under widely varying lighting conditions.


Our visual system has a large dynamic range
	We can resolve both light and dark things at the same time
	One mechanism for achieving this is that we sense light intensity on a logarithmic scale
		an exponential intensity ramp will be seen as a linear ramp
	Another mechanism is adaptation
		rods and cones adapt to be more sensitive in low light, less sensitive in bright light.


	Tired photoreceptors
		Send out negative response after a strong stimulus


Three types of cones
	Each is sensitive in a different region of the spectrum
		but regions overlap
		Short (S) corresponds to blue
		Medium (M) corresponds to green
		Long (L) corresponds to red
	Different sensitivities: we are more sensitive to green than red
	Colorblindness—deficiency in at least one type of cone


Rods and cones act as filters on the spectrum
	To get the output of a filter, multiply its response curve by the spectrum, integrate over all wavelengths
		Each cone yields one number
	Q: How can we represent an entire spectrum with 3 numbers?


	The mapping from radiance to perceived color is quite complex!
		We throw away most of the data
		We apply a logarithm
		Brightness affected by pupil size
		Brightness contrast and constancy effects
		Afterimages


	Now how about the mapping from radiance to pixels?
		It’s also complex, but better understood
		This mapping known as the film or camera response function


Method 1
	Carefully model every step in the pipeline
		measure aperture, model film, digitizer, etc.
		this is really hard to get right
Method 2
	Calibrate (estimate) the response function
		Image several objects with known radiance
		Measure the pixel values
		Fit a function






		Find the inverse: maps pixel intensity to radiance


Method 3
	Calibrate the response function from several images
		Consider taking images with shutter speeds 1/1000, 1/100, 1/10, and 1
		Q: What is the relationship between the radiance or pixel values in consecutive images?
		A: 10 times as much radiance
		Can use this to recover the camera response function


	Techniques
		Debevec: http://www.debevec.org/Research/HDR/
		Columbia: http://www.cs.columbia.edu/CAVE/tomoo/RRHomePage/rrgallery.html


What happens when a light ray hits a point on an object?
	Some of the light gets absorbed
		converted to other forms of energy (e.g., heat)
	Some gets transmitted through the object
		possibly bent, through “refraction”
	Some gets reflected
		as we saw before, it could be reflected in multiple directions at once

Let’s consider the case of reflection in detail
	In the most general case, a single incoming ray could be reflected in all directions. How can we describe the amount of light reflected in each direction?


	The Bidirectional Reflection Distribution Function
		Given an incoming ray and outgoing ray what proportion of the incoming light is reflected along outgoing ray?


	Diffuse reflection
		Dull, matte surfaces like chalk or latex paint
		Microfacets scatter incoming light randomly
		Effect is that light is reflected equally in all directions


Phong approximation of surface reflectance
	Assume reflectance is modeled by three components
		Diffuse term
		Specular term
		Ambient term (to compensate for inter-reflected light)


	Gonioreflectometer
		Device for capturing the BRDF by moving a camera + light source
		Need careful control of illumination, environment


	Captured BRDF models for a variety of materials
		http://www.cs.columbia.edu/CAVE/curet/.index.html


Ongoing research in BRDF’s seeks to:
	Recover BRDF’s from “just a few” images, model global light transport
		Yu, Debevec, Malik and Hawkins, “Inverse Global Illumination”, SIGGRAPH 1999.
	Model semi-transparent, refractive surfaces
		Zongker, Werner, Curless, and Salesin, “Environment Matting and Compositing”, SIGGRAPH 99, pp. 205-214.
	Model sub-surface scattering
		Jensen, Marschner, Levoy and Hanrahan: “A Practical Model for Subsurface Light Transport”, SIGGRAPH'2001.