Announcements

Project 2 due tomorrow night

Project 3 out Wednesday

Jiun-Hung will take photos beginning of class

Recognition

Readings

Szeliski Chapter 14

Let’s start simple

Today

skin detection

eigenfaces

face detection with adaboost

Face detection

How to tell if a face is present?

One simple method: skin detection

Skin pixels have a distinctive range of colors

Corresponds to region(s) in RGB color space

for visualization, only R and G components are shown above

Skin detection

Learn the skin region from examples

Manually label pixels in one or more “training images” as skin or not skin

Plot the training data in RGB space

skin pixels shown in orange, non-skin pixels shown in blue

some skin pixels may be outside the region, non-skin pixels inside. Why?

Skin classification techniques

Probability

Basic probability

X is a random variable

P(X) is the probability that X achieves a certain value

or

Conditional probability: P(X | Y)

probability of X given that we already know Y

Probabilistic skin classification

Now we can model uncertainty

Each pixel has a probability of being skin or not skin

Learning conditional PDF’s

We can calculate P(R | skin) from a set of training images

It is simply a histogram over the pixels in the training images

each bin R_i contains the proportion of skin pixels with color R_i

Bayes rule

In terms of our problem:

Bayesian estimation

Bayesian estimation

Goal is to choose the label (skin or ~skin) that maximizes the posterior

this is called Maximum A Posteriori (MAP) estimation

Skin detection results

General classification

This same procedure applies in more general circumstances

More than two classes

More than one dimension

Linear subspaces

Classification can be expensive

Must either search (e.g., nearest neighbors) or store large PDF’s

Dimensionality reduction

Linear subspaces

Principal component analysis

Suppose each data point is N-dimensional

Same procedure applies:

The eigenvectors of A define a new coordinate system

eigenvector with largest eigenvalue captures the most variation among training vectors x

eigenvector with smallest eigenvalue has least variation

We can compress the data by only using the top few eigenvectors

corresponds to choosing a “linear subspace”

represent points on a line, plane, or “hyper-plane”

these eigenvectors are known as the principal components

The space of faces

An image is a point in a high dimensional space

An N x M image is a point in R^NM

We can define vectors in this space as we did in the 2D case

Dimensionality reduction

The set of faces is a “subspace” of the set of images

Suppose it is K dimensional

We can find the best subspace using PCA

This is like fitting a “hyper-plane” to the set of faces

spanned by vectors v₁, v₂, ..., v_K

any face

Eigenfaces

PCA extracts the eigenvectors of A

Gives a set of vectors v₁, v₂, v₃, ...

Each one of these vectors is a direction in face space

what do these look like?

Projecting onto the eigenfaces

The eigenfaces v₁, ..., v_K span the space of faces

A face is converted to eigenface coordinates by

Recognition with eigenfaces

Algorithm

Process the image database (set of images with labels)

Run PCA—compute eigenfaces

Calculate the K coefficients for each image

Given a new image (to be recognized) x, calculate K coefficients

Detect if x is a face

If it is a face, who is it?

Choosing the dimension K

How many eigenfaces to use?

Look at the decay of the eigenvalues

the eigenvalue tells you the amount of variance “in the direction” of that eigenface

ignore eigenfaces with low variance

Issues: metrics

What’s the best way to compare images?

need to define appropriate features

depends on goal of recognition task

Metrics

Lots more feature types that we haven’t mentioned

moments, statistics

metrics: Earth mover’s distance, ...

edges, curves

metrics: Hausdorff, shape context, ...

3D: surfaces, spin images

metrics: chamfer (ICP)

...

Issues: feature selection

Issues: data modeling

Generative methods

model the “shape” of each class

histograms, PCA, mixtures of Gaussians

graphical models (HMM’s, belief networks, etc.)

...

Discriminative methods

model boundaries between classes

perceptrons, neural networks

support vector machines (SVM’s)

Generative vs. Discriminative

Issues: dimensionality

What if your space isn’t flat?

PCA may not help

Issues: speed

Case study: Viola Jones face detector

Next few slides adapted Grauman & Liebe’s tutorial

http://www.vision.ee.ethz.ch/~bleibe/teaching/tutorial-aaai08/

Also see Paul Viola’s talk (video)

http://www.cs.washington.edu/education/courses/577/04sp/contents.html#DM

Feature extraction

Large library of filters

AdaBoost for feature+classifier selection

Want to select the single rectangle feature and threshold that best separates positive (faces) and negative (non-faces) training examples, in terms of weighted error.


		Project 2 due tomorrow night
		Project 3 out Wednesday
			Jiun-Hung will take photos beginning of class


Skin pixels have a distinctive range of colors
	Corresponds to region(s) in RGB color space
		for visualization, only R and G components are shown above


Learn the skin region from examples
	Manually label pixels in one or more “training images” as skin or not skin
	Plot the training data in RGB space
		skin pixels shown in orange, non-skin pixels shown in blue
		some skin pixels may be outside the region, non-skin pixels inside. Why?


Basic probability
	X is a random variable
	P(X) is the probability that X achieves a certain value








	or


	Conditional probability: P(X \| Y)
		probability of X given that we already know Y


	Now we can model uncertainty
		Each pixel has a probability of being skin or not skin


We can calculate P(R \| skin) from a set of training images
	It is simply a histogram over the pixels in the training images
		each bin R_i contains the proportion of skin pixels with color R_i


Bayesian estimation
	Goal is to choose the label (skin or ~skin) that maximizes the posterior
		this is called Maximum A Posteriori (MAP) estimation


	This same procedure applies in more general circumstances
		More than two classes
		More than one dimension


	Classification can be expensive
		Must either search (e.g., nearest neighbors) or store large PDF’s


Suppose each data point is N-dimensional
	Same procedure applies:



	The eigenvectors of A define a new coordinate system
		eigenvector with largest eigenvalue captures the most variation among training vectors x
		eigenvector with smallest eigenvalue has least variation
	We can compress the data by only using the top few eigenvectors
		corresponds to choosing a “linear subspace”
			represent points on a line, plane, or “hyper-plane”
		these eigenvectors are known as the principal components


	An image is a point in a high dimensional space
		An N x M image is a point in R^NM
		We can define vectors in this space as we did in the 2D case


The set of faces is a “subspace” of the set of images
	Suppose it is K dimensional
	We can find the best subspace using PCA
	This is like fitting a “hyper-plane” to the set of faces
		spanned by vectors v₁, v₂, ..., v_K
		any face


PCA extracts the eigenvectors of A
	Gives a set of vectors v₁, v₂, v₃, ...
	Each one of these vectors is a direction in face space
		what do these look like?


	The eigenfaces v₁, ..., v_K span the space of faces
		A face is converted to eigenface coordinates by


Algorithm
	Process the image database (set of images with labels)
		Run PCA—compute eigenfaces
		Calculate the K coefficients for each image
	Given a new image (to be recognized) x, calculate K coefficients
	Detect if x is a face


	If it is a face, who is it?


	How many eigenfaces to use?
	Look at the decay of the eigenvalues
		the eigenvalue tells you the amount of variance “in the direction” of that eigenface
		ignore eigenfaces with low variance


	What’s the best way to compare images?
		need to define appropriate features
		depends on goal of recognition task


Lots more feature types that we haven’t mentioned
	moments, statistics
		metrics: Earth mover’s distance, ...
	edges, curves
		metrics: Hausdorff, shape context, ...
	3D: surfaces, spin images
		metrics: chamfer (ICP)
	...


Generative methods
	model the “shape” of each class
		histograms, PCA, mixtures of Gaussians
		graphical models (HMM’s, belief networks, etc.)
		...

Discriminative methods
	model boundaries between classes
		perceptrons, neural networks
		support vector machines (SVM’s)


	Case study: Viola Jones face detector
	Next few slides adapted Grauman & Liebe’s tutorial
		http://www.vision.ee.ethz.ch/~bleibe/teaching/tutorial-aaai08/
	Also see Paul Viola’s talk (video)
		http://www.cs.washington.edu/education/courses/577/04sp/contents.html#DM


	Want to select the single rectangle feature and threshold that best separates positive (faces) and negative (non-faces) training examples, in terms of weighted error.