Announcements

Questions on the project?

Updates to project 1 page and lecture slides from 1/18

Midterm (take home) out next Friday

covers material up through next Friday’s lecture

have one week to do it

Late policy is now online

3 free late days over the quarter

can use on any of the projects (not midterm)

Help session on Photoshop at the end of lecture

Segmentation (Part 2)

Today’s Readings

Shapiro, pp. 279-289 (handout)

Watt, 10.3-10.4 (handout)

http://www.dai.ed.ac.uk/HIPR2/morops.htm

Dilation, erosion, opening, closing

From images to objects

What Defines an Object?

Subjective problem, but has been well-studied

Gestalt Laws seek to formalize this

proximity, similarity, continuation, closure, common fate

see notes by Steve Joordens, U. Toronto

Image Segmentation

We will consider a few of these

Last Friday:

Intelligent Scissors (contour-based)

E. N. Mortensen and W. A. Barrett, Intelligent Scissors for Image Composition, in ACM Computer Graphics (SIGGRAPH `95), pp. 191-198, 1995

Normalized Cuts (region-based)

Discussed in Shapiro (handout), Forsyth, chapter 16.5 (supplementary)

Today:

K-means clustering (color-based)

Discussed in Shapiro (handout)

Hough transform (model-based)

Discussed in Watt (handout)

Image histograms

How many “orange” pixels are in this image?

This type of question answered by looking at the histogram

A histogram counts the number of occurrences of each color

Given an image

The histogram is defined to be

What is the dimension of the histogram of an RGB image?

What do histograms look like?

Photoshop demo

Histogram-based segmentation

Goal

Break the image into K regions (segments)

Solve this by reducing the number of colors to K and mapping each pixel to the closest color

photoshop demo

Clustering

How to choose the representative colors?

This is a clustering problem!

Break it down into subproblems

Suppose I tell you the cluster centers c_i

Q: how to determine which points to associate with each c_i?

K-means clustering

K-means clustering algorithm

Randomly initialize the cluster centers, c₁, ..., c_K

Given cluster centers, determine points in each cluster

For each point p, find the closest c_i. Put p into cluster i

Given points in each cluster, solve for c_i

Set c_i to be the mean of points in cluster i

If c_i have changed, repeat Step 2

Java demo: http://www.cs.mcgill.ca/~bonnef/project.html

Properties

Will always converge to some solution

Can be a “local minimum”

does not always find the minimum our objective function:

Cleaning up the result

Problem:

Histogram-based segmentation can produce messy regions

segments do not have to be connected

may contain holes

How can these be fixed?

Dilation operator:

Dilation operator

Demo

http://www.cs.bris.ac.uk/~majid/mengine/morph.html

Erosion operator:

Erosion operator

Demo

http://www.cs.bris.ac.uk/~majid/mengine/morph.html

Nested dilations and erosions

What does this operation do?

Model-based segmentation

Suppose we know the shapes that we’re looking for?

The Hough transform

Option 1:

Search for the object at every possible position in the image

What is the cost of this operation?

Option 2:

Use a voting scheme: Hough transform

Finding lines in an image

Connection between image (x,y) and Hough (m,b) spaces

A line in the image corresponds to a point in Hough space

To go from image space to Hough space:

given a set of points (x,y), find all (m,b) such that y = mx + b

What does a point in the image space map to?

Hough transform algorithm

Typically use a different parameterization

d is the perpendicular distance from the line to the origin

q is the angle this perpendicular makes with the x axis

Why?

Basic Hough transform algorithm

Initialize H[d, q]=0

for each edge point I[x,y] in the image

for q = 0 to 180

H[d, q] += 1

Find the value(s) of (d, q) where H[d, q] is maximum

The detected line in the image is given by

What’s the running time (measured in # votes)?

Extensions

Extension 1: Use the image gradient

same

for each edge point I[x,y] in the image

compute unique (d, q) based on image gradient at (x,y)

H[d, q] += 1

same

same

What’s the running time measured in votes?

Extension 2

give more votes for stronger edges

Extension 3

change the sampling of (d, q) to give more/less resolution

Extension 4

The same procedure can be used with circles, squares, or any other shape

Summary

Things to take away from this lecture

Graph representation of an image

Intelligent scissors method

Normalized cuts method

Image histogram

K-means clustering

Morphological operations

dilation, erosion, closing, opening

Hough transform


	Questions on the project?
	Updates to project 1 page and lecture slides from 1/18
	Midterm (take home) out next Friday
		covers material up through next Friday’s lecture
		have one week to do it
	Late policy is now online
		3 free late days over the quarter
		can use on any of the projects (not midterm)
	Help session on Photoshop at the end of lecture


Today’s Readings
	Shapiro, pp. 279-289 (handout)
	Watt, 10.3-10.4 (handout)
	http://www.dai.ed.ac.uk/HIPR2/morops.htm
		Dilation, erosion, opening, closing


What Defines an Object?
	Subjective problem, but has been well-studied
	Gestalt Laws seek to formalize this
		proximity, similarity, continuation, closure, common fate
		see notes by Steve Joordens, U. Toronto


We will consider a few of these
Last Friday:
	Intelligent Scissors (contour-based)
		E. N. Mortensen and W. A. Barrett, Intelligent Scissors for Image Composition, in ACM Computer Graphics (SIGGRAPH `95), pp. 191-198, 1995
	Normalized Cuts (region-based)
		Discussed in Shapiro (handout), Forsyth, chapter 16.5 (supplementary)
Today:
	K-means clustering (color-based)
		Discussed in Shapiro (handout)
	Hough transform (model-based)
		Discussed in Watt (handout)


How many “orange” pixels are in this image?
	This type of question answered by looking at the histogram
	A histogram counts the number of occurrences of each color
		Given an image

		The histogram is defined to be


		What is the dimension of the histogram of an RGB image?


Goal
	Break the image into K regions (segments)
	Solve this by reducing the number of colors to K and mapping each pixel to the closest color
		photoshop demo


	How to choose the representative colors?
		This is a clustering problem!


	Suppose I tell you the cluster centers c_i
		Q: how to determine which points to associate with each c_i?


K-means clustering algorithm
	Randomly initialize the cluster centers, c₁, ..., c_K
	Given cluster centers, determine points in each cluster
		For each point p, find the closest c_i. Put p into cluster i
	Given points in each cluster, solve for c_i
		Set c_i to be the mean of points in cluster i
	If c_i have changed, repeat Step 2

Java demo: http://www.cs.mcgill.ca/~bonnef/project.html

Properties
	Will always converge to some solution
	Can be a “local minimum”
		does not always find the minimum our objective function:


Problem:
	Histogram-based segmentation can produce messy regions
		segments do not have to be connected
		may contain holes

How can these be fixed?


	Option 1:
		Search for the object at every possible position in the image
		What is the cost of this operation?

	Option 2:
		Use a voting scheme: Hough transform


Connection between image (x,y) and Hough (m,b) spaces
	A line in the image corresponds to a point in Hough space
	To go from image space to Hough space:
		given a set of points (x,y), find all (m,b) such that y = mx + b
	What does a point in the image space map to?


Typically use a different parameterization

	d is the perpendicular distance from the line to the origin
	q is the angle this perpendicular makes with the x axis
	Why?
Basic Hough transform algorithm
	Initialize H[d, q]=0
	for each edge point I[x,y] in the image
		for q = 0 to 180

			H[d, q] += 1
	Find the value(s) of (d, q) where H[d, q] is maximum
	The detected line in the image is given by
What’s the running time (measured in # votes)?


Extension 1: Use the image gradient
	same
	for each edge point I[x,y] in the image
		compute unique (d, q) based on image gradient at (x,y)
			H[d, q] += 1
	same
	same
What’s the running time measured in votes?

Extension 2
	give more votes for stronger edges
Extension 3
	change the sampling of (d, q) to give more/less resolution
Extension 4
	The same procedure can be used with circles, squares, or any other shape


Things to take away from this lecture
	Graph representation of an image
	Intelligent scissors method
	Normalized cuts method
	Image histogram
	K-means clustering
	Morphological operations
		dilation, erosion, closing, opening
	Hough transform