Announcements

Project 1 is out today

help session at the end of class

Segmentation

Today’s Readings

Intelligent Scissors

http://www.cs.washington.edu/education/courses/490cv/02wi/readings/book-7-revised-a-indx.pdf

From images to objects

What Defines an Object?

Subjective problem, but has been well-studied

Gestalt Laws seek to formalize this

proximity, similarity, continuation, closure, common fate

see notes by Steve Joordens, U. Toronto

Segmentation using edges

Edges ¹ Segments

Spurious edges

Missing edges

Extracting objects

How could this be done?

Image Segmentation

Many approaches proposed

color cues

region cues

contour cues

We will consider a few of these

Today:

Intelligent Scissors (contour-based)

E. N. Mortensen and W. A. Barrett, Intelligent Scissors for Image Composition, in ACM Computer Graphics (SIGGRAPH `95), pp. 191-198, 1995

Normalized Cuts (region-based)

J. Shi and J. Malik, Normalized Cuts and Image Segmentation, IEEE Conf. Computer Vision and Pattern Recognition(CVPR), 1997

Discussed in Forsyth, chapter 16.5

Intelligent Scissors

Intelligent Scissors

Approach answers a basic question

Q: how to find a path from seed to mouse that follows object boundary as closely as possible?

A: define a path that stays as close as possible to edges

Intelligent Scissors

Basic Idea

Define edge score for each pixel

edge pixels have low cost

Find lowest cost path from seed to mouse

Path Search (basic idea)

Graph Search Algorithm

Computes minimum cost path from seed to all other pixels

How does this really work?

Treat the image as a graph

Defining the costs

Treat the image as a graph

Defining the costs

c can be computed using a cross-correlation filter

assume it is centered at p

Also typically scale c by it’s length

set c = (max-|filter response|) * length(c)

where max = maximum |filter response| over all pixels in the image

Dijkstra’s shortest path algorithm

Dijkstra’s shortest path algorithm

Properties

It computes the minimum cost path from the seed to every node in the graph. This set of minimum paths is represented as a tree

Running time, with N pixels:

O(N²) time if you use an active list

O(N log N) if you use an active priority queue (heap)

takes < second for a typical (640x480) image

Once this tree is computed once, we can extract the optimal path from any point to the seed in O(N/2) time.

it runs in real time as the mouse moves

What happens when the user specifies a new seed?

Results

How about doing this automatically?

Images as graphs

Fully-connected graph

node for every pixel

link between every pair of pixels, p,q

cost c_pq for each link

c_pq measures dissimilarity

dissimilarity: difference in color and position

this is different than the costs for intelligent scissors

Segmentation by Graph Cuts

Break Graph into Segments

Delete links that cross between segments

Easiest to break links that have high cost

similar pixels should be in the same segments

dissimilar pixels should be in different segments

Cuts in a graph

Link Cut

set of links whose removal makes a graph disconnected

cost of a cut:

Interpretation as a Dynamical System

Treat the links as springs and shake the system

elasticity proportional to cost

vibration “modes” correspond to segments

Color Image Segmentation

Normalize Cut in Matrix Form

After lots of math, we get:


Today’s Readings
	Intelligent Scissors
		http://www.cs.washington.edu/education/courses/490cv/02wi/readings/book-7-revised-a-indx.pdf


What Defines an Object?
	Subjective problem, but has been well-studied
	Gestalt Laws seek to formalize this
		proximity, similarity, continuation, closure, common fate
		see notes by Steve Joordens, U. Toronto


Many approaches proposed
	color cues
	region cues
	contour cues
We will consider a few of these
Today:
	Intelligent Scissors (contour-based)
		E. N. Mortensen and W. A. Barrett, Intelligent Scissors for Image Composition, in ACM Computer Graphics (SIGGRAPH `95), pp. 191-198, 1995
	Normalized Cuts (region-based)
		J. Shi and J. Malik, Normalized Cuts and Image Segmentation, IEEE Conf. Computer Vision and Pattern Recognition(CVPR), 1997
		Discussed in Forsyth, chapter 16.5


	Approach answers a basic question
		Q: how to find a path from seed to mouse that follows object boundary as closely as possible?
		A: define a path that stays as close as possible to edges


Basic Idea
	Define edge score for each pixel
		edge pixels have low cost
	Find lowest cost path from seed to mouse


	Graph Search Algorithm
		Computes minimum cost path from seed to all other pixels


c can be computed using a cross-correlation filter
	assume it is centered at p

Also typically scale c by it’s length
	set c = (max-\|filter response\|) * length(c)
		where max = maximum \|filter response\| over all pixels in the image


Properties
	It computes the minimum cost path from the seed to every node in the graph. This set of minimum paths is represented as a tree
	Running time, with N pixels:
		O(N²) time if you use an active list
		O(N log N) if you use an active priority queue (heap)
		takes < second for a typical (640x480) image
	Once this tree is computed once, we can extract the optimal path from any point to the seed in O(N/2) time.
		it runs in real time as the mouse moves
	What happens when the user specifies a new seed?


Fully-connected graph
	node for every pixel
	link between every pair of pixels, p,q
	cost c_pq for each link
		c_pq measures dissimilarity
			dissimilarity: difference in color and position
			this is different than the costs for intelligent scissors


Break Graph into Segments
	Delete links that cross between segments
	Easiest to break links that have high cost
		similar pixels should be in the same segments
		dissimilar pixels should be in different segments


	Link Cut
		set of links whose removal makes a graph disconnected
		cost of a cut:


	Treat the links as springs and shake the system
		elasticity proportional to cost
		vibration “modes” correspond to segments