1	Announcements Photos right now Project 3 questions Midterms back at the end of class
2	Image Segmentation Today’s Readings Shapiro, pp. 279-289 http://www.dai.ed.ac.uk/HIPR2/morops.htm Dilation, erosion, opening, closing
3	From images to objects What Defines an Object? Subjective problem, but has been well-studied Gestalt Laws seek to formalize this proximity, similarity, continuation, closure, common fate see notes by Steve Joordens, U. Toronto
4	Image Segmentation We will consider different methods Already covered: Intelligent Scissors (contour-based, manual) Today—automatic methods: K-means clustering (color-based) Normalized Cuts (region-based)
5	Image histograms How many “orange” pixels are in this image? This type of question answered by looking at the histogram A histogram counts the number of occurrences of each color Given an image The histogram is i.e., for each color value c (x-axis), plot # of pixels with that color (y-axis) What is the dimension of the histogram of an NxN RGB image?
6	What do histograms look like? Photoshop demo
7	Histogram-based segmentation Goal Break the image into K regions (segments) Solve this by reducing the number of colors to K and mapping each pixel to the closest color photoshop demo
8	Histogram-based segmentation Goal Break the image into K regions (segments) Solve this by reducing the number of colors to K and mapping each pixel to the closest color photoshop demo
9	Clustering How to choose the representative colors? This is a clustering problem!
10	Break it down into subproblems Suppose I tell you the cluster centers c_i Q: how to determine which points to associate with each c_i?
11	K-means clustering K-means clustering algorithm Randomly initialize the cluster centers, c₁, ..., c_K Given cluster centers, determine points in each cluster For each point p, find the closest c_i. Put p into cluster i Given points in each cluster, solve for c_i Set c_i to be the mean of points in cluster i If c_i have changed, repeat Step 2 Java demo: http://home.dei.polimi.it/matteucc/Clustering/tutorial_html/AppletKM.html Properties Will always converge to some solution Can be a “local minimum” does not always find the global minimum of objective function:
12	Cleaning up the result Problem: Histogram-based segmentation can produce messy regions segments do not have to be connected may contain holes How can these be fixed?
13	Dilation operator:
14	Dilation operator Demo http://www.cs.bris.ac.uk/~majid/mengine/morph.html
15	Erosion operator:
16	Erosion operator Demo http://www.cs.bris.ac.uk/~majid/mengine/morph.html
17	Nested dilations and erosions What does this operation do?
18	Nested dilations and erosions What does this operation do?
19	Nested dilations and erosions What does this operation do?
20	Graph-based segmentation?
21	Images as graphs Fully-connected graph node for every pixel link between every pair of pixels, p,q cost c_pq for each link c_pq measures similarity similarity is inversely proportional to difference in color and position this is different than the costs for intelligent scissors
22	Segmentation by Graph Cuts Break Graph into Segments Delete links that cross between segments Easiest to break links that have low cost (low similarity) similar pixels should be in the same segments dissimilar pixels should be in different segments
23	Cuts in a graph Link Cut set of links whose removal makes a graph disconnected cost of a cut:
24	But min cut is not always the best cut...
25	Cuts in a graph
26	Interpretation as a Dynamical System Treat the links as springs and shake the system elasticity proportional to cost vibration “modes” correspond to segments can compute these by solving an eigenvector problem for more details, see J. Shi and J. Malik, Normalized Cuts and Image Segmentation, CVPR, 1997
27	Interpretation as a Dynamical System
28	Color Image Segmentation